- Team Update and Travel Plans
Quick recap. The meeting was an open office hours session led by Neil, with Shawn and others joining for discussions on cloud security, AI, and related topics. Shawn announced the addition of a "Buy Me a Coffee" link for group support due to unsuccessful sponsored mail experiments. The group discussed email service options for a small law office, weighing Google and Microsoft solutions with considerations for data sovereignty and ease of management. A significant portion of the conversation focused on recent AI security incidents involving OpenAI and Anthropic, with participants critiquing the companies' security practices and marketing strategies. The discussion also covered the broader implications of AI on software development, code understanding, and organizational workflows, with varying opinions on its utility and risks. Other topics included the upcoming Black Hat conference, personal experiences with AI tools, and the economic and regulatory challenges posed by AI technology.
Show 10 discussion topics
Team Update and Travel Plans
The meeting began with technical setup issues as Shawn had difficulty accessing his system and needed to share a link with Dave. The participants discussed their current work situations, with Dave reporting being extremely busy with multiple clients and deliverables while experiencing fatigue, while Neil mentioned having limited work and no new leads. Dave shared his upcoming travel plans to Florida in November, including attending IMATS/IT Nation and visiting customers near Vero Beach, and mentioned he would be in Boston on November 3rd and 4th, potentially with some free time for Shawn to visit.
Newsletter Support Strategy Update
Shawn announced that the group's email newsletter experiment with sponsored links was unsuccessful, so he set up a "Buy Me a Coffee" donation option on the website as an alternative way for members to support the group's expenses. The discussion then shifted to casual conversation about doom scrolling, with participants sharing their experiences and definitions of what constitutes doom scrolling behavior.
Law Firm Email Service Options
The group discussed email service options for a small law firm whose current provider (mail.com) is discontinuing lawyer.com service. The main options considered were Google Gmail and Microsoft 365, with Microsoft being recommended for its better integration capabilities and security features, though it requires more management expertise. The discussion also covered business email compromise risks specific to law firms, with participants agreeing that technical controls alone cannot prevent these attacks and that strong processes and user training are essential.
AI Company Security Breaches
The team discussed recent security incidents involving AI companies, particularly focusing on breaches at OpenAI and Anthropic that were revealed through CTF competitions. Neil criticized OpenAI's security practices as inadequate, noting they had no proper air-gapping, firewall rules, or monitoring that would be expected in a mature security environment. The group agreed these incidents represented poor security practices rather than innovative attacks, with Juninho observing that the companies were using the breaches for marketing purposes while highlighting existing security failures.
AI Security Vulnerability Discussion
The group discussed security issues with OpenAI and Anthropic's AI models, focusing on recent incidents where the models were found to have security vulnerabilities. Alex expressed concerns about OpenAI's profitability and lack of valid pathway to profitability, while Matt and Neil highlighted the companies' irresponsibility in securing their models. The discussion concluded with Neil emphasizing that enterprises using AI tools must implement strong monitoring, controls, and security measures around their AI implementations to mitigate risks.
AI Security and Governance Challenges
The group discussed challenges around AI security and governance, with Dave sharing that the CEO was rejecting strict AI usage policies, preferring a more permissive approach. Participants discussed various security measures including CASB rules, managed settings files, and training approaches, with Steve sharing his experience implementing Anthropic's Claude at his company. The conversation highlighted concerns about AI reducing critical thinking skills and creating alert fatigue, with participants noting that perfect security controls are impossible and risk management focuses on probability reduction rather than prevention. Neil announced plans for an informal CSOH breakfast at Black Hat on Wednesday at 8 AM in the Luxor food court, asking attendees to confirm in the signal group.
AI in Coding and Automation
The group discussed the use of AI and automation in coding and business processes. Matt shared his experience using AI to assist with recognizing programming patterns in x86-64 assembly, while Neil and others agreed that AI can be useful for repetitive, well-defined tasks but should not be relied on for critical or complex work without human oversight. The discussion included examples of successful AI implementation in specific contexts, such as converting Docker files, while emphasizing the importance of understanding the underlying processes and maintaining human involvement in important decisions.
AI Tools in Development Workflows
The group discussed the practical applications and limitations of AI tools in software development workflows. Matt shared his experience using AI for reproducing product issues and writing Terraform configurations, while Neil expressed skepticism about the hype surrounding AI tools. JD argued that many developers lack deep understanding of their own code, particularly in large organizations, making AI tools potentially valuable for code analysis and documentation. The discussion evolved into a debate about whether AI adoption is creating new bottlenecks by requiring more human oversight and validation of AI-generated code, with participants considering how traditional specialized roles like system engineers and security experts might become more valuable in AI-driven development environments.
AI Implementation in Enterprise Settings
The group discussed the challenges and potential future of AI implementation in enterprise settings, with JD sharing an experience about using AI tools like Claude Code to assist with tasks such as maintaining tickets and writing updates. Matt and Brian explored how organizations might function with increased AI adoption, comparing it to historical technological transitions like the automobile and power industry, and debated the economic and regulatory implications of AI development. The conversation touched on the need for regulations around AI usage, drawing parallels to past technological innovations and their associated risks and safety measures.
AI Safety and Surveillance Challenges
Matt and Brian discussed the challenges of AI systems achieving goals without proper constraints, comparing this to human behavior in corporate settings. They explored how AI models can produce unwanted outputs when trained on vast amounts of internet data, with Matt explaining the concept of "obliterated models" that have safety constraints removed. The conversation also touched on government surveillance practices in China versus the US, with Matt arguing that the main difference is whether surveillance is conducted by government or private corporations.