Friday, July 19, 2024 — Meeting Recap

Equinix Team Addresses Infrastructure Outage

Shawn, Matt Alvarez, and Cooper from Equinix discussed the recent infrastructure outage caused by an automatic software update. They discussed the difficulties in fixing the issue, including the need to boot into safe mode and the potential lack of access to the necessary 48-bit locker key. The conversation also touched on the negative impact on the company's reputation. Shawn shared his experience in the data center industry, revealing that he was part of the team that created the Colo business at Exodus Communications, which grew to a valuation of $28 billion. The team also discussed the trend of customers preferring to rent server space rather than manage their own infrastructure.

Team Updates and New Members Discussed

The team discussed recent events including a Reddit outage and issues with CrowdStrike. Shawn introduced new members, Thomas and Cooper, who are looking to expand their knowledge and network. Shawn also encouraged the group to take advantage of the forum for networking and learning. Jay shared that he had released a blog and was open to discussions about it.

AI, Container Configuration, and CrowdStrike

Jay, Shawn, Jim, and Galan discussed the potential of AI and its impact on container configuration. Shawn praised the research team's work, while Jay admired a particular solution's elegance. The team also shared their experiences during the recent CrowdStrike incident, with Galan sharing his personal impact due to issues with hotel key cards. The group agreed to stay updated on future developments in the field. The team also discussed the global impact of a recent update to the CrowdStrike software, which caused many systems to crash and display a blue screen of death, akin to the disruption caused by the '90s worms, Sobig and Slammer.

Microsoft Azure Mitigation Concerns and Solutions

Neil expressed concerns about Microsoft Azure's guidance for mitigation, noting that some customers had to reboot their machines multiple times. Adam St., Shawn, and Galan discussed the effectiveness of suggested solutions and the challenges of repairing Azure-based machines. The team also considered the potential of buying Crowdstrike stock due to issues with Azure and discussed its extensive integration with tech companies and the US government. The conversation concluded with Neil and Jay speculating on the breadth of Microsoft's Defender tool and the significant number of companies affected by recent events.

Global Outage Implications and Backup Solutions

The team engaged in a discussion about the potential implications of a global outage, drawing from their experiences at Crowdstrike and in the medical sector. Shawn highlighted the financial and liability concerns, while Samantha explained that her hospital has backup systems in place to ensure continuity of care during network downtime. The team also discussed the potential change in the company's operating procedure to a multi-cloud system for backup purposes, prompted by the risk of data loss. Concerns were raised about the lack of a backup system for Windows-based medical devices, and the team acknowledged the current efforts to address this issue.

System Outage Consequences and Solutions

The team discussed the potential consequences of a system outage, focusing on its impact on the health industry and the stock market. Don raised concerns about the inability to access medical records in emergencies, while Alhaji highlighted the potential loss of life and the drop in share prices for companies like Crowdstrike. The team also explored the potential for customers to seek alternatives and the risk of lawsuits following an outage. Lastly, they discussed the potential reevaluation of technology, particularly the use of eBPF over kernel-level solutions, and the challenges of regular updates for mission-critical systems.

Crowdstrike Senate Hearing and Share Price Impact

Matt Alvarez opened a discussion about setting a date for the Crowdstrike Senate hearing. The conversation then shifted to the recent security issue that led to a significant drop in share prices. Kurt expressed curiosity about how long it would take for Crowdstrike to publish the root cause of the issue, with Jay suggesting it could take 3 to 6 weeks. Neil shared a past experience from Microsoft about a similar issue, emphasizing the importance of thorough testing before rolling out new software. Brandon then discussed the impact of the incident on cyber insurance claims, highlighting that it was a significant uptick. The conversation concluded with SLuengo@BankUnited.com acknowledging that such incidents could happen to any company.

Crowdstrike Challenges and Google Acquisition

The team discussed the recent challenges faced by Crowdstrike and possible solutions, with a focus on the importance of a deployment strategy. They also explored the potential implications of Google's proposed acquisition of Palo Alto, with Shawn, Neil, Jay, and Kurt expressing their views. The team agreed that such acquisitions are common in the tech industry and should not significantly alter their relationships, though Palo Alto might emphasize its independence more. Kurt added that any initial disruptions would likely be settled quickly as all parties aim to remain profitable.

Co-Opetition Discussion and Feedback

Shawn led a discussion about co-opetition, referencing Ray's competitive strategy towards Microsoft. Dane shared his perspective as an Orca and Google customer, expressing his appreciation for Google's recent efforts. Shawn then invited feedback and suggestions for future topics, mentioning that all members would receive a lapel pin. The conversation ended with Shawn sharing an example of co-opetition from his own experience and announcing the next meeting.