EP267 AI SOC or AI in a SOC? Cutting Through Hype, Pricing Models, and SIEM Detection Efficacy with Raffy Marty

The hosts discuss the ongoing relevance of traditional Security Information and Event Management (SIEM) systems, emphasizing that claims of their obsolescence are more marketing hype than fact. Rafi Marty highlights that SIEMs have some gaps, but these are often temporary and can be addressed by the incumbents through fixing existing issues. The conversation emphasizes the importance of locality for correlation, stateful, real-time detection, which challenges the idea of fully federated or decoupled architectures delivering the same fidelity. The rise of data pipelines is seen as creating a 'Trojan horse', making SIEM components potentially swappable, and vendors should have adapted earlier to these shifts.

The podcast explores whether AI can truly revolutionize detection engineering (DE) in security. Marty notes that despite historical efforts—from rules and heuristics to ML and now AI/agent-based approaches—the core issues of environment variability and customization remain. He stresses the importance of context graphs for entity monitoring and risk assessment, seeing this as a potential 'holy grail.' The discussion also touches on how detection rules could become more intelligent by verifying the presence of relevant data, thus reducing false positives and improving effectiveness. The influence of AI in automating and improving DE, but with the caveat that fundamental challenges still exist, is a key point.

The hosts delve into the flaws of volume-based pricing for SIEM products, noting that it often penalizes good security hygiene, as organizations that send more data pay more. Marty advocates for pricing models that fairly account for compute, enrichment, and retention, rather than volume alone. The discussion covers how vendors encouraging customers to bring their own compute and storage could reduce costs but shift responsibilities. The importance of providing effective management tools for different customer sizes, especially SMBs, is also emphasized. The conversation highlights that pricing strategies influence market adoption and vendor sustainability.

The episode considers whether modern detection engineering can be automated or significantly improved through AI. Marty recalls the history of detection rule creation—from manual rule writing to ML—and emphasizes the need for systems that verify whether data will trigger specific detections, potentially providing recommendations for optimization. The challenge of environment variability remains, and advanced concepts like context graphs for entity and risk management are proposed as future directions. The discussion underscores that better metrics beyond coverage, such as quality and relevance of detection engineering output, are needed for evaluating vendors. AI's role in automating detection content remains promising but not yet fully realized.

The hosts discuss the resurgence of Network Detection and Response (NDR) technologies, emphasizing their importance as telemetry sources rather than standalone solutions. Marty explains that network data remains valuable for visibility, especially in environments where endpoint data is unreliable or inaccessible, such as cloud migrations or sensitive regions with strict data laws. The conversation clarifies that NDR is not likely to replace other security telemetry sources but complements them, offering critical insights into network activity and potential threats. The idea of response capabilities at the network level is also acknowledged as an important aspect.