State of AppSec (Panel)

State of AppSec (Panel)

Speakers: Seth Law, Ken Johnson, Lakshmi Sudheer, Ariel Shin

Conference: BSidesSF 2025 — April 26-27, 2025, San Francisco

YouTube: https://www.youtube.com/watch?v=ScE4PMn6IZM

Reading time: ~8 minutes

TL;DR

At BSidesSF 2025, an experienced AppSec panel examined where application security has genuinely improved, where it remains stubbornly broken, and how AI-driven development is reshaping both the threat landscape and the practice of securing it. The consensus: secure-by-default approaches have moved the needle, developer-security relationships remain the hardest unsolved problem, and "vibe coding" may accelerate breaches before it accelerates fixes.

Introduction

Absolute AppSec's Ken Johnson and Seth Law, joined by Netflix's Lakshmi Sudheer and consultant Ariel Shin, assembled for a live in-person recording of their show at BSidesSF 2025 — a format that gave the panel a distinctly candid, unscripted texture. The session covered ground from secure-by-default program design to the concrete security implications of vibe coding, modern breach patterns, developer education, and predictions for where AppSec is heading in the next twelve months.

What emerged was a picture of a field that has made genuine, measurable progress on specific technical fronts — particularly around eliminating whole classes of vulnerabilities — while remaining fundamentally challenged by organizational, cultural, and relational problems that tools and frameworks alone cannot solve.

Secure by Default: Progress and Limits

▶ Watch: Secure-by-default programs and what's still broken (02:00)

Ariel Shin opened by crediting secure-by-default initiatives as the most meaningful advancement in AppSec in recent years — the ability to eliminate entire vulnerability classes through platform-level defaults rather than chasing individual instances. Ken Johnson cited GitHub as a concrete example: heavy investment in cross-site scripting prevention via Content Security Policy and secure-by-default HTML rendering reduced XSS from a persistent category of findings to a substantially managed problem.

But the panel converged quickly on the limits of this approach. Lakshmi Sudheer, drawing on her work at Netflix, invoked a well-known framing from Jason Chan: "We create paved roads — it's our responsibility as security engineers that they don't lead to a cliff." The challenge is not building the paved road; it's keeping it maintained as technology domains evolve. Netflix is moving rapidly into AI-driven product development, and the security team must build new paved roads for AI infrastructure at the same speed the business adopts it.

Risk prioritization was flagged as the category still most broken. Sudheer identified two failure modes: the persistent presence of unknown unknown risks (by definition absent from any risk register), and the uncomfortable reality that the next incident is often completely unrelated to whatever appears at the top of the risk register. Persistent medium and low risks accumulate without adequate remediation resources, and the most imminent threats frequently arrive through vectors the team hadn't fully characterized.

Vibe Coding: Net Positive or Liability Factory?

▶ Watch: Panel debate on vibe coding and AI-generated code (14:00)

No AppSec conversation in 2025 can avoid vibe coding — the practice of using generative AI to produce code with minimal traditional development expertise. The panel's discussion was notably nuanced.

Ken Johnson drew a distinction between two definitions. The first: AI-assisted development used by experienced engineers as an accelerated template generator, with competent review of outputs. The second, more troubling definition: purely prompt-driven code generation by people without the skills to meaningfully review what's produced, deployed directly toward production. Johnson considered the first a net positive; the second a genuine risk amplifier.

Ariel Shin focused on scale as the critical variable: the same organizational maturity issues that have always plagued AppSec will be reproduced at much larger scale and with greater accessibility. "If we can build a vibe coding solution trained properly on good code rather than Stack Overflow, I'd have my engineers use that." But she was clear that the current state doesn't meet that bar — and that breaches related to vibe coding are inevitable. She cited a real example of a developer who built and deployed an entire SaaS product through AI-generated code, which was subsequently compromised.

Sudheer pointed to a more subtle, structural risk: the scale of AI-generated code means that even low false-positive-rate security tooling will be overwhelmed by absolute finding volumes. Current SAST and DAST tools weren't built for the pace at which AI assistants can produce code changes. The organizational question of how to triage and govern at that scale remains unanswered.

Modern Breaches: Still Not Novel

▶ Watch: Breach patterns and what keeps panelists up at night (20:01)

When asked whether modern breaches are exposing new blind spots or simply revealing failures to execute on the basics, the panel was largely unequivocal: it's mostly the basics. Ariel Shin noted that the majority of high-profile incidents in recent memory were not caused by novel vulnerabilities — they were caused by exposed credentials, leaked secrets, and supply chain attacks exploiting well-understood vectors.

Shin invoked a comparison to a BSidesSF talk from the previous day about AI apocalypse scenarios: "How has everything been the same since the Morris worm from ages ago?" The attack surface has shifted — from front-end XSS and credential stuffing, toward CI/CD pipeline compromise, cloud misconfiguration, and now AI infrastructure — but the root causes remain consistent. Credentials get leaked. Configurations get misconfigured. Secrets end up in repositories.

Ken Johnson made the risk prioritization mismatch concrete: organizations are spending significant security resources on AI chatbot risk (often a minimal-consequence marketing-site RAG application) while credentials continue to leak on GitHub. "That is the pattern we're dealing with — and the risk prioritization is off." The challenge is that as an external consultant, he can identify the critical risks, but changing organizational prioritization requires internal political leverage that third parties rarely have.

Developer Education: What Actually Works

▶ Watch: Security training debate and what resonates with developers (34:02)

A question from the audience asked the panel to "discuss why security training is a waste of time" — and the panel mostly agreed with the framing, with important caveats. The consensus: generic security awareness training is largely ineffective. Effective training is tangible, contextual, and connected to real consequence.

Ken Johnson described an approach used at GitHub: training sessions built around actual bug bounty reports, walking engineers through the technical root cause and attack chain of vulnerabilities that had been submitted against GitHub's own systems — and the dollar amounts paid for those vulnerabilities. That tangibility changed how engineers engaged with the material. Shin reinforced this with her own experience: engineers respond when you show them a real $1,000 bug bounty finding and explain how a medium-severity vulnerability was chained with a low-severity one to create a critical exploit path. The sky-is-always-falling tone of generic security training produces the opposite effect — desensitization.

Sudheer added a more fundamental question: before designing training, a team should ask why they're training. Is the goal to fill a gap that tooling genuinely can't address? Is it to build a security-first culture? Is it to change behavior for a specific class of risk? Generic training answers none of those questions well. Training embedded in developer workflow — threat model guidance that appears as a developer writes code, rather than as an annual compliance module — may be more effective, though the tooling to deliver this contextually is still emerging.

Predictions: The Next Year in AppSec

▶ Watch: Panel predictions for AppSec trends in 2026 (40:02)

Each panelist offered a prediction for the dominant AppSec trend of the next year — and what they wished would take off.

Sudheer predicted more investment in secure-by-default platforms specifically for AI infrastructure — "AI paved roads" — and expressed a desire to stop seeing products falsely marketed as "powered by AI."

Shin predicted AI would continue to dominate the conversation, wished for greater industry focus on governance frameworks around AI coding tools, and hoped for better approaches to risk scoring that incorporate real-world exploitability data rather than pure CVSS severity.

Johnson flagged the CVE ecosystem as an area under stress, and predicted progress on reachability analysis — understanding whether a vulnerable library function is actually called in an application's execution path — as enriched vulnerability metadata becomes more available.

Seth Law predicted "agentic AI" would become the next term everyone gets sick of hearing, and expressed a genuine wish that security prompt sharing would take off as a community practice — arguing that organizations hoarding their LLM prompts as proprietary advantages are missing an opportunity for collective improvement.

Notable Quotes

"We create paved roads. It's our responsibility as security engineers that they don't lead to a cliff."

— Lakshmi Sudheer, citing Jason Chan ▶ 04:00

"The breach has occurred, and they're pulling everything via your API because your credentials got leaked on GitHub — that is the pattern we're dealing with — and the risk prioritization is off."

— Ken Johnson ▶ 24:01

"If you provide them a $1,000 bug bounty finding and show how an attacker was able to chain a low and medium vulnerability together — that's actually really, really impactful to engineers."

— Ariel Shin ▶ 34:02

Key Takeaways

• Secure-by-default programs work — but need sustained partnership. Eliminating vulnerability classes at the platform level is the most leveraged AppSec investment, but it requires deep collaboration with platform engineering teams and ongoing evolution as technology domains shift.
• Risk prioritization remains the field's most persistent failure mode. The next breach is often unrelated to the current risk register's top five. Organizations need honest assessments of exploitability and should be willing to walk away from low-value security reviews.
• Vibe coding scales existing problems before it solves them. AI-assisted development amplifies both productivity and the rate at which insecure code can reach production. Security tooling and governance must adapt to the new pace before the next wave of AI-related breaches arrives.
• Developer security training needs to be contextual and real. Generic training fails. Bug bounty walkthroughs, internal vulnerability post-mortems, and hands-on tool engagement with actual payloads produce meaningfully different outcomes.
• The fundamentals keep winning. Exposed credentials, leaked secrets, and supply chain attacks continue to drive the majority of significant breaches. Novel attacks exist, but the unsexy basics remain the highest-leverage area for defense.