AI With Receipts: Evidence-Backed AI for High-Stakes Professional Work
Pete Werner (Founder · Penumbra AI)
[un]prompted.au 2026 · Day 1 · Main Hall
Overview
In his talk, "AI With Receipts: Evidence-Backed AI for High-Stakes Professional Work," Pete Werner, founder of Penumbra AI and former Head of AI at Leonardo, tackles one of the most pressing challenges in modern AI adoption: establishing trust and reliability in AI-generated outputs, especially for critical professional tasks. Werner argues that as AI moves beyond novelty applications like image generation, its application in demanding fields like security, finance, and legal work necessitates a robust framework for verification and accountability. The core problem, he posits, is the inherent tendency of AI models to always produce a result, regardless of its accuracy, leaving users uncertain about its correctness.

Key moments
- 0:45 The core challenge: AI confidence in high-stakes work
- 2:35 Solution: Decomposing tasks and automatic evaluation (Evals)
- 4:00 Fixing AI failures: representation, presentation, model capability
- 7:50 Recommended tools and approach for building robust pipelines
- 10:00 Strategic benefits: agents, custom models, competitive edge
- 12:00 Q&A: Balancing eval costs against severe error risks
- 13:45 Q&A: Python choice and managing eval agent hallucinations
AI With Receipts: Evidence-Backed AI for High-Stakes Professional Work
Speakers: Pete Werner, Founder, Penumbra AI
Conference: [un]prompted.au
YouTube: https://www.youtube.com/watch?v=xQJSaWmdME8
Overview
In his talk, "AI With Receipts: Evidence-Backed AI for High-Stakes Professional Work," Pete Werner, founder of Penumbra AI and former Head of AI at Leonardo, tackles one of the most pressing challenges in modern AI adoption: establishing trust and reliability in AI-generated outputs, especially for critical professional tasks. Werner argues that as AI moves beyond novelty applications like image generation, its application in demanding fields like security, finance, and legal work necessitates a robust framework for verification and accountability. The core problem, he posits, is the inherent tendency of AI models to always produce a result, regardless of its accuracy, leaving users uncertain about its correctness.
Werner introduces a systematic approach to mitigate this uncertainty, advocating for the decomposition of complex AI tasks into smaller, manageable steps, each with transparent inputs, outputs, and automated evaluation mechanisms. This methodology, which he terms "evidence-backed AI," aims to provide "receipts" or verifiable trails for every AI decision. By doing so, organizations can identify precisely where and why AI systems fail, thereby enabling targeted improvements and fostering confidence in AI's capabilities for high-stakes scenarios where errors can have significant consequences.
This talk is particularly pertinent for security professionals, developers, and business leaders looking to integrate AI into their operations. It shifts the focus from simply deploying "better" models to building resilient, auditable AI pipelines that can withstand the rigors of real-world application. Werner's insights offer a practical blueprint for transforming AI from an enigmatic black box into a transparent, dependable tool capable of delivering consistent value and maintaining integrity in critical domains.
Background
▶ Watch: The core challenge: AI confidence in high-stakes work (0:45)
The pervasive challenge in leveraging AI for professional tasks stems from a fundamental characteristic of large language models (LLMs) and other generative AI: they are designed to always provide an answer. Whether asked to "find a vulnerability in Chrome" or summarize a complex legal document, an AI will generate a response. The critical issue, as highlighted by Werner, is the inherent difficulty in assessing the confidence level of that response. Is it correct? Mostly correct? Or entirely fabricated, a phenomenon commonly known as hallucination? This lack of inherent self-assessment or "receipts" for its reasoning makes AI outputs problematic for high-stakes environments where accuracy is paramount.
The consequences of unverified AI outputs can be severe and far-reaching. Werner illustrates this with a stark warning: undetected errors can persist for months, leading to wasted effort, flawed decisions, or even significant operational risks. Imagine an AI falsely reporting "no vulnerabilities in Chrome," leading a security team to a false sense of security, or an AI misinterpreting financial data, resulting in erroneous investment advice. The traditional approach, often involving a single, complex, "one-shot prompt" to an AI, exacerbates this problem by creating a black-box scenario where the internal workings and reasoning are opaque. When a failure occurs in such a system, answering the crucial questions – where did it fail, why did it fail, and what can be done about it – becomes exceedingly difficult, if not impossible.
The problem exists because current AI models, despite their impressive capabilities, are still probabilistic systems. They generate output based on patterns learned from vast datasets, but without a built-in mechanism for factual verification or logical consistency checking against external ground truth. This necessitates an external framework to instill reliability. Werner's talk builds upon the growing recognition that AI systems need more than just raw intelligence; they require traceability, auditing capabilities, and mechanisms for self-correction to be truly valuable and trustworthy in professional contexts. His approach directly addresses this gap by imposing structure and verification on the AI workflow, moving beyond simple assistant functions to create robust, scalable, and verifiable AI solutions.
Key Findings
▶ Watch: Fixing AI failures: representation, presentation, model capability (4:00)
Pete Werner's presentation distills several key findings essential for building reliable AI systems in demanding professional environments. The central tenet is that trust in AI outputs can only be achieved through a systematic, evidence-backed approach rather than relying solely on the inherent capabilities of large models.
Firstly, Werner advocates for the decomposition of large, complex AI tasks into numerous smaller, more manageable subtasks. This modular approach allows for granular control and inspection at each stage, transforming an opaque, monolithic AI process into a series of transparent, verifiable steps. This is a fundamental shift from the "one-shot prompt" paradigm.
Secondly, a critical component of this methodology is the capture of all inputs and outputs for each individual subtask within the pipeline. This data collection serves as the "receipts" – a detailed, auditable trail that documents the AI's processing journey. This historical record is invaluable for debugging, analysis, and future improvements.
Thirdly, Werner introduces the concept of automatic evaluation using "evals," also known as scorers or judges. These evals are AI models or rule-based systems specifically designed to assess the reasonableness and correctness of the output from each subtask, given its input. By deploying multiple evals per task (initially one, ideally three to five), the system gains a robust, multi-faceted validation layer. These evals are instrumental in precisely pinpointing where a failure occurred within a multi-stage pipeline, thereby addressing the "where did it fail?" question.
Fourthly, the captured inputs and outputs, combined with the feedback from evals, provide the necessary data to diagnose why a failure happened. By examining the source data, interim steps, and comparing failed examples with similar successful ones, developers can gain deep insights into the root causes of issues. This diagnostic capability is crucial for effective problem-solving and system refinement.
A significant finding related to problem resolution is the identification of three primary levers for improvement: representation, presentation, and model capability. Werner emphasizes that often, the problem isn't solely with the underlying AI model. He demonstrated a system where an initial 95% hallucination rate was dramatically reduced to 3% by solely optimizing how input data was represented (structured) and presented (how the task and data were described) to the same model. This highlights that developers often prematurely assume they need a "better model" when optimizing data handling can yield substantial improvements at lower cost and faster execution.
Finally, Werner stresses that building these robust, evaluated pipelines creates a powerful competitive edge and a value moat. Such systems enable rapid experimentation with new models and methods, quickly determining their impact on specific problems and data. Furthermore, the captured data from these pipelines can be leveraged to train custom, domain-specific AI models that are often cheaper, faster, and more consistent than general-purpose foundational models, allowing organizations to achieve scale and capabilities unique to their operations.
Technical Deep Dive
▶ Watch: Recommended tools and approach for building robust pipelines (7:50)
The technical core of Pete Werner's approach lies in its structured, modular, and verifiable design, moving away from black-box AI interactions towards transparent, auditable processes. This section elaborates on the methodologies and tooling proposed for implementing "AI with receipts."
The foundational step is task decomposition. Instead of formulating a single, complex prompt, a large AI task is broken down into numerous smaller, independent subtasks. For example, a security analysis task might be decomposed into: "extract relevant code snippets," "identify potential vulnerability patterns," "cross-reference with known CVEs," "summarize findings," and "generate a remediation recommendation." Each subtask is designed to be atomic and focused, making its inputs, outputs, and expected behavior easier to define and evaluate.
Central to the verification process are evals (evaluators, scorers, or judges). An eval is essentially a mechanism – which can be another AI model, a rule-based system, or even a human-in-the-loop – that determines if the output of a given subtask is "reasonable" or correct, given its input. Werner recommends starting with at least one eval per subtask and ideally scaling to three to five evals as the system matures. This multi-eval approach provides a more robust assessment, reducing the risk of a single faulty eval leading to incorrect conclusions. Evals are critical for identifying precisely where in a multi-stage pipeline a failure occurred. For instance, if the "extract relevant code snippets" subtask fails its eval, the subsequent steps are not even attempted or are flagged as operating on bad data.
When a failure is detected, the detailed capture of inputs and outputs for every subtask becomes invaluable. This allows developers to trace the data flow, inspect the original source data, and review all intermediate steps. By comparing the data of failed examples with similar successful ones, engineers can pinpoint patterns or specific data characteristics that lead to errors.
Werner identifies three primary dimensions for addressing AI failures:
- Representation: This refers to how the input data is structured. Is it plain text, natural language, YAML, JSON, or source code? For highly specialized tasks, the representation can be much more complex. Werner provides an example of a learned representation of compiled code, derived from a specific platform, compiler, and compiler flags. This highly specialized representation, generated from vast examples, can be used to train a model exceptionally good at identifying a specific class of issue within that compiled code. The effectiveness of the AI is often dramatically improved by tailoring the data representation to the task's inherent structure.
- Presentation: This concerns how the data is made available to the model and how the task is described. This includes prompt engineering, context window management, and the overall framing of the problem for the AI. Werner cites a compelling case where, using the same input data and the same model, changing the representation and presentation alone reduced a system's hallucination rate from 95% to 3%. This underscores that often, the interaction design with the AI is more impactful than simply upgrading to a larger, more expensive model.
- Model Capability: Only after optimizing representation and presentation should one consider whether the task is genuinely beyond the current model's capabilities. In such cases, further task decomposition might be necessary, or a more capable (and often more expensive/slower) model might be required. Werner cautions against the common assumption that "I just need a better model," highlighting it as a last resort rather than a first.
For implementing these pipelines, Werner suggests a practical toolkit:
- Python: Chosen for its ease of use, extensive libraries, and status as the de facto language for AI development. It integrates seamlessly with tools like OpenTelemetry for instrumentation and observability. During the Q&A, Werner addressed the choice of Python over type-safe languages, stating that while types solve some problems, they also create others, and Python's ecosystem and flexibility often outweigh the perceived benefits of strong typing for rapid AI development, especially given that Python offers optional typing.
- Taskfile: A YAML-based alternative to Makefile, used for orchestrating the entire pipeline. It simplifies defining and executing complex sequences of commands and scripts, making the pipeline definition human-readable and version-controllable.
- MLflow: A platform for managing the machine learning lifecycle. Werner uses it specifically for observability, capturing all inputs and outputs of subtasks, running evals, and providing dashboards for monitoring pipeline performance and identifying bottlenecks or failures. Other tools exist in this space, but MLflow is a strong choice for developers.
- "Improve Judge" Agent: A custom-built AI agent designed to iterate on and refine the evals themselves. Recognizing that evals can also "hallucinate" or be imperfect, this agent helps to automatically improve the prompts and logic of the judges, ensuring their accuracy and reliability over time. This meta-evaluation approach addresses the recursive problem of trusting the evaluators.
Werner advises a phased approach: start simple. Build the tooling and the initial pipeline with a straightforward task to gain confidence in the system's reliability before tackling highly complex problems. This iterative development ensures that the underlying infrastructure is robust before adding layers of complexity. The resulting system is not only robust and reliable but also inherently scalable via agentic workflows, where well-defined tasks with structured data are highly amenable to automated processing by multiple agents. Furthermore, the continuous capture of high-quality, task-specific data provides a rich dataset for building and training custom, more efficient, and specialized models, creating a significant competitive advantage.
Demo / Proof of Concept
▶ Watch: Q&A: Balancing eval costs against severe error risks (12:00)
While Pete Werner's talk did not feature a live, interactive demo of a system being built from scratch, he provided compelling examples that serve as strong proof points for the efficacy of his proposed methodology. The most notable example cited was a system that initially exhibited an alarmingly high hallucination rate of 95%. By meticulously applying the principles of optimizing data representation and presentation – without changing the underlying AI model – this rate was drastically reduced to 3%. This concrete result underscores the profound impact that careful engineering of AI inputs and task descriptions can have on model reliability, often circumventing the need for more expensive and time-consuming model upgrades.
Additionally, Werner mentioned the development of an "improve judge" agent within his own work. This agent is designed to automatically refine and enhance the performance of the evals (the AI judges themselves), acknowledging that even evaluators can be imperfect or "hallucinate." While not demonstrated in real-time, the concept of using AI to improve AI's own evaluation capabilities serves as a practical demonstration of the recursive self-improvement possible within a well-structured pipeline. These examples, drawn from real-world experience, illustrate the tangible benefits and practical application of the "AI with receipts" framework.
Defensive Implications
▶ Watch: Q&A: Python choice and managing eval agent hallucinations (13:45)
For security professionals and organizations leveraging or considering AI, Pete Werner's "AI with Receipts" framework provides crucial defensive implications, shifting the paradigm from blind trust to verifiable outcomes.
Firstly, demand transparency and evidence for AI outputs. Security teams should no longer accept AI results without an accompanying "receipt" or audit trail. When evaluating AI-powered security tools (e.g., vulnerability scanners, threat intelligence platforms, SIEM augmentations), inquire about their internal validation mechanisms. How does the tool verify its findings? Is there a traceable path from input data to the final conclusion? This is especially critical for high-stakes decisions where false positives or, worse, false negatives can have severe consequences.
Secondly, prioritize building observable and auditable AI pipelines. For organizations developing their own AI solutions, adopting Werner's approach is paramount. Decompose complex security tasks (e.g., malware analysis, incident response triage, policy enforcement) into smaller, manageable subtasks. Implement evals at each stage to automatically check the correctness and reasonableness of intermediate outputs. Tools like MLflow or similar observability platforms should be integrated to capture all inputs, outputs, and evaluation results. This creates a forensic trail that is invaluable for post-incident analysis, compliance auditing, and demonstrating the integrity of AI-driven security operations.
Thirdly, focus on data representation and presentation before model choice. Before investing in the latest, most powerful (and expensive) foundational models, security engineers should critically assess how their security data is being structured and presented to the AI. Can vulnerability reports be converted into a more structured format (e.g., YAML, JSON, or a custom learned representation)? Are prompts clearly defining the security context, specific threats, and expected output format? Optimizing these aspects can significantly reduce hallucination rates and improve the accuracy of security AI, often at a lower cost than simply upgrading models. For instance, a highly specialized "learned representation of compiled code" (as discussed by Werner) could drastically improve an AI's ability to spot specific compiler-level vulnerabilities.
Fourthly, leverage captured data to build domain-specific security models. The continuous capture of inputs, outputs, and evaluation results from robust AI pipelines provides a rich dataset. This data can be used to train custom, highly specialized AI models tailored to an organization's unique threat landscape, codebases, or operational environment. These custom models are often cheaper, faster, and more reliable for specific security tasks than general-purpose LLMs, offering a competitive advantage and enhanced security posture. This also allows for the development of highly accurate internal tools that understand proprietary systems and specific security policies.
Finally, be prepared for AI's inherent imperfections, and build mechanisms to address them. Werner acknowledges that evals themselves can sometimes "hallucinate." This highlights the need for continuous monitoring and refinement of the evaluation system, potentially using meta-agents like his "improve judge" agent. Security teams should understand that AI is an imperfect science and art, and therefore, human oversight and iterative improvement processes are still essential, even within highly automated pipelines. The goal is to make AI failures transparent and actionable, not to eliminate them entirely.
Key Takeaways
- Trust in AI requires evidence: For high-stakes professional work, AI outputs must be backed by verifiable "receipts" or audit trails, not just an answer.
- Decompose complex tasks: Break down large AI problems into smaller, manageable subtasks to enable granular control, inspection, and evaluation.
- Implement robust evaluation (Evals): Use automated "judges" or "scorers" at each stage of an AI pipeline to verify the reasonableness and correctness of inputs and outputs.
- Optimize Representation and Presentation: Significant improvements in AI reliability and hallucination reduction (e.g., 95% to 3%) can be achieved by carefully structuring input data and crafting clear prompts, often more effectively than simply using a "better" model.
- Build an observable pipeline: Tools like Python, Taskfile, and MLflow facilitate the creation of auditable, scalable AI systems that capture data, run evaluations, and provide insights into failures.
- Foster a competitive edge: Robust AI pipelines enable rapid iteration, efficient debugging, and the creation of valuable, domain-specific models from captured data, leading to a strong competitive advantage.
About the Speaker(s)
Pete Werner is the founder of Penumbra AI, a company specializing in applying AI to demanding, domain-specific professional use cases. With a deep background in artificial intelligence, he previously served as the Head of AI at Leonardo, a prominent diffusion model company focused on image and video generation. During his tenure at Leonardo, Werner was instrumental in scaling the platform from zero to 20 million users. He led the design and construction of the platform's AI services, established its AI research capabilities, and built the large-scale training cluster. Under his leadership, Leonardo produced one of Australia's first foundational models and was subsequently acquired by Canva in 2024. Werner's expertise spans various model types and industries, with a particular focus on financial services. He helps organizations with AI strategy, ideation, model building and scaling, and go-to-market initiatives, demonstrating a passion for exploring the practical and enduring value of AI technology.
Reviews
Dr. Zero (Offensive Security Researcher) — SOLID
Thank god. An AI talk that admits LLMs just confidently lie to your face. Werner skips the vendor hype and gives a concrete engineering blueprint for operationalizing language models using task decomposition, explicit state tracking, and automated evals.
Heather Calloway (CISO) — STRONG ACCEPT
This is the blueprint for how we stop treating enterprise AI like magic and start managing it like an auditable IT system. Werner provides a concrete engineering framework for verifying AI outputs in high-stakes environments. I'd make this required viewing for any team asking for budget to build or buy generative AI tools this year.