AI Engineering That Ships

Securing Agentic AI Systems: What Two Rounds of Adversarial Testing Taught Us

Securing Agentic AI Systems: What Two Rounds of Adversarial Testing Taught Us 27 attacks. 14 defense patches. 550 lines of security hardening. Two rounds proved the same thing from opposite directions: targeted patches drop the attack success rate from 65% to 20% against known vectors. Structural weaknesses keep it at 85.7% for new ones. Patching and architecture are complements, not substitutes. Figure 1 - The Two-Round Journey: From 65% CRITICAL to 47% HIGH, but the headline obscures the real story. Regression ASR (20%) proves patches work. Escalation ASR (85.7%) proves architecture doesn't. The gap between these two numbers is the gap…

The Escalation Wave: Why Patches Work but Architecture Doesn't

The Escalation Wave: Why Patches Work but Architecture Doesn't The regression wave confirmed everything we hoped: 8 of 10 Round 1 attacks were blocked. ASR dropped from 65% to 20%. Patches hold. Then the escalation wave confirmed everything we feared: 6 of 7 new attacks succeeded. A zero-width Unicode space between the letters of "ignore" made the word invisible to every regex in the system. Figure 1 - The Two-Wave Story: The regression wave (left, green) proves patches work -- 8 of 10 original attacks blocked. The escalation wave (right, red) proves architecture doesn't -- 6 of 7 new attacks confirmed. The overall 47.06% ASR is a blend of…

65% Attack Success Rate Against an Unpatched Target

65% Attack Success Rate Against an Unpatched Target The Red Team exfiltrated our Anthropic API key, OpenAI API key, PostgreSQL password, and YouTube API key in a single curl command. The exploit: encode an absolute file path as base64, pass it as a URL parameter, and the server reads whatever file you want. A validation function existed in the codebase -- it just wasn't applied to this endpoint. Figure 1 - Round 1 Scorecard: ASR 65% rates as CRITICAL -- more than half of Red Team's attacks succeeded. Blue Team scored 67.86/100 with a perfect 100% detection rate, but the ASR reflects the target's vulnerability before Blue intervened. The…

Adversarial Agent Testing: When Your AI Agents Attack Each Other

Adversarial Agent Testing: When Your AI Agents Attack Each Other Five Claude Code agents. Three teams. One target. The Red Team found 7 vulnerabilities in 5 minutes. The Blue Team patched every one. Then Round 2 proved that patches alone aren't enough. Figure 1 - The Adversarial Architecture: Red Team (recon + exploit agents) attacks the target from one worktree. Blue Team (monitor + hardener agents) defends from another. The Referee scores both sides from a read-only vantage point. Information asymmetry is enforced by hooks -- not prompts. Two rounds of adversarial exercises against a real application. 27 total attacks across 10 OWASP…

WordPress to Astro: Migrating a Production Site with AI-Assisted Infrastructure

WordPress to Astro: Migrating a Production Site with AI-Assisted Infrastructure 41 WordPress articles, 187 images, a design-matched dark theme, and a Projects section -- all extracted from a SQL backup file and rebuilt in Astro. This is the story of migrating dotzlaw.com from WordPress to a modern static site, and what the Bootstrap Framework actually contributed. Figure 1 - Before and After: The WordPress Kicker dark theme (left) and the rebuilt Astro Fuwari site (right). Same near-black backgrounds, same teal accents, same full-bleed card layouts. The source material was a SQL backup file and a wp-content directory. The Starting Point:…

Securing Agentic AI: How We Found 11 Security Gaps in Our Own Framework and Built Defense-in-Depth to Close Them

Securing Agentic AI: Building Security-Conscious Agent Systems with Claude Code We found 11 security gaps in our own production framework -- then closed every one with 6 new hooks, 2 JSON schemas, 7 per-archetype security patterns, and a 3-tier trajectory monitoring system. All 10 OWASP Top 10 for Agentic Applications items are now addressed. Figure 1 - Defense in Depth: The Security Architecture: Four concentric rings protect the pipeline core. Ring 1 (red/amber) fires on every tool call. Ring 2 (blue/purple) monitors behavior patterns over time. Ring 3 (gold) enforces architectural guarantees that prompts cannot bypass. Ring 4 (green)…

Securing Agentic AI: How We Found 11 Security Gaps in Our Own Framework and Built Defense-in-Depth to Close Them

From Prototype to Platform: How a Framework Learned to Improve Itself

From Prototype to Platform: How a Framework Learned to Improve Itself The 14-document analysis we generated for metabase-server claimed that was 1,620 lines, that a CORS wildcard appeared at line 70, and that a Redis command appeared at lines 486-489. An independent reviewer, given no context about how these documents were built, spot-checked all five claims against the actual source code. Every one was accurate. The framework didn't just generate documentation -- it generated documentation that was verifiably correct. Figure 1 - The Gap Analysis Matrix: Eight missing capabilities plotted by value and effort. Round 1 targeted the…

An Agent Swarm That Builds Agent Swarms: How We Used Claude Code to Generate Claude Code Infrastructure

An Agent Swarm That Builds Agent Swarms We used Claude Code to build a framework that generates Claude Code infrastructure for any project. Then we proved it works by migrating two production apps -- the second more complex but completed faster. 67 Python files, an AI/ML pipeline, a full architectural redesign, 8 sessions, zero framework build time. Figure 1 - The Three-Folder Architecture: The framework reads the source project but never modifies it. All generated infrastructure lands in a fresh target project. This READ-ONLY invariant held across 10 sessions and was never violated. A user types "top 10 customers in revenue for the last…

Claude Code Security: Building Defense-in-Depth with Five Primitives

Claude Code Security: Building Defense-in-Depth with Five Primitives Most Claude Code projects ship with zero security infrastructure. The building blocks for comprehensive defense-in-depth are already in the toolkit. Hooks enforce deterministically. Agents restrict by capability. Skills encode security knowledge. Commands scan on demand. Teams validate across boundaries. Five primitives, five security layers -- none turned on by default. Figure 1 - Five Primitives, Five Security Layers: Each Claude Code building block -- hooks, agents, skills, commands, and teams -- map directly to a security capability. Hooks provide deterministic…

Claude Code Agent Teams: Building Coordinated Swarms of AI Developers

Claude Code Agent Teams: Building Coordinated Swarms of AI Developers 16 parallel Claude agents. 100,000 lines of Rust. A C compiler that builds the Linux kernel across 3 architectures. No single agent could hold the codebase in context. The team succeeded because each agent only needed to hold its piece. Figure 1 - Single Agent vs Agent Team: A single agent trying to hold a 100,000-line codebase degrades as context fills with competing concerns from every layer. An agent team gives each specialist a focused context window: the lexer agent thinks only about lexing, the parser agent thinks only about parsing. Focused context produces better…

Claude Code Hooks: The Deterministic Control Layer for AI Agents

Claude Code Hooks: The Deterministic Control Layer for AI Agents A CLAUDE.md instruction says "always run the linter." The agent usually complies. A PostToolUse hook runs the linter after every file write, every single time, no exceptions. That gap between "usually" and "always" is where production systems fail. Figure 1 - The Reliability Gap: Prompts vs Hooks: Prompt-based instructions achieve 70-90% compliance. The agent usually follows them but can skip under context pressure, long sessions, or competing priorities. Hooks achieve 100% compliance. They execute at the system level, outside the LLM's reasoning chain. For anything that must…

Claude Code Skills: Building Reusable Knowledge Packages for AI Agents

Claude Code Skills: Building Reusable Knowledge Packages for AI Agents A project with 8 skills loads 500 tokens at startup. Loading everything would cost 70,000 tokens. That 140x difference is progressive disclosure, and it is the reason agent teams can carry deep domain knowledge without drowning in context. Figure 1 - The Progressive Disclosure Advantage: Without skills, every agent loads all domain documentation into context at startup, consuming 70,000 tokens before any work begins. With skills, only a lightweight index loads at startup (500 tokens), and full skill content loads only when relevant to the current task. This 140x…

Building Effective Claude Code Agents: From Definition to Production

Building Effective Claude Code Agents: From Definition to Production The team behind Claude Code's C compiler project needed 60 sessions across parallel agents to compile a working C compiler. The biggest challenge wasn't the prompts. It was designing the environment around the agents. Figure 1 - From Chat Sessions to Agent Teams: The fundamental shift from interactive chat (constant human supervision, one task at a time) to autonomous agents (parallel specialists, each with bounded scope and shared progress tracking). This architectural change is what makes it possible to tackle projects that span 60+ sessions and thousands of…

Orchestrating AI Agent Teams: How Skills, Hooks, and Context Flow Make Autonomous Coding Reliable

Orchestrating AI Agent Teams: How Skills, Hooks, and Context Flow Make Autonomous Coding Reliable An orchestrator breaks a task into pieces. Specialized agents pick up work items, each carrying skills that define what they know and hooks that enforce how they behave. Context flows from session start to task completion through a deterministic pipeline. Here is how the pieces fit together. Cover - The Orchestrator Pattern: A central orchestrator coordinates specialized agents, each equipped with domain-specific skills and hooks. Context flows bidirectionally: the orchestrator assigns tasks with context, agents execute with skill-guided…

Backend Development & Data Analytics

The Dotzlaw Team

Two skilled engineers building advanced agentic AI projects and research alongside me. They contribute directly to the systems, articles, and tools published on this site.

Katrina Dotzlaw

katrina.dotzlaw.com ↗

Building AI-powered data pipelines and full-stack applications at the intersection of machine learning and real-world business problems.

Python Java Machine Learning UI Design

Visit Site →

Ryan Dotzlaw

ryan.dotzlaw.com ↗

Software Developer & Data Analyst

Applying statistical analysis, neural networks, and modern UI to extract insight from complex datasets and build compelling data-driven applications.

Python R Neural Networks UI Graphics

Visit Site →

Latest Insights

View all →

Part 4 of 4 Adversarial Agent Testing

Securing Agentic AI Systems: What Two Rounds of Adversarial Testing Taught Us

27 attacks across 2 rounds, 14 defense patches, 550 lines of security hardening. The transferable lesson: patching fixes yesterday's attacks, architecture survives tomorrow's. Here is what we learned about building, testing, and defending agentic AI applications.

Part 3 of 4 Adversarial Agent Testing

The Escalation Wave: Why Patches Work but Architecture Doesn't

Round 2 re-ran all 10 original attacks against patched code -- 8 were blocked (20% ASR). Then 7 new attacks hit structural weaknesses: Unicode zero-width characters bypassed every regex, 5 rapid requests crashed the server, and a pattern gap between security layers let 11 injection techniques through. Escalation ASR: 85.7%.

Part 2 of 4 Adversarial Agent Testing

65% Attack Success Rate Against an Unpatched Target

Round 1 of our adversarial exercise: 10 attacks in 5 minutes, 7 confirmed vulnerabilities, one critical credential exfiltration. The Red Team read our API keys through a base64-encoded path that nobody thought to validate. Blue Team detected everything -- but the damage was already done.

Part 1 of 4 Adversarial Agent Testing

Adversarial Agent Testing: When Your AI Agents Attack Each Other

We built a platform where five Claude Code agents operate as Red Team attackers, Blue Team defenders, and an impartial Referee -- then pointed them at a real target. The first exercise found 7 confirmed vulnerabilities in 5 minutes. The second proved that patches work but architecture doesn't.

Part 4 of 4 Building the Bootstrap Framework

WordPress to Astro: Migrating a Production Site with AI-Assisted Infrastructure

41 WordPress articles, 187 images, a design-matched dark theme, and a Projects section -- all extracted from a SQL backup file and rebuilt in Astro. This is the story of migrating dotzlaw.com from WordPress to a modern static site, and what the Bootstrap Framework actually contributed.

2026-02-27 Read Article →

Part 3 of 4 Building the Bootstrap Framework

Securing Agentic AI: How We Found 11 Security Gaps in Our Own Framework and Built Defense-in-Depth to Close Them

We built a framework with 18 skills and 11 hooks. A security audit found 11 gaps. We closed all of them with 6 new hooks, 2 JSON schemas, a 3-tier trajectory monitoring system, and per-archetype security patterns across 7 project types.

2026-02-26 Read Article →

Part 2 of 4 Building the Bootstrap Framework

From Prototype to Platform: How a Framework Learned to Improve Itself

After two production migrations, we turned the framework on itself. A systematic gap analysis identified 8 missing capabilities. Round 1 added 3 of them, expanding the pipeline from 7 to 10 steps. An independent review graded the work A-. The compound returns operate not just project-to-project but within the framework itself.

2026-02-25 Read Article →

Part 1 of 4 Building the Bootstrap Framework

An Agent Swarm That Builds Agent Swarms: How We Used Claude Code to Generate Claude Code Infrastructure

We built a framework where Claude Code agents analyze an existing codebase, generate tailored agent teams, hooks, and skills. Two migrations later -- the second harder but faster -- the compound returns are real.

2026-02-11 Read Article →