Writing on Nick Lackman — Distributed Systems & AI

Your Voice Agent Passed Every Eval. The Caller Hung Up.

Thu, 08 Oct 2026 00:00:00 +0000

Your voice agent passed every eval you have. The transcript looks perfect. The tool calls all fired. And the caller hung up on it.

I’ve spent two years building production voice AI, and this is the failure that took me longest to get a handle on — not because it’s rare, but because our entire evaluation stack was structurally incapable of seeing it.

The transcript said the agent handled the call. What actually happened was four seconds of dead air, a robotic monotone, and the agent talking over the caller mid-sentence. None of that is in the transcript. None of it could be.

The Audio Gateway: The Production Pattern for Real-Time Voice AI

Thu, 16 Jul 2026 22:30:48 +0000

If your voice agent stutters under load, talks over callers, or freezes mid-sentence while a tool call runs — the fix probably isn’t a better model. It’s an architectural split.

In my last post, Python Is Lying to You, I walked through every way Python’s async model quietly sabotages a real-time audio pipeline: the async-looking guardrail that froze the event loop, the logger doing blocking I/O on every frame, GIL contention that looks exactly like network jitter. The fixes worked. But they are all defensive. Move this off the hot path. Offload that to a thread. Audit every innocent-looking call. You’re playing whack-a-mole against your own process.

Python Is Lying to You: Async Pitfalls in Real-Time Audio Pipelines

Fri, 12 Jun 2026 04:14:07 +0000

If you’re building voice AI with async Python, your audio quality is probably worse than it needs to be — and the bottleneck isn’t where you think it is.

I’ve spent the last two years building production voice AI systems — real-time pipelines handling thousands of calls per day over PSTN telephony. The stack is what you’d expect: STT, LLM reasoning, TTS, Speech-to-Speech, guardrails, tool calls, all wired together with async Python and streaming over websockets.

Voice AI: Fast and Dumb or Slow and Smart — Why Not Fast and Smart?

Mon, 06 Apr 2026 23:26:29 +0000

Many voice AI demos connect the browser directly to a real-time audio model API and lets the server decide when you’ve stopped talking. That’s a demo architecture with a built-in latency tax that quickly breaks down in production. Here’s the production alternative: a backend-mediated, multi-thinker voice system with local voice activity detection that owns the entire audio pipeline end-to-end.

I spent the last year and half building production voice AI systems that handle thousands of calls per day. This post covers the architecture I wish someone had documented when I started: how to make your voice AI product fast and smart, what the Responder-Thinker pattern is, why single-thinker breaks, how to build multi-thinker with your backend in the middle, and why local VAD is the key to making it feel instant.

Cutting 600ms from Every Voice AI Turn with Local VAD

Sat, 21 Mar 2026 02:45:02 +0000

If you’re building voice AI on OpenAI’s Realtime API, your agent is slower than it needs to be — the main bottleneck is certainly inference but there’s additional overhead to cut.

I spent the past week instrumenting a production telephony voice pipeline, measuring where latency actually lives, and testing whether local voice activity detection (VAD) could meaningfully reduce response time. The answer is yes — by 689ms per turn on substantive responses — and the methodology is cleaner than I expected.

I Tested Our WebSocket Audio Pipeline with WebRTC. Here's Why I Switched It Back.

Wed, 11 Mar 2026 04:01:01 +0000

There’s a prevailing assumption in the voice AI space that WebRTC is inherently better than WebSockets for real-time audio. Better latency, better quality, better everything. I built a full proof-of-concept to test that assumption on an enterprise scale production AI voice system.

I found a few things surprising.

The Setup

Our system takes inbound phone calls, pipes the audio through an AI agent (OpenAI Realtime API), and sends the response back to the caller. The current architecture uses Twilio Programmable Voice with WebSocket media streams — G.711 μ-law audio at 8kHz using WebSocket protocol.