The AI Was the Prototype

A few months ago my team ran an offsite with an open brief: take a week, find a way to fold AI into how your team works day to day, present what you built on Friday. Small teams, no shared plan, little coordination by design. The point was to see what people reached for when you handed them room.

By Friday there were not six different projects. There were two, built six times.

Half the teams had built a dashboard of some kind. The commercial team called theirs a reporting tool, the partnerships team called theirs a pipeline view, but under both labels sat one need: reach the data faster so the next move with a customer or partner is informed instead of guessed. Nobody used the same words. Everybody built the same thing.

The other half had built capture. Engineers had noticed that the most valuable output of a hard week was not the code, it was the reasoning: the back-and-forth with the model that took them from a vague symptom to a root cause. They built ways to save that thread, hand it off, and let the next person see how the conclusion was reached instead of inheriting only the conclusion. Different teams, different tools, one shape: make the path to an answer visible to whoever needs it next.

Nobody set out to converge. The convergence is the evidence. When six independent teams land on two problems, those were never six teams’ problems. They were two shapes the whole organization carried, wearing six sets of local vocabulary. (I wrote about this convergence from a different angle in Your Issue Is Not Unique. The presentations did the abstraction no single team had done alone.)

The part that matters for this piece: the sameness was invisible until Friday. Each team spent the week certain its problem was its own. It took standing them next to each other for the shared shape to surface.

The offsite did not have time to reach the next step, which is what happens when you watch four teams build the same dashboard. You do not let them keep building four different dashboards. You build it once. You make it shared infrastructure. You decide which parts keep the model and which parts become plain code.

This is the move that is hardest to make, because AI is so good at the first half. It is the fastest discovery layer we have ever had. You describe the shape of the problem, run a prompt, and within twenty minutes you have a working prototype. The instinct is to leave it there. The prototype works. It answers the question. It even looks like a real tool.

But AI is a poor runtime. It is slow, nondeterministic, and costs a call every interaction. The mistake is leaving a tool on the AI runtime after AI has already done its job of revealing what the tool should be. Once the shape is clear, you need to graduate it: crystallize the discovered spec into deterministic infrastructure that runs without paying the AI tax on every use.

The Dashboard Worked Example

Look closer at what those teams built and a second lesson shows up. Most of it was not an AI solution at all. A dashboard that pulls numbers and arranges them on a screen is deterministic. It is a SQL query and a render. There is no judgment in it that needs a model at runtime, no ambiguity for a language model to resolve. The intelligence was spent on the way in, not on every run.

The partnership teams used Claude Cowork to aggregate brand performance. The sales and CSM teams used it to aggregate account state: the status of their cases, the performance of their users, the overview of conversations with other teams. Cowork outputs static HTML assets that render the data nicely, so they can drop them into presentations and share them with stakeholders. It works.

But the data they need is deterministic to some degree. It is a static SQL query with variable date ranges and enum filters. If you keep an AI service answering questions on demand, you are burning tokens to regenerate a dashboard that a parameterized query could return for a fraction of a cent. The non-technical person who built the Cowork prototype is the perfect person to maintain the SQL query. They know the exact cuts of data that mattered. They know which fields drive their decisions. They just need the tool to stop asking them to prompt it every time they need fresh numbers.

The graduation is straightforward. Build an actual dashboard with filters the user manipulates directly. The AI surfaced which cuts of the data mattered. Now serve those cuts deterministically. The user clicks a filter; they do not re-prompt a model. The model found the requirement. Architecture decides the runtime.

Where AI Stays vs Graduates Out

Deciding what to graduate requires a clean line between the parts that need judgment and the parts that do not.

Where AI stays: the parts that vary every time, the parts that genuinely benefit from a model reading novel input, the parts that require a human to interpret context and make a call.

Where AI graduates out: deterministic queries, fixed filters, repeatable transforms, anything where the answer is the same function of the same inputs every time. You do not need an LLM to run a GROUP BY.

The meeting-prep projects from the offsite illustrate this split. AI gathers and structures source data every time. The graduation is to predefine the source data and the gathering pipeline so AI consumes a tight, consistent input instead of re-discovering the inputs each run. Move the nondeterministic gathering step to a deterministic one. Let AI do only the part that genuinely needs judgment.

The Cowork-Runs-Locally Trap

A file generated inside a chat is not a system. It does not run on a schedule. It does not flex when the team’s question shifts next month. And if a model stays in the loop at runtime it burns tens of thousands of tokens to produce a table a query would return for a fraction of a cent.

Many of the offsite demos were built with Claude Cowork. They run fine locally for the person who built them. They are not easily shared. They are not deployed. They re-run the AI every time. That “runs locally, not shared, AI-on-every-call” state is exactly the prototype stage. The piece is about what comes after.

The mistake is not building a skill or a Cowork script that outgrows itself. The mistake is skipping the prototype entirely and building a webapp before you know if anyone needs it, or holding onto the prototype after it is clear that the team needs a real tool.

Build small first. Let AI reveal the shape. Then pay attention to who starts asking for what you built. When four teams build the same dashboard, it is time to graduate.

The Throughline

The thread across the arc keeps pointing at the same place. Building got cheap. The skill that survives is architectural, and that skill compounds when it scales across people rather than staying inside one head. The graduation ladder shows the technical path from script to webapp. This piece is the non-technical, ops-facing cut of the same idea.

The offsite proved that AI is a brilliant way to find the shape and a wasteful way to be the shape. The teams that converged on the dashboard knew they needed a better view of their data. They just used the fastest tool available to get there. The next challenge, the one the offsite did not have time to reach, is conversion. Take the spec the prototype encodes and graduate it into something deterministic that runs cheaply, predictably, and with the flexibility the team needs.

The companion piece to this one argues that collaboration is the next gap. When AI prototypes become shared tools, the session itself needs to be shareable. But before you share it, make sure it is worth sharing. If your team is building deterministic tools on top of a nondeterministic runtime, you are paying a tax that you do not need to pay. Graduating the prototype off the AI runtime is the architecture decision that turns a clever demo into a real system.

If you want help spotting which of your AI-assisted prototypes are ready to graduate into deterministic tools your whole team can use, reach out at hello@escapecommand.com.