Back to Blog
leadershipstrategyai-agentslocal-aibuild-in-public

Pilots, Not Passengers

June 14, 20266 min readBy Bruce Canedy

The build that prompted this

Last month I gave one of my local agents memory. It's the same espresso-cart build I've been writing about in the M4 Max series. route-ops plans tomorrow's stops. bar-prep reads that plan and writes the prep checklist. Now the system also remembers what happened on past visits, so it can tell me a stop sold out four of its last five mornings instead of treating every day like the first. It runs in OrbStack against a local Ollama model and stores its memory as files on a disk. No cloud, no framework.

The build was small. The question it forced was not. An agent with memory can remember anything. So what should it remember? That one decision shaped the whole thing, and it turned out to have almost nothing to do with code.

A leadership question wearing an engineering costume

On the surface, "what should the agent remember" looks like a schema decision. Pick the fields, define the table, move on. Underneath, it's a decision about whose reality the system encodes. Remember sales totals and you've built a tool for the owner. Remember which stops felt rushed and understaffed and you've built a tool for the person working the cart. Same agent, same architecture, completely different center of gravity.

That is the pattern I keep running into. The decisions that look technical are often leadership decisions that got handed to whoever was closest to the keyboard. Nobody meant to make a strategy call. They just needed to name the columns.

This matters because the answer is visible to the person on the other end. They can tell whether the system was built around their work or around a number someone above them wanted to watch.

What employees decode when an agent lands

The HBR piece I anchored this series to makes the case that AI strategy is not a private executive deliberation. It's a signal, and it travels fast. The moment an agent touches someone's workflow, they decode it. Is this here to help me do my job better, or to do my job without me?

What they decide drives what they do next. The authors surveyed 1,294 desk workers and found that people who felt forced to adopt AI reported a 65% higher rate of "workslop" — low-effort, low-quality AI output that someone downstream has to clean up — than people who felt encouraged. Same tools. Different perceived intent. Different results.

There's a second number worth sitting with. The survey found a wide gap between how senior leaders and individual contributors describe the same rollout. 81% of senior leaders think their organization is all-in on augmentation. At the individual-contributor level, only 53% perceive augmentation, and 40% suspect automation. The strategy can be augmentation in the boardroom and automation at the desk, and the desk is where the work actually happens.

Pilots versus passengers

I think about this as pilots versus passengers. A pilot helped design the aircraft they fly. They know why the controls are where they are because they were part of putting them there. A passenger just rides. They had no say, so they have no ownership, and when something feels wrong their only move is to complain or to quietly check out.

The difference between the two is not motivation. It's not that pilots are the engaged employees and passengers are the lazy ones. The difference is whether they were in the room when the decisions got made. You can turn a pilot into a passenger by scoping the agent without them. You can turn a passenger into a pilot by bringing them in before the spec is written.

Co-design is a default, not a workshop

The fix is co-design, and the word gets misused. Co-design is not a workshop you run once to check a box. It's a default posture. The person whose work the agent touches is a designer of the agent, not a stakeholder of it. A stakeholder gets a demo and a feedback form. A designer gets to decide what the thing does.

On the memory build, co-design meant answering whose question the memory should serve before writing the schema. When the affected person is real, that's a conversation you have with them at the whiteboard, not a guess you make on their behalf. On a solo build like mine, the discipline is to name out loud whose reality I'm encoding, and to bring the real user in the moment there is one rather than handing them a finished system to react to.

Remember the wrong things well and you've built a confident, useless agent. It will answer fast and sound sure of itself, and it will be optimized for a question nobody on the ground was asking.

Three signals you're building passengers

You don't need a survey to know which one you're building. Three signals show up early.

  1. The affected person never opens the repo. They've heard about the agent in a standup. They've never looked at what it actually does. The distance is the tell.

  2. Demo feedback is polite, not specific. "Looks great, thanks" usually means they don't see their own work in it yet. Real pilots argue with you. They point at the one field that's wrong because they live in that field every day.

  3. The team keeps fixing the same agent every sprint. That's not iteration. That's paying down the co-design you skipped. Each "fix" is the system finally learning something the affected person could have told you before you wrote a line.

The asymmetry

The reason this is a leadership issue and not a nice-to-have is the math. Adding a co-design step costs about an hour per agent. One conversation, before the spec. Skipping it costs 65% more workslop, the rework of agents that solve the wrong problem, and the slow attrition of people who concluded the tools were built to replace them rather than help them. An hour against all of that is not a close call.

What this means for our build

We're a small operation experimenting with private local models. That smallness makes this choice more visible, not less. There's no layer of four thousand employees between a design decision and the person whose Tuesday it changes. When I decide what an agent remembers, I'm deciding whose job it's built around, and there's nowhere for that decision to hide.

The architecture decision was what to store and where. The leadership decision was whose question it answers. The first one lives in a config file. The second one lives in who I invite to the table before the config file exists.

References

This is part of my leadership series, "Leading Teams That Build Agents," the companion to my M4 Max local-AI build series. I'm building in public at Technology Playground. If you're thinking about AI for your team — or just want to argue with my take — I'm at bruce.canedy@technologyplayground.com.

Back to all posts