The Human In The Loop

Heidi sat across from the two engineers on the video call, their faces split across her monitor. Marcus on the left — methodical, unhurried, his fingers drumming against his desk. Dmitri on the right — leaning forward, intensity radiating through the screen.

The braking system design from the automotive company sat between them, invisible but present.

Marcus said the AI ran the full simulation suite — three thousand test iterations. Edge cases, wet conditions, sensor degradation. It all checks out. The safety margin is solid.

Dmitri shook his head. He'd been through the model assumptions. The AI is making inferences about tire grip coefficients that aren't validated for European road conditions. And when he traced back the sensor failure protocols, there's a gap in how it's handling cascade failures. Real world, that could matter.

Heidi said nothing. She was used to disagreement — engineers disagreed all the time. But they disagreed about the same thing, using the same language, the same tools. This felt different. This felt like they were arguing in two different dialects entirely.

She asked Dmitri how confident he was. He said not confident enough to sign off.

She turned to Marcus. And he said the math is there, that the AI doesn't lie.

That last sentence drove something cold into her chest. She thought of her cousin Stefan. Seventeen years old. Highway outside Krakow. A braking system that failed in the rain.

Stefan had been driving home from university. It was October. The roads were wet. The investigation afterward said the braking system had degraded past safe tolerances, but no one had caught it. No one had looked closely enough. For twenty-three years, Heidi had carried that fact like a stone in her pocket.

She became an engineer because of it. Then a safety engineer. Then she moved into regulation because she wanted to be the person who did look closely enough. Who wouldn't let it happen again.

Now she was staring at a design that two brilliant people had opposite conclusions about, and one of them was telling her that a machine had validated it.

"I need you both to send me your full workings," Heidi said quietly. "Not summaries. Everything. The assumptions, the model inputs, the validation datasets the AI used."

"That's going to take time," Marcus said.

"Then it takes time," Heidi replied. "We're not approving this until I understand it. Not because I don't trust you. But because I need to trust what we're doing."

She ended the call and sat alone in the meeting room, the silence pressing in. Outside, Brussels moved on. But in here, in this room, the weight of Stefan's death — and every death that might follow a decision made too quickly — settled onto her shoulders.

Heidi opened her laptop and pulled up the design files. The data was immense — thousands of pages of simulation outputs, model parameters, sensor specifications. She knew what she was looking for, even if she didn't know how to find it yet. The gap between what Marcus trusted and what Dmitri feared. The place where certainty collapsed into assumption.

She thought about calling someone. A colleague who understood AI better. Someone at headquarters. But that felt like abdication. This was her decision. Her responsibility. If she handed it off now, she'd never know if she'd made the right call or just the easy one.

Instead, she opened a new document and started writing down questions. Not answers. Questions. What datasets trained the AI? Were they European road conditions or global? What happens when the model encounters something it's never seen before? How does it fail?

And underneath all of it, the question she couldn't quite articulate: How do I know?

Not how does Marcus know. Not how does Dmitri know. How do I know?

Because that's what regulation was supposed to mean. It was supposed to mean someone — her — stood between a design and the public and said, "I have looked at this. I understand it. It is safe." Not, "A machine says it's safe."

She pulled out her phone and texted both engineers the same message: "Tomorrow, nine a.m. My office. Bring your work. Bring your doubts. We're going to understand this together."

Then she sat back and waited for morning.

Think of a decision you've made recently where you had help — a tool, a system, an algorithm, a recommendation engine. Something that gave you an answer.

Now ask yourself, honestly: Did you understand it? Or did you accept it?

There's a difference. Understanding means you could explain the reasoning, trace the logic, defend the conclusion. Accepting means it came back with a number, a result, a green light — and you moved on.

Heidi couldn't accept. Not with what she was carrying. Not with what was at stake.

You don't have to question every tool you use. But it might be worth knowing which decisions you'd want to truly understand — and which ones you're quietly delegating without realizing it.

Try this question.

"When I stopped blindly trusting the output and started asking how, I realized ________."

Hat tip to my colleague who handed me this one. They just didn't know it yet. Thank you!!