The Oversight Illusion: Rethinking “Human in the Loop”

The common refrain among HR leaders integrating artificial intelligence into their workflows is that they always keep a “human in the loop”—a statement meant to reassure legal teams, business leaders, and employees alike that a real person is checking the work.

But what is that human actually doing?

“Humans in the loop is a necessary but insufficient condition,” said Ken Matos, PhD, director of market insights at HR tech platform HiBob. “Everybody can say ‘there’s a human here,’ but can that human actually affect anything?”

Having a person involved might seem like a safeguard, but if the work moves too quickly for the human to keep up, or that person isn’t qualified to review output—the safeguard becomes a scapegoat, or, at worst, a liability.

Just because a person is licensed to drive a car doesn’t mean they’re qualified to inspect that car before it leaves the factory floor.

And even if they are qualified, there’s only so much they can do. Imagine asking someone to inspect 50 cars in three minutes. “If the AI is making a decision every 30 microseconds, it doesn’t really matter if a human’s there,” said Matos. “If you want to have a human in the loop, you need to make sure that the loop moves at a speed where the person can process what’s happening and that they are able to interfere with that loop.”

During a From Day One webinar on rethinking the notion of having a “human in the loop” when it comes to AI, Matos challenged HR leaders to think more critically about what human oversight looks like, and what it takes to make it work.

Rewarding Accuracy vs. Rewarding Speed

One question leaders should ask is whether they’re rewarding speed or accuracy.

Think about a car maker. If workers are rewarded for speed and penalized for stopping production, an error could be repeated for hours, producing hundreds of cars that aren’t fit for purpose. But if quality is the priority, workers have a reason to stop the line and fix the problem.

The same goes for AI workflows. If employees are under pressure to work faster, how likely are they to stop and question an AI-generated result? And if they do catch a mistake, do they have the authority to do anything about it? Otherwise, the human in the loop may be little more than a scapegoat.

Can Employees Spot the Mistakes?

HiBob surveyed 1,200 decision-makers about what AI skills they consider most valuable for their non-technical employees. At the top of the list was proactive output review. Can employees confirm that AI-generated results are accurate?  Did they document workflows for consistency? Did they appropriately handle sensitive information?

“It’s all about judgment, reliability, and governed execution,” Matos said.

But AI makes it possible to produce work you don’t necessarily understand. Someone might use AI to create a graph, for example, but struggle to explain what it shows. This makes reviewing the work especially important. Yet when speed is the priority, employees may have little choice but to rely on spot checks. 

But spot checks aren’t always enough. Matos gave the example of a company that made layoffs based on an AI analysis of productivity scores. The analysis didn’t account for medical leave, so employees who had been out of work received low scores and were let go. A thorough review might have caught the fact that previously high-performing workers were rated at the bottom of the list. A spot check didn’t catch it.

How Many Jobs Can One Person Do?

Some leaders are using AI to fold specialized tasks into other jobs. If AI can handle much of the work, why not give one employee several responsibilities? But there’s a limit to how much any one person can reasonably take on, even when that work is carried out by agents.

“Just how many roles can we squeeze into one person before it’s just too much, and they’re not doing it well?” Matos said. “Have I asked this person to do too many jobs? Can I get one employee to do risk management and set out the standards that everyone else follows, or am I expecting everybody to be an independent risk management expert?”

Not everyone needs to become an expert. Matos suggested that peer mentoring may be a better approach. Companies might identify a handful of employees who understand AI and how it applies to their work, then give them time to keep learning and share what they know, freeing their colleagues to focus on the AI skills they actually need, rather than trying to become experts in a technology that’s constantly changing.

Keeping a human in the loop should mean having someone who can question the work, not simply approve it. That’s especially important when tools can produce convincing answers even when they’re wrong. As Matos observed, “AI is really good at sounding correct, much better than it is at being correct.” 

Editor’s note: From Day One thanks our partner, HiBob, for sponsoring this webinar. 

Emily McCrary-Ruiz-Esparza is an independent journalist and From Day One contributing editor who writes about business and the world of work. Her work has appeared in the Economist, the BBC, The Washington Post, Inc., and Business Insider, among others. She is the recipient of a Virginia Press Association award for business and financial journalism. She is the host of How to Be Anything, the podcast about people with unusual jobs.

(Photo by triloks/iStock)