Judgement by Design starts with a simple question: in AI-enabled hiring, is the human genuinely flying the decision, or simply sitting in the cockpit?

“Don’t worry, we keep a human in the loop.”

I understand why that line has become the default reassurance for AI in hiring. It signals responsibility. It signals care. But in any system with automation, the question is not only whether a human is present. It is whether they are equipped to interpret what the system is doing, recognise when conditions have changed, and decide what to do next.

A human can be “in the loop” and still be a passenger.

That is why Judgement by Design matters. The real upgrade is not simply adding a person at the end of an automated workflow. It is building a decision process where people have the evidence, context and permission to challenge the recommendation before it becomes an outcome.

Why Judgement by Design Matters in AI Hiring

Gartner’s human-centred AI framing is useful because it moves the conversation beyond reassurance language. It asks leaders to consider whether AI should be involved in a task at all, not simply whether a human touches the output. That is a design question.

The regulatory direction is similar. In its Recruitment Rewired work, the UK Information Commissioner’s Office warned that many employers using automated recruitment are likely relying on solely automated decisions without meaningful human involvement. Where those decisions have a legal or similarly significant effect, they can fall within the UK GDPR provisions on automated decision-making.

Put those two points together and the implication is clear. Human involvement is not a vibe. It is something an organisation may need to evidence, explain and defend.

Judgement by Design makes that practical. It asks: what did the person review, what did they add, what did they challenge, and who owns the final call?

The Quiet Risk: Fluency Can Masquerade as Truth

AI is getting very good at producing outputs that feel complete. Summaries. Rationales. Shortlists. Recommendations.

In Talent Acquisition, that can be genuinely useful. TA is a constant mix of high volume, time pressure, imperfect data, shifting role requirements and real human consequences.

But tidy outputs can create a particular kind of confidence. They remove friction, which is often useful. They can also remove questioning, which is where judgement lives.

A 2025 study from Microsoft Research and Carnegie Mellon found that confidence in GenAI can shape whether people engage in critical thinking and how much mental effort they invest. In plain language, when a tool feels reliable, it becomes easier to stop interrogating it.

This is not an argument against AI. It is an argument for Judgement by Design, so the system keeps human thinking active rather than quietly replacing it.

Where “Human in the Loop” Falls Short

1. When AI Meets AI, New Forms of Advantage Appear

We are moving into a world where candidates use AI to write applications and employers use AI to evaluate them.

A recent study into AI self-preferencing in algorithmic hiring found that LLM-based evaluators could favour CVs generated by the same model, even when the underlying content quality was controlled. Its simulations suggested that candidates using the same LLM as the evaluator could be more likely to be shortlisted than equally qualified applicants submitting human-written CVs.

That is still emerging research, rather than a universal rule across every product and hiring process. But the signal is important. Compatibility effects may now be part of the hiring landscape, and they are not always visible on the surface.

A polished shortlist can look better while becoming subtly narrower. It can become less open to different ways of communicating, less curious about unconventional evidence of potential, or over-optimised for what the system finds easy to read.

Without Judgement by Design, teams may never spot that change.

2. Agentic AI Raises the Risk of Approval Drift

As AI moves from copilots to agents, workflows become chains of micro-decisions. Scheduling, messaging, stage movement, interview packs and recommendations may all be handled in sequence.

The human role can quietly drift from judgement to throughput.

Gartner has warned that more than 40% of agentic AI projects may be cancelled by the end of 2027 because of rising costs, unclear value or inadequate risk controls. That does not mean agentic AI has no place in Talent Acquisition. It does mean teams should establish where automation adds value, where people need to intervene, and what should trigger an exception.

In TA, the greater risk may not be one obvious mistake. It may be slow erosion. Approvals become faster. Exceptions become rarer. The process feels smoother, but less reflective.

Judgement by Design gives teams a way to notice when that is happening.

3. Cognitive Offloading Can Weaken Judgement Over Time

This is the most human failure mode, and probably the hardest to spot.

When AI takes on more thinking-shaped work, people can start to delegate the uncomfortable part of decision-making: sitting with ambiguity, asking the second question, challenging the neat narrative and dealing with the trade-offs.

In hiring, that can mean fewer second looks at unconventional profiles, less curiosity about what the system did not see, and more acceptance of rationales that sound structured but are not necessarily grounded.

Again, this is not a reason to reject AI. It is a reason to protect the muscle we do not want to lose.

Judgement by Design: A Better Test for AI Hiring

Judgement by Design is the idea that human judgement is not a last-minute safety net. It is deliberately built into how decisions are framed, challenged and owned.

It changes the question from “who reviewed the output?” to “how did we produce a decision we can stand behind?”

For me, it comes down to four principles.

Evidence Before Confidence

Fluent outputs can feel right. Judgement asks, what is this based on? What evidence is missing? What would change the recommendation?

Context Before Conclusion

Hiring is full of context that no model sees by default. Role nuance. Team dynamics. Local market realities. Constraints. The difference between potential and polish.

Trade-Offs in the Open

Good decisions are rarely perfect answers. They are choices between competing priorities. Judgement makes those trade-offs visible rather than hiding them behind a score or a rationale.

Ownership Stays Human

Tools can recommend. Accountability cannot be delegated.

This is the heart of Judgement by Design. The point is not to slow every decision down. It is to focus human attention where context, risk or uncertainty is highest.

It also matches the wider direction of enterprise AI. The real work is not only what a model can produce. It is how an organisation shapes the operating model, feedback loops, rules and accountability around it.

Calibration Checkpoints: How Judgement Stays Real

If Judgement by Design is the philosophy, calibration checkpoints are how it stays true over time.

A calibration checkpoint is a deliberate pause where the AI output stops being treated as an answer and starts being treated as an instrument reading. You validate assumptions. You adjust. You learn.

AI systems do not fail only in dramatic ways. More often, they drift. The market shifts. Candidate behaviour changes. Hiring priorities evolve. What worked last quarter quietly stops working this quarter.

Everest Group describes Human + AI as a feedback dynamic: people can flag failure patterns, improve prompts and adjust fine-tuning priorities as enterprise norms, markets and regulatory expectations change. That is calibration as a capability, not a compliance step.

There are three moments where this matters most.

Input Calibration

Are we feeding the right signals in?

In TA, this is where you sanity-check the definition of “good” before you optimise for it. Is the brief clear? Have we separated essential skills from preferences? Are we using relevant, current and lawful data? Do we understand what the model is being asked to infer?

Input calibration stops us automating a weak brief at speed.

Output Calibration

Does this recommendation still make sense when reality is added back in?

This is where experience pressure-tests the output against context, constraints and edge cases. It is where a hiring manager or recruiter can say: “That might be true in the data, but it misses something important about this role, market or person.”

Output calibration is Judgement by Design in action. It turns review from a passive approval into active interpretation.

Outcome Calibration

Did we get the result we wanted, and what did we learn?

This is where teams look beyond whether a role was filled. Did the shortlist hold up? Did candidates progress fairly? Did we see a pattern in false positives, false negatives or exceptions? What should change before the next hiring cycle?

Outcome calibration turns Judgement by Design into a learning system, rather than a one-off intervention.

Judgement by Design as a Talent Advantage

LinkedIn’s 2026 Talent Velocity Advantage report reinforces what many leaders are sensing. Just 14% of organisations are pulling ahead, and these talent velocity leaders are more likely to develop both AI literacy and the human skills that help people work with AI well.

The edge is not access to AI. Most organisations will get access to similar tools.

The edge is interpretation and action.

Two teams can deploy the same technology. One treats outputs as shortcuts. The other builds Judgement by Design into how it sets the brief, reviews recommendations, handles exceptions and learns from outcomes. Only one of those builds a durable advantage.

That also links to AMS’s work on human judgement and AI in recruitment, and to the superworker idea. The value of AI is not making people move faster through the same admin. It is creating capacity for people to operate at the top of their licence: interpreting signals, handling exceptions, building relationships and making the calls that matter.

What We Cannot Outsource

There is a useful point in Simon Sinek’s reflections on AI: reducing friction is not the same as removing the work that changes us.

In decision-making, that work is the struggle with ambiguity. The second question. The willingness to say, “I am not yet convinced.” The ability to make a hard call when the right answer is not obvious.

AI can reduce admin. It can widen search. It can structure information. It can remove friction.

But it cannot own responsibility. It cannot hold the full human context of a hiring decision. And it cannot do the work of judgement for us.

So yes, keep a human involved.

Just do not stop at involvement.

Design for judgement. Keep it calibrated. Build decisions you can stand behind.

Judgement by Design. Calibration checkpoints. Decisions you can stand behind.

A human can be ‘in the loop’ and still be a passenger.