At Yuzu Communications, we're constantly asked about AI implementation. But here's what we've learned: the conversation shouldn't start with "What can AI do?" It should start with "Who's in control, and why does that matter?"

The question no one's asking

Recent research from the Association for the Advancement of Artificial Intelligence (AAAI) Conference on Artificial Intelligence poses a critical question that challenges how we think about AI in business: Are we building human-in-the-loop (HIL) systems or AI-in-the-loop (AI2L) systems? And more importantly, do we even know the difference?

This isn't semantic hair-splitting. It's fundamental to successful AI integration.

Two systems, two realities

Human-in-the-loop (HIL): AI makes the decisions, and humans provide feedback and corrections. Think recommendation algorithms, fraud detection systems, or automated content moderation. The AI is driving; humans are course-correcting.

AI-in-the-loop (AI2L): Humans make the decisions, and AI provides support and insights. Think physician treatment planning, investment advisory, or quality assurance strategies. The human is driving; AI is the sophisticated co-pilot.

The distinction matters because these systems fail differently, scale differently, and integrate differently into your business processes.

Why This matters for business integration

Here's where many organizations stumble: they evaluate AI systems purely on technical performance—accuracy, precision, recall—without considering how these systems will actually work within existing business processes and organizational structures.

From our experience at Yuzu Communications, the failure point isn't usually the AI itself. It's the integration gap between what AI can do and how businesses actually operate.

The integration questions we should be asking:

For HIL Systems:

• How do we maintain human oversight without creating bottlenecks?

• What happens when the AI encounters adversarial inputs or manipulation?

• How do we assess the credibility of human feedback at scale?

For AI2L Systems:

• Does the human decision-maker trust and understand the AI's recommendations?

• Can we explain why the AI suggested a particular course of action?

• What happens when AI outputs don't align with human expertise or intuition?

The Real Challenge: process, not just performance

Consider two scenarios from the research:

1. Early diagnosis of Alzheimer's disease—This is HIL territory. AI analyses MRI scans and detects anomalies automatically. High accuracy is paramount, human oversight is confirmatory.

2. Treatment plan formulation—This is AI2L. A physician selects and tailors the final treatment plan from AI-generated candidates, considering patient context, values, and factors AI cannot fully comprehend.

Using HIL evaluation methods (accuracy, F-scores) for an AI2L system would be like judging a hammer by how well it types. You're measuring the wrong thing.

What this means for your business

When we work with clients on AI integration, we've learned that success depends on three critical factors:

1. Honest assessment of control

Who should be making the final decision in your process? Is automation appropriate, or is collaboration essential? This isn't always obvious and varies even within departments.

2. Evaluation that matches reality

If humans are in control (AI2L), your evaluation must go beyond model performance. You need to measure:

• Impact on decision quality and outcomes

• User trust and adoption

• System explainability and transparency

• Ability to handle edge cases and exceptions

3. Integration architecture that reflects purpose

Your technical architecture should match your control model. HIL systems need robust feedback loops and adversarial safeguards. AI2L systems need transparent reasoning, user-friendly interfaces, and seamless workflow integration.

The Yuzu Communications perspective

We've seen organisations invest millions in AI capabilities only to achieve minimal business impact. Why? Because they treated AI as a plug-and-play solution rather than a fundamental shift in how work gets done.

The most successful AI implementations we've supported share a common trait: they started with process understanding, not technology selection. They asked:

• What decisions are we making now, and who makes them?

• Where do bottlenecks and errors occur?

• What would improve our outcomes—automation or augmentation?

• How will this change daily workflows and responsibilities?

Only after answering these questions did they consider which AI capabilities to deploy and how to integrate them.

Moving forward

The research community is starting to recognize that "human-in-the-loop" has become a catch-all term that obscures more than it clarifies. As businesses, we need to be equally precise.

Before implementing your next AI initiative, ask yourself:

1. Is this an automate or collaborate scenario?

2. Who should maintain control, and why?

3. How will we measure success beyond technical metrics?

4. What organizational changes are required for successful integration?

The bottom line

AI capability is abundant. What's scarce is thoughtful integration into business systems and processes that respects both human judgment and machine capability.

At Yuzu Communications, we believe the future isn't about choosing between humans and AI. It's about designing systems where each contributes what they do best—and being crystal clear about who's in the driver's seat.

The question isn't whether to use AI. It's about how to integrate it in ways that genuinely improve outcomes, respect human expertise, and align with your organizational realities.

Because at the end of the day, technology is only as good as the systems and processes it serves.

This article draws on research from "Human-in-the-loop or AI-in-the-loop? Automate or Collaborate?" (Natarajan et al., AAAI-25). While we've highlighted one academic perspective, the broader conversation about AI integration continues to evolve. We'd love to hear your experiences: Are you building systems that automate or collaborate? What integration challenges have you faced?

‍

‍