AI Moderation: A Game-Changer or a Gimmick?

At the last three research conferences I attended, AI-powered qualitative research - often called ‘Qual at Scale’ - was everywhere. Research tech companies are positioning it as a way to make qual more scalable, efficient, and cost-effective. At the heart of this shift is AI moderation - chatbots conducting real-time interviews via text or voice. The pitch is compelling: AI moderators work 24/7, never get tired, and scale research at a fraction of the cost of human moderation. But does this still deliver real qual?

This article explores what the evidence so far suggests that AI moderation can and can’t do. We’ll look at the gains in efficiency and scale but also at what’s lost when the human moderator is removed from the conversation. AI moderation is already being used in a growing number of studies - so how do we work with it effectively while maintaining research quality?

AI Moderation vs. Human Moderation

In The Essence of Good Qual, we explored the skills that define the best moderators – the ability to use all their senses and intuition and subjectivity to guide conversations where they naturally need to go, which can differ for each participant.

AI moderation is built for efficiency. It can run hundreds of interviews at once, follow a structured guide, and apply the same prompts consistently. It’s scalable, fast and cost-effective - particularly useful for large-scale studies where speed and structured analysis are priorities.

But moderation is more than just following a guide. A skilled human moderator listens between the lines, detects contradictions, probes unexpected responses, adapts dynamically – letting the conversation unfold naturally. AI, in contrast, follows a script. It can analyse sentiment but doesn’t truly understand emotions*. It can probe but won’t detect hesitation or discomfort. It can ask follow-up questions, but it won’t challenge an assumption or spot an insight buried in an offhand remark. And it can’t pivot when a topic is well understood, or a new issue emerges. In many ways it acts like a novice moderator – structured, but inflexible.

There’s also the issue of cultural nuance. AI moderators work by processing language patterns, but of course they lack lived experience. They don’t understand humour, irony, or how cultural references shape meaning.

* As I write, one provider in this space is claiming to offer conversational agents with “emotional intelligence”, that can “truly perceive, listen, understand and engage in a deeply human way”. I’m sure the models will get better – whether they can get close to matching the emotional intelligence of an experienced human moderator remains to be seen and feels a long way off (but I could be wrong!).

Where AI Moderation Works Best

Despite its limitations, I believe AI moderation should have a place in the researchers’ toolkit There are clear cases where its strengths in scale, efficiency, and cost-effectiveness outweigh its lack of depth. In Episode 4, we discussed the cultural shift towards immediacy and ‘good enough’ research, which is also driving the move towards quicker, cheaper methodologies.

At the heart of identifying potential use cases is the question “How much depth is needed?”. Some briefs are straightforward, don’t require deep exploration, and lend themselves to the semi-structured approach often referred to as ‘Qual Light’. Here are some of the strongest use cases we see:

• Some UX testing is one area where AI moderation works well, guiding participants through digital experiences or assigning online tasks.

• Rapid exploratory research to unpack a topic – where quick answers matter more than deep probing

• Iterative concept testing where brands need quick feedback loops on ideas, messaging or creative executions

• Conversational surveys can bridge the gap between a traditional survey and an in-depth interview, providing more engaging participant experiences and greater depth than an online survey, at the scale of quantitative surveys.

• Online communities often require a light touch – either to keep engagement high or to probe answers provided to tasks and questions.

We shouldn’t just think about use cases but users. One of the biggest advantages of AI moderation is accessibility. Because it’s built into DIY research platforms, it allows a broader range of people - marketers, product teams, and UX designers - to run qualitative studies without needing formal research training. This means more people conducting qualitative-style research, though not necessarily at the same level of depth or rigor as traditional qual. If this allows more people to experience the benefits of qualitative insights (albeit a light version), this may not be a bad thing for our industry.

How to Use AI Moderation Effectively

If we decide to use this method, how can we get the best possible outcomes? Quite simply – the best outcomes will be achieved by using an experienced qualitative researcher at every stage – by designing the discussion guide, monitoring the interviews, adjusting the questioning as needed during the project, and being involved in the analysis and reporting.

While any single experienced human moderator can iterate during their allocated interviews, this has traditionally been a challenge in multi-country design, using multiple moderators. The centralisation of ‘Qual at Scale’ platforms allow a centralised team to review early interviews across markets and pivot as needed. Indeed, the research can even be rolled out in stages, to build in moments of reflection and iteration.

Hybrid approaches hold real potential for studies requiring both depth and scale. This is where experienced qualitative practitioners can lead - integrating human-led and AI-led interviews. Just as we mix qual and quant in the same study, there are endless possibilities, including designs that combine human moderation with ‘Qual at Scale’ in place of Quant.

A Blurring of the Boundaries

Hopefully by now it’s clear that AI-powered moderation doesn’t fit neatly into either qual or quant terminologies. It’s more open-ended than traditional quant, but it lacks the depth and adaptability of human-led qual. It can be analysed qualitative or quantitatively (or both).

This is perhaps a new category of research that introduces a new kind of trade-off. The depth, emotional intelligence, and contextual sensitivity that define real qual aren’t fully there - but in exchange, a ‘lighter’ form of qual research can be conducted at a scale, speed, and cost that was previously out of reach.

For clients and research buyers, understanding this trade-off is key. AI moderation isn’t a substitute for human moderation – rather it’s an addition to the researcher’s toolkit, offering a lighter-touch, scalable option that works well in the right circumstances. The challenge for researchers is to position it correctly - to educate clients on what it can and can’t do, and to ensure that when depth is required, it’s built into the research design.

Conclusion

AI moderation is already being used by clients and agencies. It offers scale, efficiency, and cost-effectiveness, making qualitative-style research more accessible to a wider range of users. But it comes at a cost - losing the human depth, contextual sensitivity, and nuanced interpretation that traditional qual provides.

This isn’t about rejecting AI moderation – it’s about knowing where it fits. The best qualitative projects will always be led by experienced qualitative researchers, ensuring that AI is used strategically rather than as a blunt instrument. As AI evolves, so must the role of the qualitative researcher. We need to get comfortable working alongside AI, understanding its strengths and limitations and expanding our toolkit accordingly. If we do that, we’ll continue to thrive.

Next week in The Campaign for Real Qual, we’ll look at how AI is being used in qualitative analysis – one of the areas where many people are already experimenting. Once again, we will explore where automation adds value, and where human expertise remains irreplaceable.