Scaling Qualitative Research Without Losing Its Soul? Lessons from Using R on Big Qual Data
As qualitative researchers, we’re often told that if we want to work at scale, we need to give something up. Nuance, context, or depth. But what if we didn’t? What if, instead of flattening our methods to fit the data, we expanded our toolkit?
That’s exactly what Dr Emma Davidson, Prof Lynn Jamieson, Dr Susie Weller, Prof Rosalind Edwards, and Justin Chun-ting Ho set out to explore in a recent feasibility study focused on care and intimacy across the life course. Working with six core studies from the Timescapes Qualitative Longitudinal Data Archive, the team faced the challenge of interpreting huge volumes of interview data, too large for conventional qualitative methods alone.
They responded with a hybrid approach, combining computational text analysis in R with semantic mapping in Leximancer. The results reveal a thoughtful, iterative methodology that preserves interpretive richness while making large-scale analysis not just possible, but productive.
Moving Beyond Volume: Analysing Differently
At the heart of their project was a breadth-and-depth strategy: using machines to scan and map thematic patterns across the data corpus, and humans to interpret and refine those findings.
They began by using R to develop pre-defined thematic clusters (such as “conflict”, “relationship work”, and “formal childcare”) and comparing their presence across gender and generation. This surfaced broad trends that manual coding could have missed. For instance, older women spoke more often about family, younger women focused on friendship, and men leaned toward work and leisure.
But even with R’s customisability, the team quickly recognised its limitations… both technical and logistical.
The Practical Role of Leximancer
When working with R, the team encountered barriers typical of qualitative researchers adopting computational tools: steep learning curves, limited experience in statistical programming, and difficulty finding collaborators who understood both the technical and interpretive demands of the work. The team noted that many experts in natural language processing weren’t particularly interested in projects driven by qualitative theory and narrative structure.
Leximancer provided a more accessible, ready-to-use solution for exploratory thematic mapping. Unlike R, Leximancer doesn’t require programming expertise. It allowed the team to visualise semantic relationships between concepts, discover emergent patterns without imposing a fixed ontology, and generate maps that could direct further interpretive work.
By using Leximancer to scan the broader thematic terrain, they could identify areas worth deeper exploration. Then return to R or manual methods as needed. This was particularly valuable for supporting the shallow test pit strategy they developed: surfacing potential themes across the dataset, sampling them at a shallow level to test their significance, and then deciding whether to drill deeper.
Crucially, Leximancer enabled continued progress when the limitations of R (time, skill gaps, and complexity) threatened to slow the project down. It became a practical bridge between intent and execution.
Interpretation Still Matters
The team remained clear-eyed about the risks of relying too heavily on automated outputs. For example, certain words like “sibling” appeared frequently not because of their thematic relevance across interviews, but because they featured prominently in the design of a single study within the dataset. In other words, not every high-frequency term is an insight.
Computational tools can amplify what’s already present, but without critical interpretation, they can also amplify noise. This is where the recursive movement between Leximancer, R, and human-led analysis became essential.
The tools didn’t replace interpretation. They shaped it.
Lessons for the Future of Big Qual
This study offers a compelling model for qualitative researchers facing the realities of scale in the digital age. It doesn’t pretend that more data is inherently better. It doesn’t suggest that automation can do our jobs for us. Instead, it shows how a deliberate interplay between machine and human analysis, between tools like R and Leximancer, can produce better, more reflective research.
A few key takeaways:
Flexible tools like R offer precision and control, but they come with a steep learning curve. Making the most of them often requires technical expertise and genuine collaboration across disciplines. Something not always easy to find or fund.
User-friendly platforms like Leximancer lower the barrier to entry, enabling teams to quickly map themes and visualise patterns without needing to code. This makes them especially valuable for interdisciplinary teams working within tight timelines or resource constraints.
No matter how sophisticated the tool, meaningful insights still depend on the researcher’s ability to question, contextualise, and critically assess what the software reveals.
Investing in training, time, and multi-method fluency is essential if qualitative research is to thrive at scale. This means knowing when to zoom out, when to dig in, and which tool is best suited for each stage of the journey.
This was never a simple question of tool selection. It was a lesson in research design that balanced scalability and with depth. And one that offers a guide for other researchers navigating Big Qual challenges.
📖 Read the full post in Big Qual Analysis Resource Hub
