The History of Qualitative Research

5 min readSep 6, 2018

Originally posted in 2015

There are three immense changes happening right now that should spur any business into a revaluation of their approach to qualitative research.

The first is the digital revolution. Twitter, Tumblr, Facebook, Pinterest, WhatsApp and all the rest are creating new forms of content that allow us to interact with research participants in fantastic new ways. By leveraging community building tools we can keep the research conversation going and build rich and rewarding relationships.

The second is the explosion of academic theories. Qualitative research now exists in a radically new theoretical landscape. The rational consumer has been replaced by the irrational and biased one. The emphasis now is on understanding the personal attitudes, previous experiences, social norms and environmental factors involved in decision making — all of which have long been considered the domain of qualitative research.

Finally, the emergence of big data is making qualitative research imperative. In a world where everything is measured, where sentiment and topics can be analysed on mass, and with increasingly sophisticated algorithms — providing a human context is crucial.

A quick history of qualitative research

To understand where qualitative research is heading, it pays to have a quick glance at where it came from.

You can see the emergence of qualitative research at the start of the 20th century as the influence of psychoanalysis started to enter into the commercial world. By 1945 the father of qualitative research, Paul Felix Lazersfield, had shown how psychology could provide aframework to interpret human behaviour. He introduced the world to unstructured interviewing and group discussions, and stressed the importance of answering the important ‘why?’ question.

More famous than Lazersfield was his pupil, Ernest Dichter, the ‘Freud of the supermarket age’. Dichter developed his ideas in a prosperous post-war America when mass production and an abundance of consumer goods meant that companies began to rely on branding to differentiate themselves. Arguing that people didn’t know why they made the purchasing decisions they did, he began to uncover the personality behind brands and the hidden desires and urges behind purchasing decisions.

During the 60’s and 70’s projective techniques such as word associations, sentence completion, speech bubbles and role playing all began to gain traction. This era also saw the ‘qual vs quant’ debate heat up as qualitative methods started getting established. Up until this period scientific, provable, quantitative research was the dominant paradigm.

And it would continue to be so until the 80’s when qualitative research really started to deliver what clients wanted. The explosion throughout this decade in qualitative research encouraged people with different skillsets to add to the mix, and it was no longer necessary to have a psychology degree to be a qualitative researcher.

Today, qualitative research makes up 18% of global research spend and has grown to embrace a myriad of ideas and methods from linguistics, social sciences and psychology.

What does qualitative research do?

Although it is basically true that qualitative = ‘words’ and quantitative = ‘numbers’, it is important to dive a bit deeper into the role of qualitative research.

Qualitative research is, by its very nature, an agile and reactive discipline. Over time, a host of different business questions have been asked of it. This has meant that it operates in a variety of different modes.

On one level, it can listen to how people answer straight-forward questions. This can allow you to capture superficial, but often crucial, information. More usefully, it dives deeper and works out what people don’t realise they think. It is concerned with mapping out the mental associations of the human mind. It reveals the invisible and complex motives and needs of our inner emotional and unconscious world.

Some things are difficult for people to say. They may be difficult to admit, misrepresented or exist behind a mental block. Qualitative research can disclose this problematic data by digging deeper into participant’s lives, connecting different parts of interviews and leveraging contradictions.

Beyond interviewing, qualitative research also uses behavioural observations to see what really happens in people’s lives. The automatic actions of people in their day to day lives provide glimpses that could be difficult to express in words.

Creativity, future forecasting and insight

Whilst the above description outlines how qualitative research can map out the richness of the human mind, it can also serve other key purposes.

By leveraging workshops and by providing tools to stimulate thoughts, a qualitative researcher can assist in any creative process. Whether it is generating entirely new ideas, building a brand story, or refining communication — qualitative research can provide the much needed spark of inspiration.

Qualitative research also helps prepare for the future by allowing researchers to use a repertoire of techniques to interrogate the effects of future events and understand future behaviours.

In both creative and future-facing exercises, it is often the case that only a handful of people will provide anything of use. These outliers could be early adopters or laggards, or they could be particularly creative or analytical minded people. Either way, it is the value of unique personalities, extracted through qualitative tools that provide the value.

Finally, it is lacing all this together into a compelling narrative that can spur on business decision-making that makes qualitative research crucial and valuable.

So what now? The next stage of qualitative

What I hope to have illustrated above is that qualitative research has had a short, but exciting lifespan. Having only really been popularly accepted as a method since the 80’s, I would suggest that it is yet to fully mature.

It is a method that has at its heart an ability to embrace other disciplines. This is important in a era where there is a proliferation of academic ideas - where a whole new picture of the human mind is appearing. More than ever, the judgement of a qualitative researcher is needed to navigate the increasingly complex maze of the mind.

Technology is both a threat and an opportunity for qualitative research. The rise of big data and the intelligence it delivers could bring back the ‘quant vs qual’ debate in a new form. On the other hand — mobile devices, a continually connected population, and new technological advances could herald a golden age for insight.

Previously, qualitative research consisted of focus groups, interviews and observation. It took a while for the benefits of these tools to filter into common business practice. Now it is mobile ethnography, social listening, online communities, commercial semiotics and more that make up the research toolbox. Let’s just hope that it doesn’t take as long this time around.