Hi there, my name is Kitty Xu, and I am a quantitative user experience researcher (a.k.a., Quant UXR). What is Quant UX, you ask? Over the years, I've published 3 articles on it, and you can read them here:
What is quantitative user experience research at Pinterest?
You’re a STEM major? Your dream job might be in Design
From brain science to quantitative UX
Today, I want to tell a different story, one that involves doing qualitative UX research (a.k.a., qual UX) and how it made me a better Quant UXR.
For clarification, I’m not talking about partnering with a qual UXR. That’s often beneficial, especially if you have the privilege of working with multiple researchers (Lesson #4 here)! What I'm additionally arguing is that Quant UXRs can develop and execute qual UX research, and this will reciprocally improve their Quant UX research capabilities.
If you consider yourself someone who always defaults to quant research methods and you are skeptical about whether building up some qual research experience in your portfolio will actually make you a better Quant UXR — or if you're just too intimidated to execute research outside your methodological comfort zone — read on ...
The limitation of technical expertise
In the mid-2010s, Quant UX research was still defining its professional identity. When I entered the field, I thought what differentiated Quant UXRs from their qual counterparts was technical expertise — particularly the ability to design surveys, query log data, and program in languages like R or Python to format, manipulate, and analyze large datasets, and to run statistical modeling and create data visualizations. Back then, I believed that advancing these technical capabilities was the key to becoming a better Quant UXR.
Early in my career, my work reflected this mindset. I wanted to be an "numbers expert," showcasing advanced statistical models, intricate SQL queries, and meticulously crafted surveys. I built strength in transforming survey responses and behavioral log data into actionable user insights at scale.
However, technical mastery alone doesn't guarantee research impact. As my career progressed, I faced increasingly complex and ambiguous challenges that couldn't be solved with complex data analysis alone. Such research required a deep understanding of nuanced spaces and the ability to identify opportunities for meaningful product improvement, balancing both user experience and business needs.
At the same time, I came to appreciate a hierarchy of research excellence:
A good researcher answers a question effectively.
A great researcher selects the most appropriate tools to answer a question effectively.
An amazing researcher ensures they are tackling the right question, then selects the most appropriate tools to answer it.
An exceptional researcher ensures they are tackling the right question, then selects the most appropriate tools to answer it, and crafts compelling narratives that inspire action.
Modifiers used to describe a researcher shall not be used for a survey scale question :).
My work evolved from technical execution to uncovering deeper user insights that could shape business strategy. Technical skills remained critical, but they became tools in service of a more fundamental goal: understanding user experience at its deepest, most nuanced level.
The true measure of research impact, I realized, isn't the complexity of the analysis — it's the potential for meaningful change in both user experience and business outcomes.
First steps into qualitative research
“A fundamental principle of innovation or creative thinking is to start with empathy.” – IDEO
People SAY, DO, THINK and FEEL, and these don’t always align. Quant UX research focuses on understanding what (and how much) people DO and SAY they feel about a product or experience at scale by collecting and analyzing empirical evidence. In contrast, qual UX research delves deeper, exploring why people THINK and FEEL the way they do, uncovering nuanced motivations and underlying emotions. Combining them gives me a richer data set to analyze patterns, synthesize the meaning, identify opportunities, and build a compelling narrative.
My initial experience with qual research came through collaborations with skilled qual UXRs. This was done in two ways: (a) observing and leveraging the work of my qual UXR colleagues, and (b) designing and commissioning qual studies done by external research vendors. Through projects ranging from consumer shopping journeys and advertiser trust to teen safety and user segmentation, I developed a deeper appreciation for the nuanced and layered insights that qual research provides, extending far beyond what my data charts and dashboards alone can reveal.
These early collaborations were eye-opening. While I wasn't conducting the user interviews directly, I learned to:
Leverage qual findings to complement and enhance my quant analysis.
Triangulate quant and qual insights to craft narratives that are both data-driven and human-centered, combining breadth with depth.
Recognize and value contextual details from the periphery of observations and interviews, uncovering hidden insights that often prove invaluable during research synthesis.
I thought I had gained a solid understanding of qual research through these collaborations, but conducting my own qual research projects taught me far more. Over the past couple years, I led several international research initiatives, employing both remote and in-person methodologies. The remote studies consisted of in-depth interviews (IDIs) conducted over video calls, working across multiple time zones and languages with the help of simultaneous interpreters.
For field research, I sourced and partnered with local research agency partners to operationalize the study, and facilitate on-site market deep-dives. I brought cross-functional teammates across the world to meet and interact with users in their own local context. My travel-mates spanned exec leadership, product managers (PMs), engineers, designers, analysts, and marketers. These studies were conducted across China, South Korea and Turkey.
For a Quant UXR who revels in the structure and predictability of quant methods, the unstructured nature of qual research initially felt messy, disorienting and deeply uncomfortable. Yet, this discomfort was a catalyst that allowed me to grow.
I was fortunate to learn from veteran qual UXRs who provided invaluable mentorship throughout my growth. They shared essential resources like recruiting screeners and interview protocols, which gave me a strong foundation to build upon. By shadowing their interview sessions, I gained firsthand experience with effective interviewing techniques and learned to handle unexpected challenges, from technical issues to participant no-shows. Their guidance on working with international research vendors proved especially valuable, helping me navigate cross-cultural research complexities.
With their support and encouragement, I developed confidence in leading end-to-end research processes, developing skills in:
Research design and methodology: Crafting nuanced interview protocols and designing diverse research methodologies, including IDIs, focus groups, expert interviews, co-creation workshops, and dinner mixers that facilitate authentic conversations between my cross-functional teammates and research participants (who are English language learners in our target markets).
Global research operations: Sourcing, vetting, and relationship managing of local qual UX agencies and facilities in target markets.
Participant recruitment: Developing and localizing screeners to recruit research participants across complex cultural landscapes, languages, and timezones. (This one was particularly hard for me, not going to sugar coat it.)
Moderation and facilitation: Conducting IDIs through simultaneous interpreters across multiple languages. In one challenging case, I conducted an interview in both English and Mandarin simultaneously to accommodate both the participant and stakeholders observing the session.
Qualitative data analysis and synthesis: Synthesizing qual data, including individual stories, observations, and unstructured data, into a cohesive narrative.
Leadership in cross-functional collaboration: Guiding large cross-functional teams during intensive international research trips, and facilitating cross-functional team debriefs and synthesis sessions.
These projects offered far more than just data collection. By conducting qual research myself, I learned to step back, to shift my focus from being the primary driver of the research to truly listen — not just to what the participants said, but also what they didn't say, and the nuances between the lines.
This shift in perspective taught me to set aside my own preconceptions and embrace genuine curiosity about what I might discover through these intimate exchanges across languages and cultures. The resulting data, synthesized insights, and the stories we crafted felt more human and more personally meaningful, because they were grounded in real human experiences.
The power of qualitative research for Quant UXRs
Qual research has been instrumental in elevating my work as a Quant UXR. By delving into the nuances of user experience, it has enabled me to reframe research questions, develop holistic research strategies, and channel user empathy for greater outcomes, empowers me to deliver more impactful insights, influencing stakeholders to prioritize the most critical opportunities with conviction.
Reframing research questions
In quant research, I often started with preset questions. However, qual research taught me the importance of understanding the user's world first. By listening to users' unfiltered experiences and identifying pain points they might not articulate in a survey, I am able to challenge assumptions and uncover deeper issues.
For example, when I worked on an educational product that connects English language learners with native English-speaking tutors through 1-on-1 online lessons, one data analysis revealed that despite offering free trial lessons to all new users, free trial lesson usage was low. The growth product team initially focused on optimizing the free trial's visibility on the homepage. However during an international research trip, we observed new users struggling to understand how to use the product. This led us to reframe the problem from "why aren't new users using free trial lessons?" to "how can we improve new user comprehension during onboarding?" The team shifted its focus to educating new users on how to navigate the product, and take key actions earlier in their learning journey, which resulted in a more significant impact on free trial lesson adoption, and ultimately increased the conversion rate.
Developing holistic research strategies
As a Quant UXR leading qual studies, it allowed me to strategically design research that investigated the "why" behind my statistical findings. Rather than participating in other qual UXRs’ work and following someone else's research protocol, I could craft interview questions that directly probed into patterns I'd identified in my data, creating a powerful feedback loop between methods.
For example, while my quant research report suggested that English language learners highly value access to native English-speaking tutors, the product team, based in Silicon Valley, initially prioritized AI technology in our new user acquisition strategy. When I took the team on an international research trip and immersed ourselves in non-English-speaking contexts, the team realized that many learners faced significant barriers to accessing native speakers, far greater than the team had initially imagined. One learner poignantly expressed, "Before discovering your product, learning English meant waiting 45 minutes in a classroom for a chance to speak one sentence with my English teacher."
Armed with both the statistical evidence from the quant report and these powerful firsthand experiences, my team was compelled to reevaluate our strategy and re-emphasize the value of native speaker access in our user acquisition efforts.
Channeling user empathy for greater outcomes
Perhaps, most significantly, qualitative research is pivotal in cultivating a shared, profound empathy for users among myself and stakeholders. By directly engaging with the people we design for, we collectively gain a deeper understanding of their motivations and frustrations — and the often unspoken anxieties that drive their behavior.
For example, in my survey study, we discovered that many English language learners in the APAC region highly valued the curriculum during lessons with native English-speaking tutors. Initially, our takeaway was that APAC learners have a tendency to prefer structured lessons over free-form speaking. However, after conducting interviews with learners, we uncovered a deeper insight: structured lessons with a pre-selected curriculum served as an avoidance mechanism for these learners’ anxiety, alleviating the dreaded fear of running out of topics to discuss with their tutor — a fear that significantly impacted their learning experience! This insight helped our product team not only build more experiences that allowed structured English lessons, but also explore other ways they could address the more impactful pain point and opportunity — helping reduce learner anxiety.
Tips for getting started with qual work
For Quant UXRs hesitant to dive into qual work, if you’ve read this far, I hope you are feeling more ready to take on the new challenge! Here are a few tips to get you started:
Get exposure
Change mindset/attitude
Approach individuals with curiosity and an open mind, recognizing that even small samples can offer valuable insights and unexpected discoveries
View it as diversifying the tools in your toolkit with new ones, not replacing them
Take action
UX managers?
Give your Quant UXRs room for qualitative work! It’s not something they do as a distraction from Quant UX research. Rather, it is a core part of great research, regardless of role.
The evolving research identity
My research identity is no longer about being "quant" or "qual”, because these are tools. It's about being a T-shaped, entrepreneurial-minded UXR who specializes in quant research methods — the vertical stroke of the “T”, the depth of my quant tools – but also is experienced in crafting and executing qual research – the horizontal stroke of the “T”, the breath of my research toolkit, AND is willing to do whatever it takes to get the insight needed to understand complex users and businesses, and explore opportunities at their intersection.
This essay was inspired by the many many long conversations I had with Gabe Trinofi, Altay Sendil, Cassandra Rowe, Chris Chapman, Scott Tong and Omar Seyal, and is dedicated to all the qualitative UXRs whose work inspired me to be a better researcher.
Special shout-out to Altay Sendil for his meticulous edits, and Jeff Miao for his creative illustration!
