Quantitative UX is a mixture of many domains, including data science, human factors, social psychology, cognitive science, human-computer interaction, computational social science, and computer science.
Quantitative user experience researchers take their expertise with data, combine it with domain knowledge from behavioral science, and solve problems related to how users think about and interact with products. These problems often include questions such as:
- What product features might users prefer?
- How can an interface best guide a user toward completing a certain task?
- How might we measure a certain type of interaction with a product?
- What metrics matter most for evaluating overall product success?
- How can we generate measurement frameworks for abstract user phenomena, such as “trust” and “confidence” with a product?
- Which is the best way to experimentally test new feature ideas?
- How can we identify and meaningfully interpret natural experiments that arise with a product?
- How might we predict future disengagement with a product or feature?
- How can a heterogenous population of users best be construed as a discrete set of common user-types (i.e., segments, clusters, or “personas”)?
Skills needed to be a Quantitative UX Researcher:
Though it varies, the typical background for a quantitative UX researcher at a technology company includes:
- Domain expertise in a field of behavioral science.
- Fluency in R/Python and SQL.
- Experience with data engineering – developing automated data workflows for database development and management.
- Advanced statistical knowledge, in particular familiarity with:
- Nested data structures
- Non-parametric modeling
- Over-dispersed/highly-skewed data
- Basic machine learning and classification modeling
- Field experiments, quasi-experiments, and deriving causal inference from observational data
- Experience with mixed-methods research.
- Strong data visualization skills.
- An ability to communicate abstract or technical data phenomena to non-technical audiences or stakeholders.
- Familiarity with geospatial data.
However, these are just the “resume requirements”. Beyond formal qualifications and technical skills, to be a successful quantitative UX researcher requires:
- Sound data intuition. You need to have a firm grasp of how data is measured, processed, and ingested into any database before you start running fancy statistics over it. No amount of sophisticated data science can overcome problems with a dataset that is fundamentally flawed due to “upstream” issues with the data. How we interpret and communicate findings must take into consideration the full end-to-end nature of the data.
- Methodological flexibility. UX research problems often call for one to pull from a variety of methodologies… survey development, psychometrics, machine learning, inferential modeling, and so on. Being flexible across a number of methodological domains will ensure you don’t fall into the trap of seeing every problem as a nail in need of a hammer.
- User-centered thinking. How you interpret behavioral data must account for how users construe the same behavior. An observation of a certain type of behavior coming from one type of user may mean something totally different than the same behavior observed in another type of user. At Uber, I often observed driving styles and behaviors occurring in some countries due to contextual factors unique to those markets (e.g., road and infrastructure conditions); Blindly comparing the same behavioral data (e.g., the frequency of sudden stops) across different markets without this user-level understanding of how certain behavior was actually manifesting would lead to profoundly flawed analyses.