In our Influencing Strategy by Design course, Tom Chi and I spent a fair amount of time discussing the impact of metrics on product design and strategic influence.
Metrics are used by the organizations that track them to decide what initiatives to pursue, to understand the impact of behavioral trends on decisions, and to interpret the impact of past decisions. Though different teams measure different metrics, most metrics that impact design fall under one of three categories:
- Quantitative metrics (actual numerical values that are usually tracked over time) such as market share, audience growth, or engagement
- Qualitative metrics (subjective data gathered directly from users) such as customer satisfaction, focus group feedback, or field studies
- Guerilla metrics (quick calculations used to set direction) such as smart estimates, samplings, or novel measurements
These metrics can affect product design and strategy by helping identify: key audience segments, structural or social norms, the boundaries of a problem space, entry & exit points, an appropriate feature mix, distribution requirements, product positioning, and lots more.
While business-driven metrics are often top of mind for companies, consumer focused metrics are frequently less utilized. In fact, most companies have clearly defined metrics optimized to track the health of their business. But when these metrics are used as the sole measure for product success, they may create an oppressive environment for product design. Consider a Web company that measures its business success by the number of pages views it maintains per month. Optimizing a design for this metric could lead to an over-indulgence in pagination to keep the numbers high amounting to what is probably not a great experience for the company’s customers.
To account for metrics that don’t explicitly focus on user experience and thereby have the potential to negatively impact product design, ask: how can these metrics be adjusted to take user experience into account, what other metrics might help us make better decisions about design, and can we access or estimate those metrics?
Metrics can be gathered technically (databases, logs, statistical sampling), though business channels (analysts, models, external vendors), and directly from customers (usability testing, market research). Good metrics are:
- Actionable - What would I do if the metric is out of line? Do I have the levers that can impact it?
- Accessible, creditable data - The data can be acquired with modest effort from a source that people trust.
- Common interpretation – People in the organization recognize what the metric means.
- Transparent, simple calculation – How the metric is generated is shared and easy to understand.
Taking the initiative to measure and track explicit customer experience metrics provides designers with a significant amount of leverage during strategic planning and product design. Being able to support opinions with metrics that reflect an “outside in” perspective (how customers see your products and organization) goes a long way toward influencing decisions.