Peter Drucker wrote 39 books on modern business management. He is widely regarded as the greatest management thinker of all time. He is credited with two of the most important quotes in business management. The first one – If you can’t measure it, you can’t improve it. This quote is of tremendous value.
If you can’t measure the effectiveness of user experience, you can’t manage or grow it. Analyze how users are using your product or service. Compare different user interfaces in A/B tests. Collect the right set of data and make informed decisions based on it.
Peter Drucker’s second quote – Management is doing things right. Leadership is doing the right things. It’s tempting to brainstorm a long list of metrics, but it will quickly get unwieldy and hard to prioritize. Focus on a small set of key metrics that everyone on the team cares about. Choose the right metrics to measure the effectiveness of user experience
UX researchers at Google have developed couple of useful methods to define metrics appropriate to measure the effectiveness and impact of user experience. It’s called the HEART framework and Goals-Signals-Metrics (GSM) process. Let’s look at the framework and process.
How to measure the quality of user experience
The HEART framework measures the quality of user experience. The Goals-Signals-Metrics process measure the goals of your product or service.
The HEART framework
The HEART framework measures user experience in five categories – Happiness, Engagement, Adoption, Retention, Task success.
The HEART framework can be applied at different levels – product level, feature level, etc. For example, you may measure the adoption of new subscription service, but also its specific features.
Happiness
Happiness measures user satisfaction, ease of use, net promoter score, etc. User satisfaction and ease of use can be measured via surveys. Net promoter score is how likely users are spreading a positive message about your product. How does your product’s net promoter score compare with the industry ?
Engagement
Engagement measures the frequency of user involvement. Metrics include number of visits per user per day, number of photos and videos uploaded per user per day, number of product downloads per user per day, etc.
Adoption
Adoption measures number of new users adopting your product. Metrics include number of new user sign ups in last month, etc.
Retention
Retention measures the rate at which existing users return. Metrics include the number of users still active within a given time period. Distinguish new users from returning users by measuring adoption and retention. You can measure how quickly your user base is growing.
Task Success
Task success measures the efficiency and effectiveness. Metrics include time to complete a task, percent of tasks completed, error percentage, etc.
Recommendation
Not all of these may be equally important for your product or service. Choose the most meaningful ones. In an enterprise, the “heart” of HEART has minimal value, if any. Happiness and Task Success (HT) retain obvious value, but Engagement, Adoption, and Retention (EAR) become almost moot as users have no choice but to use the product. You may still measure what percentage are using key features of the product. Bolster the HT with more EAR … or perhaps something else more important to enterprise product users and business owners ?
How to measure the goals of your product
There’s no magical HEART dashboard that can be applied to any product. The most useful metrics are likely to be specific to your product. The Goals-Signals-Metrics (GSM) process measures the goals of your product.
Goals
Identify goals so you choose metrics to measure progress towards those goals –
| Vimeo Videos | Goal |
| Engagement Category | You want users to enjoy the videos they watch and keep discovering more videos |
| Vimeo Search | Goal |
| Task-Success Category | When a user enters a search, you want them to quickly and easily find the videos or channels that are most relevant |
Another question to ponder – are you interested in increasing the engagement of existing users, in attracting new users, both? Different members of your team may have different ideas about the goals of your product. This is an opportunity to build consensus about the direction for your product. You can’t design for a disagreement.
Signals
Map your goals to lower-level signals. There are usually a large number of useful signals for a goal. Research and analyze which ones to choose. How easy or difficult is each signal to log and track ?
| Vimeo Videos | Goal | Signal |
| Engagement Category | You want users to enjoy the videos they watch and keep discovering more videos | Amount of time they spend watching the videos |
| Vimeo Search | Goal | Signal |
| Task-Success Category | When a user enters a search, you want them to quickly and easily find the videos or channels that are most relevant | Entering a query but not clicking on any of the results |
Metrics
| Vimeo Videos | Goal | Signal | Metric |
| Engagement Category | You want users to enjoy the videos they watch and keep discovering more videos | Amount of time they spend watching the videos | Average number of minutes spent watching the videos per user per day |
Recommendation
The GSM process leads to prioritization of the metrics. Track the metrics related to your top goals and that help you make decision. Use the drawing worksheet like the one above to help you work through the steps. Don’t include all the HEART categories if some don’t make sense for your product. Feel free to add more categories of your own, if needed.
What has most value
At Microsoft Engineering India, we are a team of user researchers and designers who create effective and deep research-driven innovative design solutions for Microsoft internal line of business applications in domains like HR, Finance, Legal, etc.
Our goal is to create, inspire, and support compelling and effective experiences through deep research driven user understanding and innovative design, best practices and continuous improvement in quality in use.
We have developed a method to measure the quality of the applications we design by measuring –
- Satisfaction
- Effectiveness
- Ease of use
We do this during design work (to predict success) and after major releases (to benchmark). We want to understand –
- How good are our existing apps ?
- The effectiveness, efficiency and satisfaction
- Measure scenario health
- Cost reduction- fewer priority 1 UX bugs, less rework, etc.
- How well do we do our work ?
- What are the benefits of having a UX team ?
- How does UCD improve the user experience ?
- How do we extend UX across IT ?
- ROI – to grow the responsibility for UX across IT ?
- What training, guidelines, level of understanding are needed ?
Below slide shows the quality in use improving for Microsoft’s internal Careers Web site by adopting user centered design methodology. Read more about How to improve usability by leveraging design best practices.
How do you measure the effectiveness of user experience and goals for your product ?
What metrics do you track ?
Do share in the comments!
Like this article ? How about giving it a like and share ? Thank you!
Digital Garage 