Measuring the User Experience
Regular readers of SM probably have seen my rants against “the usability industry.” My opinion is that most so-called usability professionals are so narrowly focussed that their studies are irrelevant from a business perspective.
In the User Experience group on LinkedIn, I saw a reference to a new book titled Measuring the User Experience by Tom Tullis and Bill Albert. Could it be that someone was finally writing about real UX metrics, not just the same tired usability study crap?? Sadly, a glance at the table of contents reveals this is just another usability book with very little focus on UX, despite the title. The authors discuss task-based usability methods: Was a task completed successfully, in how much time, were there errors, did users recover, can they navigate, etc.
All of these task-based academic measures miss the point: users aren’t buying tasks, they’re buying a product that promises a unique value proposition. The buying decision is a complex one that involves attitudes about the brand, perceptions of the product’s strengths and weaknesses, return on investment, total cost of ownership, and the user’s sense of identity with the product. Are there aspects of the product that creates loyalty, passion, and a desire to talk about the experience? What are the social dimensions of the product? How well does the product fit within its intended target market? Does a product integrate easily with other systems likely to exist in the user’s environment? what about the esthetics - do they add or take away from the user’s perception of the product?
Usability is still an important component of the overall UX but the stagnant state of usability testing doesn’t provide answers that actually matter. Maybe I’m missing something…
Popularity: 11%
This is so true! Usability testing is akin to crash testing cars. it’s something that needs to be done, but unless there’s something obviously unsafe about the cars, people are more concerned about the colour and the optional leather trim than how it’ll perform in a crash.
Actually, I think you are missing something. Completing tasks is not an academic measure, it is about the bottom line, whether it is making a purchase or improving effciency to reduce costs.
All the questions you raise are important ones, but most are not about the experience. Experience is about USING a product. For example, how the brand is preceived is important, and may impact their actual experience, but by itself it is not an experience. Same with color or other esthetics. Market researchers have been looking into these questions for decades. People may have different emotional reactions to color. The real question is… how does this manifest itself in interacting with the product and impact future behaviors.
Rather than look at the table of contents, I suggest you actually skim the book, or even pick up a copy. I think you might be surprised at the breadth.
Good suggestion, I should skim the actual book.
You point about market research is interesting. I have more negative experience with standard MR methodology!!
I still have a big issue with focussing on the “USING” of a product… especially without all of the ROI/economic/social/Psycho-demographic context…
big deal. using the product. yes, this Beet-Masher is very usable.
I hate beets!