Testing User Comprehension

[00:00:00]
>> So let's talk about the first test that I like to do, which is testing comprehension, right? So when you're testing comprehension, you're essentially attempting to understand what the users understand what the page is about and what action they can take from it, right? So it's more on the usability side of things.

[00:00:18]
We're not worried about whether people like it, we're just worried about whether they understand it and can use it. One of the tests that I've really liked is called an eight-second test, when it's not, it's called a five-second test, right? Which really annoys me because I never do it for five seconds, always do it for eight seconds, and I'll explain why.

[00:00:39]
So users spend on average about eight seconds assessing a page when they hit it. This is based on some possibly dubious research that was done previously, but it's the best that we've got by time, which has been published in various places. Who knows how accurate it is and how represented it is overall, but it seems to be a reasonable number.

[00:01:07]
You could look at your own analytics if you really wanted to get an idea of it, but yeah, about eight seconds assessing a page before they decide if it's helpful or not. If they don't think it's gonna be helpful, they're gonna abandon it, right? So one of the things that we want to ensure is that they consider this page helpful, right?

[00:01:27]
And they understand what's going on. So with this test, which we'll call the eight-second test, we first of all need to understand who we need to do this test with, right? Obviously, with all testing, in an ideal world, you wanna test with your average user. However, it doesn't necessarily need to be that way.

[00:01:51]
You can get away with testing with people that aren't as familiar with it. And I said this before, and I just wanna repeat it here that there is a lot of good reasons for making something usable and comprehensible by everybody. They might be new to the sector. English might not be their first language.

[00:02:08]
They might have a cognitive disability, etc. So long and short of it is you can do this first eight-second test with anybody you want that's outside of the organization and that is approximately equally able as your target audience, right? I regularly use friends and family to the point now where there are certain family members I can't use anymore because they've become so competent at doing these tests that they're not representative of the average user anymore, problematic.

[00:02:40]
So yeah, it's officially called the five-second test. I show it for eight because that's the average time somebody says on the site. So essentially, a five-second test involves showing the user your design concept for eight seconds, after which you ask them to recall specific details or their first impression or both, right?

[00:03:02]
It's totally up to you what you wanna ask. And all of this is about assessing the effectiveness of the visual communication. So typically, what I will do is I normally ask them two questions. The first question was, what was that page about, right? And the second question is, what do you remember from that page?

[00:03:24]
What elements do you remember seeing from the page, all right? That's basically it. And it's really easy to set up, right? You just go into the site and you select five-second test. You give it a name. You upload your image. You write your questions. You press publish, and that's it.

[00:03:43]
It gives you a URL, and that URL you can give out to friends and family or whoever else, right? And then what they will see is this, look at the interface for eight seconds and remember as much as you can. That's what I've asked them to do in this particular case, right?

[00:03:58]
And when they hit that button, the image then appears for eight seconds. And then at the end of the eight seconds, it disappears and they're given a box to fill in what they remembered, right? Such a quick and easy test, okay? You can see how you can get results in a few minutes from a test like that, takes few minutes to set up, few minutes to run.

[00:04:20]
And what you'll get back is that, basically a word cloud saying all of the different words that people used and different phrases. So in this case, I was pretty happy with this because it was a site about doing direct mail, right? Something, that word worried me slightly. [LAUGH] So I did have to go in and check the individual results to find out why people, they were saying things like something about direct mail.

[00:04:48]
So that's fair enough. I didn't need to worry about that too much. So yeah, and you can get a very quick idea very quickly about whether people comprehend what they're looking at, brilliant, wonderful, super. I mean, an alternative approach if you've got a little bit more time is you could do some user interviews and do it that way because user interviews are always very enlightening.

[00:05:11]
I mean, they're more in-depth for a start. You can ask follow-up questions. And you could also see people's non-visual cues cuz sometimes you get great reactions. You showed the design and they go like that. [LAUGH] And then you know, or they lean clubs, which is maybe an indication, it's not as obvious as it should be.

[00:05:29]
And there are things like that which are quite nice. But let's be honest, it's a long lead time to set up something like that. It takes time to actually run it. And it's often smaller numbers you're looking at. So do what you feel is appropriate in those situations.

[00:05:47]
Yeah, user interviews are great, but they do come with their drawback. But I tell you, that five-second test is a corker for giving you solid advice that says, no, people did see the logo, we don't need to make it bigger, right? Or, yes, people did understand what the site is about.