Working with the Results

[00:00:00]
>> Right, once you've done your testing, however you've done it, working with the results is really important, it's invaluable but only if you actually use the results, right? Again, there's no point in doing testing for the sake of it unless you can do something. First thing I do is prioritize the issues that come up, the things that I've observed.

[00:00:20]
I prioritize them in three ways. First of all, I prioritize them on the user impact. I start by categorizing issues based on their severity with a focus really on the showstoppers, the things that, okay, I'm done. If this was a normal situation, I would've given up now. Those that you really wanna pay attention into those, obviously.

[00:00:41]
The second thing you need to pay attention to is the business impact. So some of those issues will have a bigger impact on the profitability of the business. They might not be showstoppers but if someone is really struggling to check out, that's fairly important. And then finally, effort, cuz there'll be some things that okay, they're not as big a deal but they're so quick to fix we might as well kind of attitude.

[00:01:08]
So you're balancing those three different criteria when trying to prioritise everything out in terms of how you're gonna adopt them. The other often overlooked benefit of usability testing is the huge impact that it can have on culture and your stakeholders and how they think. If you want people to care more about usability and care more about the user experience, if you can get them to watch a usability test session, it will transform them.

[00:01:37]
It is incredible. How many of you have actually sat in a usability test session. See now it's quite interesting, right? And that's no criticism by the way, most people haven't, right? It changes you. That sounds so metaphysical. Yes, it's a life changing experience. But it really does watching somebody struggle with the thing that you have made is the most demoralizing, heart wrenching, awful experience but it valves you, you vow, that you will do better.

[00:02:16]
Because you really see the pain in the frustration of people. So it's a really good thing to do, even if you just sit in on one day. So we wanna pass this on to other people, right? So first of all, encourage attendance, right? Get other people involved, if you're gonna run a session don't do it behind closed doors.

[00:02:36]
Well, obviously you don't want a room full of people looking down and watching the attendee but if you couldn't stream it online or something like that. In person it's great to even if they're sitting in another room and you're just streaming it through to the other room, really good if they can all sit around and talk about it, people get quite animated and it's why did she do that?

[00:02:56]
And what did she mean by that and they're all bouncing it It creates a great atmosphere. So if you can get people to turn up, that is amazing. You can also stream it more widely to people in the office and people can just tune in. Some organizations I know have a big screen like this one here, and they just have it on in the background, right?

[00:03:15]
Really low volume and it just plays in the background and it's amazing how much people stop and watch it. And then you can also discuss it with their attendees, right? Then you get them discussing the results, what came out, what they think we should do about it. All that kind of engagement works really well if you can get people involved, but people are often reluctant to get involved.

[00:03:37]
So a kind of initial step, yeah go for it.
>> And what people would you want to be there? Or even are there some groups where you wouldn't care to have them there? Are you saying that any sort of developer, any sort of administrative assistant, a stakeholder
>> Everybody.

[00:03:59]
>> Anybody.
>> There is nobody I would not want to watch at least one user session. There are some people that it's worth their time to go more regularly and by the way I would include developers in that. Because I think designers can be very snobby about it's our job to do design, it's developer's job to do development and they've got nothing to add to it and I call BS on that.

[00:04:31]
I think developers are incredibly, they probably in fact, I've got an article on my site that argues that developers impact the user experience more than designers do. In things like performance, in things like error messaging, in things like security, logging. All the complicated interface elements of many applications are actually designed by developers and not touched by designers.

[00:04:53]
So I think developers play a crucial role and should be involved. And most senior stakeholders I think should at least see one or two sessions it's good. Stakeholders if they're making decisions about the usability and functionality they should be involved. Anybody really anybody that's interested or any if they're not interested forced them.

[00:05:15]
But the other very valuable tool if you can get people involved in they are often reluctant because everybody's busy. They've all got loads of meetings, blah blah blah. Maybe pizza isn't their thing and bribing them with pizza didn't do it. Recordings, recordings are really worthwhile and I touched on this before, but I do just wanna touch on it very briefly again.

[00:05:34]
Highs and lows, low light videos, good things that happened, all of that kind of it show edited videos of what went well, what went badly distributing that around really valuable people watch that. Then they're more likely to come to the next user session that you've seen because it inspires them and excites them.

[00:05:54]
Also, I just crowbar clips into every presentation I ever give. We all have a little clip to show just to force people to pay attention to this stuff. So the prototype stage is an amazing stage for doing kind of more traditional usability testing. Yeah, it does take a little bit more effort, especially if you're gonna do in person or you're gonna do facilitated.

[00:06:19]
But it's absolutely worth it and do it at that time before you stop building stuff. Once you start building stuff, the cost of making changes shoots up, and nobody's willing to act upon what they discover in usability testing. But if you do at the prototype stage, the costs are relatively low for making changes at that stage.

[00:06:38]
So just to summarize, prototyping would probably be the best opportunity to do your testing that you can have. Remote unfacilitated testing is often the fastest option facilitated in person does provide significantly more insights if you've got the time to do it. So that's basically my attitude. And yeah, but most of all, don't bother doing any of that if you're not gonna make good use of the results, either through improving the service or engaging with stakeholders, ideally both.

[00:07:11]
If you can do both of those things through it, then you double the value of doing the usability testing, basically
>> I'm curious about the dynamic of observers.
>> Yes.
>> Now I can imagine that somebody who is acting as the person doing the testing.
>> Yeah.
>> That they might be unnerved with people in the room watching, and if you've got like ten people.

[00:07:38]
>> No, you mustn't do that.
>> Okay.
>> Right.
>> Great, could you clarify then when you're saying, okay, I wanna encourage observers.
>> Yeah, sorry I got in a mess when I was saying that. So let me repeat what I was getting at much clearer. So in the room, if you're doing inperson usability testing, there's the participant who's in front of the computer.

[00:07:59]
There is a facilitator that will sit next to them and be their guide if you're doing facilitated. Then everything on their screen, okay, and the audio is being pumped through to another room via Lookback or Zoom or whatever you choose to use into an extra room where you can have as many people sitting and chatting as you want, right?

[00:08:24]
So that's the kind of replacement for the old fashioned single way mirror way of doing it where. So no the most I ever have sometimes I will have a third person in the room, which is a note taker. But even that these days I very rarely do because it's been recorded anyway and I can watch it back afterwards.

[00:08:47]
But there are occasions where I know if it's just say me doing some testing, and I know that I'm not gonna be editing the video down it's really just for my own learning. Then sometimes I prefer for notes to be taken as I go along because otherwise I have to watch all those videos back, right?

[00:09:07]
So that doubles the amount of time I'm spending doing it. So in those occasions, it's worth having someone taking notes. It's very hard to facilitate and take notes at the same time. I find anyway, perhaps that's me, all right? Yeah, go for it.
>> I have a more general question.

[00:09:25]
How do you manage expectations when you're refactoring or remaking an existing product to remove the initial shock and bias of things being different?
>> They're right [LAUGH] that's quite a complicated question is the honest answer. So there are two sides to it. There is the shock to internal stakeholders, right?

[00:09:47]
And then there's the shock to the end users. I'm perfuming you're talking about the end users there.
>> You said that this bias can come from stakeholders or
>> Yes, she did see I said stakeholders, right? With stakeholders it's easier in a way, because you drip feed them all the way along one of the big.

[00:10:06]
And this goes back to this ta-da moment thing that's kind of been traditional in doing design, where the designer goes away and does magic in a room, right? And then they come back and go, ta-da! Look at my amazing design. And everybody goes that wasn't what I was expecting.

[00:10:21]
But if you have been showing them wireframes, if you've been showing the mood boards, if you've been showing them design comps, if you've been showing them prototypes as they're being developed, then they're being drip fed all the time and they're seeing the design evolve. And that is far less of a shock to their system as they see it coming about, especially if you're building alongside that a communications about why you're doing what you're doing, etc.

[00:10:52]
So one of the things I recommend with clients that are doing big redesigns. Again, a universities is a great example where there's basically hundreds and hundreds of people across the organization that are all go completely lose their minds when you launch this new site and they go it's because you're wondering.

[00:11:17]
So what I do with organizations like that is actually, I send an email to everybody in the organization and say. If you care about the website, sign up for our mailing list and we'll keep you informed as things go. And I run in parallel to the development, an entire comms campaign, basically saying, this is what we're doing, this is why we're doing it.

[00:11:36]
Here's where we're at now and I keep I'm informed so when it goes live, there's no shock. Now, when it comes to end users, it's a little bit more complicated because in that situation there will be a shock moment. When you first launch a new website, oftentimes you break people's procedural knowledge.

[00:11:57]
So we've learned to do something in a particular way, right? And even if that has been improved upon, the fact that it's broken the way that we know, means that we get upset. So there are several things you can do. First of all, is not panic, right? Normally I say, ignore anything feedback for the first two weeks, right?

[00:12:21]
Every time Facebook redesigns, everybody goes epileptic for ten minutes. And there are Facebook groups created complaining about Facebook and every just goes nuts. But you give it two weeks and everybody's over it because two weeks is about enough time feel procedural knowledge to adapt to the new interface.

[00:12:41]
And at that point, you stop caring, right? So that's one thing but ideally, you wanna soften the blow to begin with. So there are a couple of ways of doing that. But one of my preferences is to actually open up a beta. So sometimes you'll go to a website and it says, hey, check out the new version of our website.

[00:12:59]
And you can click on that and off you go to the new version but there's always a button that says I wanna go back to the old version and you can go roll back. So techniques like that can help. But a lot of it comes down to stakeholder management as you rightly said and preparing stakeholders for users flipping out when you first change things.

[00:13:22]
And I always tell stakeholders look when this goes live every one of go nuts for two weeks and then calm down. And just knowing it's coming gives stakeholders a little bit more confidence not to panic.