Don’t use one way mirrors for ux research

Do you use one way mirror labs? Do you value research that gets you the best results? Then you might want to re-consider using one way mirrors. Here’s why…

Talking to users is fascinating! It’s something I still love doing despite having conducted thousands of them over the last 10 years. When it comes to location, you can test almost anywhere but there’s one place that I now advise against, and that’s one way mirror labs.

What is a one way mirror lab?

A one way mirror lab (also known as two way) consists of two adjoining rooms with a mirror between them. One room is used to interview people and the mirror functions as a normal mirror from this side. On the other side of the mirror is the observation room where people watch the research taking place, from this side the mirror behaves as a window, enabling the observers to secretly observe what’s happening in the research room.

The negative consequences for research

I’ve used this setup many times and I’ve sat on both sides of the mirror. These are the problems:

Nervous users

As a researcher you should always tell the participant that there are observers behind the mirror. However, if I say to you now, don’t think of a pink elephant, the first thing you think about is a pink elephant. In psychology we call this the Ironic Process Theory or the White Bear Principle and it refers to the human tendency to continue to think about something after being told not to think about it.

So we’re ethically bound to tell people there are people behind the mirror but by doing that their attention is being drawn to it. Many users are fine with this and they’ll forget about the mirror. There are other users who will interview ok but afterwards they will mention how they felt like they were being watched and finally, some people simply do not interview well. They may appear nervous, glance at the mirror throughout, whisper some answers to you because they don’t want the people behind the mirror to hear any negative feedback, etc. And the mirror is a difficult thing for people to get over once they have a problem with it, because it’s such a huge object in the room and therefore a constant reminder.

Ask yourself, wouldn’t you feel uncomfortable knowing there were people watching you behind a mirror in the room?

Positively biased responses

If you knew there were a group of people watching you behind a mirror wouldn’t you be more inclined to give positive responses and to withhold negative opinions? But it’s really important that we understand negative feedback in order to make things better.

Sound leakage

Your observers need to be relatively quiet. I’ve seen labs provide headphones so that observers can turn audio volume up without sound leaking into the testing suite.

When two rooms are next to each other, it’s impossible to soundproof them completely. If the observers next door get quite loud, the sound can leak into the adjoining room. This can be fatal if they laugh and the user hears this. In some labs, the doors don’t close quietly either – this is then another reminder to the participant that there are people watching them.

Noisy cameras

One way mirror labs almost always have cameras that can be controlled in the observation room. These aren’t always silent though. You might be in the middle of a really interesting insight when suddenly you hear the buzz of the camera. Off-putting to say the least and yet another reminder to the user that they are being watched.

Dark, uninspiring observation room where no one speaks

Observation rooms in labs are awful places really. There are no windows and therefore no natural light, the lights have to be turned off (otherwise you can see straight through the mirror) so it’s a dark, dull, uninspiring room to be sat in all day. In one way mirror labs sometimes the observers can be much quieter than in labs without a mirror, because you can see how close the participant is to you.

The problem is these are great setups for observing research, especially focus groups, not UX research. If you have a team of designers observing research, the one thing they’re guaranteed to want to do is sketch, but how do they do that well when they’re sat in a dark room? It’s not an environment that encourages team collaboration, makes a team feel energised, inspired and creative. Conversation and teamworking should be encouraged – now’s the perfect time for the team to collaborate and get to work on designs.

Ironically, no one really observes what’s happening through the mirror!

We spend most of our time watching the TV screens, which give us consistent detail, clarity and control. The glass, for all its glamour, doesn’t always fulfil its worth.

In UX research, the most important interaction to focus on is that between the user and what’s being tested, and in this regard you can’t see anything through the mirror, the detail is through the cameras pointing at what the user is doing. Therefore, the majority of the time, observers are focussed on the tv screen – where the action is. Compare it to UX design…if you want the users attention to focus on something you might give it a more central position, make it bigger, put everything else around it. So when the UI is the most important thing for people to observe, why do labs show this on a small tv screen and give the highest visual prominence in the room to the mirror? It’s crazy!

The solution

The alternative, better solution is to use two rooms that have all the same technology to record and observe the user and their interaction but in the observation room, there are TV screens and no mirror. GDS (Government Digital Services) also use this setup which you can see here. Without a mirror, you’ll get better insights from your more relaxed users and the observation room can now be a creative haven. You can turn up the lights, have natural daylight (windows), have dynamic team discussions and work together on sketches and ideas.

It suddenly becomes an exciting and inspiring workshop to turn user feedback into better designs! And this, is the whole purpose of user research.

A note from me:

This is a post I recently published on Linkedin Pulse. If you like it, you can follow my future posts or connect with me Linkedin


Never use the word ‘Test’

When I first started life as a user researcher, it was commonplace (and it it still is) to refer to research as user testing or usability testing. I soon observed that when you use the word ‘test’ it:

a) Implies that you’re testing the end user (which is wrong, you’re testing the interface, you’re understanding of the customer, your user journeys, etc).

b) As soon as you mention the word ‘test’ to a participant they instantly tense up and worry. I used to say something along the lines of ‘please don’t be concerned, we’re not testing you, we’re testing the software’ and even this was too much. It’s a bit like if I say to you, don’t think of a pink elephant, the first thing you think of is a pink elephant – you just can’t help it, it’s how the human brain works.

I also noticed that when I used the word ‘test’ sometimes participants would ask me during the research session how they were doing or ask whether they’d got something right. In effect, they were treating it like a test. I haven’t experienced this since I stopped all use of the word ‘test’.

Now, when speaking with participants I always use the word ‘research’ which has a much more positive connotation. Of course, clients still use terms like user testing, and that’s absolutely fine, let’s not undo all the hard work ux professionals have done over the years to gain awareness of what we do, but let’s keep in mind that we’re always researching and aiming to understand things from the perspective of your target audience.

Have fun researching!:)

UX Booth guest post: 5 Useful Lies to Tell User Research Participants

user research liesDo you read UX Booth? If you’re interested in User Experience then bookmark it now! It’s been my favourite site for reading interesting and useful UX articles for a number of years now. As such, when I decided to write a guest post, they were my first choice. I decided to write about 5 little (white) lies that can be told during user research interviews to gain higher validity data.

I’ve conducted hundreds of research interviews and I’ve picked up a few useful techniques along the way to encourage the best out of the people I interview. This includes making them feel more at ease, increasing rappor, gaining trust and encouraging an open dialogue where it is ok to be 100% honest.

Active and passive deception has been used in research for a long time. In the past it was unfortunately used unethically and there are a lot of examples out there of how not to use deception. The Milgram experiment is one of the most known for the psychological and physical damage it caused.

Of course, all the lies I use and recommend are incredibly nice. They’re white lies and many UX researchers use some or even all of them. You don’t have to use any but they are a useful tool to have in your UX toolbox. Enjoy!:)

Read the article: 5 Useful Lies to Tell User Research Participants


Eye tracking – Should your website use it?

This week I’ve been visiting local usability labs that have eye tracking capabilities as we will soon be offering this added value service to our clients.

What is eye tracking?

Eye tracking uses non-intrusive technology to track where the user’s eyes are looking. It is often used for usability testing websites, software, mobiles, adverts, even shopping malls (there’s a lot of psychology that goes into where to place that can of Heinz!).

Testing involves the user using a website to perform tasks and is often accompanied by a satisfaction questionnaire. Afterwards, the users gaze path and fixations (basically what they looked at and for how long) can be watched and analysed. All the users data can also be combined into a heatmap so you can see which areas of each page attracted the most attention from users – did they even look at the big advert in the middle of your homepage?

Is it worth the extra time, effort and cost?

There’s a lot of disagreement amongst professionals as to whether eye tracking is worth the extra time and effort required to analyse the data and whether it really brings more value to usability testing. From speaking to those professionals that do use it, it is clear that businesses and in particular marketing departments are more persuaded and engaged by this testing method. Perhaps this is because marketing departments are more concerned about users noticing their promotions and advertising than finding out the hows and whys of users that visit their site.

I’m of the opinion that some websites are better suited to standard usability testing methods, some are better suited to eye tracking and others may benefit most from a mixture of both.

Video showing user’s scanpath
(the bigger the circle, the longer the user is looking at that point)

Video showing the use of heatmaps

Eye tracking is most useful when you want to test:

  • Page elements – To assess how much people notice ui elements onscreen, such as logos, promotions, calls-to-action, etc.
  • Navigation – To identify any conflicting terms as well as how different navigation layouts interact with each other.
  • Page layout – To show how a page layout and colour scheme affect the way users scan a page.

Eye tracking is least useful if you want to:

  • Gather user feedback – In order to generate an accurate heat map users can’t be asked too many questions as they’ll often look  away from the screen when answering questions.
  • Know why – Eye tracking will tell you what people look at and what they don’t look at but it won’t tell you why.
  • Test on a budget – Eye tracking is more expensive as it involves the hiring of technology and extra time to analyse the results. Stick with regular methods if you’re budget is limited.

I hope this basic introduction to eye tracking has been useful. If you are interested in seeing how real users actually use your website (with or without eye tracking) then give me a shout. You can observe the user testing in person (in the observation room) or we can stream the sessions live from the testing lab directly to your computer.