Eyetracking Studies — 7 Traps to Avoid

In my last post I talked about eyetracking. I don’t actually do a lot of eyetracking work, but this past week I was asked to give a talk on a panel about eyetracking at the SES (Search Engine Strategy) conference in Chicago, so it is kind of on my mind. So one more post about eyetracking, and then I’ll move on to different topics!

At the conference I talked about the 7 traps to avoid if you are contemplating conducting an eyetracking study. In my last post I briefly explain what eyetracking is:

“Eye tracking is a technology that allows you to see and record what a person is looking at, and for how long. One way it is used is to study web sites to see where people are looking on a web page, where they look first, second, etc. It’s a pretty interesting technology, one of the benefits being that you don’t have to rely on what people SAY they are looking at, but can collect the data directly.”

Here’s a brief summary of my talk at the conference:

Trap #1: Underestimating the effect of what you ask people to do on where they look — When you are doing an eyetracking study you are interested in where people are looking on the screen. But the research shows that where they look depends on what you have asked them. See my last post for details and examples on this.

Trap #2: Assuming that where people are looking is what they are paying attention to – At the SES conference I believe Shari Thurow (who spoke on the panel with me) called this the ketchup effect (or maybe it was the milk effect or the refrigerator effect). Have you ever opened the refrigerator to get the ketchup out and you can’t find it? Even though it is right in front of you? People often look at something without paying attention to it. And we have peripheral vision too, so it is also possible for people to be looking at one thing and actually paying attention to something nearby.

Trap #3: Underestimating the effects of your protocols – When you do any kind of usability study (including eyetracking) you use a “protocol”. The protocol describes what you will do and say with the participant, what the steps are that you will have the participant go through during the study etc. In an eyetracking study the protocol you use becomes critical. For instance, if you stop and ask them a question they will likely look away from the screen and mess up the eye tracking data collection. So there are special protocols you need to follow in an eyetracking study.

Trap #4: Underestimating the time needed to analyze the data — There is an enormous amount of data that is collected by the computer automatically when you do an eyetracking study.

So Much Data Collected

So Much Data Collected

Make sure you are ready to deal with all that data and that you don’t get in trouble wtih “ANALYSIS PARALYSIS”

Trap #5: Underestimating the time/cost to do the study — Because there is a lot of thought you have to give to the protocol, and because there is a lot of data to analyze, it is easy to underestimate the time it will take and how much it will cost to do an eyetracking study. If it’s your first time doing one I suggest you multiply your estimates by at least a factor of 2.

Trap #6: Overwhelming people with the data – Not only do you have to worry about becoming overwhelmed with the data, you have to guard against overwhelming others.

Gaze plot data

Gaze plot data

When you go to present the data to your client or your colleagues beware of showing them too much too fast, especially if they are new to eyetracking too.

Trap #7: Neglecting to draw useful, meaningful decisions and actions from the study — Eyetracking is interesting and useful and different and unusual, but at the end of the day, what is most important is that you can draw relevant and accurate conclusions from your study. Don’t get so caught up in the technology that you fail to interpret and apply.

As Alvin Toffler said, “You can use all the quantitative data you can get, but you still have to distrust it and use your own intelligence and judgment.”

——————————————-

Did you find this post interesting? If you did, please consider adding your comment, subscribing to the blog via RSS or email, signing up for the Brain Lady newsletter, and/or sharing the post.

Tagged with: ,
Posted in eye tracking, research, usability
3 comments on “Eyetracking Studies — 7 Traps to Avoid
  1. Jon Dodd says:

    Hi Susan,

    Thanks for pointing out the above (and there are more!) – these kind of basic errors are what risk giving eyetracking a bad name.

    I have read many posts by so called usablity experts who dismiss eyetracking – in most cases it is because they are doing it badly or using it for the wrong things (or in many cases I suspect because they do not have any expertise in it and are behaving like luddites) .

    Like any tool (and that is all that it is a tool) it has to be used correctly and appropriately – otherwise it is indeed problematic and of limited worth. And a poor protocol is a poor protocol whether it has eyetracking included in it or not.

    BTW I would contend that in goal driven behaviour ‘the ketchup effect’ is actually very rare (although may be present in a poor design with many visual distractors and few organisational cues) – people do tend to look at what they are interested in – BUT it is a good reminder that raw gaze data alone can be misleading and should be used along with other data (e.g. recognition and recall exercises, qualitative restrospective protocols etc.).

    Anyway great article – I shall be pointing people to it regularly…

  2. Susan Weinschenk says:

    Thanks for writing in Jon. It would be good to find out how rare the ketchup phenomenon is, wouldn’t it?

3 Pings/Trackbacks for "Eyetracking Studies — 7 Traps to Avoid"
  1. [...] What Makes Them Click » Blog Archive » Eyetracking Studies — 7 Traps to Avoid [...]

  2. [...] Studies — 7 Traps to Avoid | UX Booth UX BoothEyetracking Studies — 7 Traps to AvoidShare This Share Browse UX BoothPreviousNext Close This function resizeIframe() { var height = [...]

  3. [...] usability expert and author Dr. Susan Weinschenk said that people sometimes choose to not fixate on an item because their peripheral vision [...]

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Subscribe to the Newsletter

News & research on human behavior: Subscribe to our FREE newsletter.



Welcome to The Brain Lady Blog

I'm a Ph.D. psychologist and I write and videoblog about how to apply psychology and brain science research to understand how people think, work, and behave. For more information about me and about the Weinschenk Institute, check out the Weinschenk Institute website.

Susan Weinschenk, Ph.D.
The "Brain Lady"

Learn from the Brain Lady’s In-Person Seminars

The Science of Persuasive and Engaging Design and The Science and Art of Creativity -- fast-paced, hands-on, one-day seminars in Chicago.

Connect

  • RSS Feed
  • Twitter
  • YouTube
  • Facebook
  • Google+

Online Video Courses

Dr. Weinschenk has created several online video courses. You can preview the courses at the Weinschenk Institute website.

Archives

Categories

Contact Us

Weinschenk Institute, LLC
625 N. 4th Avenue
Edgar, WI 54426
USA
info@theteamw.com

Susan
Email Susan
@thebrainlady
847.909.5946

Guthrie
Email Guthrie