Week 11: Usability Testing Ends!

Hurray! I have survived. I finished up my usability testing this week with one current OT customer and with a former OT employee who used to use an older version of the system. Unfortunately, I never heard back from the other customers I reached out to, but I think still getting someone who had used the system in the past was just a good. These two tests were very different in feedback and flow. The currently customer is a developer, not so much an end-user of the GUI, so she had a lot to say about other aspects of the product beyond just what I was testing. I had to be diligent about capturing her other feedback for the team that extended beyond being able to choose objects in the system. She lamented all the clicking, even though she was able to perform each task, and really wanted to be able to drag stuff around, like dragging a channel over to the "My Selections" area instead of clicking checkboxes. I saved an excerpt of the recording where she was performing task six and complains about the clicking saying I was torturing her by making her do that (all in good fun, though) that I will use in my presentation.

The former employee did not have much to say in comparison. She mostly said things seemed simple enough. What I found interesting, though is she had a real problem with task eight where I ask the users to try and navigate just using the folder grid with the tree hidden. She got very frustrated that she could not see where to go or what to do and it was SO HARD for me to sit there and watch her struggle. I was worried she was going to get mad at me for not stepping in. In reality, it only lasted about two minutes but it felt much longer. She did eventually discover the "up" button and was able to complete the task, though I marked this as a failure simply because she was ready to give up long before and I think would have in a normal use situation—at least she would have used another method. In fact she tried going back and looking at the tree and it was on that screen where she first noticed the "up" button for folder navigation but the button was not active for that screen. I was most surprised that she struggled with this task since in the version of the system she used, that is how navigation worked. There was no folder tree to show hierarchy, but there was a breadcrumb. It has been interesting for me just how much having a semi-working prototype influences the testing. I'd be interesting in running a test with just printed out pieces of paper having the users explain to me what they would like to have happen.

I think overall that the testing portion of my project went very well. It was a new experience to do everything myself: coming up with the design, writing the test plan, administering the tests and acting as both facilitator and note taker at the same time. The spreadsheet really helps and allowed me to capture time on task, success or failure of tasks, difficulty ratings and general comments very easily. There are only three weeks left in my project and I will use the remaining time to write up a "formal" report of my findings, basically translating all the boring data into a story with my designs and recommendations. If this small sample tells me anything it is that I was able to come up with some real improvements over what is in the current system. I'm ready for all the questions I'm sure to get from the business stakeholders about the results.

I want to leave you, the reader, with something fun or interesting each time so here's a short clip of a friend of mine climbing a lamppost to replace a geocache called Climb #6. This cache has a terrain rating of 4.5/5. I have only personally done a 4 so far.