Evaluation of Google Maps for Mobile

Evaluation of Google Maps for Mobile

Location-based services on mobile devices are a new and fascinating form of computing. My team wanted a better understanding of some of the unique challenges in this area, including developing new ways of testing.

The Facts

What: A complete system evaluation of Google Maps for Mobile on the BlackBerry
When: Spring 2008
Who: Myself along with Gaurav Anand, Jackie Cerretani, and Beilei Zhang
Where: SI622 – Evaluation of Systems and Services with Mark Newman

The Story

The goal of this project was to build and explore our toolkits as usability professionals, concentrating particularly on both formative and summative evaluation. Our team was interested in how to apply our design strategies to mobile devices, prompting us to choose Google Maps for Mobile (GMM) as our application for evaluation. We began by creating a complete interface map of the application in order to understand the domain. We then conducted user interviews and created a set of personas and scenarios for focusing our further evaluation. By performing a competitive analysis, we were able to see how GMM differentiated its self from similar applications via specific design choices. We administered a survey to gain a sense of user satisfaction with these design choices and where our personas were located within a larger sample population. We also performed a heuristic evaluation, user tests, and a visual analysis in order to identify usability problems are areas where the interface could be refined.

In conducting these evaluations, we faced many challenges. Many of these challenges were centered around our choice to test a location-based mobile service and our desire to do so in a way representative of actual use. In particular, performing user testing on GMM required us to think creatively and led us to constructing a mobile usability lab where we could observe participants using the application around town. Such challenges forced us to critically examine which information was important from an evaluative standpoint and how we could best capture that data.

My Role

Our team decided use a divide and conquer strategy when approaching the various phase of our evaluation, with each member appointed to lead selected phases. In particular, I was most involved with user testing, user interviews, and persona/scenario creation. Through this project, we all gained a comprehensive understanding of the strength and weaknesses of these usability tools and techniques.

Artifacts