Video Prototyping: The MatchMaker case study
What if you could evaluate a system with users without getting into costly and lengthy field testing? In this case study we tried to compare user evaluation based on video prototypes of a system versus actual field testing of the system. Our study concerned the use of video prototyping as an early design technique for surveying user reactions and attitudes to a design concept. The question that arises is how reliable is the feedback that might be obtained: viewers of such videos evaluate a future they have not experienced yet, for which they are asked to address issues such as: fit to daily life, emerging experiences, appropriation, etc.
More specifically, we took the case of the MatchMaker, an ambient intelligence application that notifies people in different locations that a connected other is engaging in a similar activity at that very moment. The aim is to increase connectedness between senior parents and their children living remotely. This system was built through an iterative design process and was tested in a home-simulation environment in a prior study.
We created a video prototype for the MatchMaker, one that would show clearly its use, how it works and how the users interact with it. We then planned a user evaluation based on that video prototype, with interviews and questionnaires. The qualitative data from the interviews were gathered in an affinity diagram and analysed further. The results that came out from this video prototype study did not differ from the original MatchMaker (field) user studies in their majority, thus showing good evidence that video prototyping can be a good method for user testing especially in the early stages of the concept development and bring very valuable feedback.
This project in keywords:
brainstorming, storyboarding, video prototyping, user evaluation planning, user evaluation, questionnaires, interviews, affinity diagram, qualitative data analysis.
What I did:
This was a 2-week group project during which I had different roles. I was involved in the initial brainstorming to come up with ideas for the structure of the video prototype, created storyboards and took part in planning and filming the video prototype. I analysed the interviews with the participants in open code to identify different categories of responses. Based on these I was in charge of building an affinity diagram to identify patterns and categorize them according to themes, in order to get our final, concrete results. I also, co-wrote a paper for this study (“Comparing user feedback from video prototypes to field deployments of ubicomp systems”)which will be presented at the International Conference on Human-Computer Interaction in July 2013.
Stan Ackerman’s Institute – User System Interaction, Eindhoven, The Netherlands in collaboration with Philips Research.