User experience research and such

Last week at Nokia, I presented some findings on the user experience research we did through mobilemuse.ca using N80 mobile devices and the Metrocode application for the Vancouver Sculpture Biennale cell phone tour. The audience was very N-Gage-ing (sorry, couldn't resist), were a great bunch, and asked a number of really good and interesting questions.
Some that I'd like to share here were oldies but goodies that qualitative researchers like myself often encounter in quantitative-oriented audiences on a regular basis, like:

1. How is this research representative, given such a small sample size (like, say n=30 versus 1000)? Short answer: Qualitative research often doesn't claim to be representative. Rather, the objectives of this type of research are to spur inquiry, inspire, and give examples of 'exceptions to the rules.' Example: wheelchair ramps, though made for a 'select' group, are ultimately good for everybody so designing for that is a good thing. Besides, if random large samples and their quantitative analyses were the be all and end all, wouldn't we know everything we need to know by now? Ultimately, in this type of research, randomness, generalizability, and largeness of sample size would not have helped.
   
2. Isn't this research biased? Short answer: glad you asked. The cheeky answer would be to say that all research is inherently biased and qualitative researchers just acknowledge those biases more...? Um... the not so cheeky answer is yes, research absolutely needs to be evaluated and considered amidst its funders and standpoint epistemologies of all participants, especially the researchers. In this research, I personally know that I designed the project with certain 'controls'. For example, instead of using just Nokia products, we had some focus groups that used their own mobile phones.
   
3. Vancouver is a specific test-case, as are your general participant demographic of twenty-somethings. Isn't this too specific? Short answer: This is a similar concern to #2, and #1. The plain truth of the matter is that we're not after generalized data in this type of research. The types of insights generated by focusing on very specific cases is to be able to better observe and articulate contingencies in product/service design plans that might not have been accounted for (which happens all too often). Why is Vancouver different from Seoul? SHOULD we design technologies geared at a generalized average or disregard one 'fringe' group altogether? Probably not.

I think that once these differences in types of research are generally understood for their respective strengths, they are used in amazing ways to improve things. Grant McCracken writes about just such strategies used by Nokia here. I'm optimistic.
My blog on the Biennale's latest appearance in the news, including links to related discussions on the research by team members is here.