{"rowid": 94, "title": "Using Questionnaires for Design Research", "contents": "How do you ask the right questions? \n\nIn this article, I share a bunch of tips and practical advice on how to write and use your own surveys for design research.\n\nI\u2019m an audience researcher \u2013 I\u2019m not a designer or developer. I\u2019ve spent much of the last thirteen years working with audience data both in creative agencies and on the client-side. I\u2019m also a member of the Market Research Society. I run user surveys and undertake user research for our clients at the design studio I run with my husband \u2013 Mark Boulton Design.\n\nSo let\u2019s get started!\n\nWho are you designing for?\n\nGood web designers and developers appreciate the importance of understanding the audience they are designing or building a website or app for. I\u2019m assuming that because you are reading a quality publication like 24 ways that you fall into this category, and so I won\u2019t begin this article with a lecture.\n\nSuffice it to say, it\u2019s a good idea to involve research of some sort during the life cycle of every project you undertake. I don\u2019t just mean visual or competitor research, which of course is also very important. I mean looking at or finding your own audience or user data. Whether that be auditing existing data or research available from the client, carrying out user interviews, A/B testing, or conducting a simple questionnaire with users, any research is better than none. If you create personas as a design tool, they should always be based on research, so you will need to have plenty of data to hand for that.\n\nWhere do I start?\n\nIn the initial kick-off stages of a project, it\u2019s a good idea to start by asking your client (when working in-house you still have a client \u2013 you might even be the client) what research or audience data they have available. Some will have loads \u2013 analytics, surveys, focus groups and insights \u2013 from talking to customers. Some won\u2019t have much at all and you\u2019ll be hard pressed to find out much about the audience. It\u2019s best to review existing research first without rushing headlong into doing new research. Get a picture of what the data tells you and perhaps get this into a document \u2013 who, what, why and how are they using this website or app? What gaps are there in existing research? What else do you need to know? Then you can decide what else you need to do to plug these gaps. Think about the information first before deciding on the methodology. The rest of my article talks mostly about running self-completion online surveys. You can of course do face-to-face surveys, self-completion written questionnaires or phone polls, but I won\u2019t cover those here. That\u2019s for another article.\n\nWhy run a survey?\n\nSurveys are great for getting a broad picture of your audience. As long as they are designed carefully, you can create an overview of them, how they use the site and their opinions of it, with an idea of which parts of this picture are more important than others. By using a limited amount of open-ended questions, you can also get some more qualitative feedback or insights on your website or app. The clients we work with surprisingly often don\u2019t have much in the way of audience research available, even basic analytics, so I will often suggest running a short survey, just to create a picture of who is out there.\n\nOK, what should I do first?\n\nBefore you rush into writing questions, stop and think about what you\u2019re trying to find out. Remember being in school when you studied science and you had to propose a hypothesis? This could be a starting point \u2013 something to prove or disprove. Or, even better, write a research brief. It doesn\u2019t have to be long; it can be just a sentence that encapsulates what you\u2019re trying to do, like a good creative brief. For the purposes of this article, I created a short, slightly silly survey on Christmas and beliefs in Father Christmas.\n\nMy research brief was:\n\n\n\tTo find out more about people\u2019s beliefs about Father Christmas and their experiences of Christmas.\n\n\nInevitably, as you start thinking of what questions to ask, you will find that you go off at tangents or your client will want you to add in everything but the kitchen sink. In order for your questionnaire not to get too long and lose focus, you could write lists of what it is and what it\u2019s not. This is how I\u2019d apply it to my Christmas questionnaire example:\n\nWhat it is about\n\n\n\tHow people communicate with Father Christmas\n\tIf someone\u2019s background has affected their likelihood of believing in Father Christmas\n\n\nWhat it is not about\n\n\n\tWhat colour to change Father Christmas\u2019s coat to\n\tFather Christmas\u2019s elves\n\n\nLet\u2019s get down to business: the questions. \n\nKinds of questions\n\nThere are two basic kinds of questions: open-ended and closed. Closed questions limit answers by giving the respondent a number of predefined lists of options to choose from. Typically, these are multiple-choice questions with a list of responses. You can either select one or tick all that apply. Another useful type of closed question I often use is a rating scale, where a respondent can assess a situation along a continuum of values. These can also be useful as a measure of advocacy or strength of feeling about something. There is a standard measure called the Net Promoter score, which measures how likely someone is to recommend your product or service to a friend or acquaintance. It\u2019s a useful benchmark as you can compare your scores to others in a similar sector.\n\n\n\nOpen-ended questions often take the form of a statement which requires a response. Generally, respondents are given a text box to fill in. It\u2019s useful to limit this in some way so that people have an idea of how long the expected response should be; for example, a single line for an email address (Q18), or a larger text area for a longer response (Q6).\n\nIf you plan to send your survey out to a large number of people, I would suggest using mostly closed questions, unless you want to spend a long time wading through comments and hand-coded responses. I\u2019d always advise adding a general request at the end of a survey (\u2018Is there anything else you\u2019d like to tell us?\u2019). You\u2019d be surprised how many interesting and insightful comments people will add.\n\nThere are times when it\u2019s better to provide an open-ended text box rather than a predefined list makes assumptions about your audience\u2019s groupings. For example, we ran a short survey for our Gridset beta testers and rather than assume we knew who they were, we decided to ask an open-ended question: \u201cWhat is your current job title?\u201d\n\n\n\nThe analysis took quite a bit longer than responses using a predefined list, but it meant that we were able to make sure we didn\u2019t miss anyone. And next time we run a survey for Gridset, I can use the responses gathered from this survey to help create a predefined list to make analysis easier.\n\nWhat to ask\n\nThe questions to ask depend on what you want to know, but your brief and lists of what the survey is and isn\u2019t should help here. I always ask the design team and client to give me ideas of what they are interested in finding out, and combine this with a mix of new and standard questions I have used in other surveys. I find Survey Monkey\u2019s question bank a very useful source of example questions and help with tricky wording.\n\nI always include simple demographics so I can compare my results to the population at large or internet users as a whole \u2013 just going on age, gender and location can be quite illuminating. For example, with the Christmas survey, I can see that the respondents were typical of the online design and dev community, mainly young and male.\n\nIf appropriate, I add questions on disability, ethnic background, religion and community of interest. Questions about ethnicity, religion, sexual preference, disability and other sensitive subjects can feel awkward and difficult to ask. This is not a good reason to not ask them. Perhaps you\u2019re working for a public sector client, like a local council, so it\u2019s likely you will need to consider groups of people who maybe under-represented, who may have differing views to others, or who you need to look at specifically as a subset.\n\nHow to ask\n\nAlthough they may seem clunky and wordy, it\u2019s often best to use the census wording or professional body wording for such demographic questions. For example, I used the UK census 2011 wording for Wales on my Christmas questionnaire in my questions on religion [PDF] (Q16) and ethnicity [PDF] (Q17). I had to adapt them slightly for the Survey Monkey format \u2013 self-completion online, rather than pen and paper \u2013 which is why \u201cWhite Welsh\u201d came up as the first option for the ethnicity question. For similar questions for US audiences, try the Census Bureau website.\n\nWhen conducting a survey for a project that has a global audience, you need to consider who your primary audience is. For example, I recently created a questionnaire for a global news website. A large proportion of its audience is based in the USA, so I was careful to word things in a way Americans would find familiar. I used the US ethnic background census question wording and options, and looked at data for US competitor news websites to decide which to include.\n\nYou should also consider people whose first language isn\u2019t English. Working as an audience researcher at BBC Wales, every survey we did was bilingual. I\u2019ve also recently run a user survey in Arabic using Google Forms. During this project, we found that while Survey Monkey supports different languages, including Arabic, the text ran left to right with no option to change it to right to left \u2013 an essential when it comes to reading Arabic! If research is a deliverable in a client project, and you know you\u2019ll need to conduct it in a foreign language, always build in extra time for translation at both the questionnaire design and analysis stages. Make sure you also allow for plenty of checks. In this case we had to change to Google Forms after initially creating our survey with Survey Monkey to get the functionality we needed.\n\nLook and feel\n\nThink about the survey as another way your audience will experience your brand. Take care getting the tone of voice right. There are plenty of great articles and books out there about tone of voice \u2013 try Letting Go of the Words by Ginny Redish for starters, or Brand Language by Liz Doig. The basic rule of thumb is to sound like a human, and use clear and friendly language. If, like me, you are lucky enough to work with journalists or copy editors, you should ask for their help, particularly in the preamble, linking text and closing statements. I find it helpful to break my questions down into sections and to have a page for each. I then have an introductory piece of text for each section to guide the respondent through the survey.\n\nYou should also make sure you check with your designers how your survey looks \u2013 use a company logo and branding, and make the typography legible. Many survey apps like Survey Monkey and Google Forms have a progress bar. This is helpful for users to see how far through your survey they are. I generally time the survey and give an indication in the preamble: \u201cThis survey will only take five minutes of your time.\u201d\n\nYou also need to think about how you will technically serve the questionnaire. For example, will it be via email, social media, a pop-up or lightbox on your website, or (not recommended but possible) in an ad space?\n\nEthical considerations\n\nSomething else to think about are any local laws that govern how you collect and store data, such as the Data Protection Act in the UK. As a member of the Market Research Society, I am also obliged to consider its guidelines, but even if you\u2019re not, it\u2019s always a good idea to deal with personal data ethically.\n\nIf you collect personal data that can identify individuals, you must ask their permission to share it with others, and store it securely for no longer than two years. If you want to contact people afterwards, you must ask for their permission. If you ask for email addresses, as I did in question 18, you have a ready-made sample for a further survey, interviews or focus groups. Remember, you shouldn\u2019t survey people under sixteen years old without the permission of their parents or legal guardians, so if you know your website is likely to be used by children, you must ask for verification of age early on, and your survey should close someone answers that they are under sixteen. The ESOMAR guidelines for online research [PDF] are well worth reading, as they go into detail about such issues, as well as privacy guidelines \u2013 using cookies, storing IP addresses, and so on.\n\nTools\n\nUnless you work in-house and have proprietary software, or at a market research agency and you\u2019re using specialist software such as Snap or IBM SPSS Statistics (previously just SPSS), you will need to use a good tool to run your survey, collect your responses and, ideally, help with the analysis. I like Survey Monkey because of the question bank and analysis tools. The software graphs your results and does simple cross-tabbing and filtering. What this means is you can slice the data in more interesting ways and delve a bit deeper. For example, in the Gridset questionnaire I mentioned earlier, I cross-tabbed responses to questions against whether a person worked in-house, for an agency or as a freelancer. \n\nOther well known online tools that I also use from time to time are Wufoo and Google Forms. Smart Surveys is a similar service to Survey Monkey and it\u2019s used by many leading brands in the UK. Snap Surveys mentioned above is a well-established player in the market research scene, used a lot for face-to-face surveys and also on tablets and smartphones.\n\nAnalysis\n\nAnalysis is often overlooked but is as important as the design of the questionnaire. Don\u2019t just rely on looking at the summary report and charts generated as standard by your form or survey software. Spend time with your data. Spend at least a week now and then if you can, looking at the data. Keep coming back to it and tweaking or cutting it a different way to see if there are any different pictures. Slice it up in different ways to reveal new insights. Here is the data from my dummy survey (apart from the open-ended responses). \n\nFor open-ended questions, you can analyse collaboratively. Print and cut out the open-ended responses and do a cluster analysis or affinity sort with a colleague. \n\n\n\nDiscussing the comments helps you to understand them. You will also find the design team are more likely to buy into the research as they have uncovered the insights for themselves. Always make sure to treat open-ended responses sensitively and don\u2019t share anything publicly in a way that identifies the respondent.\n\nWrite a report\n\nNever hand over a dataset to your client without a summary of the findings. Data on its own can be skewed to suit the reader\u2019s needs, and not everyone is able to find the story in a dataset. Even if it\u2019s not a deliverable, it\u2019s always a good idea to capture your findings in a report of some sort. Use graphs sparingly to show really interesting things or to aid the reader\u2019s understanding. I have written a quick dummy report using the data from the Christmas questionnaire so you can see how it\u2019s done.\n\nI highly recommend Brian Suda\u2019s book A Practical Guide to Designing with Data for tips on how to present data effectively, but that\u2019s a subject that benefits a whole article (indeed book) in itself. \n\nI am not a designer. I am a researcher, so I never write design recommendations in a report unless they have been talked about or suggested by the designers I work with. More often, I write up the results and we talk about them and what impact they have on the project or design. Often they lead to more questions or further research.\n\nSo that\u2019s it: a brief introduction to using questionnaires for design research. Here\u2019s a quick summary to remind you what I have talked about, and a list of resources if you\u2019re interested in reading further.\n\nTop 10 things to remember when using questionnaires for design research:\n\n\n\tStart by auditing existing research to identify gaps in data.\n\tWrite a research brief. Work out exactly what you\u2019re trying to find out \u2013 what is the survey about, and what is it not about?\n\tThe two basic kinds of questions are open-ended and closed.\n\tClosed questions limit responses by giving the respondent a number of predefined lists of options to choose from (multiple choice, rating scales, and so on).\n\tOpen-ended questions are often in the form of a statement which requires a response. Always ask one at the end of a questionnaire.\n\tAlways include simple demographics to enable you to compare your sample against the population in general.\n\tIt\u2019s best to use official census or professional body wording for questions on ethnicity, disability and religion.\n\tBe sure to think carefully about your tone of voice and the look of your questionnaire.\n\tPay attention to guidelines and laws on storing personal data, cookies and privacy.\n\tInvest plenty of time in analysis and report writing. Don\u2019t just look at the obvious \u2013 dig deep for more interesting insights.\n\n\nSome useful resources for further study\n\nOnline research\n\n\n\tDesign Research: Methods and Perspectives edited by Brenda Laurel\n\tOnline Research Essentials by Brenda Russell and John Purcell\n\tHandbook of Online and Social Media Research by Ray Poynter\n\tESOMAR guidelines for online research [PDF]\n\tOnline questionnaires\n\n\nMarket research books on questionnaire design\n\n\n\tUsing Questionnaires in Small-Scale Research: A Beginner\u2019s Guide by Pamela Munn\n\tQuestionnaire Design by A N Oppenheim\n\tDeveloping a Questionnaire by Bill Gillham", "year": "2012", "author": "Emma Boulton", "author_slug": "emmaboulton", "published": "2012-12-14T00:00:00+00:00", "url": "https://24ways.org/2012/using-questionnaires-for-design-research/", "topic": "business"} {"rowid": 96, "title": "Unwrapping the Wii U Browser", "contents": "The Wii U was released on 18 November 2012 in the US, and 30 November in the UK. It\u2019s the first eighth generation home console, the first mainstream second-screen device, and it has some really impressive browser specs.\n\nConsoles are not just for games now: they\u2019re marketed as complete entertainment solutions. Internet connectivity and browser functionality have gone from a nice-to-have feature in game consoles to a selling point. In Nintendo\u2019s case, they see it as a challenge to design an experience that\u2019s better than browsing on a desktop.\n\n\n\tLet\u2019s make a browser that users can use on a daily basis, something that can really handle everything we\u2019ve come to expect from a browser and do it more naturally.\nSasaki \u2013 Iwata Asks on Nintendo.com\n\n\nWith 11% of people using console browsers to visit websites, it\u2019s important to consider these devices right from the start of projects. Browsing the web on a TV or handheld console is a very different experience to browsing on a desktop or a mobile phone, and has many usability implications.\n\nConsole browser testing\n\nWhen I\u2019m testing a console browser, one of the first things I do is run Niels Leenheer\u2019s HTML5 test and Lea Verou\u2019s CSS3 test. I use these benchmarks as a rough comparison of the standards each browser supports.\n\nIn October, IE9 came out for the Xbox 360, scoring 120/500 in the HTML5 test and 32% in the CSS3 test. The PS Vita also had an update to its browser in recent weeks, jumping from 58/500 to 243/500 in the HTML5 test, and 32% to 55% in the CSS3 test. Manufacturers have been stepping up their game, trying to make their browsing experiences better.\n\nTo give you an idea of how the Wii U currently compares to other devices, here are the test results of the other TV consoles I\u2019ve tested. I\u2019ve written more in-depth notes on TV and portable console browsers separately.\n\n\nYear of releaseHTML5 scoreCSS3 scoreNotes\nWii U2012258/50048%Runs a Netfront browser (WebKit).\nWii200689/500Wouldn\u2019t runRuns an Opera browser.\nPS3200668/50038%Runs a Netfront browser (WebKit).\nXbox 3602005120/50032%A browser for the Xbox (IE9) was only recently released in October 2012. The Kinect provides voice and gesture support. There\u2019s also SmartGlass, a second-screen app for platforms including Android and iOS.\n\n\nThe Wii U browser is Nintendo\u2019s fifth attempt at a console browser. Based on these tests, it\u2019s already looking promising.\n\nWhy console browsers used to suck\n\nIt takes a lot of system memory to run a good browser, and the problem of older consoles is that they don\u2019t have much memory available. The original Nintendo DS needs a memory expansion pack just to run the browser, because the 4MB it has on board isn\u2019t enough. I noticed that even on newer devices, some sites fail to load because the system runs out of memory.\n\nThe Wii came out six years ago with an Opera browser. Still being used today and with such low resources available, the latest browser features can\u2019t reasonably be supported. There\u2019s also pressure to add features such as tabs, and enable gamers to use the browser while a game is paused. Nintendo\u2019s browser team have the advantage of higher specs to play with on their new console (1GB of memory dedicated to games, 1GB for the system), which makes it easier to support the latest standards. But it\u2019s still a challenge to fit everything in.\n\n\n\t\u2026even though we have more memory, the amount of memory we can use for the browser is limited compared to a PC, so we\u2019ve worked in ways that efficiently allocates the available memory per tab. To work on this, the experience working on the browser for the Nintendo 3DS system under a limited memory constraint helped us greatly.\nSasaki \u2013 Iwata Asks on Nintendo.com\n\n\nIn the box\n\nThe Wii U consists of a console unit which plugs into a TV (the first to support HD), and a wireless controller known as a gamepad. The gamepad is a lot bigger than typical TV console controllers, and it has a touchscreen on the front. The touchscreen is resistive, responding to pressure rather than electrical current. It\u2019s intended to be used with a stylus (provided) but fingers can be used.\n\nIt might look a bit like one, but the gamepad isn\u2019t a portable console designed to be taken out like the PS Vita. The gamepad can be used as a standalone screen with the TV switched off, as long as it\u2019s within range of the console unit \u2013 it basically piggybacks off it.\n\n\n\nIt\u2019s surprisingly lightweight for its size. It has a wealth of detectors including 9-axis control. Sensors wake the device from sleep when it\u2019s picked up. There\u2019s also a camera on the front, and a headphone port and speakers, with audio coming through both the TV and the gamepad giving a surround sound feel.\n\nUp to six tabs can be opened at once, and the browser can be used while games are paused. There\u2019s a really nice little feature here \u2013 the current game\u2019s name is saved as a search option, so it\u2019s really quick to look up contextual content such as walk-throughs.\n\nControls\n\nOnly one gamepad can be used to control the browser, but if there are Wiimotes connected, they can be used as pointers. This doesn\u2019t let the user do anything except point (they each get a little hand icon with a number on it displayed on the screen), but it\u2019s interesting that multiple people can be interacting with a site at once.\n\n\n\nSee a bigger version\n\nThe gamepad can also be used as a simple TV remote control, with basic functions such as bringing up the programme guide, adjusting volume and changing channel. I found the simplified interface much more usable than a full-featured remote control.\n\n\n\nI\u2019m used to scrolling being sluggish on consoles, but the Wii U feels almost as snappy as a desktop browser. Sites load considerably faster compared with others I\u2019ve tested.\n\nTilt-scroll\n\nHolding down ZL and ZR while tilting the screen activates an Instapaper-style tilt to scroll for going up and down the page quickly, useful for navigating very long pages.\n\nSecond screen\n\nThe TV mirrors most of what\u2019s on the gamepad, although the TV screen just displays the contents of the browser window, while the gamepad displays the site along with the browser toolbar.\n\nWhen the user with the gamepad is typing, the keyboard is hidden from the TV screen \u2013 there\u2019s just a bit of text at the top indicating what\u2019s happening on the gamepad.\n\nPressing X draws an on-screen curtain over the TV, hiding the content that\u2019s on the gamepad from the TV. Pressing X again opens the curtains, revealing what\u2019s on the gamepad. Holding the button down plays a drumroll before it\u2019s released and the curtains are opened. I can imagine this being used in meetings as a fun presentation tool.\n\n\n\n\n\tIn a sense, browsing is a personal activity, but you get the idea that people will be coming and going through the room. When I first saw the curtain function, it made a huge impression on me. I walked around with it all over the company saying, \u201cThey\u2019ve really come up with something amazing!\u201d\nIwata \u2013 Iwata Asks on Nintendo.com\n\n\nText\n\nWriting text\n\nUnlike the capacitive screens on smartphones, the Wii U\u2019s resistive screen needs to be pressed harder than you\u2019re probably used to for registering a touch event. The gamepad screen is big, which makes it much easier to type on this device than other handheld consoles, even without the stylus. It\u2019s still more fiddly than a full-sized keyboard though. When you\u2019re designing forms, consider the extra difficulty console users experience.\n\n\n\nAlthough TV screens are physically big, they are typically viewed from further away than desktop screens. This makes readability an issue, so Nintendo have provided not one, but four ways to zoom in and out:\n\n\n\tDouble-tapping on the screen.\n\tTapping the on-screen zoom icons in the browser toolbar.\n\tPressing the + and - buttons on the device.\n\tMoving the right analogue stick up and down.\n\n\nAs well as making it easy to zoom in and out, Nintendo have done a few other things to improve the reading experience on the TV.\n\nSystem font\n\nOne thing you\u2019ll notice pretty quickly is that the browser lacks all the fonts we\u2019re used to falling back to. Serif fonts are replaced with the system\u2019s sans-serif font. I couldn\u2019t get Typekit\u2019s font loading method to work but Fontdeck, which works slightly differently, does display custom fonts.\n\n The system font has been optimised for reading at a distance and is easy to distinguish because the lowercase e has a quirky little tilt.\n\nDon\u2019t lose :focus\n\nUsing the D-pad to navigate is similar to using a keyboard. Individual links are focused on, with a blue outline drawn around them.\n\nThe recently redesigned An Event Apart site is an example that improves the experience for keyboard and D-pad users. They\u2019ve added a yellow background colour to links on focus. It feels nicer than the default blue outline on its own.\n\n\n\nMedia\n\nThis year, television overtook PCs as the primary way to watch online video content. TV is the natural environment for video, and 42% of online TVs in the US are connected to the internet via a console. Unfortunately, the