rowid,title,contents,year,author,author_slug,published,url,topic 17,Bringing Design and Research Closer Together,"The ‘should designers be able to code’ debate has raged for some time, but I’m interested in another debate: should designers be able to research? Are you a designer who can do research? Good research and the insights you uncover inspire fresh ways of thinking and get your creative juices flowing. Good research brings clarity to a woolly brief. Audience insight helps sharpen your focus on what’s really important. Experimentation through research and design brings a sense of playfulness and curiosity to your work. Good research helps you do good design. Being a web designer today is pretty tough, particularly if you’re a freelancer and work on your own. There are so many new ideas, approaches to workflow and trends and tools to keep up with. How do you decide which things to do and which to ignore? A modern web designer needs to be able to consider the needs of the audience, design appropriate IAs and layouts, choose colour palettes, pick appropriate typefaces and type layouts, wrangle with content, style, code, dabble in SEO, and the list goes on and on. Not only that, but today’s web designer also has to keep up with the latest talking points in the industry: responsive design, Agile, accessibility, Sass, Git, lean UX, content first, mobile first, blah blah blah. Any good web designer doesn’t need to be persuaded about the merits of including research in their toolkit, but do you really have time to include research too? Who is responsible for research? Generally, research in the web industry forms part of other disciplines and isn’t so much a discipline in its own right. It’s very often thought of as part of UX, or activities that make up a process such as IA or content strategy. Research is often undertaken by UX designers, information architects or content strategists and isn’t something designers or developers get that involved in. Some people lump all of these activities together and label it design research and have design researchers to do it. Some companies, such as the one I run with my husband Mark, are lucky enough to have someone with specialist research knowledge (yup, that would be me folks) who can lead all or most of the research work undertaken by the company. See also Mule Design, GOV.UK, the BBC, Mailchimp, Facebook and Twitter. What if you’re not lucky enough to have your own researcher or team of researchers? Often research is the kind of thing that’s nice to have, or it can be cut from scope when doing the budget dance with a client. It often forms part of the discovery phase of a project and sometimes just becomes a tick-box exercise. But research isn’t just user testing and it shouldn’t just live in a report on Basecamp that no one reads. I would argue that research and experimentation is a way of working or an approach to how you design. Research can be used during the whole design process and must be a vital part of a designer’s workflow on every project. Even if you work in a small studio, you can still create a culture of audience insight. Even if you work on your own, you can still absorb yourself in as much audience data as you can throughout the project life cycle. Here’s how. Research is everyone’s job There is a subtle difference between writing a research report and delivering it to a client, and them actually using it and applying the insights to their thought process. In my experience of working in the audiences team at the BBC, research was most effective when the role was embedded in the production team and insights were used as part of the editorial process. In this section I’ll talk through some common problems you might encounter in a typical project life cycle and show you ways you can use research to help you. For the sake of this article, let’s imagine that we’re talking about a particular project here and not ongoing product development. The same principles can of course be applied then, but even if you work in-house rather than on the agency side, you’re probably used to working on distinct projects or phases of work. 1. Problem: I want to come up with a new product idea. Solution: Inspiration through insights. Before you begin a new project, a good way of quickly absorbing all the existing knowledge that there maybe about a theme, product type or website is to literally surround yourself with it. This is especially relevant for new ideas or product development. Create an incident room if you can: fill the walls of your meeting room, the walls near your desk, or even just use a pinboard or online pinboard if space is tight or you’re working with a dispersed team. The same process can be used throughout a project’s or product life cycle — read about how MailChimp has applied this idea. Let’s take a new product idea as an example. Say you wanted to develop a responsive tool for web designers but you weren’t sure what aspect of responsive design to focus on. First of all, you should pose a hypothesis or problem statement to gather ideas around. For example: “How to speed up a designer’s responsive workflow.” You would then need to gather insights around this topic. You could run some interviews with freelance designers about how they work responsively. You could shadow a development team for the day to understand their processes. You could observe conversations on Twitter or IRC or wherever your target audience interact to see what people talk about. You could search out industry data and articles currently available. The next stage is to comb through this data and extract insights from it. You can use good old Post-it notes and a sharpie: capture one insight or thought per Post-it. If one insight leads into another, use two Post-its. The objective is volume. Try to ensure clarity in each Post-it so you don’t have to go back and reference material again (maybe you could use a key if you think it’ll get confusing). After this, stick them all up and synthesise the same way you would for any kind of cluster or affinity sort. Organise into broad themes. These themes then become springboards for further exploration and idea generation. You might see a gap or opportunity in one particular area, both from a workflow perspective but also from a business perspective. Bingo. Your insights then become the fuel for ideas generation. This method doesn’t just have to be used for new products — it works particularly well in a discovery phase for new projects or for new features in an existing product. We’re doing something similar for our own responsive tool, Gridset at the moment. Resources: Sticky Wisdom by Dave Allan, Matt Kingdon, Kris Murrin, Daz Rudkin The Science of Serendipity by Matt Kingdon The Art of Innovation by Tom Kelley 2. Problem: You’re starting a new project and need to know the basics before you get headlong into designing or building. Solution: Quantitative survey. Common questions might be: Who are the users? How many are there? What are they like? Why do they use the site? What do they need from the site? What are their goals? Print out and stick up what you already know and have in your project space or ‘incident room’: any reports you have found or been given, analytics graphs, personas, pen portraits, as well as screengrabs of the current website, product or branding. Spend time looking through it all and identify the gaps. If you have very little existing audience data, a quick and easy way to get some baseline information is to run a quick user survey on a current website. You can establish basic demographic information, appreciation and views of the website as it stands, as well as delve a little deeper into needs and wants. This is also vital if you want some kind of trackable measures to go back to once you have designed and built your shiny new website for your client — read more in my article for 24 ways last year.) We use surveys a lot at Mark Boulton Design for our client work. Here’s a screen grab of one we ran in March on http://info.cern.ch before we redesigned the site and did the work on the First Website Project. We repeated the survey after the new website went live and were able to compare the results. Both surveys were a great source of insight to the project team as well as for the project stakeholders who needed to pitch the idea of the hack days and fundraise for them. Once you’ve run your survey, you should always write up a short summary for yourself and your client to refer to. If you’re not a trained researcher, you should try to read up on analysis techniques or data visualisation. It can be easy to misinterpret data and make it bend to the story you are trying to tell. You should be looking for the story in the data and present it without bias. If you’re using the ‘incident room’ method I mentioned earlier on, you can also extract the insights onto post it notes and add them to your growing body of knowledge. Resources: Using Questionnaires for Design Research by Emma Boulton Data-driven Design with an Annual Survey by Aarron Walter Research Methods for Product Design by Alex Milton and Paul Rodgers A Practical Guide to Designing with Data by Brian Suda 3. Problem: You have a prototype of a new design and you need some feedback from real users. Solution: User interviews and task based testing. Interviewing is a staple research method that every designer should master as it can be used throughout a project life cycle. Erika Hall recently wrote a great article on the basics for A List Apart. From stakeholder interviews in a discovery phase, to initial user research, right through to task based testing and iteration, interviews can be enormously helpful. They are very time-consuming, however, and although speaking to someone is better than speaking to no one, it’s always better to plan to do a few interviews at once, rather than one or two. I generally find that patterns only start to emerge after I’ve spoken to 4 or 5 people. Interviews are another thing we do a lot of at Mark Boulton Design. Most of the interviews we do are remote due to the location of our clients and their users. Rigour is an important consideration in all research activities and especially if you’re a non-researcher. Interviews particularly can be easily skewed by an inexperienced facilitator, which is why pairing can be a good approach. Building rapport, questioning, time keeping, note taking and thinking on your feet can be difficult to do all at once, so having a colleague take notes while you concentrate on leading the conversation can work really well. It’s important for the note taker to sit in on more than one interview so that they get a more rounded view of the feedback. The same person should also be involved in the analysis of the data. Interviews can be analysed and written up in a report or summary as with other types of research. I often use the same kind of collaborative process detailed earlier for deciding on themes, particularly if multiple members of the team have been involved in interviewing. Interviews are particularly useful for our incident room and can provide much colour and insight to an exploratory process. I often find verbatim quotes to be the most insightful type of data. You might find that an inexperienced researcher (or designer who is used to solving problems) will jump to interpretation too soon and forget to just listen to what the interviewee is saying. Capturing the exact form of words a person uses can help get away from this. Resources: Interviewing Humans by Erika Hall A Pocket Guide to Interviewing for Research by Andrew Travers Interviewing Users by Steve Portigal 4. Problem: How successful have I been with this new design? Solution: Key performance indicators Once your new design has been realised, it’s important to evaluate it. What works, what doesn’t work so well? As well as a straightforward design crit, don’t forget to introduce audience insights into a review meeting or project wash up. Work out what your KPIs — your key performance indicators — will be beforehand and then you can start to track them over time. For example, number of visits, appreciation of the site, willingness to recommend the site to a friend, number of sales, and number of conversions are all sensible measures to track. Interviews can again be helpful but cold, hard numbers are often better here. Read Corey Vilhauer’s take on this on A List Apart. Consistency is key here. If you have looked at your analytics and done a survey beforehand, you will have a baseline to start from. Don’t keep changing your measures and questions, or your data will not be comparable. Pick a few key questions or a set of measures, create a survey and then run it once a month, once a quarter, every six months or annually. You’ll start to see changes over time as the design beds in. You may see seasonal trends and spot patterns in the data related to other activities like marketing, promotion and so on. Keeping a record of all of this will increase your understanding of your audience. We’ve created a satisfaction survey for Gridset with a number of measures that we track on an ongoing basis. MailChimp has also created an annual survey with the aim of tracking their audience measures over time Resources: Search Analytics by Louis Rosenfeld A Primer on A/B Testing by Lara Swanson Lean UX by Jeff Gothelf Anyone can do research Research can be brought into the project life cycle at any stage. And of course, anyone can do research — you don’t need to be a researcher. Some of the main skills most designers possess are also key research skills: inquisitive nature, problem solving, playfulness, empathy, and so on. We have a small team at Mark Boulton Design. Most of the team are designers and the rest of us focus on supporting the team and clients both in terms of billable work (research, content strategy, project management) as well as the non-billable things like finance and studio management. Despite my best intentions, in the past I’ve undertaken research for clients in isolation — first being briefed by the design lead, carrying out the research and then delivering the findings back, trusting the design team to take the findings on board. This was often due to time and availability of resources. We’ve been trying hard to join up our processes and collaborate even more across the team. Undertaking heuristic or design reviews collaboratively; taking part in frequent critiques of our work and the work of others together; pairing a researcher and a designer to run interviews; workshopping results from interviews to come up with recommendations; working closely together on questionnaire design; shadowing each other on tasks that don’t fall within our core skills. A little thing like moving our desks around has also helped us have more conversations that we can all be a part of. I’ve come to the conclusion that my role as the research director at Mark Boulton Design is actually a facilitator of research. As well as carrying out research, I am responsible for ensuring that research happens consistently across the team. I am responsible for empowering and training our designers so they feel confident in carrying out their own user, audience or design research for clients. So they know what to look for, when to listen, when to probe and when to take note of something. So they know how to look for themes, how to synthesise insights from research and how to apply them to their work. Better research leads to better design So, are you a designer who can do research? Are you a researcher who can design? The best designers are a lucky combination of researcher and designer. If you’re not one of those, look at ways of enhancing the skills you lack. Because there’s no doubt in my mind, that becoming a better researcher will make you a better designer. General resources: Seeing the Elephant by Louis Rosenfeld Connected UX by Aarron Walter Beyond Usability Testing by Devan Goldstein Just Enough Research by Erika Hall The User Experience Team of One by Leah Buley Undercover User Experience Design by Cennydd Bowles and James Box A Pocket Guide to Psychology for Designers by Joe Leech A Pocket Guide to International User Research by Chui Chui Tan Remote Research by Nate Bolt and Tony Tulathimutte A Pocket Guide to Experiments for Designers by Colin McFarland",2013,Emma Boulton,emmaboulton,2013-12-22T00:00:00+00:00,https://24ways.org/2013/bringing-design-and-research-closer-together/,ux 94,Using Questionnaires for Design Research,"How do you ask the right questions? In this article, I share a bunch of tips and practical advice on how to write and use your own surveys for design research. I’m an audience researcher – I’m not a designer or developer. I’ve spent much of the last thirteen years working with audience data both in creative agencies and on the client-side. I’m also a member of the Market Research Society. I run user surveys and undertake user research for our clients at the design studio I run with my husband – Mark Boulton Design. So let’s get started! Who are you designing for? Good web designers and developers appreciate the importance of understanding the audience they are designing or building a website or app for. I’m assuming that because you are reading a quality publication like 24 ways that you fall into this category, and so I won’t begin this article with a lecture. Suffice it to say, it’s a good idea to involve research of some sort during the life cycle of every project you undertake. I don’t just mean visual or competitor research, which of course is also very important. I mean looking at or finding your own audience or user data. Whether that be auditing existing data or research available from the client, carrying out user interviews, A/B testing, or conducting a simple questionnaire with users, any research is better than none. If you create personas as a design tool, they should always be based on research, so you will need to have plenty of data to hand for that. Where do I start? In the initial kick-off stages of a project, it’s a good idea to start by asking your client (when working in-house you still have a client – you might even be the client) what research or audience data they have available. Some will have loads – analytics, surveys, focus groups and insights – from talking to customers. Some won’t have much at all and you’ll be hard pressed to find out much about the audience. It’s best to review existing research first without rushing headlong into doing new research. Get a picture of what the data tells you and perhaps get this into a document – who, what, why and how are they using this website or app? What gaps are there in existing research? What else do you need to know? Then you can decide what else you need to do to plug these gaps. Think about the information first before deciding on the methodology. The rest of my article talks mostly about running self-completion online surveys. You can of course do face-to-face surveys, self-completion written questionnaires or phone polls, but I won’t cover those here. That’s for another article. Why run a survey? Surveys are great for getting a broad picture of your audience. As long as they are designed carefully, you can create an overview of them, how they use the site and their opinions of it, with an idea of which parts of this picture are more important than others. By using a limited amount of open-ended questions, you can also get some more qualitative feedback or insights on your website or app. The clients we work with surprisingly often don’t have much in the way of audience research available, even basic analytics, so I will often suggest running a short survey, just to create a picture of who is out there. OK, what should I do first? Before you rush into writing questions, stop and think about what you’re trying to find out. Remember being in school when you studied science and you had to propose a hypothesis? This could be a starting point – something to prove or disprove. Or, even better, write a research brief. It doesn’t have to be long; it can be just a sentence that encapsulates what you’re trying to do, like a good creative brief. For the purposes of this article, I created a short, slightly silly survey on Christmas and beliefs in Father Christmas. My research brief was: To find out more about people’s beliefs about Father Christmas and their experiences of Christmas. Inevitably, as you start thinking of what questions to ask, you will find that you go off at tangents or your client will want you to add in everything but the kitchen sink. In order for your questionnaire not to get too long and lose focus, you could write lists of what it is and what it’s not. This is how I’d apply it to my Christmas questionnaire example: What it is about How people communicate with Father Christmas If someone’s background has affected their likelihood of believing in Father Christmas What it is not about What colour to change Father Christmas’s coat to Father Christmas’s elves Let’s get down to business: the questions. Kinds of questions There are two basic kinds of questions: open-ended and closed. Closed questions limit answers by giving the respondent a number of predefined lists of options to choose from. Typically, these are multiple-choice questions with a list of responses. You can either select one or tick all that apply. Another useful type of closed question I often use is a rating scale, where a respondent can assess a situation along a continuum of values. These can also be useful as a measure of advocacy or strength of feeling about something. There is a standard measure called the Net Promoter score, which measures how likely someone is to recommend your product or service to a friend or acquaintance. It’s a useful benchmark as you can compare your scores to others in a similar sector. Open-ended questions often take the form of a statement which requires a response. Generally, respondents are given a text box to fill in. It’s useful to limit this in some way so that people have an idea of how long the expected response should be; for example, a single line for an email address (Q18), or a larger text area for a longer response (Q6). If you plan to send your survey out to a large number of people, I would suggest using mostly closed questions, unless you want to spend a long time wading through comments and hand-coded responses. I’d always advise adding a general request at the end of a survey (‘Is there anything else you’d like to tell us?’). You’d be surprised how many interesting and insightful comments people will add. There are times when it’s better to provide an open-ended text box rather than a predefined list makes assumptions about your audience’s groupings. For example, we ran a short survey for our Gridset beta testers and rather than assume we knew who they were, we decided to ask an open-ended question: “What is your current job title?” The analysis took quite a bit longer than responses using a predefined list, but it meant that we were able to make sure we didn’t miss anyone. And next time we run a survey for Gridset, I can use the responses gathered from this survey to help create a predefined list to make analysis easier. What to ask The questions to ask depend on what you want to know, but your brief and lists of what the survey is and isn’t should help here. I always ask the design team and client to give me ideas of what they are interested in finding out, and combine this with a mix of new and standard questions I have used in other surveys. I find Survey Monkey’s question bank a very useful source of example questions and help with tricky wording. I always include simple demographics so I can compare my results to the population at large or internet users as a whole – just going on age, gender and location can be quite illuminating. For example, with the Christmas survey, I can see that the respondents were typical of the online design and dev community, mainly young and male. If appropriate, I add questions on disability, ethnic background, religion and community of interest. Questions about ethnicity, religion, sexual preference, disability and other sensitive subjects can feel awkward and difficult to ask. This is not a good reason to not ask them. Perhaps you’re working for a public sector client, like a local council, so it’s likely you will need to consider groups of people who maybe under-represented, who may have differing views to others, or who you need to look at specifically as a subset. How to ask Although they may seem clunky and wordy, it’s often best to use the census wording or professional body wording for such demographic questions. For example, I used the UK census 2011 wording for Wales on my Christmas questionnaire in my questions on religion [PDF] (Q16) and ethnicity [PDF] (Q17). I had to adapt them slightly for the Survey Monkey format – self-completion online, rather than pen and paper – which is why “White Welsh” came up as the first option for the ethnicity question. For similar questions for US audiences, try the Census Bureau website. When conducting a survey for a project that has a global audience, you need to consider who your primary audience is. For example, I recently created a questionnaire for a global news website. A large proportion of its audience is based in the USA, so I was careful to word things in a way Americans would find familiar. I used the US ethnic background census question wording and options, and looked at data for US competitor news websites to decide which to include. You should also consider people whose first language isn’t English. Working as an audience researcher at BBC Wales, every survey we did was bilingual. I’ve also recently run a user survey in Arabic using Google Forms. During this project, we found that while Survey Monkey supports different languages, including Arabic, the text ran left to right with no option to change it to right to left – an essential when it comes to reading Arabic! If research is a deliverable in a client project, and you know you’ll need to conduct it in a foreign language, always build in extra time for translation at both the questionnaire design and analysis stages. Make sure you also allow for plenty of checks. In this case we had to change to Google Forms after initially creating our survey with Survey Monkey to get the functionality we needed. Look and feel Think about the survey as another way your audience will experience your brand. Take care getting the tone of voice right. There are plenty of great articles and books out there about tone of voice – try Letting Go of the Words by Ginny Redish for starters, or Brand Language by Liz Doig. The basic rule of thumb is to sound like a human, and use clear and friendly language. If, like me, you are lucky enough to work with journalists or copy editors, you should ask for their help, particularly in the preamble, linking text and closing statements. I find it helpful to break my questions down into sections and to have a page for each. I then have an introductory piece of text for each section to guide the respondent through the survey. You should also make sure you check with your designers how your survey looks – use a company logo and branding, and make the typography legible. Many survey apps like Survey Monkey and Google Forms have a progress bar. This is helpful for users to see how far through your survey they are. I generally time the survey and give an indication in the preamble: “This survey will only take five minutes of your time.” You also need to think about how you will technically serve the questionnaire. For example, will it be via email, social media, a pop-up or lightbox on your website, or (not recommended but possible) in an ad space? Ethical considerations Something else to think about are any local laws that govern how you collect and store data, such as the Data Protection Act in the UK. As a member of the Market Research Society, I am also obliged to consider its guidelines, but even if you’re not, it’s always a good idea to deal with personal data ethically. If you collect personal data that can identify individuals, you must ask their permission to share it with others, and store it securely for no longer than two years. If you want to contact people afterwards, you must ask for their permission. If you ask for email addresses, as I did in question 18, you have a ready-made sample for a further survey, interviews or focus groups. Remember, you shouldn’t survey people under sixteen years old without the permission of their parents or legal guardians, so if you know your website is likely to be used by children, you must ask for verification of age early on, and your survey should close someone answers that they are under sixteen. The ESOMAR guidelines for online research [PDF] are well worth reading, as they go into detail about such issues, as well as privacy guidelines – using cookies, storing IP addresses, and so on. Tools Unless you work in-house and have proprietary software, or at a market research agency and you’re using specialist software such as Snap or IBM SPSS Statistics (previously just SPSS), you will need to use a good tool to run your survey, collect your responses and, ideally, help with the analysis. I like Survey Monkey because of the question bank and analysis tools. The software graphs your results and does simple cross-tabbing and filtering. What this means is you can slice the data in more interesting ways and delve a bit deeper. For example, in the Gridset questionnaire I mentioned earlier, I cross-tabbed responses to questions against whether a person worked in-house, for an agency or as a freelancer. Other well known online tools that I also use from time to time are Wufoo and Google Forms. Smart Surveys is a similar service to Survey Monkey and it’s used by many leading brands in the UK. Snap Surveys mentioned above is a well-established player in the market research scene, used a lot for face-to-face surveys and also on tablets and smartphones. Analysis Analysis is often overlooked but is as important as the design of the questionnaire. Don’t just rely on looking at the summary report and charts generated as standard by your form or survey software. Spend time with your data. Spend at least a week now and then if you can, looking at the data. Keep coming back to it and tweaking or cutting it a different way to see if there are any different pictures. Slice it up in different ways to reveal new insights. Here is the data from my dummy survey (apart from the open-ended responses). For open-ended questions, you can analyse collaboratively. Print and cut out the open-ended responses and do a cluster analysis or affinity sort with a colleague. Discussing the comments helps you to understand them. You will also find the design team are more likely to buy into the research as they have uncovered the insights for themselves. Always make sure to treat open-ended responses sensitively and don’t share anything publicly in a way that identifies the respondent. Write a report Never hand over a dataset to your client without a summary of the findings. Data on its own can be skewed to suit the reader’s needs, and not everyone is able to find the story in a dataset. Even if it’s not a deliverable, it’s always a good idea to capture your findings in a report of some sort. Use graphs sparingly to show really interesting things or to aid the reader’s understanding. I have written a quick dummy report using the data from the Christmas questionnaire so you can see how it’s done. I highly recommend Brian Suda’s book A Practical Guide to Designing with Data for tips on how to present data effectively, but that’s a subject that benefits a whole article (indeed book) in itself. I am not a designer. I am a researcher, so I never write design recommendations in a report unless they have been talked about or suggested by the designers I work with. More often, I write up the results and we talk about them and what impact they have on the project or design. Often they lead to more questions or further research. So that’s it: a brief introduction to using questionnaires for design research. Here’s a quick summary to remind you what I have talked about, and a list of resources if you’re interested in reading further. Top 10 things to remember when using questionnaires for design research: Start by auditing existing research to identify gaps in data. Write a research brief. Work out exactly what you’re trying to find out – what is the survey about, and what is it not about? The two basic kinds of questions are open-ended and closed. Closed questions limit responses by giving the respondent a number of predefined lists of options to choose from (multiple choice, rating scales, and so on). Open-ended questions are often in the form of a statement which requires a response. Always ask one at the end of a questionnaire. Always include simple demographics to enable you to compare your sample against the population in general. It’s best to use official census or professional body wording for questions on ethnicity, disability and religion. Be sure to think carefully about your tone of voice and the look of your questionnaire. Pay attention to guidelines and laws on storing personal data, cookies and privacy. Invest plenty of time in analysis and report writing. Don’t just look at the obvious – dig deep for more interesting insights. Some useful resources for further study Online research Design Research: Methods and Perspectives edited by Brenda Laurel Online Research Essentials by Brenda Russell and John Purcell Handbook of Online and Social Media Research by Ray Poynter ESOMAR guidelines for online research [PDF] Online questionnaires Market research books on questionnaire design Using Questionnaires in Small-Scale Research: A Beginner’s Guide by Pamela Munn Questionnaire Design by A N Oppenheim Developing a Questionnaire by Bill Gillham",2012,Emma Boulton,emmaboulton,2012-12-14T00:00:00+00:00,https://24ways.org/2012/using-questionnaires-for-design-research/,business