January 2013 - June 2013
About the Company
Kicksend (YC S11) was a mobile first application that let you send, receive and print photo albums.
UX Designer & Research. I joined the company one month after a 2.0 relaunch and put in place a UX research process. I ran all usability studies and customer development research and assisted our lead designer with the UX.
When I joined Kicksend the company had just launched it's first photo printing product and had no customer research process at the time. My role was to create a UX research process that brought customer feedback to every product decision we made. I was also in charge of creating low fidelity wireframes that would inform the final design.
After having set up our two main research frameworks (described in detail below) I was challenged with increasing the research velocity to allow us to run two separate usability studies a week. This meant recruiting, prototyping, interviewing, and synthesizing results twice a week. I put in place a couple different processes to allow us to do this, by the end of my Co-Op I was interviewing up to 22 users a week (in-person), had conducted over 300 usability studies that directly lead to dramatic improvements in mobile retention and conversion rates from the single digits to over 40%. I left behind two separate research processes that were used up until the company was acquired two years later.
Exploratory Research Guidelines
When I first arrived at Kicksend they had no research processes in place for exploratory or usability research. It was my job to create the right research frameworks for the company as well as tweak them to make sure they kept up with the pace the company was working at.
When I first joined we were trying to better understand how we could target particular demographics (teachers, scrapbookers, etc). Using a lot of the principles from the Human Centered Design toolkit as well as the concept testing I had done back at Zeo I created an Exploratory Research Framework that would allow us to recruit, test, and synthesize results with a week turnaround on any new concept that we wanted to explore. Here's how we did it.
1. The Design Challenge
The design challenge is one sentence, phrased in "human" terms with a sense of possibility, that will guide the rest of the questions asked throughout the research.
- Identify criteria for the design challenge: Does it need to explore an existing initiative/product? Are we trying to better understand a certain demographic? Are there customer behaviors/problems we’re trying to learn more about?
- Make a list of all the challenges/questions we have with PostIt notes on the board.
- Rephrase those from the customer’s perspective.
- Select the top three.
- Determine the Design challenge with input from key stakeholders.
- Write a one sentence Design Challenge that will guide the team for the rest of the research.
Design Challenges We've Used Previously:
- “Create a better way for families stay in touch over long distances”
- “How can we help people showcase their most precious memories”
- “Build a product that will help people share and remember important events in their
- “How can we help people decorate their homes?”
2. Recognize Existing Knowledge
At the start of every research phase it's important to document what we already know so that we can focus on what we don't know.
- Write down the Design Challenge on the whiteboard.
- Write down all the things we know about the design challenge on postits, based off
- of previous research, feedback, what other products are doing in this area.
- Identify similarities and strengths of understanding.
- Write down what you don’t know and what you need to learn about the design
- challenge in a seperate column.
- Group the notes into common themes.
- Where are the biggest needs for research?
- How should the recruiting strategy be tailored?
- How is this going to inform the interview guide.
We come out of this phase with solid questions that inform the recruiting strategy.
3. Create The Recruiting Strategy:
The recruiting strategy is the description of the desired participants we want in the study will be. The recruiting strategy should include: the desired age/demographic, the number of participants and why, the desired spectrum of participants.
Choosing the Participants: Extreme participants help to unearth the unarticulated behaviors, desires and needs of the rest of the population, but are easier to observe and identify because they feel the effects more powerfully than others. By including both ends of the spectrum as well as some people in the middle, the full range of behaviors, beliefs, and perspectives will be heard even with a small number of participants.
- Define both ends of the spectrum and develop the spectrum on which to recruit.
- Identify relevant locations from which to recruit the desired participants.
I conducted the recruiting on Taskrabbit, Craigslist, Amazon Mechanical Turk and from a company database of 50+ in-person beta testers that I had created.
4. Choose The Research Method
Based on our Design Challenge we had to choose the right research method that would best explore the answer. Here were the different kinds of methods at our disposal:
- Individual Interview: The individual interview involves interviewing one participant
- at a time and enables use to do a deep dive into their behaviors.
- Group Interview: Interviewing a group of (48) people at once. Great at gaining an
- understanding of a community quickly. Group members build off each others answers and help paint a broader picture of the community as a whole. Not good at gaining deep understanding.
- Expert Interview: Interviewing someone who can be considered an “expert” in the field. I.e a professional photographer, photo printing plant operator. This is a great way to gain a better understanding at the upcoming technologies and trends that are coming up.
- Seeking Inspiration in New Places: Looking at similar experiences in different contexts, instead of focusing too narrowly on the research topic. The simple act of looking at different contexts can bring to mind new ideas. I.e studying the IKEA showroom to get inspiration on how to help people decorate their homes with photos.
5. Create The Interview Guide
For each research phase we created two different documents. The Interview Guide which serves to guide the discussion with the customers, including all of the context setting and questions that will be asked during the research. The Notetaking Document is used by the notetaker to capture everything that the participant says. This was usually an excel file with a different cell for each question.
The guidelines I created for each interview:
- The most powerful insights come from user stories, push them to tell you stories.
- Always open up with warm up questions and engage the user in conversation. The more comfortable they feel the more they’ll open up.
- Keep your opinions out of it. If they tell you something you disagree dig deeper into it.
- Always ask followup questions. If you start noticing themes be sure to prod further to understand them better.
Synthesis is the process by which we review all of the data and the notes that come out of the interview. It’s where you go from inspiration to ideas, from stories to strategic direction. We're looking to find all the common signals that were given out throughout the interview process.
- Using postits write out all the different answers of the study and put them up onto the whiteboard.
- Identify and group all postit notes by themes, pay very close attention to all the stories told.
- Write a key statement that summarizes each group of themes.
- Write out a 12 page summary of all these themes, including some of the more powerful customer stories.
View the full Kicksend Exploratory Research document here.
UX Research At Kicksend
During my time there Kicksend there was a company wide focus on becoming a much more customer centered company. We wanted every new feature or iteration we built to have been tested and validated with customers. It was my job to create the right research framework to provide that validation at the speed that we needed it, which is why I put in place two usability research rounds a week. Some of what's below is taken out of the UX Research At Kicksend document hence the change of tense.
The framework I created had us running two research rounds a week testing every new feature or iteration in each of the rounds. This dramatically improved the quality of the product that we shipped.
- Round 1: Scheduling participants for round one should begin Friday afternoon by emailing out our current list of UX participants. Taskrabbit tasks should be put up both on Sunday Morning as well as Monday morning.
- Round 2: Scheduling for Round 2 should begin on Tuesday night with an email out to the participant list. Taskrabbits should be set up on Wednesday and Thursday mornings.
- Wednesday: Wednesday is your day “off” it’s when you can work on updating the Flinto mockups, helping the design team with any balsamiq work or animating any new screens that need to be tested in the next round.
We have two research rounds a week, Round 1 spans Monday and Tuesday while Round 2 spans Thursday and Friday. There are several reasons for this structure:
- Having set days for research allows you to pace yourself and make sure you have time to recruit good quality participants.
- It sets a pace for the design and product team to have certain specs/wireframes done by then.
- It will prevent you from scrambling to find participants which can lead to lower quality studies.
Conducting Usability Studies
Ideal preparation for a Usability Study should go like this:
- Read the specs carefully and ask the Product/Design team if you have any questions.
- Take the existing screen/mockups and make them interactive.
- Spend some time looking at how the current iteration is performing and where there are dropoff points. (If applicable).
- Create a short list of things you have doubts about. This can be informed by pieces of the flow you’re skeptical about, areas the designer is worried about or data about how that specific piece is currently performing.
- Identify the right people to get feedback from.
I created this document at the end of my co-op while training the UX researcher that would take my place. When it came to actually conducting the guidelines and advice I passed on to her based on what I had learned.
Things to remember:
- Usability Studies MUST be conducted in person. If you can’t see the participants eye movement, hand movement and body language then you’re missing out on a ton of data.
- From the moment the participant arrives to when you begin the study you should be focused on one thing: Making them feel comfortable.
- You need to make sure the participant understands that they are NOT being tested. They are there to use our products, critique them and give us their honest feedback.
- The best way to do this is to joke around with them, ask them about their background, how their day is going, etc.
- Pay attention to their body language and their hands specifically. If you see their hands shaking or sense them being nervous it’s okay to go off script ask them personal questions to help them relax.
- Always begin the study with our ground rule: “We have one ground rule for today, as you go through the product try to think aloud for me. If you’re not sure where a button goes, if something is ugly please say so. Be as honest and critical as you can, that’s what we’re looking for. My job is not affected by anything you say so be as critical as possible.”
- Tell them only what they need to know about the app to begin with. I.e if they’re coming in to test Home Delivery provide context like this: “Kicksend is an app that allows you to take the pictures off your phone have them printed and shipped to yourself, your family members and your friends. They receive an envelope of photo prints straight to their door.”
- While in the Usability study pay attention to what they tap on, and where they click. Ideally you can replicate the exact order they did things in when reporting results to the designer.
- Your notes serve one purpose: to remind you where anxiety points and confusion came up. They should be clear enough that you can understand what you wrote and compile the results later on. But at the same time they should be concise enough that you’re not breaking the flow of the study and are able to keep paying attention to the user/asking them questions.
- Some participants use products by tapping through everything rapidly and seeing how things work. Let them do this and pay attention to where they’re tapping and once they’ve reached a certain point it’s okay to bring them back to a screen and ask them to critique it further.
At the end of each research round we reported results via an email to the entire company. This kept everybody informed on where the state of usability of our product was, as well as how new features were being received. Most importantly, everybody was hearing customer quotes about our product on a bi-weekly basis. This sometimes also surfaced areas we needed to improve outside of product and design, if for example a feature wasn't introduced or marketed well.
Here is a sample email I would send out:
Here are the results from Round 1 of testing this week! We tested Home Delivery on the web.
- 5 out of 5 participants brought up cropping while in the uploaded photos screen.
- "There are basic functions I'm used to seeing on photo printing sites like Wallgreens and one of them is cropping."
- "I need to be able to crop photos. I know that they are always cropped when they are printed on 4x6's, I want to have control over that."
- Suggestion: The fact that this came up in every single interview today tells me that it's a user expectation to have cropping when ordering photos on the web. We should consider doing this as soon as we can.
- 2 out of 5 participants clicked on the "Order 10 prints or more for free shipping" banner on the landing page expecting it to be a button.
- Suggestion: Consider modifying the design of that banner so that users click on "Get Started" instead.
- 3 out of 5 participants were confused about the shipping details in the "Choose Addresses" screen.
- "I don't care about most of these bullets. I expect a sturdy package, I'm assuming someone must be processing my order. The only important thing you tell me is that it'll take 3-8 days to be shipped to me."
- "How much is shipping? Since I qualified for free shipping does that mean it's free to ship to multiple people as well? That'd be pretty cool.."
- Suggestion: Remove extraneous copy other than the shipping estimate. If they qualify for free shipping we need to mention that it's free to ship to anyone that's a big incentive.
- 4 out of 5 participants were thrown off by the "Pick up photos in store" copy.
- "So here you tell me that it'll take 3-8 days to get my photos and then you give me the option to get photos faster by sending them to a store. Of course I'm going to print them to a store!"
- "This pick up in store is right next to the shipping details, this doesn't make sense it looks like they belong together."
- "I've chosen the recipients above and now you're telling me that I can send the photos to a store? How does that work? Does it look for stores near my recipients and then force them to go and pick it up?"
- Note: Having this option on the screen really threw users off. We should think through what the right place for this is and how to position it.
I will be extending this round of research into tomorow as well so more results will follow.
View the full Kicksend UX Research Framework here.
While I was at Kicksend I learned that UX research was crucial to the success of a product. Our checkout flow dropout rate went from >30% to less than 10% in the months that we implemented the UX research framework. I learned how to create a solid round of UX research that could get us answers within a day or two and not within weeks. While I loved conducting the research, I found that I got the most joy out of prototyping and designing, after my experience at Kicksend I began to look to for more opportunities to delve deep into product design.