Feedback Add On
December 2015 - January 2017
Principal Designer & UX Researcher. I was the sole designer for this project soliciting feedback from our design team at our weekly critiques. In addition I organized and ran all the UX research conducted during the project. For V 2.0 a second designer was brought on to work on Club Portal while I focused on the module.
Netpulse's overall business strategy was what we called Land & Expand. We sold our base app with features such as check in, find a class, workout tracking, challenges, etc at a low price with the goal of dominating the market horizontally and snatching up as many locations as possible. With over 600 brands and 9,000 locations worldwide, the base product had found tremendous market fit and it was only a matter of time until we fully dominated the market, especially considering there were no competitors offering anything similar.
The second part of the plan, Expand, hinged on upselling our customers to add-ons that would add tremendous functionality targeted at one of the pillars of a gym's business. We defined these pillars as Marketing & Operations. In October of 2015, we launched our first Add-On mobile acquistion at 2.5x the cost of the base app. It was a strong first step and within the first year of operation was purchased by 8% of our user base. 6 Months post launch I was put in charge of all design for our second add-on, Feedback, targeting the Operational side of a gym's business.
Key business metrics for gyms:
There are two key metrics that a gym monitors to determine the health of their business.
- New member acquisition
- 90 day retention
New member acquisition is something our Mobile Acquisition add on targeted, to help gyms generate more referrals, guest passes and ultimately new member sign-ups. 90 Day retention is also incredible important, industry wide 80% of customers who join a gym cancel their membership after 5 months.
The conception of the Feedback Add-On was directed at helping gyms identify at-risk members and reduce churn, based on Uber & Lyft's ride reviews system our goals were:
- Member-side: Collect feedback on their experience at the gym, beginning only with "How was your last visit?" with the goal of expanding to be platform wide, where could ask questions like "How was your Personal Training session with Marcello?", "How did your Zumba class go yesterday?"
- Gym Manager: Take in all this feedback and create a platform where they could get real-time feedback on all aspects of their gym. Identifying at-risk members and being able to contact them after a bad review to turn a negative into a positive. Leveraging positive feedback to use on their website and in marketing materials as testimonials.
Design Objectives & First Iterations
With our directive given my PM and I got to work, we began by defining the following design objectives for both experiences:
Our trigger for the feedback ask would be based on a couple hours after they checked in to the gym. Which means it would come up to the user unprompted. It needs to feel light and easily dismissable, since we're essentially getting in the way of whatever the user was trying to accomplish.
We (too) quickly began to obsess over this idea of real-time feedback and response. We wanted to enable an experience where if I left a bad review telling you the showers were broken, the gym manager could call me within a couple hours to thank me for my feedback and let me know that they had a plumber on the way. In addition to this, we wanted to answer the question "How is the experience I'm providing my members?" by providing a map or a score of the health of one's gym across all it's different offerings. We saw NPS as our biggest competitor.
Feedback Module Exploration
In my initial exploration I went with a modal design so it didn't feel too intrusive (versus bringing up a full screen) in addition I wanted it to feel as delightful as possible, as if the manager of the gym geniunely wanted to know what you thought and taking inspiration from officevibe went for a very illustrative design.
club manager portal - gym 360
Early on we became, obsessed with the idea of creating a separate app that could be on any gym employee's phone, allowing them to respond to incoming feedback in near real-time regardless of where they were. We gave the app a nickname of Gym 360, branded it in Netpulse colors, (the first of our products to have Netpulse branding as an effort to increase staff awareness of our products). The two main design objectives of Gym 360 were to provide a dashboard showing the health of your gym based on reviews and a ticketing system allowing a staff member to contact the customer, resolve the ticket or assign it to another team member.
With the first cut of the designs completed it was time to move forward to beta test. Goodlife Australia one of our more tech-savvy brands agreed to run a beta test of Feedback in four of their locations. Going into the beta test, here's what were looking for.
- Is the modal intrusive?
- Where do they believe their feedback is going?
- What kind of feedback (positive and negative) is left?
- Do they interact with the modal everytime? Is the cadence right?
- How often do gym managers access the app?
- Do they receive enough feedback volume to justify the focus on real-time ticketing?
- Which tickets do they respond to?
- Does it give them all the information they need to address their problems?
Working with our front-end engineer we built a stripped down, mobile-web version of Gym 360. Here's what we learned after our month-long beta test:
- There were no complaints in terms of the trigger of he feedback Modal.
- "This made it easier for me to report issues to my gym, especially issues I didn't feel comfortable reporting in person."
- Gym managers did not use the app everyday, per location there simply wasn’t enough volume to warrant checking it daily.
- Unanimously all club managers wanted us to email them when there was a very negative ticket that came in.
- When there was an issue, managers did contact the customers, by phone or by email.
Coming out of the Beta test we had the following objectives:
- Clean up the Feedback Module
- Adapt the design of Gym 360 to fit into our web portal
- Create a robust email system to alert club managers when there was a review they really needed to address
I was able to combine two steps in the negative review flow to remove an extra step for the user, bringing in categories and comments into one screen. Since the feedback modal comes in unprompted it's important we get out of the user's way quickly. In addition, for positive reviews, after overwhelming demand from our gyms, we added the ability for a user to copy their review and post it to Yelp.
Leveraging the card concept we adapted the design to work primarily on desktop and to fit the style of our existing Admin Portal, used by gyms to manage the rest of their Netpulse app.
Launch & Reception
We launched Feedback in July of 2016, here's what happened:
- 19 Brands purchased within the 1st month (compared to 4 brands with our Mobile Acquisition Add-on)
- Active within 100 locations in the first month alone
- ~18,000 ratings within 3 months
- Chuze Fitness cancelled 2 months in
- AM's put a freeze on selling feedback after Chuze pulls out
The initial excitement and speed of adoption of the Add-On signaled a clear market need for the product, however with Chuze cancelling it was clear the product wasn't there yet. We got on the phone with all of the customers that purchased Feedback to.. get feedback on Feedback!
Distilling these down, here are the issues we wanted to address:
One of the more alarm complaints we got was that the feedback prompt would come up whenever members were trying to check into the gym via the app. This made sense in hindsight since for a lot of members, the primary reason for accessing the app is check-in.
Where is my feedback going?
When we spoke to members many of them weren't sure where their feedback was going, if anywhere, and what would be done with it. The app felt disconnected from the gym staff.
"What does 2, 3 or 4 stars mean?"
A lot of members either gave 1 star or 5, they had trouble describing what the stars in between meant.
Negative Ratings Without Comments
"When I get a one star review without a comment or a category it's absolutely useless to me, I have no idea if they're just having a bad day, or if there's anything I can particularly address, those ratings are useless."
"This data isn't as useful as our quarterly surveys"
We heard from a couple of our customers, it's an existential piece of feedback (on Feedback). We're competing with Listen 360, Qualtrics and more, with our deep integration into the app we should be surfacing way more actionable useful feedback for our gyms.
Only 18% leave text reviews
Reviews that contain text are the most valuable. They can be used as testimonials, or to point to something that went wrong that the gym can act on, 18% was a good start, but we needed to improve it.
v1.5 - Stop The Bleeding
The biggest reason for Chuze and several others cancelling or pausing their enrollment in Feedback was the complaints they received from members about the modal being too intrusive and getting in the way of them checking into the gym. Before we could address some of the more existential issues mentioned above we needed to do something, quick, to stop the bleeding, here's what we focused on:
- Reduce the cadence of asking for Feedback to 3x a week at a maximum.
- Provide an opt-out function, or an "ask me less"
- Update the design so it feels more optional/easier to dismiss and move on
Above is what we came up with, featuring these notable changes:
- A more transactional design that doesn't overtake the dashboard.
- More ways to dismiss the modal, if you don't have feedback to give you shouldn't be forced to. We included a countdown times that would dismiss the modal if you didn't interact with it. A prominent 'x' button, and we surface more of the dashboard instead of blurring it with an overlay.
- If you dismiss the modal 3 times in a row we ask you if you'd like to opt out or move to a smaller frequency.
Based on the feedback we received above, here's what we needed to address in version 2.0:
- Targeted Feedback > Quantity. Our superpower is the ability to integrate deeply with our app and ask gym goers very specifically about their experiences in a gym. We can get more targeted, higher quality feedback than any quarterly survey tool. So we made the conscious decision to forgo our "Real-Time" feedback approach and focus on getting more detailed, specific feedback.
- Re-architect the question we ask the users. We heard from enough gym goers that they didn't particularly know what the different star ratings meant. So we decided to focus on a more real-world metric, when we ask someone how their gym experience went they answer in one of three ways:
- It was exceptional, something really great happened! (Testimonial)
- The usual, everything was fine nothing to report here.
- Something terrible happened, what a bad experience! (Feedback)
- Don't make me type. When a gym goer has feedback to give they shouldn't need to type, with 18,000 reviews collected we could begin to make out the different categories and common pieces of feedback given and make them easy for a member to report in without having to tap into the comment box.
With these goals we laid out the problem:
I took a look at how other apps were requesting feedback, in particular interest was how they were getting the user to give them more specific feedback without relying on the comment box, using different input methods.
As I began ideation I storyboarded the experience into three seperate "Chapters".
At this point, with the problem laid out, clear goals, I began to explore with the goal of bringing a couple different ideas to a usability study where we could test the different interaction methods.
3 Potential Directions
For all the negative ratings that came in, the initial 3 directions explored were:
- Multi-Step Survey. One question at a time, in a modular format that can support up to 4 questions, customizable based on the trigger for the feedback.
- Full Screen Survey. When giving us a negative review, we assume you have something to say, pop up a full screen survey with three questions of varying input types.
- Chat Bot. A conversational chatbot that helps a user get to the bottom of what went wrong.
Here is the invision prototype which we took to Concept Testing:
Concept Testing Results
Going into the usability study we had high hopes for the chatbot, there was something that felt right about having the bot help get to the bottom of what went wrong, but, as concept testing can do sometimes, users overlwhemingly felt different. Here's what we learned:
- Chatbot felt way too long and tedious to get to a simple answer.
- Multi-Flow survey also felt very long, since we're asking users for feedback unprompted it quickly became clear that anything that involved multiple steps and more than a couple taps would feel like a heavy interaction to the user.
- Our full screen survey performed best, organized by categories with parameters pulled from the most commonly stated complaints in the 18,00 reviews we had received so far.
Final Iteration & Modal Deliverable
Armed with our findings from Concept Testing I set off to rapidly iterate to a final solution. Specifically in the iterations below I explored:
- Hierachy. The pieces to prioritize:
- Recipient of Feedback
- Top Level Category (Atmosphere, Instructor, Content)
- Sub Category (Temperature)
- Responses (Too hot!, Too cold!)
- Input Methods. Stars, sliders, prewritten answers, open text fields, etc.
We settled on a card design to represent the high level categories, to begin with there are three total top-level categories for a user to choose from. Each card exposes the sub-categories and contains a section for comments/photo upload (which are automatically tagged to that top level category).
A big goal of V2.0 was to give users a quick way to say what's on their mind without needing them to type. So I pulled all 18,000 reviews we had received. Analyzed them and generated the most common responses which are revealed under the sub-categories as a quick tap for a user to submit them. The more reviews we receive the more we can update these answers and add to them.
To address the "Where are my reviews going?" feedback we received in usability testing we allow you to choose to whom you'd like to send your feedback to. Giving feedback on a class you recently took? Send some feedback to the instructor, the club manager, or the facilities supervisor.
Give it a whirl below!
Feedback 2.0 was presented to our largest clients and given the greenlight to move to development in February of 2017.
Club Portal 2.0
While I focused on redesigning the modal, another Product Designer was brought in to redesign and add new features to our admin portal. Her and I conducted daily reviews and here's what we delivered:
Analysis & Ticket response
Club portal was completely redesigned to focus on analysis of all the feedback a club receives as well as ticket responding. The most important tickets (negative reviews with comments) are surfaced first giving you the ability to respond to them, other less important tickets are batched together.
Up until this point Feedback had a strong focus on triaging and handling negative reviews. While it gave our gyms the ability to receive testimonials and positive reviews, we didn't enable them to do anything with those positive ratings. In addition, from a sales perspective the product was only used to help prevent members from dropping out, leveraging testimonials would allow us to help gyms attract new members. Here are the two features we added to club portal:
- Testimonial Widget. We built a widget that a gym could customize to their brand colors and place on their websites that would should a live stream of the testimonial coming in through feedback.
- Social Media Campaigns. We designed over a dozen templates that would feature that week's highest rated class, instructor, or particularly well written testimonials to be auto-posted to Facebook. These templates were also designed to pull a gym's brand colors and logo.