Stars in their eyes – why we changed our emoji ratings

We all know the importance of measuring the quality of your interactions with your customers.

Asking for feedback from customers helps managers spot and reward star performers and identify opportunities to improve the quality of support their team provides. There are numerous industry standards for this – Customer Effort Score (CES), Customer Satisfaction (CSAT), Net Promoter Score (NPS), etc. At Intercom, we built what we call “conversation ratings”.

When working on this feature over two years ago, we wanted to design a solution that was in line with our mission, making internet business personal. Thus, while we explored options such as simple thumbs up/thumbs down designs and five-star satisfaction ratings, it was no surprise that we were really drawn to using an emoji scale, with a row of faces – 😠 🙁 😐 😃 😍 – representing a five-point scale from “Terrible” to “Amazing”. The prototype was well received by both our own teammates and end users.

Managers felt it allowed them to better understand how their customers felt, and end users felt it was modern and fun, and offered clarity around what they were being asked. While most feedback was positive, there were some customers who felt it was too expressive or not professional enough for their business.

However, after thoughtful consideration of the pros and cons of each solution, we decided to pursue the emojis as our mode of conversation rating. As we put it at the time, “Using emoji faces felt expressive, modern and in line with the Intercom messenger style.”

Why were people reluctant to send 😍?

Before we shipped the feature, we expected customers to ask to allow them to customize or change the emoji set altogether. Surprisingly, when we carried out the research a couple of months later, this was not a common request. However, we did discover that 30% of the conversations we analyzed included a request to kill 😍. As avid users of this emoji at Intercom, we were a little confused by the request.

As time went by, it became apparent why our customers wanted to change the heart eyes emoji specifically:

Cody Keith@codykeith

Nothing like missing out on perfect customer service scores because @intercom uses , which most professionals find inappropriate.

Yo @intercom, any way to change that?

7:56 PM - Oct 9, 2018

Twitter Ads info and privacy

See  Cody Keith's other Tweets

View image on Twitter

MODassic@modassic

Not sure this customer support rating system by @intercom with emojis is a good idea. I’m not giving my support rep heart eyes but I would of given him 5 stars.

2

9:02 AM - Aug 1, 2018 · Salida, CO

See MODassic's other Tweets

Twitter Ads info and privacy

Customers were apprehensive to click on 😍 for two main reasons:

  • They only used this emoji in communication with a romantic partner.

  • Males did not feel comfortable sending it to another male.

The explanation was clear – the heart eye emoji was perceived as being inappropriate or having romantic connotations, which is probably not what you want to suggest when rating a customer support interaction. Indeed, the great thing about emojis is that they are very expressive in a succinct manner.

“This reluctance to use the 😍 emoji resulted in customer support agents and teams getting lower ratings than their performance deserved”

What we did not realize when choosing the scale was that the usage and interpretation of certain emojis would differ so much across cultures and even gender. There are even models to infer gender based on usage of emojis.

This reluctance to use the 😍 emoji resulted in customer support agents and teams getting lower ratings than their performance deserved, largely because of a design decision rather than any fault of their own. This trend was also noticed by our internal customer support team.

From 😍 to 🤩

We knew we had to change this particular emoji and after careful deliberation, we decided 🤩 was the perfect alternative. It was differentiated enough from the other emojis in the scale to suggest very high satisfaction, while communicating less passion and more excitement. Both our own customer support team and customers were unanimous in their appreciation. As one Intercom user put it: “Thumbs up for changing the 😍 to a starry eyed face 👍. Some of our customers have said they would have rated maximum, but the lovely eyes were just too much 😂”.

Beyond the positive reaction, the results we saw were staggering.

Looking at Intercom’s own Customer Support team, we saw an 11% increase in the number of “Amazing” CSAT ratings, almost all of which came from customers shifting from 😃 to 🤩.

 

Even more impressive was what happened when you looked at just the males on the Customer Support team, as this group saw a 17% increase in the number of “Amazing” ratings when we moved from 😍 to 🤩. This increase was enough to close the small gap between the rates at which male and female customer support representatives received 🤩 and 😃 ratings. This had a real impact on people’s perceived performance.

What we have learned

In text-based communication, emojis can act as substitutes for our body language and facial expressions, and can be used to convey emotion and clarify a message’s meaning. It was fascinating to find such a clear example of how, just like nonverbal cues, the usage and interpretation of emojis changes across cultures and, furthermore, how someone’s choice of emoji can be dependent on the gender of the recipient.

Above all, though, whether it’s expressed with stars or hearts or smiles, we love enabling great customer support and the very real personal interactions that can occur with thoughtful design.

5 strategies to drive your next wave of growth

You often hear great talks about the founding story and how to drive that first phase of growth. At SaaStr, I talked about what happens after that, how you can drive your next phase of growth.

When you are starting a business, or joining a new one, you are excited and full of ideas. You probably have a goal in mind. It might be to launch a new product, to define a new category or to hit $1M in revenue.

“What you thought was the finish line was actually just the starting line”

You work hard and get there, which is amazing! You hit your goal – you launched that new product, defined that new category, hit $1M in revenue. But it turns out, that what you thought was the finish line was actually just the starting line. It turns out there is a lot more work to do. How do you keep growing?

Over the years, I’ve learned a lot through both successes and failures, and these are my top 5 strategies in scaling SaaS businesses from $1M to $500M in revenue:

  1. Expand your market. Figure out how your product can solve the problem you originally solved, but for a broader set of customers. If you only serve your initial customers, you are stunting your future growth. If you start by serving smaller customers, the best way to expand upmarket is to do that one step at a time.

  2. Build your next big product. Your next big product is probably right in front of you. Find it by watching what your customers do, not what they say. Ask yourself: What other important problems do your customers have? Which could you solve well?

  3. Find product market fit again and again. It is something you need to keep focused on and find over and over, for every set of new target customers you go after, for every new product you want to create and for everything you’ve already created and every customer you already have, every year or two.

  4. Decide your platform play. We live in a deeply interconnected world. You will never win if you choose to go it alone. Your customers don’t want you to go it alone. They want you to work with all of the other things they already use. As you get bigger the reverse is also true: they want all of the other things they use to work with you.

  5. Build your team. You can’t do it all yourself. Invest time to build the team to take you to the next level. If you are hiring for a role that you aren’t super familiar with, or at a scale you haven’t been at before, learn and benchmark first. Then apply Lean Startup principles to hiring by putting people in the role. Have them do the job you want them to do for a day. That way, you both know what you are getting into – it isbetter for you and for them.