This article only gets two stars, will you read it?

5 stars, Let’s go with this. 4 stars, not sure anymore. 3 stars, something is fishy. 2 stars, forget it.

Does anyone really know what the stars are all about? What are they really rating?

The mechanism behind why we easily get influenced by the shimmering stars is called Social Proof. Social proof is what makes us think we make rational decisions when buying that toaster as other customers have said that it’s a wise decision. Therefor we feel good about ourselves and our choice. Using the Likert-scale (or similar) is a quick, comfortable way of getting an overview when comparing products or services. We also take a trusting comfort knowing that it is not the producers or service providers that makes the rating but rather the actual users which should add credibility to the product.

But is it really always an accurate system, can it get out of control and can it be enhanced?

Writing about this phenomena came to me when I recently purchased an online service. After a few minutes of tinkering I was in need of support. I wrote the developers who were actually really quick with replying. I said thank you and what followed seems to become more common when doing online business today:

–No problem, we want to give you good service. Can you please give us a rating?

Sure, I thought. They did provide me with good help. But then again, I’ve had this service for like 15 minutes. Can I really rate it without trying it out or comparing to others? I therefor selected 4 stars. After a few minutes the following mail conversation took place:

-Hello again, we noticed you only put 4 stars. Is it something we can improve? Is it something you didn’t like?

I wrote back:

-I just don’t have anything to compare to and not sure what I can expect from your service. Since you asked me so early on I couldn’t really give a 5 star rating yet.

Then came the reply:

-We as developers work really hard and need that 5 star rating. Can you please reconsider?

The best part is, I did change it. It felt uncomfortable and almost a bit extorting. Why? I paid for a service and therefor a good customer to their business. But as soon as I didn’t give 5 stars I wasn’t that good of a customer anymore, I was more of a tool in their marketing business.

I thought it was a strange behavior coming from a business to a customer but didn’t think much of it as it might have been an isolated incident until I went on a trip. During the trip I was asked in almost every diner and hotel if I could rate them and on every front door there were stickers that yelled that Tripadvisor users approved them. When I got back home I decided to actually rate a few of the places I actually liked. I went on to Tripadvisor and searched for my hotel and started to look at some earlier review to see if I could identify that others good experience where similar with mine. Almost everyone gave 5 star but there were a few that gave lower. I particularly looked into one of those.

This customer described her visit as very pleasant and everything was nice but she only gave 3 stars. The owner replied with something in line with:

-Thank you for your review but your rating does not match your described experience. You are not helping us, you are actually doing the opposite.

This, combined with a pretty good episode from the show Black Mirror made me want to look further into this world of ratings. I was surprised to see that this happens all around the web. Airbnb sends warnings to hosts if guests leave to many stars below 5 in a row even though the written reviews themselves are top-notch.

What is an online rating? If her text described the experience as very nice but her choice of stars said something else. Why use 1-5 when everything under 5 is terrible.

But are ratings that important?

Well, according to a study made by Yotpo, 94% of all purchases are made for products with an average rating of 4 stars and above (Social Proof at work). The products that tend to receive a review are also the ones with better rating. This means, contrary to popular belief, that people rather rate products they are pleased with. This also mean that everything below 4 stars is not selling very well, and yet, there are three more steps down. No wonder the businesses are chasing customers that give too few stars.

What is being rated?

Is it the customer service? Prices? Delivery services? Quality? The general feeling of the business?

The last example comes to mind when Grand Hotel in Stockholm allowed an extreme right-wing party to host an event there. A war broke out on their Facebook page where protesters gave them 1 star and their supporters gave 5 stars. In this case it was one action from the company that made people react this way. Grand Hotel later removed the rating function from their Facebook page.

Some sites, like Booking.com or Airbnb, actually specifies what is being rated.

In their cases it’s facilities, location, cleanliness etc. This I think is a good thing (Not sure how to rate Free Wifi from 1-10 though) as we are getting closer to understand what the stars is all about. We rate things differently. A movie is of course rated differently than a toaster. 

The game distributor Steam goes with a two choices rating system when reviewing a game you own. Do you recommend this game? Yes or no. Together with a review that’s enough. The system then counts the most votes and can for example say that a game recently got most Negative reviews.

Real or not?

According to some sources review sites like Trustpilot are riddled with people taking money from business to have a full time job of rating 5 stars. There are Chinese click farms dedicated to this purpose. Yelp has apparently made fake reviews into a business by extorting business owners.

A lot of sites have implemented a Certified customer function that let potential buyers know that the review is from a real customer. How that works is not really specified though. In a lot of cases the customer can still be totally anonymous when rating. There’s a balance between integrity for the reviewing user and trust in ratings. Sites like Reviewmeta try to tackle this by analyzing reviews as they tend to follow a certain pattern.

So, got any ideas?

Customer Journey stories are probably better than stars. This is why many businesses with landing pages always have an image of a customer together with a success story. It actually raises the conversion-rate. The easy, comfortable overview on review sites would suffer a bit but the customer could make their own judgment on what part of the business/product/service that is being specified in the text. Steam is also on track with just a two way rating system where a review only can be written once you own the game. Also, no one should be able to give stars without a proper review attached. This also give business owners a chance to reply instead of having to ask the customer why he gave a low star rating.

Sure, replies to bad reviews should be handled by someone with a little communication skills in the public relations area (so we can avoid that infamous Amys Bakery meltdown review replies). However not everyone have those skills, the time, or can afford someone that have both. Should they get off the net because someone didn’t like that they were closed on Sundays at 9 pm, the product was delivered one day too late or maybe some driver didn’t offer a water bottle and therefor decided to give the business (or person) a one star rating. This is also why the rating system used should match what is being purchased like booking.com does for example. Another way too address the issue is to put the control in the hands of the user. Say a user could sift through the reviews and hide the ones that are not relevant. The system could then show the new rating based on the reviews that the user did not hide.

Small design changes to the star-rating system can also enhance what people actually think. You can for example, use a color scheme from red to green instead and show the distribution of opinion. Let’s look at what a toaster rating would look like with that when we let a customer get to answer a few question specifically for that product with steps from Fully agree to Fully disagree. In this case we have three questions (even though I have no idea what to ask about when talking about toasters). Then display results with with same color scheme.

Letting users rate business (or people for that matter) is a good thought but at the same time we need to be aware of that we are heavily influenced by Social Proof. Therefor, as we seem to be able to rate more and more things (like drivers and employees) we need to be able to count on what we read is accurate and true. Also, business can’t rely more on their reviews and push for more stars than trying to improve the actual product or service. Why not? Mainly because the rating system then will lose its meaning. Because in the so called Goodhart’s law it is stated that: “When a measure becomes a target, it ceases to be a good measure.”. 

Update

Since this post, Netflix has decided to change their rating system to look more like the kind Steam uses.. yay.