At the end of Part I, I promised to talk about some results I’d found relating to the influence of reviews on sales. Well here it is, and it’s a beauty.
The research is titled Estimating the Helpfulness and Economic Impact of Product Reviews: Mining Text and Reviewer Characteristics.
Put into layman’s terms, the researchers were looking for ways to reliably predict :
a) the types of reviews that would influence customers to buy,
b) and the types of reviews that would help merchants to sell.
I won’t pretend to understand the methodology, or the statistics, however I think it’s important to look at what the researchers were studying, and where. The ‘what’ included three categories of popular consumer goods :
1. Audio and video players (144 products),
2. Digital cameras (109 products), and
3. DVDs (158 products).
At first glance, these products don’t look terribly relevant to authors. However if you look more closely at number 3 [DVDs] you will realise that reviews of DVDs are exactly like reviews of books. Both describe the users’ personal experience of the content. As such, the results have direct relevance to us as authors/marketers.
Now for the interesting part, the ‘where’. The data, which was collected between March 2005 and May 2006, all came from Amazon! I told you this was good.
In terms of the results, I’ve cherry-picked the bits that would be of interest to authors. They include the following :
1. the perceived reliability of reviewers affected how their reviews were received by customers. Indicators of reliability included information such as ‘real name’, geographic location, and the badges provided by Amazon. These badges include the reviewer’s ranking – e.g. Top 10 Reviewer, etc. In other words, customers did not rate anonymous reviews as highly as those from ‘real people’.
2. objective reviews worked better for functional products – such as digital cameras etc – while more subjective reviews worked for DVDs, where readers wanted to know what other people thought of the content.
To me, however, the truly interesting part of the study is found in this sentence from the conclusion:
“Based on our findings, we can identify quickly reviews that
are expected to be helpful to the users, and display them
first, improving significantly the usefulness of the reviewing
mechanism to the users of the electronic marketplace.”
As soon as I read ‘…and display them first…’, I had an ‘ah hah!’ moment. You see, once I started looking at Amazon reviews more closely, I noticed that the order in which reviews are displayed is determined not by star rating or date, but by how many customers found the review ‘helpful’.
The screenshot shows the reviewer rating feature that appears beneath each published review on Amazon. A perfect score would be something like 3/3 – i.e. 3 customers found the review helpful and none found it unhelpful. A 3 out of 4 would rate lower because I response was negative.
Now, although no one knows for certain how the Amazon algorithms work, I suspect Amazon may use the methods developed in the study to track the influence of reviews against sales, and display them accordingly. In other words, the reviews you are most likely to read will be the ones most likely to make you buy.
Taken all together, the results do seem to validate Paul Drakar’s idea that higher ranked reviewers will be more influential.
In the final part of this series I’ll be looking at Drakar’s strategy for enlisting the help of top ranked Amazon reviewers, and asking the question ‘Do you have the chutzpah to try?’
Very interesting discovery! Good research; thanks for digging into this.
You’re very welcome Melissa. Once I started digging, I couldn’t stop. I truly would love to be a fly on the wall inside Amazon, to see how it /really/ works.
A.C., I read Draker’s post and I believe there points to be gleaned from his strategy. It is incredibly lengthy, as a writer I don’t know where I’d find the time to follow his step by step. Personally, I spent one week researching top Amazon reviewers who actually had a contact email, leant towards my genre, wrote polite queries acknowledging I spent time finding out a bit about them, and nada. To be honest don’t expect a reply, but it could be my work wasn’t suitable for them?? I don’t think that there is a one size fits all to obtain a review. Chutzpah and patience, the combination might work.
Thank you for this informative post, I’m looking to the finale.
I had a niggling sense of unease when I read Drakar’s article, that’s what started me on this bit of sleuthing. I suspect my my conclusions will be very similar to yours. 🙂
This is fascinating stuff. Now I wonder if we can actually influence our number and ranking of reviews.
lol – I think the answer is ‘maybe’. 😀
Interesting stuff, AC. Right now Amazon uses the votes to decide which reviews to display and in what order on the initial product page. What their exact algorithm is, I don’t know (for example the page I’m looking at right now has a 30 of 31 first and a 20 of 20 second). But I think we can surmise that it is a combination of the number of helpful votes and the ratio of helpful to not helpful with some weighting and possibly other factors that aren’t as obvious.
If you want to see more reviews there is a link to see all reviews at the bottom of that section and it displays newest first, but with two at the very top, the top ranked positive review and the top ranked critical review.(1, 2, or 3 star, if I’ve figured things out correctly).
Having said that, what I find interesting is the research you reference indicates that they are able to tell with some degree of accuracy which reviews will be most helpful. Amazon uses the up and down votes on reviews for two different things that I’m aware of. Ordering of the reviews, as discussed, and for reviewer rankings. (It is those votes with a few other factors applied to them that determine those rankings.)
So, in theory, Amazon *could* use the results of this research to order the reviews instead of the actual user voting. I know of authors who try to game the review ranking in various ways (for example get all their people to upvote a lukewarm 3 star and downvote a detailed negative 1 star to elevate the first to the “highest ranked critical review” and bury the second). I would imagine Amazon would rather not do that since those votes also figure into reviewer ranking and it gives the appearance of transparency in the process, but, as in other situations in the past when they felt too many people were gaming the system, if they thought that was happening they’d have an alternative.
Something I didn’t mention in part 2 is that the researchers tried to gauge the review to sales relationship by using publicly available data, and correlating from that. They did this to determine if their process could reliably predict sales.
Now I suspect that Amazon took the process and added real sales figures to the equation. I believe, but can’t prove, that part of the ‘hidden’ data that determines rankings is the result of this real sales data – i.e. a review that was helpful /and/ lead to an actual sale would have greater weight than one that was just helpful.
It would be really interesting to run a survey, asking readers how often they provide feedback on reviews, and if that feedback was always followed by a sale.
One thing is for sure, Amazon treats its top ranked reviewers like royalty!
Oops – Apologies Al, this comment was meant for you.
Interesting stuff, I was hoping for more info of how to find these top ranking reviewers for potential authors to ask for a review from them but we will see! thank you for this interesting read!
I can see you enjoy this sleuthing, AC, you may have missed a calling? You are revealing a lot of information but I’m not sure whether knowing more about the how and why of Amazon’s methodology is less or more frustrating.
Excellent article, so far, and makes one wonder what the climax will uncover.
A really interesting article – looking forward to Part III
I don’t always review (books & everything else) but now I’m reconsidering. Perhaps I can become one of the uber-reviewers? Strange thought…
Apologies – my pc literally blew up yesterday so I only had my old one to work with and that didn’t have any of these links on it. So I’m very behind. Going to catch up with your comments on Part III now. Thanks so much for persevering!