Lies, Damn Lies, and Statistics

Earlier this month Stephen Hise gave his take on a survey of readers coordinated by bestselling author Marie Force. On the subject of reviews, Hise summarized a few of the survey’s findings with these words:

Reviews are important, but readers pay far more attention to other reader reviews on retail sites than to reviews from publications and review sites.

This seemed like a fair summary of the six survey questions related to reviews and their impact although I did have one nit to pick, which we’ll get to shortly. In the comments I saw this exchange between Hise and IU contributor Lin Robinson.

When I saw Lin’s comment my initial take was that his “professional reviews” comment would be interpreted to mean publications (like Kirkus, mentioned in Stephen’s comment) along with review sites not run for profit by book bloggers. Sites like mine. I liked Stephen’s response and had started expanding on why even if we take this survey and Stephen’s summary at face value that an author could easily use this to make what I think would be an incorrect decision.

My concern is that an author could see this and think it means he or she should focus their energy on getting reviews from readers while ignoring book blogs. I see this as a mistake. Here are some of the reasons.

A review on a blog is usually also a reader review

In addition to IU and my review site I run a second website, The IndieView, One of its main draws is a listing of Indie friendly review sites. An item we collect from each site as part of the listing is whether their reviews are posted somewhere other than their review blog. Of the close to 300 entries currently in our database, just shy of three quarters of them also post their reviews to Amazon. Many also post their review to other retail sites and reader oriented review sites such as Goodreads or Barnes & Noble.

Not all reviews are equal

We’ve all read 5 star reviews that contain a couple sentences about the plot (on Amazon this satisfies the 20 word minimum) followed by “I loved it.” The only thing worse is the 1 star review that is identical except for saying “it sucked” instead. These reviews help or hurt you with readers who are influenced by the “average review ranking,” but aren’t of much use otherwise. While there are many great reviewers who review exclusively on Amazon or other retail sites, I don’t think it is out of line to say that the typical book review site is going to have reviews that are more helpful to the reader trying to make a purchasing decision than a large percentage of the reader reviews.

A review from a book blogger helps discoverability

Stephen’s comment that he thinks “book bloggers and reviewers hold significant sway with their followers” is true. At least I hope it is. Readers who follow a review site tend to do so because they’ve found the reviewer’s taste and theirs align fairly well. A review blog also attracts traffic from Google and other search engines. A reader has to be on your book’s page on the retail site before they can read the reviews there. Amazon’s algorithms are great for that, but do you want to put all your eggs in one basket?

Author: Big Al

Big Al (who insists he only has one name, like Cher, Sting, and Madonna) spends his days writing computer programs that are full of typos, homonym errors, and incorrect verb usage. During his evenings, he writes reviews of indie books for BigAl’s Books and Pals and has recently taken over The IndieView, a website founded by indie author Simon Royle as a resource for indie authors, indie reviewers, and those who read either.

13 thoughts on “Lies, Damn Lies, and Statistics”

  1. Great post, Al. I, for one, am indebted to those book bloggers who took a chance on me in the early days and gave this struggling author an opportunity to reach readers. Now, I couldn’t imagine NOT working with bloggers (are double negatives allowed? I haven’t had enough caffeine apparently…)

    As for the paid reviews, I’ve heard they don’t move a lot of books no matter if you’re trad pubbed or indie (although, as with most things, your mileage may vary). It’s still paid advertising which sometimes works, sometimes doesn’t.

    1. Thanks for the comment, DV. I think there are people more influenced by reviews from Kirkus than from readers, but I agree, if you’re paying the big question is whether it is cost effective for the average author.

  2. I think Mr. Monk summed it up best when he said, “It’s a blessing, and a curse.” I’ve written a lot of reviews, and received a lot of reviews on Amazon. Most of the reviews you come across don’t really tell you much. They give the star rating and “I liked it” or “I hated.” Some of the reviews are really great, and go into a lot of detail about the book, and as an author, those are the reviews I like to see, whether they’re 3 stars or 5 stars. It doesn’t matter to me.

    I also get really excited when a “Vine Voice” author, or Top 100, Top 500, or Top 1000 Reviewer decides to review one of my books. It seems to give it that extra bit of credibility.

    What bothers me is when I come across a two or three day old book with six to ten reviews. It just doesn’t happen that way, unless your Steve Scott, or Steven King. Especially if they’re all five star reviews.

    I had an author contact me one days because I wrote a four star review about his book. He was having fits, because i ruined his reputation. His thought was because I authored a book similar to his, I was out to make him look bad. Other than the four stars, the review I gave was a glowing recommendation for his book. No matter, he was talking about crying foul to Amazon.

    I eventually took the review down. I don’t know about you, but I love it when I get 200 or 300 word reviews, even if they are four stars. They give people a new perspective on my book.

    What do you think?

    1. Thanks, Nick. I write reviews rather than receive them, so my thoughts might not be applicable, but based on the feedback I’ve seen reasonable authors would rather see an informative 4 star review than a 5 star review that doesn’t help a potential reader. Some of them object to informative reviews that are less than 3 or 4 stars though, although I’m not sure the issue there is how informative the review is or isn’t.

      As for authors who have what seems to be a lot of reviews right after their book is released, that’s often (although not always) because they arranged to have readers get their book in advance with the agreement to give an honest review on release. Given the universe they’re drawing from to get these (typically among fans of their FB page or people on their newsletter) you’d expect the initial reviews to be decent more often than not because these are people who have established that they are in the author’s target audience.

    2. Nick, your story about the author who protested the 4-star review amazes me. I had one reviewer take a few potshots at one of my books, but she ultimately gave it 4 stars and while I wished she could have been more positive in her review, I was happy with the star rating. I would certainly never complain about that! I do like it when the reviewers are fairly specific about what they like or don’t like. I think that’s more helpful to readers.

  3. It’s essential people look at the reasons for the three- or four-star review. I am a book blogger (so I do so hope authors value my reviews, or I might as well give up now), and one very miffed author was ‘surprised’ by my three-star review on Goodreads. How could I possibly have thought his book was ‘mediocre’? I pointed out that, according to the Goodreads star ratings, three stars was ‘I liked it’. Four stars was ‘I loved it’. I didn’t ‘love’ it, I just ‘liked’ it. But in my review, I ensured that the reasons I ‘liked’ it were clearly listed along with a short mention of why the four-star ‘I loved it’ and the much-coveted five-star ‘It was amazing’ just didn’t apply (in my opinion). There was enough praise, I think, in my review to encourage readers to give the book a go. I think it’s crucial to read the review.

    1. Cathy, I hate the star system. Even mention it right off the top in the explanation of my rating system. What is said should matter more than the ranking. Part of that is the significant difference between rating systems that (by being 5 star based) appear to be the same, but really aren’t. If someone posts to both Amazon and Goodreads and follows their respective definitions, the Goodreads ranking *should* be almost a full star less with the exception of some of the Amazon 5 star reviews. Different reviewers handle (or don’t handled) that difference in many ways.

  4. I read a lot and mostly find I enjoy what I choose to read, but in reality there are probably only one or two books a year which make me think ‘wow’ when I’ve finished them – and to me that would be the definition of a top rated book. Surely the average rating for the average novel should be around 3 which would indicate that most readers would enjoy it – some might think it was brilliant, some might not like it, everyone’s taste is different, but when you get to the point where everyone feels they have to rate everything highly it just makes a travesty of the system.
    And the problem with advance reviews from early readers from Facebook etc is that they are not going to be as unbiased as they should be – how many of those readers are going to feel as if they can post anything less than a glowing review without upsetting the author, no matter how much the author insists they want an honest review?

    1. Mel, thanks for the comment. I agree with you (at least to a point) on the advance reviews. I know I’ve not reviewed books in the past in that situation where I felt some obligation to do a good review because I wasn’t willing to give a good review to a book I didn’t think deserved it. Others might make a different decision.

      I think it depends a lot on the person and situation. I know that some of the sites that solicit readers to read and review books (Book Rooster, for example) generate their share of negative reviews. But generally speaking advance reader reviews are going to lean more positive for a lot of reasons. Bias in this case isn’t always going to be reflective of something unethical, either. For example, the point I made above that the pool of people solicited are normally going to be people who have read and liked the author’s previous books. Many negative reviews are not because of a fault with the story or writing in a book, but because it was a bad match for the taste of the reviewer. That scenario would be under represented among advance readers solicited from an author’s facebook page.

      As for your contention that an average review should be 3 stars, I disagree, although I see why you would think that. For reviews using Goodreads 5 star system, that is close (I’d actually expect the average to be higher than that. Maybe 3 1/2.) For Amazon, the average should be closer to 4 stars or a slightly higher because they define the meaning of the stars differently. One of my first posts for IU discussed the reason why I see it that way in my depth. Here’s the link if you’re interested.

  5. Al, Thanks for the link – it’s a great post (even as it brought back memories of the horrors of learning standard deviations!). And yes, you’re right that the average should be higher than 3 with the given systems, because as you say the reasons people choose particular books and review them is more likely to produce a skewed distribution in favor of the positive.

  6. On my blog, I review books that I have loved, and over the course of the last year and a half, many of the people who read my blog have grown to trust my reviews. This is what word-of-mouth is all about. Without that trust factor, I doubt any review will have much impact. However how, or what, you trust can vary quite a bit. I trust the book recommendations of my ‘kindred spirits’ in a way I probably wouldn’t trust a Kirkus review, or an Amazon one either.

    1. Absolutely, A.C. I describe book bloggers as amplified word of mouth. It’s the same process except instead of telling 2 or 3 or 5 people what they thought about a book they tell 20 or 200 or 2,000. But their followers know how well their tastes line up.

Comments are closed.