Letters to the Editor
-
You're dead wrong about Fred's.
Fred's is one of the best breakfast places around. Definately the best in Sausolito.
Were you coming off of amphetamines or something? That will make any breakfast taste nasty.
-
what about shills?
Many book reviews on Amazon and many movie reviews on IMDB have that "written by the PR dept" stink to them.
Restaurant reviews on various sites often come across like they were posted by friends of the owners.
I suppose there is no way to control for this.
-
Another factor in response bias
There's another factor which review sites don't seem to account for the passion of the reviewers. It's really easy to get good reviews of things that people are passionate about: cars, digital cameras, and food. But when I was looking for a small wireless router the reviews didn't tell me much, because not too many people care about routers other than whether they work or not.
I think reviews would be a lot better if people simply had to rate things compared to similar things they had tried.
Why not rate breakfast restaurants as better or worse than other breakfast places that you've eaten at in the same year. No more absolute 5 star system, simply better or worse than a set of cafe's. Purchaser's bias and response bias would be factored out to a certain degree.
-
Why is a five-star news magazine serving one-star design?
Three separate times I clicked the link for this article from the Salon main page before I finally grokked that I wasn't being misdirected offsite to some website titled "Machinist" that was simply affiliated with Salon - might Salon have given just a wee indication that they are (once again) revamping their magazine's design? Just to reduce any possible confusion?
Regarding the "Machinist" web design, the smaller font isn't kind to these ageing eyes, and neither is red-on-black or dark-gray-on-black text in the left nav column. Did Salon, perhaps, mean to title this new section "Machination" instead?
-
Bought It On Amazon
Farhad mentioned that you had to buy an item on Amazon before you could review it. That's not completely true - as long as you have an Amazon account, you can review any item, whether you bought it there or not. I've reviewed items that I bought elsewhere.
However, I generally don't bother to review something unless it was exceptionally bad or exceptionally good, so I guess I'm part of the problem.
-
4.5/5
Tasty topic, good presentation, meaty analysis. Interesting thoughts about the response bias... I've often wondered how many people simply put five stars automatically when they're -somewhat- satisfied, or conversely, scored an item/service at one or zero ("tihs suxs") when they're only mildly dissatisfied. The red-yellow-green meter thingy is intriguing. Hope to see that sort of thing around a lot more.
-
Yet another element of response bias
Is that people who buy something not only expect to like it, they WANT to like it. It's hard to say something sucks if you were really excited about it unless it really, really sucks, thus you get the 1 and 5 star breakdown.
-
Gaming the system
Reviews have always been a matter of trusting your fellow netizens. Early on, when web reviews flew under the radar of partisans, the reviews could be trusted. That didn't mean they were good or accurate, but they were (for the most part) someone's actual opinion. Reading their other reviews was a good way to assess their taste and talents. Not unlike movie reviews in the newspaper.
This is less true now, as businesses and their competitors have gotten wise. Reviews are still handy, but you simply can't trust the first few opinions listed. You have to dig deeper. Some sites have a "Trusted Reviewer" type category. Partisans are gaming the system, hoping to catch the unaware and credulous. Sometimes they do.
On a related note: Did you ever look at the spread of ratings on IMDB? No matter what the movie, no matter how good or bad you think it is, the scores are stacked at 10 and 1. Too often, a nuanced review takes too much brainpower, and it's "all or nothing". This is why five-star restaurants can get one-star reviews, and why greasy spoons can get five star reviews.
-
Response bias isn't all bad
In many cases reviewers should have the same bias as anyone to whom they'd be useful.
For example, suppose I were a fan of Jason Mraz and was interested in buying his new album. I wouldn't want to know what a random person on the street thought of it, let alone someone who thought Jason Mraz was crap. I'd want to know what other Jason Mraz fans thought, with expectations much like my own.
The album could be the darling of press critics but lividly distasteful to fans of his old material. Or vice versa. For anyone already interested in making a purchase, the biased reviews are the valuable ones.
-
The Rating Game
I have long been frustrated with Amazon's surpluss of 5 star ratings. My favorit genre is overflowing with barely readable tripe, but you would never know from the endless supply of over the top reviews, praising nearly every book that makes it into print as the most amazing book ever. I use to do a great deal of reviews, but gave up when my somewhat less than completely favorable reviews got sunk to the bottom of the list by rabid fans (or perhaps marketing people with a bottomless bowl of accounts) who marked my very detailed and thoughful reviews as unhelpful. I don't pay too much attention to the Amazon ratings, but I do find reading the reviews themselves is quite useful. I always sort tham in from lowest to highest rating and pay most attention to the two and three star reivews which tend to have the most useful feedback. I think Amazon and other rating sites would be more useful if the system could create a list of users who similiar tastes to you, rather than having to sort though random people whose criteria for liking or disliking something quite probably has little to do with you own. I don't really find their current method of asking you to rate something and than suggesting other things very helpful. Netflix has begun expermenting ways to narrow opinions to those that are likely to be the most useful to you.