Most people gain value from reading reviews about products and videos and such online. We read a review of a book on Amazon.com and believe we can trust most of the reviews to provide a narrative that we either find helpful or not.
When it comes to product websites, and even some vendor sites like Circuit City, you get something less and you get something more. Usually the narratives are less detailed and informative, but you get a numerical rating which gleams respectability and validity. (To read about the truth of these numbers, however, read my concerns about the reliability and validity of online ratings in general.)
When we go to YouTube and see people rated a video 3 out of 5 stars, that tells us something instantly. Whether its reliable or not probably doesn’t matter so much, since a video isn’t likely to change any person’s life one way or another.
But when it comes to rating people on their livelihood, like a doctor (or CEO), should their entire life experiences and career be whittled down to a single number (or a bunch of numbers) that try and capture the wide range of variables in their career and professional lives?
Imagine someone who is an expert in the same field as you comes to your workplace, reviews your personnel file, and after a 30 minute interview, puts up a rating on an employee website about how well they think you do as an employee. Imagine employees rating their bosses, or managers rating their vice-presidents, or vice-presidents rating their CEOs. Or a company’s board of directors rating their CEO.
Yeah, we thought that might jog a few brain cells.
The Problem with Rating People on Their Jobs
The problem with rating individuals on their livelihood is that even if we were to spend hours with that individual, observing them do their daily routine, even for days at a time, we still wouldn’t be in a position to rate that individual on their work. Could you imagine someone boiling your work, your entire career, down to such a review? Very few people could. People have bad days. Heck, some people have bad years. We’re only human, after all.
We expect the law of averages to, well, average out the lows and highs in such online ratings, and that the “truth” will somehow be magically revealed by the sheer quantity of people rating a thing online. And for things, this may or may not be true, because there are literally millions of people who may own or use that thing. But there are not millions, or even thousands of people, who use or interact with a single person in their profession. Even a waitress only interacts with a few dozen people on any given day. Same with a doctor.
Grohol, J. (2007). Hey, It’s 2007! Let’s Rate Doctors!. Psych Central. Retrieved on July 30, 2014, from http://psychcentral.com/lib/hey-its-2007-lets-rate-doctors/000905
Last reviewed: By John M. Grohol, Psy.D. on 30 Jan 2013
Published on PsychCentral.com. All rights reserved.