Vaughan has, unsurprisingly, found no randomized controlled trials (RCTs) on “management techniques.” I say “unsurprisingly” because effective management of people in a work setting is not something that has been studied in that manner, since that’s not how researchers conduct studies in all fields and disciplines.
Not to say there isn’t research in this area, because, of course, there is. There’s an entire field of study called industrial and organizational psychology that studies how psychology works in organizations and business.
So I think it’s a bit unfair holding this field up to another field’s gold standard. Sociologists also generally don’t employ RCTs, nor do epidemiologists. Does that make their science any less valuable, legitimate, or useful? Of course not. These fields also offer evidence-based research, it’s just that their evidence is based upon something other than a RCT.
While I sympathize with the feeling that a lot of “management techniques” are nothing more than pop psychology repositioned for business use, you’re not going to win friends or influence people by suggesting their techniques are grounded in bs. I guess it really depends on what you’re looking to get out of a job or a career…
Read the full enty, In search of evidence-based bulls–t over at Mind Hacks.
(2 votes, average: 5 out of 5)
Loading ...You can leave a response, or trackback from your own site.
Links to This Article
Job Search: Build a Career, Job Listings, Employment » Evidence-based Management Techniques? (12/2/2007)
Economics Topics News » Blog Archive » Evidence-based Management Techniques? (12/2/2007)
6 Comments to
“Evidence-based Management Techniques?”
Evidence based medicine was thought up as a way to help doctors get up to speed with what was going on in research so that they could make better decisions (and not simply do what they’d been taught in med. school). To my mind evidence based practice is horribly misused when it becomes something like: you may only do what has been proven in RCTs - nothing else is permitted. I suppose it’s a way of trying to contain anxiety.
If, on the other hand, evidence based practice can 1. guide one to consider practices have been shown to work and away from those which have been proven not to work, fine.
Hi Jon,
My surprise is because RCTs are the single best way of determining the causal effect of any intervention on an outcome and the *only* one from which a causal effect can be confidently inferred.
There is nothing clinically specific about this technique, the logic is inescapable. To recommend one intervention over another, you should have evidence that it improves functioning in a similar group of people when compared to the alternative.
Sociologists and epidemiologists don’t employ RCTs because they don’t research the effectiveness of interventions, whereas organisational and clinical psychology’s main raison d’etre is to intervene to improve functioning.
So, I’m not suggesting organisational psychology is bullshit, just expressing surprise that they’re not using the best tools available.
A hammer is no good for driving in a screw, and RCTs are not particularly the best tool to judge organizational or leadership effectiveness.
Your statement that the RCT is the “single best way of determining the causal effect of any intervention” with no qualifications suggest a somewhat narrow reading of research methods. It is far harder to conduct realistic, real-world RCTs that answer the research question and don’t make things so artificial that the results are generalizable. I’d love to hear how this could be done in a work setting (without any type of negative impact on the actual business).
And if you don’t think sociologists and epidemiologists, as just two examples, don’t research the effectiveness of interventions, I suggest a more in-depth look into those fields. They just don’t do so at the individual level (they do so at the group or a population level). So I’m not sure how you can say that an epidemiologist, a researcher that looks at trends in population data, isn’t doing so to study the effectiveness of various policies and interventions?
To find out how generalisable, realistic RCTs are conducted in the work setting, you simply need to consult the occupational health literature, as they’re conducted frequently. So the task is not impossible, and neither is it particularly uncommon.
Apparently, using RCTs to assess (for example) the effectiveness of a stress management programme on productivity is possible, but doing one on the effectiveness of a new management technique on productivity, is not.
Just seems like a missed opportunity to me.
Perhaps “missed” for a reason?
Perhaps one detail is that to do randomized controlled trials on interventions with organizations would require, I don’t know, an infinite number of parallel universes to get the sample size that would give a meaningful degree of statistical power for all of the variables involved. And who’s got that kind of funding?
Join the Conversation! Post a Comment:
Last reviewed: By John M. Grohol, Psy.D. on 3 Dec 2007
