Podcast: Do Personality Tests Actually Work? Exploring the Validity of Psychometric Assessments

Inside Mental Health Podcast Guest and Host Bio

Lindsay Oberleitner, PhD, is a clinical psychologist and the Education Director at SimplePractice Learning, as well as an Assistant Professor Adjunct at the Yale University School of Medicine. Prior to joining SimplePractice, Dr. Oberleitner was full-time faculty at the Yale University School of Medicine where she oversaw an interdisciplinary team of behavioral health providers and developed and managed grant-funded programs to increase access to integrated treatments for addiction, mental health, and physical health, and conducted clinical and forensic psychological evaluations.

Our host, Gabe Howard, is an award-winning writer and speaker who lives with bipolar disorder. He is the author of the popular book, “Mental Illness is an Asshole and other Observations,” available from Amazon; signed copies are also available directly from the author.

Gabe makes his home in the suburbs of Columbus, Ohio. He lives with his supportive wife, Kendall, and a Miniature Schnauzer dog that he never wanted, but now can’t imagine life without.

To book Gabe for your next event or learn more about him, please visit gabehoward.com.

Episode Transcript

Producer’s Note: Please be mindful that this transcript has been computer generated and therefore may contain inaccuracies and grammar errors. Thank you.

Announcer: You’re listening to Inside Mental Health: A Psych Central Podcast where experts share experiences and the latest thinking on mental health and psychology. Here’s your host, Gabe Howard.

Gabe Howard: Welcome to the podcast, everyone. I’m your host Gabe Howard. Calling in today, we have Doctor Lindsay Oberleitner. Dr. Oberleitner is a clinical psychologist who has worked at the intersection of addiction, trauma and chronic health conditions, and she’s currently the education director for Simple Practice. Dr. Oberleitner, welcome to the podcast.

Lindsay Oberleitner, PhD: Thank you Gabe. I appreciate you having me on today. Thank you for the opportunity to speak with you.

Gabe Howard: Oh, you are very, very welcome. The pleasure is all ours. Now, today we’re going to be talking about the Minnesota Multiphasic Personality Inventory, better known as the MMPI. I want to be up front and say that, that I don’t have a lot of faith in standardized testing. And I was kind of shocked to find out that, per the National Institute of Health’s website, the Minnesota Multiphasic Personality Inventory, or MMPI, is the most common psychometric test devised to assess personality traits and psychopathology in use today. And it’s in use everywhere. It really determines whether people are going to get hired or not in many types of jobs across our country. Now, for our listeners who may be unfamiliar, can you give us a quick history of the MMPI?

Lindsay Oberleitner, PhD: Absolutely, happy to. The MMPI has existed for decades at this point. Originally developed back um, in the 1930s, uh, not really publicly made until the 1940s, and it’s gone through multiple iterations. It is really one of the major tools that we look to as, uh, within the field of mental health and being able to help define different, um, categories of psychopathology. So, of diagnoses it doesn’t directly diagnose, but gives us some insight into those, as well as personality. The MMPI-2, now transitioned to the MMPI-3. Um, and it’s a questionnaire that uh, clients can complete that you respond in true and false, uh, responses to a wide variety of questions, a few hundred questions. Some of which may seem completely unrelated to mental health as you go through these items.

Gabe Howard: I think people really like this idea of, hey, I can take a test and have all the answers. And we see this play out in other areas of society. For example, when online dating first came out, everybody thought that you just filled out a 100-word questionnaire. Somebody else would fill out a 100-word questionnaire, the computer would do computer magic, and boom, you would immediately be on a date with your soulmate. And then eventually we realized that online dating is much like dating in the real world. You can use it to match up and it can give you some help, but ultimately you’re going to have to go on those dates and decide if you’re compatible. I’d like to see the MMPI be more understood, like the online dating questionnaire. It can give you some help. It can give you some insight into your job candidate. But ultimately you’re going to have to use your expertise and hiring and just use this as a tool to get the right candidates in the door. Because if you rely 100% on anything, you’re probably not going to get the right candidates.

Lindsay Oberleitner, PhD: I just want to say I loved your comparison to the dating apps, and I think it always makes me think like the Facebook, apps and things where people are like, what celebrity are you? Like, we all want that quick answer.

Gabe Howard: Oh yeah, I love those things. Like BuzzFeed quizzes. Right? I like to find out which Harry Potter house I belong to, which Futurama character I am, what Disney princess I should be. We take them constantly in my group of friends, and then we have a really good time discussing if we think that it got them right or wrong. Now the difference is, is this doesn’t make me qualified to be a police officer or get me a job or bar me from anything because I completely understand where it fits. Now, I don’t want to compare a BuzzFeed quiz to the MMPI, but there’s there’s some analogy that runs through there. If you heard that somebody was using a BuzzFeed quiz for hiring, you’d think to yourself, uh oh, that’s problematic. But when we hear that we’re using this other quiz, we’re like, well, yeah, that’s good. What are your thoughts on that comparison? And I understand I want to want to be the first to say, I understand that it is a ridiculous comparison, but I do think that people understand that a BuzzFeed quiz is just for entertainment. And I think that people believe 100% that the MMPI is flawless and perfect.

Lindsay Oberleitner, PhD: Yeah, that’s I, I, I love thinking about those comparisons because it often does strike me. We all want those easy answers, right? We want something to be so neat and tidy that like, we fit. You know, as you mentioned, we fit this house in Harry Potter, whatever it may be. And it feels so, so, so clean cut and it feels nice and it feels easy cognitively to say, we are this or we are not this. Um, so we’re all always looking for that. But I think, you know, as you’re mentioning, that part of the compare and contrast is the similar thing is sometimes we simplify the MMPI to think it’s going to be that easy. We’re going to be able to look at someone and say they are exactly this. And the truth is it’s never exactly this category or that category. What it is, is this person is more likely to look like an individual who responds in these ways. And, you know, many profiles aren’t even clear. So, when we talk about the MMPI, um, we talk about sets of, you know, individuals can have individual scales that that show an elevation. They can have code types is what we call them across multiple codes that go together. So, there’s multiple scales that spike. Um, and even when we drill down to those type of levels like, oh, an individual with these three scales spiked look like this commonly.

Lindsay Oberleitner, PhD: There are still, uh, pieces that this doesn’t capture about the individual. So, in the similarity between a BuzzFeed quiz and an MMPI, I think we need to be always aware that there’s much more to it than any single cut, right? So, in the contrasting the MMPI has based on tons of data and that is its strength. It is one of our most data informed tools, whereas our BuzzFeed quizzes are probably a little bit more that seems fun and someone throws it together. Not unlike those little folded paper guessing games we’d play in school. If you remember those where you’d ask someone to flip a little tab up and it would tell them something about who they’d marry or what type of house they’d have. Uh, BuzzFeed quizzes are a little more close to that with the data they’re using, which is mostly none. But it feels good. And I think we need to always fight that feeling that it just feels so good to have a nice, tight, clean answer and to know it’s very rarely only that simple.

Lindsay Oberleitner, PhD: So, we’re farther from it with BuzzFeed, we’re closer to it with MMPI, but we’re not there. It’s never just that, right? We always need to think, what does it mean for this broader contexts. So, I love the comparison. And you know what? I’ll confess, I as well love taking those. It’s really fun to get to find a way that you can take a moment and reflect on yourself and in the strength of BuzzFeed and the MMPI, to go back to one of the strengths that I really see with the MMPI is it lets us take a chance to take insight into ourselves. And you know what? BuzzFeed does sometimes, too. It may not be science based, it may not give us answers. But the fact that as you shared, you might talk with friends about what it means and where it fits and where it doesn’t. What a great opportunity to grow an insight, right? It’s still something that can help us, even if it maybe isn’t scientifically based.

Gabe Howard: I recognize that it’s one extreme to another, and that’s one of the things that makes it such a compelling analogy. You’ve got literally arguably the most ridiculous quiz, right? It reminds me of the magazine quizzes that we took, you know, back in the day. You know, Cosmo had so many quizzes, you know, five ways to know if you’re in the perfect relationship. And they were largely for fun. People took them for fun. But in sitcoms, if you watch enough television, somebody took one of those magazine quizzes and got a divorce, right, because they weren’t compatible. After ten questions. And there’s an example of course, then hilarity ensued and they got back together in the end. But we, the viewing public, are like, oh my God, why would you put so much stock into a quiz? And I really am sitting on the side of, are we really making the decision to determine if somebody’s going to be a good police officer on a bigger quiz? And again, I know the example is one extreme to the other, but I feel that it does spark a very interesting conversation.

Lindsay Oberleitner, PhD: I think it is an interesting thing to consider and how it actually plays out, and whether it even is what is truly intended for it to be used. Right. The practicality versus what we intend assessment tools to truly do. I think one of the things that stands out, though, in the differentiating between great example of, you know, someone taking a quiz and thinking like, you’re right, we don’t match, um, versus MMPI is there are some scales that we can have a lot of confidence on that maybe bring enough concern, um, that it’s at least something that we really need to pay attention to. Right. And that difference, it’s being able to take a different lens of this has been very similar to individuals who have had, um, you know, approaches who maybe this isn’t a safe career. Maybe when we’re talking about like, public safety, it’s like aggression and control and all those pieces that many, you know, and many more. Just to give some of the most common examples of what we’re trying to pay attention to is that it really does bring up like we need to look into this, right? That’s what it should do.

Lindsay Oberleitner, PhD: That’s what this tool should do, is it should tell us, you know, there’s enough going on here that looks like other individuals who have this struggle. We need to dig deep into this and ensure this isn’t the issue. Or maybe it is. Um, but it’s really that we have enough data to say a lot of people look this way, but again, not alone. Right? Um, in my opinion, in my professional opinion, I think it never stands alone. So exactly in the same way, it should be a chance for conversation. So that quiz in that magazine could be a great chance for a conversation amongst a couple, right? To say like, oh, interesting. I’d answer these ways. It says we’re not compatible. Why doesn’t this fit or why doesn’t fit for us? The same way that MMPI should give us a chance in those decisions to say like, oh, this is a real concern. This is important data. And I might have missed it in an interview, and here’s why I’m going to pay attention to it now. Here’s why I’m going to try to make sense of it for this individual right now.

Gabe Howard: I’m sincere in it’s weird. People make fun of me all the time and how much I love the I. There is not a Facebook BuzzFeed. For real if somebody’s like answer five questions and we’ll tell you that food you are, I’m like click, click click click click I it I’m like a moth to flame. And then I start like emailing my friends like, hey, hey, do you think I’m French fries? It’s like, what are

Lindsay Oberleitner, PhD: Yeah. [Laughter]

Gabe Howard: You talking about? And I send them the link and they’re like, well, they called me steak. I’m like, I don’t think you’re steak. I, I can’t explain it, but it’s it’s fun.

Lindsay Oberleitner, PhD:

[Laughter] Well, I think our minds want it right. True. I think cognitive science would actually support exactly why these are so popular, which is we like a nice clear. We really do. We like a clear story. Right? But it’s so fun to explore it for ourselves and be like, oh, did I learn something new about me and my French fries? Right? And why? Um,

Gabe Howard: Exactly,

Lindsay Oberleitner, PhD: It can be really fun.

Gabe Howard: Exactly. And the stakes are very low. Right? If nobody thinks I’m French fries, or if everybody thinks I’m French fries, I’ve lost nothing. Uh, so it’s just it’s just a for fun thing.

Lindsay Oberleitner, PhD: Right.

Gabe Howard: Yeah. It’s fun. And like you said, it is just made up. There’s no expectation that an internet quiz is anything. It’s not written by an expert. It’s not. It’s not administered by an expert. And I, I think that’s what I want the listeners to understand. You personally are not the typical person administering the MMPI to the vast majority of people who are taking it. You have a PhD, and that comes with a lot of education and years of training in human behavior. But more often than not, these tests are being administered by leadership coaches, human resource reps, hiring managers. Doesn’t that raise the potential for errors?

Lindsay Oberleitner, PhD: You know, one of the things about the MMPI is it is intended, when you purchase, that you are agreeing that you have had advanced training in assessment. So, one of the expectations is that you say yourself, either through proving you’ve gone through programing like a doctorate with focus in assessment and you understand the broader picture, or you’re attesting to having other backgrounds that has led you to this. Sometimes what we will see is psychologists who are not clinical, not licensed, but in other areas of psychology, um, who are who are um, overseeing the use of these tests in hiring management situations, for example, sometimes, though, you’re absolutely right when it comes down to the person on the ground looking at this test, even if someone had to agree and approve the use of it in this setting, who has appropriate training? It can get misused, as can any of our tools when it’s met at the ground with someone who doesn’t have that broader picture.

Gabe Howard: One of my really big concerns with the MMPI is how it was developed. It’s my understanding that they created this test using primarily white male college students from the Midwest. Now, my concern is, is that the test claims that it can predict, and it does say there is a margin of error, but it claims that it can predict whether or not a 50-year-old African American woman from the Deep South has, for example, criminal tendencies. Is that reasonable? Is it reasonable to think that you can get this from that? There’s just a world of difference between the group of initial test subjects and many of the people who are taking the test today.

Sponsor Break

Gabe Howard: And we’re back discussing the MMPI with Dr. Lindsay Oberleitner.

Lindsay Oberleitner, PhD: You’re absolutely right. The original development of this test was on a pretty narrow population of individuals. Um, so it was intended to be designed and in part of its development to look at something we call criterion groups. Um, but what these criterion groups were individuals who met for different psychiatric diagnoses, individuals who were depressed, individuals who were anxious. What it didn’t account for is the intersection of exactly what you’re speaking about, didn’t account for those intersections of gender or race or any level of identity that individuals may hold. It’s something that we have to take very seriously. We know there’s flaws. If we directly ask individuals about things like depression, uh, we know that individuals across different intersections of identity don’t all respond in the same way. Those intersections are even more important. There have been I do want to acknowledge some changes in recent years, um, in that there has been a new edition released which sought to begin to answer some of those exact issues you raise. Um, and though that work has begun, we’re far from perfect in those areas. I don’t know that any singular test could ever get to perfect on that, because part of the problem will always be that the interpretation of any item, of any word, of any piece, the narrower and narrower we get, the more intersections we consider in someone’s identity, in their experience, in their experience of the world and mental health as well. The more difficult it is to truly represent, and that needs to be taken in mind whenever we’re considering tests such as the MMPI.

Gabe Howard: One of the things that it reminds me of is something that a previous guest said, Dr. Joanne Lunceford talked about race and race relations and really where we’re going as a society and how we’ve gotten here. And one of the things that she said is let’s go back to the 1950s, when we hired predominantly white people, and then eventually we, we stopped doing that, presumably. Not to fall down a rabbit hole. But white people were primarily in charge, for example, and they tended to feel more comfortable with people who looked like them. So, it’s not that they were being racist. They weren’t trying to be anyways, but when they brought people in, they felt more comfortable, a better connection with the people who were their friends and neighbors. They also looked to their staff, who was predominantly white, and said, hey, can you recommend somebody from this job? And of course, they lived in neighborhoods that were also predominantly white and brought people in like so. So, I’m wondering if the MMPI is just contributing to this, except it’s providing some cover. They’re saying no, no, no, no, no, no, no. We used science. We had all of our job applicants take the MMPI. And it just so happens that this group tested better. But of course, there’s this inherent bias in the test.

Lindsay Oberleitner, PhD: Yeah. Thank you. Gabe. Great, great. Uh, great question and clarification. I think, you know, as we think about that right now, what has been attempted to be done with the and has been done to some extent with the newest iteration of the MMPI, was to look at um, samples that were representative across the United States. So being able to look at things like racial identity, looking at the percentage in the United States and making sure, as we do with many levels of research, that the samples that all of these questions were normed upon, uh, matched those, those similar, um, percentages and similar, um, kind of divisions across the United States. Now, the important thing to consider when we’re doing this again, though, I think as you raise is like, are we trying but not meeting? I think it is a big improvement. I think it is a big improvement from what the MMPI-2 was to the MMPI-3. But if we imagine the MMPI or any test as a standalone piece of information, it may not fully reflect any individuals sitting in front of us, any individual at any intersection of identities, any individual at intersections of these areas of psychopathology we’re assessing. The more complex we get in our understanding of an individual, the harder it sometimes is to then match up those test results to that person. Good piece of information, I think is the key. Test can provide a good piece of information to reflect back on, but if we use it in isolation, I think to your point, Gabe, it really doesn’t always reflect that person in front of you.

Gabe Howard: I can’t thank you enough for being so willing to answer these questions. I know they’re tough. I know that there’s a this is a well-respected and well researched and a long-standing tradition. Many people have taken these tests. I, I think you’d be hard pressed to find anybody working in America who hasn’t at least heard of these tests or taken it at some sort of job interview. So, I, I appreciate you answering all of the questions, but I want to make sure before we end the show that you get an opportunity to fill in any gaps with questions that maybe I didn’t ask you or anything, that you think it’s important for people to know. I just don’t want to end on the note that, hey, Gabe doesn’t like the MMPI.

Lindsay Oberleitner, PhD: [Laughter]

Gabe Howard: I want to make sure that you have an opportunity to share your views, your thoughts and everything that you’ve seen on it, because I know it’s a tool that you very much believe in.

Lindsay Oberleitner, PhD: Yeah, absolutely. Thank you for the opportunity to share a little bit more. I’ve used the MMPI many times and in multiple contexts. Um, so I mentioned that I’ve used it in forensic evaluations. I’ve used it in clinical settings. Um, and I think it can be really powerful. The problem to me really comes up with how and when it’s used, and not necessarily that the tool itself is the problem. The science behind the MMPI is one of the strongest science-based tools that we have in assessment, in clinical assessment and understanding individuals. The fact that it can be validated on thousands and thousands of people is something many of our tools can’t do. Can’t say that they’ve done. But the power that comes with that is that many people misinterpret it because of that. Is it a gold standard, what we’d call a gold standard test for psychopathology and personality? Absolutely. But no gold standard is infallible. Um, and you know, what I love about the tool is that it can give you really great insights. As I mentioned earlier, there are times that it is really brought up something when there was a client, I just couldn’t figure it out.

Lindsay Oberleitner, PhD: It wasn’t quite just a depression diagnosis they were struggling with, wasn’t quite just anxiety. Um, and using this tool helped me get better insight into things that that client couldn’t tell me either. And we could sit together. And I think the real power of the MMPI is being able to sit together with a client and say, you responded in ways that are consistent with individuals that look like this and experience these types of life situations and experience the world in this way. What do you think about that? And the power of those conversations has been hard to describe. It really brings a depth to things. And that can help some people really move forward. So, I think it really holds power in that way. But again, when it’s used as a this tool means more than anything else, it means more than the client in front of me and what they say it means more than my clinical expertise and it can stand on its own.

Lindsay Oberleitner, PhD: That’s where I really get concerned. Um, because it really is. It’s all of those pieces. I take a very person-centered approach to work. Um, in anything I do, whether it’s a forensic assessment or it’s a clinical client that I’m going to work with, um, and being able to hear what it means to that individual, being able to reflect what it means to me with the individual I’m working with is the true power of the tool, not the fact that it is, um, going to tell me all the answers, right? The minute I think I’m skeptical and this maybe says something about me as well, but I’m skeptical. The minute we say this, one thing is going to give me the answer. I don’t think that’s true. And it’s definitely not true when we’re talking about personality, mental health and complex things. Obviously I still use this tool. It doesn’t take away from my use of it, but it goes into my interpretation of it is there are potential, you know, there are assumptions we make in our diagnostic system generally.

Lindsay Oberleitner, PhD: We assume that there are clean cuts between diagnoses in the way our diagnostic manuals are written in the way we see the world. Yet we know that’s very often not true. Many people have multiple diagnoses. There’s symptoms that cut across multiple diagnoses, and it can be hard to determine what someone’s really experiencing. And maybe maybe it’s because we’ve not always cut those things quite right. And there’s a lot of debate in the field about that. But tools like this are based on our existing diagnostic belief sets at some level. Um, and we have to consider that too. Like maybe part of the problem isn’t always even the test, but some of the assumptions that sit behind the test that we’re still working out in the field. So, thank you for getting me giving me that chance to clarify some of my thoughts. I think I take a very kind of middle ground approach. I really enjoy it in the right setting. I really enjoy it, using this tool as a way to bring insight into things that are hard to pinpoint and to give additional tools, um, additional ways to look at problems. But it’s not without concern. And it definitely gold standard doesn’t mean perfect standard, um, by any means.

Gabe Howard: Thank you so much for being here. Where can folks find you on the web? And of course, where can folks find Simple Practice online as well?

Lindsay Oberleitner, PhD: Absolutely. Thank you so much. Individuals can find out more about my work and about Simple Practice at SimplePractice.com. Just as a quick some simple practice is uh, um uh, practice management solution that gives health and wellness practitioners things they need to succeed and advance in their profession and ability to help others, um, and run their practice more smoothly. And you can find even more on the learning.SimplePractice.com um, profile, which is all of the information about our continuing education and courses that we offer. And finally, for some personal information, you can find me on LinkedIn, Lindsey Oberleitner. Uh, very hard to miss with my last name, um, which is O B E R L E I T N E R. There are not many of us. So, um, please feel free to reach out and connect. More than happy always to build connections. Have discussions. I love talking about this and many other topics within mental health, so please reach out and connect anytime. Gabe. I very much enjoyed the conversation and loved the opportunity to sit here with you today. Thank you.

Gabe Howard: Well, you’re very welcome, Dr. Oberleitner. And a big thank you to all of our listeners. My name is Gabe Howard, and I’m an award-winning public speaker who could be available for your next event. I’m also the author of “Mental Illness Is an Asshole and Other Observations,” which you can get on Amazon. However, you can grab a signed copy with free podcast swag or learn more about me just by heading over to gabehoward.com. Wherever you downloaded this episode, please follow or subscribe to the show. It is absolutely free and you don’t want to miss a thing. And listen up! Can you do me a favor? Recommend the show, share it on social media, share it in a text message. Share it on a support group. Tell people about the show because sharing it is how we’re going to grow. I will see everybody next Thursday on Inside Mental Health.

Announcer: You’ve been listening to Inside Mental Health: A Psych Central Podcast from Healthline Media. Have a topic or guest suggestion? E-mail us at show@psychcentral.com. Previous episodes can be found at psychcentral.com/show or on your favorite podcast player. Thank you for listening.