Let’s imagine that, once upon a time, you signed up for an online dating service that you thought was going to provide you with the best possible matches it could based upon the information you gave it.
Let’s also imagine that the guy who ran the site decided to play a game. He decided that, “Hey, wouldn’t it be fun to show people the opposite of what we’d regularly show them as their best matches and see what happens?” He’ll do it under the guise of an “experiment” to make it all sound legit.
Maybe this guy doesn’t understand much about human ethics. Or maybe he just doesn’t care.
Sadly, you don’t have to imagine this scenario. Because if you’re a user of OKCupid, an online dating site, you may have been an unwitting participant in a research study its co-founder, Christian Rudder, decided to run earlier this year.
If you’re a user of OKCupid, I feel sorry for you. The company you signed up with is apparently run by a mathematician — Christian Rudder — who has little training as a researcher or scientist, yet feels perfectly okay with conducting social science experiments on its users. This is an equation, in my opinion, for disaster…
The controversy erupted over the summer after revelations that Facebook was also experimenting on its users without their explicit consent:
Rudder thought Facebook got a raw deal in news coverage because all Internet companies run small- and large-scale experiments to help hone their products or make sense of their data. Among other things, his post disclosed that OKCupid sometimes inverted its match percentages, showing high marks to people who weren’t supposed to be compatible, therefore implying the opposite. OKCupid then measured whether those matches were less productive (i.e. led to fewer messages) than the traditional algorithm’s.
Or, as The Guardian wrote:
To test its theory, the site lied to a portion of users about how strongly they matched with other users, and observed how many single messages led to a full conversation. Sure enough, they found that two users who actually had a 90% match but were told that they had a 30% match were less likely to carry on talking than two users who actually had a 30% match but were told they had a 90% match.
Whaat?!? So the online dating website you thought was going to use its best efforts to match you with what it thought was your best matches sometimes decided to play games with you instead. Rather than showing you the best matches, it sometimes showed you the worst matches you could be paired with. Just to see what you would do.
If I were a customer of OKCupid, here’s what I would do — cancel my account with that company immediately. Tell all of my friends on social media to stay away from OKCupid, a company that apparently cares so little about the ethics of human research, it feels perfectly at ease lying to its own customers.
It’s okay to run A/B usability experiments to test how different designs of your website function.
It’s not okay to run experiments where you directly lie to your customers about the core service you’re providing — in this case, a matching algorithm. In what universe would that ever be considered ethical?
There are two very different things. Any researcher knows this:
But while OkCupid’s blog post was seemingly an effort to demystify the process of experimenting on users, for many, it simply underscored the tech industry’s failure to “understand why some testing is ethical and some is not”, in the words of the sociologist Nathan Jurgenson.
Despite OkCupid’s personable demeanour, the firm is owned by the IAC conglomerate, a multinational corporation with a market cap of $5bn. “Tech folks routinely equate emotional manipulation and a logo change,” Jurgenson continued. “[You] need an ethics board to distinguish what requires opt-in, debriefing, etc.”
No offense, but this is exactly the kind of unethical behavior I’d expect from a college student who majored in math. In my opinion, Rudder is not a scientist or researcher — he doesn’t even have a master’s degree, much less a doctorate. He seemingly thinks that publishing his “insights” from Excel on a blog is equivalent to publishing a peer-reviewed journal article (of which I could find zero he’s published).
I know it’s the fun, hip thing to start calling yourself a “data scientist” the first time you put some data into an Excel spreadsheet and notice a correlation between some numbers. But calling yourself a name and actually having the same skillset and education as a real data scientist are two very different things. In my opinion, Rudder is not any kind of scientist, just a very good showman and marketer.
Why should Rudder’s credentials matter? Because it’s in graduate school that you learn about human subject testing. It’s in graduate school that you learn about ethics. And it’s in graduate school people generally learn about how to use real statistics programs to find data trends.
Did they do the research to share with other researchers — as most real scientists do — in a scientific journal? Nope. They did it so apparently they’d have fodder for yet another blog post. That’s right — your online dating life was manipulated so they could entertain others in a blog.
IAC, the parent company of OKCupid, should be ashamed of this guy’s behavior. It would instantly make me not trust any of the properties IAC runs — like About.com or Ask.com.
But then again, you get what you pay for. Customers of OKCupid — a free online dating site — shouldn’t be surprised that the company apparently has no problem lying to you and manipulating the matching results it shows you.
You deserve better. Pony up a few bucks to become a member of an online dating site run by professionals and who don’t conduct human subject experiments on you just for fun.
Read the full article here: Matchmaker, Matchmaker, Make Me A Spreadsheet