Web 2.0: Consistency, Relevancy and Reliability
Del.icio.us: Design for Everyone?
The challenge in social networks where tagging is a vital component and at the core of the service’s functionality is to design it from the ground-up for everyone. That translates into designing not only for the good-minded individuals who want to use the service for its intended purpose, but to also walk through typical user scenarios informed by the trials and tribulations that other services have experienced. History teaches us much, if only we would listen.
It is ridiculously simple to game Del.icio.us today. As long as you’re interested in a set of keywords or a concept that isn’t extremely popular (as apparently many health phrases aren’t currently), anyone can put any result near the top of the search results listing within 24 hours (apparently only because it takes 24 hours for Del.icio.us to re-index its database). In 5 minutes I had setup 10 new dummy Del.icio.us accounts and in another 5 minutes, had them all pointing to a dummy page for certain keywords (depression drugs). The next day I went to the site, typed in the health phrase I was targeting, and voila! There it was at number three:
Del.icio.us was, apparently, not designed for people looking to get their own pages near the top of a search result. And yet, we’ve known for over a decade now that if you place any search interface on any database, people will try and use that system to get their own results near the top. It’s unclear to me why it was so incredibly easy to do what I just did. The only thing stopping more people from doing it is that Del.icio.us isn’t yet quite on most people’s radars (but it is growing in extraordinary leaps and bounds).
But let’s assume Del.icio.us is run by some very smart people who will figure out simple controls to stop the abuse that I just illustrated. What about the regular results, ostensibly tagged by ordinary folks looking to help others out? How relevant are they and what kind of quality can we find here?
With Del.icio.us, it’s far more easier to run health and mental health searches. I chose a term that my own users often type into my website’s search engine, so I know it’s a commonly searched-on term. On the result I gamed, “depression drugs,” let’s look at the other results that bubbled up through tagging:
- Although the title is listed as “Health education: Stress, Depression, Anxiety, Drug Use” it’s actually a website that is an online book entitled, “How to Survive Unbearable Stress” which was published in 2004. It’s certainly an interesting site and appears to have useful information on a cursory review, but its relevance to “depression drugs” escapes me (I couldn’t find any content that discussed drugs commonly prescribed for depression).
- The second result is also from 2004, and is a news artcle about the publication’s belief (supported by some questionable assertions) that at one time, President Bush was taking antidepressants (and that is somehow “news”). Interesting to someone looking for political conspiracy theories, but completely irrelevant to someone searching for depression drug information.
- My gamed result pointing to a Geocities placeholder page.
- The fourth result, a journal article from PLoS Medicine, is the most interesting of the first 10 presented, by far. Well, to me as a researcher and writer in this field, it’s interesting. To someone looking for depression drug information, it’s probably a little too academic and lengthy for most to read through or care about. It’s also offering a different kind of political view, one debated within the field itself. I’d argue this result is both relevant and of high quality, but not exactly what I was expecting.
- The Erowid Splash Page is some type of personal page “documenting the complex relationship between humans and psychoactives.” It’s the sort of thing I would’ve expected to find from a search done in 1996 or so on any of the available search engines at the time, because it’s sort of a fringe website that mixes personal experience with actual information. Probably somewhat relevant, but I’d certainly question the accuracy of some of the information found on the site.
- BBC News article from Feb. 2005 that describes a study that found St. John’s wort, a common herbal remedy for depression, as effective as a specific type of antidepressant. The article pays some lip service to the overall mixed picture of research on this herbal remedy.
- A Pravda news article from March 2005 that describes “whipping therapy” for the treatment of depression. I won’t even comment on how ridiculous this is as a search result.
- This is a blog entry from Oct. 2005 about a study on depression, illicit drug use, and teens. I’m beginning to see how a simple word like “drug” used as a tag can lead to confusing and mixed results.
- A press release from Eurekalert from Dec. 2005 about how a new antidepressant drug is shown to increase the brain’s “own cannabis.”
- The last result on the first page is a blog entry about the same study, from BoingBoing.
So in this example, we have 7 of the 10 results related to a recent news article or study about some finding related either to depression, drugs, or depression and drugs, and sometimes antidepressants. We have one planted result (mine), and we have one online book that has little to no relevance to my intended search (apparently because of the many meanings of the word “drug” I chose to use). And we have one “personal page” that seems to be an interesting mix of all of the above. Clicking on further results pages doesn’t help me find any information about the drugs available to treat depression.
Okay, so that was a mixed bag. Let’s try a search phrase that is bound to be more specific and helpful, “anxiety symptoms.” Seven of the top 10 results are spam, 1 is for a specific anti-anxiety herbal remedy that is available for the treatment of practically any mental health issue you might have, and two are good quality, relevant results. Additional pages were full of more spam. Definitely not a good return on quality for this set.
“ADHD treatment” gives us two (repeated) blog entries, two (repeated) articles on ADHD treatments, two (repeated) tables of drugs prescribed for a variety of disorders (including ADHD), one news article, one blog entry, one journal article, and one very questionable treatment method. Additional pages seemed also to be of similar high quality.
Once someone is diagnosed with cancer, they often go online to find an online support group to talk about the emotions they’re experiencing. “Cancer support groups” seemed to be a reasonable way to capture this query. Five of the top ten results went to cancer support groups. Three went to large, popular websites, including one to Del.icio.us itself. One was a blog entry, one went to a news article. Only 12 results were listed.
The results from Del.icio.us show a mixed bag. Some searches provided as relevant and useful results as you’d expect from Google or the like. Other searches were full of spam. Still others were half and half. Quality seems variable and, again, idiosyncratic. Are we starting to see a pattern here?
Grohol, J. (2014). Web 2.0: Consistency, Relevancy and Reliability. Psych Central. Retrieved on May 4, 2015, from http://psychcentral.com/lib/web-20-consistency-relevancy-and-reliability/000111