Response to the HomeNet Study

John M. Grohol, Psy.D.
September 2, 1998

______________________________

This is a response to a study which has been widely reported in the press as finding that, as people spend more time online, they become more depressed, more lonely, and reduce family communications. This research is referred to as the HomeNet study. Leonard Holmes, on the International Society for Mental Health Online mailing list (where this issue is also being discussed, as well as on the Psychology of the Internet mailing list), found a draft of the study online at:

http://homenet.andrew.cmu.edu/progress/HN.impact.10.htm

He said, "I've been told that this is what will be published in American Psychologist."

Psychometric properties of measures used

As suspected, the measures used had some serious weaknesses. The most serious weakness appears to be in the 3-item scale they used to assess loneliness. A Cronbach’s alpha of .54 is poor, to say the least. Therefore, I would question the reliability of anything relating to this measure. Why they didn't, as Leonard asked, use more standard measures is beyond me. The depression measure was stronger than the loneliness measure. The CES-D has 20 items, so it seems strange they didn't go ahead and use the whole thing. The CES-D is described as "a self-report measure of 20 symptoms experienced during the past week."

Outside of the Cronbach’s alpha, I wonder if they did any testing to make sure the "new" measures weren't somehow biased in other, unknown ways relating to the measurement of online relationships. Perhaps the measures have inherent, untested biases for pulling for traditional means of communication rather than online communications. The researchers say no, but until you test for it, it's just one opinion against another.

Sample

The self-selecting sample, lack of an adequate control group, the inclusion of only one urban/suburban geographic sample (rural vs. urban use - would we expect the same results from a rural community? Los Angeles?) and the high acceptance rate ("None of the groups approached about the study declined the invitation, and over 90% of the families contacted within each group agreed to participate.") all suggest caution.

And as has been pointed out, these are not experienced users. These are newcomers not only to the Net, but computers in general. It is therefore predictable that they may spend more time exploring these uses and use them for their intended purposes! Computers do nothing to enhance family communication -- it is a tool like any other. When used, the time spent in use of it must be taken away from something else. The authors note that at the end of the study, but don't seem to understand what they are saying. What would have been interesting is to see whether computer use in general declined or increased over time, and whether there was any covarying effect in the measure of family communication. These data were not provided.

Changes Over Time

Two years is a long time. There were apparently no controls for history and maturation effects, nor for the growth and changes of the Internet during that time as well.

Did a significant amount of people move away from this subject pool during the study's time? I'm not sure we know.

Analysis of Results

The tables are not as clear as I would have liked in this draft and I hope more data is presented in the final paper. Lack of data, and just having the statistical significances in front of us, is somewhat frustrating. The path analyses are helpful.

Family income seems to be a factor significant to increased loneliness. Being a teenager is a factor significant to decreased family communication. Being depressed is related to race and stress. I'm glad they controlled for as much as they did, given what they found.

Is it surprising that one's "local social circle" will decline when someone gives you a free, new toy to enjoy for 2 years and unlimited access to the Internet? What is not clear is whether people understood new friendships they made online could then be included in their "distant social circle," which was defined as "the number of people outside of the Pittsburgh area whom you seek out to talk with or to visit at least once a year." I neither talk nor visit once a year many of my online friends, yet they have steadily grown since I've been online. It is not clear whether subjects added their new social relationships online to that group.

Conclusions Drawn

Despite their sampling problems, the lack of a control group, and some problems with the measures they used, the researchers still claim they can infer causation from their data. I strongly disagree. There are many alternative hypotheses I could go into which might adequately explain their findings. I don't buy their argument that their analysis is equivalent to "an analysis of change scores." In addition to the threat they write about, other threats to the causal relationship abound. They include confounding due to sampling problems, additional third factors (e.g., the rise of Internet use in general, or changing Internet technologies and the way people use the Internet), etc. etc.

The central findings from the study are two-fold -- real-world relationships may decline (according to the NY Times article, about a loss of 3 such relationships out of an average of 66(!)) and people who spend more and more time online become more and more depressed (statistically-speaking. According to the NY Times article, of about 1%, but it is unclear whether this decline applies to clinical observance too, in a general sense, because that data isn't provided).

The conclusions drawn seem to go beyond the data. The researchers infer a causal relationship, while I doubt such a relationship can be inferred. They also suggest, with very little data outside of anecdotal reports, that online ties are inherently weaker and are replacing stronger social ties. Where they got this from their data is beyond me. When you try and define all fruit in its relation to apples, I think you're going to eventually come to the conclusion that an orange is very much lacking the quality characteristics of "appleness." When researchers define online relationships in terms of their real-world brethren, it is not surprising they come up short.

The researchers in the end bemoan the lack of research funding to help prod the development and deployment of more "interpersonal communication applications" which help foster stronger social ties online, rather than weaker ones formed through newsgroups, mailing lists, chat rooms, and Web-based discussion forums. This is an odd call to action. E-mail is the oldest, most widely and successfully used "interpersonal communication application." Do we really need a search engine, as the researchers suggest, to find people based on our attributes? This would lead to relationships with "stronger social ties"? (I can see how such a search engine would be used... "Yes, I'm interested in meeting women who are 5'5", are intelligent, have brown hair, weigh under 140 lbs., etc. etc.")

As Holmes has pointed out, this is fairly good research despite the problems observed. The researchers did control for a wealth of possible confounds. I just think that, in the end, they made too much of their results, and didn't put them into their proper perspective. Instead, the spent their discussion section (in the draft, anyhow), propounding on how terrible online social relationships are and suggesting future policy. What about the generalizability of the results? What about how this sample was artificially created and not one you could find in the real world? What about the actual, clinical significance of these findings?

Last reviewed: By John M. Grohol, Psy.D. on 9 Oct 2013
    Published on PsychCentral.com. All rights reserved.

 

 

My doctor told me to stop having intimate dinners for four. Unless there are three other people.
-- Orson Welles