Our General Guidelines
Our guidelines have been developed over the years with an emphasis on content and understanding attribution (who is publishing what you’re reading and why). We have developed more of a secondary emphasis on design and usability, since we find most modern websites don’t suffer from the serious usability issues of earlier sites that were commonplace in the late 1990s.
We break our system down into Primary Criteria, Secondary Criteria, and Overall Impressions. Primary Criteria are of primary importance when reviewing and rating websites, because if a site fails one of these criteria in a significant way, there may be serious problems with the site. Secondary Criteria are of less significance, as they have to do with a site’s look and feel, and usability. Overall Impressions take all of these criteria together to form an overall judgment about a site.
Content is the meat of the site. It is the breadth and depth of the material, no matter what the subject matter, provided through that site. Content may be reading material, information, contact information for real-world organizations, and links. Questions to ask when rating content:
- Does the site add new and unique material to the online world, or does it simply list or link to existing material well-covered by other sites already? (e.g., we’ve seen hundreds of websites that publish little more than NIMH or NIH content, content freely and readily available already from the NIMH or NIH)
- Does it offer new insights into a disorder, or offer information not found elsewhere? For instance, a patient’s story is more interesting to others than yet another “new breakthrough” suggesting vitamins or a health product will solve all of a person’s ills.
- Does it offer refreshing perspectives, and/or regularly updated new content in the form of news articles, opinions, or other forms of communication?
- Does it include interactive features that help provide more information to an individual that otherwise would be difficult (or impossible) to provide?
- Is it just an advertisement for an organization or company or individual?
- If the site has articles that discuss research, is that research cited appropriately and fairly reliably?
- Is the content written for a consumer audience or a professional audience? Does it meet minimal grammar, spelling and English standards?
Attribution, Transparency and Biases
Everyone has their own biases, from pharmaceutical companies and the NIH, to reporters and doctors who treat people with depression. The key isn’t to look for people or companies without biases, but to look for them that clearly talk about their biases and acknowledge them up-front. Editorial content must be separated from advertising, which must also be clearly labeled and defined as such (including “sponsored content”). If a site is a larger publisher of information, we look for an editorial policy and one reaffirming their editorial independence from financial influences. We also look for sites that keep their information updated and attributed. Sites that don’t put authors’ names or dates on their articles usually don’t want you to know.
- Is every article or entry on a website authored and properly attributed?
- Does every article or entry on a website have a date?
- If an article is modified or updated, is the “last modified” date noted?
- Is advertising and advertising units on each page clearly defined and delineated, and not mixed in with editorial content? (e.g., advertising should not be mixed inline with editorial content to try and confuse readers to click on the advertisement because they mistakenly think it’s a part of the site’s content)
- Does the site offer “Sponsored health centers” or similarly labeled content that is provided by advertisers, but published under the site’s logo and in the site’s templates to suggest it is editorial content?
Community and Social Networking
As social networks have become all the rage, lots of people flock and enter in private health data into such networks without sometimes clearly understanding the ramifications of such behavior. We look for sites that help people understand how such data may be indexed in a search engine for all time. Because health information can affect future employment and other major components of a person’s life, we believe health sites need to be more aware (and help their users become more aware) of the ramifications of such information being shared.
- Does the site offer a way for a user to delete their user profile and all of their contributions to the website if they so choose?
- Who owns a user’s contributions to the website? (e.g., does the individual retain ownership and copyright of their own contributions, or do they become a part of the site and can never be removed?)
- Are user contributions, a user’s profile and other possibly personally-identifiable information indexed in search engines?
- What types of user generated content is and is not allowed? Is such a list readily defined and available? Is such content appropriate for the type of health or mental health concern?
- What user protections are available to reduce or address cyberstalking and other user harassment concerns?
- Who runs the community aspects of the site? Are support staff available 24/7 or only during business hours?
- What is the site’s disaster and backup plan? Do they even have one, or share it with their users?
Presentation is, at the heart of it, how good does a site look. A very plain site with little or no graphics can score just as high as a site with loads of fancy graphics, if it is well-arranged and -designed. Usually, though, most sites which score higher on presentation do so because of well-designed graphics with limited or no use of animated graphics (which most users find annoying).
- Is the information laid out in a logical and well-organized manner?
- Are graphics designed appropriately for the site and do they load quickly?
- Do the graphics overwhelm the user and the content?
- Is the site advertiser-sponsored (and hence, adding to the “busy-ness” of each page which sports a banner advertisement)?
- Is it arranged so the best material the site has to offer is clearly delineated from other, less-important material?
Ease-of-use denotes how easy it is for you to move around the site and find specific information relatively easily and most of all, quickly. Large sites should have or use a search engine to help you find information in this manner. All sites should be logically arranged and organized. Often it helps to pick a specific piece of information you would hypothetically be searching for. For instance, on a site devoted to depression, perhaps we’d be interested in learning more about what makes someone depressed or what treatments are available for it. Contact information, in the form of at least an e-mail address or a link to something similar, should be provided on every page.
- Can you find our way to X content quickly using the navigation aids provided, and then get back to the home page as painlessly?
- Is a search engine or utility provided to help you find information more quickly (when appropriate)?
- Can you easily find contact information on the page to provide feedback about broken links?
This category takes into account the previous categories and allows you to evaluate your overall experience and feelings about the site. Although this is a relatively subjective rating, you should be able to make it less subjective by simply taking into account the 3 previous categories.
- What kind of feelings did you have when we came away from the site?
- Was it enjoyable to read through or was it painful?
- Were you expecting more and became disappointed by the content or presentation or difficulty in navigating through the site?
- Would you bookmark the site yourself or find yourself wanting to return on a regular basis?
- Would you recommend the site to a friend or family member?
Other Codes and Rating Systems
All rating systems have their pros and cons; therefore no single system will ever be adequate for a consumer to rely on. For instance, we’ve discovered troubling and inconsistent policing of the HONCode, finding it on many websites that don’t actually subscribe to all of the 8 principles. When complaints are filed against a site, action may or may not be taken. Eight months after filing a complaint for one popular site, we still note that most of their articles don’t have dates or author attributions, two important components of the HONCode. We have found the Health on the Net Foundation (HON) is largely unresponsive to inquiries.
One of the things we encourage consumers to do is to review multiple recommendations and guidelines (e.g., from us, HONCode, the AMA, and others), and develop their own set of criteria that works for them.
Disclaimers and Considerations
Even our rating system is far from perfect. Websites are constantly undergoing revision and a site we rated even a month ago can be completely changes and turned into some sort of spam site overnight (which we’ve seen happen more times than we can count). Other times, even when you make your best efforts to figure out what a site is about, you can still make mistakes because ultimately, we’re human and so are the judgments we’re making about other sites. As humans, we have and will continue to make mistakes.
But we’re committed to doing the best job possible, given all the information we can read and glean from any website we review.
One of the things to be aware of is that an article may follow all of these guidelines to a tee, and still not be accurate. Why? Because large publishers know about these guidelines and publish their articles to publicly adhere to them, without putting the mechanisms in place to actually follow them. For instance, I’ve consulted with large online health companies where they claimed every article on their website was reviewed by a physician or medical director, but few, if any, really were. Nobody can tell you that a company is actually doing what they’re claiming to do, so at the end of the day, you will always be left with imperfect information based upon company’s own self-reports (and perhaps an external verification or two).
We hope you find this information perhaps helpful in your own online research and travels.