We love a usability study as much as the next person. But we love well-designed, elegant studies that rightfully point out their own limitations and are printed in peer-reviewed journals most of all. We have less love for studies that act as propaganda, or researchers who draw conclusions not supported by their own data.
Bad research can be found anywhere — even by consultants who make their living from doing research on usability. Well-meaning folks like Jakob Nielsen for one. Recently he released a study on his website on the usability of the Kindle, the iPad, a PC and a book for reading a piece of short fiction.
After noting there was no statistical difference between reading on the Kindle or iPad, and then noting that the data did not reach statistical significance for the iPad versus a book, the authors of this report still declare that “Books Faster Than Tablets.”
Before posting, please read our blog moderation guidelines. The comments below begin with the oldest comments first. Click on the last comments page to jump to the most recent comments.
Before posting, please read our blog moderation guidelines.
Post a Comment:
Copyright © 1995-2013 Psych Central. All rights reserved.
Site last updated: 23 May 2013
|
The most likely “culprit” in my opinion is not that either the kindle or the iPad are less familiar, but that both devices let you look up a word you are not familiar with by tapping the word (in the case of the iPad) or navigating to the word via the joystick (in the case of the Kindle), so that people reading these devices are much more apt to stop and “look up” a word than those who are using a book.
Another perfectly good and legitimate hypothesis that could readily explain the researcher’s findings. Yet not mentioned in the original report.
Those were some good points-especially the methodology that included reading the same story 4 times in a row.
I didn’t read the actual study, but thought maybe technology is affecting research quality? Perhaps researchers are trying to shorten the length of articles, and tend to not elaborate as much? just throwing that out there.
Also, it could be that the researchers did not want to invest too much in theisstudy and instead wanted to do more of a brief pilot study to find the most appropriate ways to better structure future research in this area-including adapting the methodology. Maybe they didn’t have a good foundation to start with and realize they will learn from the possible mistakes or limitations.
I still prefer reading hard copies over any type of computer screen or device although it is not good for trees. Computer monitors of various sorts bother my eyes and overall, I don’t feel well after looking at a screen for considerable lengths of time. So I still print things out before I read them and prefer books over electronic words.
The links to look up words, as pointed out by DS, come with benefits, and although not directly related to this particular study, I find it distracting when using the internet as I can never quite finish reading what I started because I click on links that lead away from the material I sought to read in the first place.
There is a major issue with the kindle for academic use: you cannot reference the page number as you would in a printed text. Is there an option to set the page numbers (will most likely affect font and image size) in iBooks to match a printed version of the book? Thank you.
In response to Michelle, there is a way to reference location in a Kindle book. There are location numbers at the bottom of the screen. The location number for a particular quote in the book will be the same for every Kindle version regardless of the text size. Unfortunately, iBooks does use page numbers which are meaningless because they depend on text size.
I wonder whether the study figured in page turning. It is more frequent and takes a litle longer on a device.
I really wonder about the page turning question also. I have a kindle and a Sony reader. I probably read 4-5 books a week, including paper ones, the LA Times on my door step every day, the NY Times on my Kindle and a ton of documentation online. I know the Kindle is slower because of the page turning but also much easier to read than a regular book because I have terrible vision and even with contacts and glasses on top of them, regular text is still difficult for me to read. Because I use the magnification on any electronic device, the number of pages (and hence, page turns required) is much higher.
Surely, God loves p <= .06 as much as he loves p <= .04
Interesting perspective. I think your points about the limitations of the subject pool are stronger than your point about the p-values and rejecting the Null.
I wrote about it here
http://www.measuringusability.com/blog/books-tablets.php