…the vast majority of picturebooks are created for children. If we wish to be clearer about the nature of the picturebook should we attend to what children make of them or will our own close reading of individual texts be sufficient? And how relevant is it to our attempts to understand picturebooks that they are often used for teaching children to read?

– from the introduction to Reading Contemporary Picturebooks: Picturing Text by David Lewis

As children we relate to our picture books in a holistic fashion, merging sensations of the eye and the ear (for first we are read to), which marries the image and the sound of the words, and later, as we learn to read, the look of the words.

How Picturebooks Work, by Maria Nikolajeva and Carole Scott

It’s not surprising that research on a new medium happens only after the new medium comes into existence and gains a foothold in culture. Since interactive storybook apps are so new, there is still relatively little research that has been done, and when making development decisions, developers are instead reliant upon our own commonsense, and inevitably, our own experience of literature and reading.

One of the assumptions to have arisen about the nature of ‘good’ storybook apps is that they include word or phrase highlighting synchronous with narration. The assumption: that word highlighting is beneficial for emergent readers.

At this point, the beneficial nature of text highlighting is an assumption. It may be of benefit. It may not be. And it is also possible that word highlighting actually does more harm than good to an emergent reader.

Why this assumption in the first place? I think word highlighting is often considered the digital equivalent to pointing at words with a finger, and many are under the impression (rightly or wrongly) that when a caring adult co-reader points to words as they read, that the child will pick up reading — as part of a much wider program to teach reading skills, of course.

So before focusing on the topic of word highlighting, I would first like to look a little harder at the finger-pointing assumption.

From an article in the Telegraph titled ‘Pointing to words helps children read in later years‘:

Researchers claim this is the first time a study has shown a link between referencing during reading and literary achievement in later life.

So, if there have been many good studies on the effect of pointing to words on emergent readers, they haven’t been widely published.

Let’s go with that and trust our parental instincts: that occasionally pointing to words in books, and drawing children’s attention to various technical aspects of reading does improve literacy. I’m not going to argue with that because I intuit this is the case.

I don’t think I’m alone in this, because there seem to be many app developers who intuit that pointing to words by a trained or careful adult can be emulated electronically in a storybook app.

This, I’m not so confident about. Pointing to words may be really quite different from animating individual words in digital stories:

  1. When pointing, the finger does not obscure the actual word. Instead, effective pointers would  surely place their fingers BELOW the word in question, not over it.
  2. Also when pointing, the fingers are not making those jerky movements reminiscent of colours flashing on a screen. The hand glides across the page unobtrusively. Emergent readers may well be less distracted by a hand than by digital animation of words.
  3. Fluent readers do not read by looking at one word at a time. We take in three words at once. While it’s clear that early readers need to learn words one by one, when it comes to training the eyes to move across the page, is it really that helpful to highlight words individually, especially when the narrator is reading fluently themselves? I wonder about what we are modelling when app developers choose to individually highlight words.
  4. It’s possible that some ways of highlighting words are better than others. We need more research into this. It’s not enough to simply assume that ‘apps with word highlighting are good’ while ‘apps without word highlighting are lacking’.
Here are some various ways of word highlighting that you’ll see in some popular storybook apps right now.

1. JUMPING WORDS

Sir Charly Stinkysocks and the really BIG Adventure

This is a storybook app produced by a large publishing house. The words ‘jump’ off the page as they are read. But when a word is jumping, it’s moving, and therefore not able to be read. All the emergent reader can see is where in the paragraph the narrator is up to; they can’t see the word itself. Not unless their own reading is actually out of sync with the highlighting.

Here is another app which makes use of the same technique:

Logan and the upside-down sea

*

2. FLOWING TEXT HIGHLIGHTING

Perhaps to avoid the choppiness which results from highlighting words individually, this app developer decided to make the word highlighting last slightly longer than the narration itself. The colour that appears around the words fades out slowly, so you end up with an ‘approximate’ highlighting of words. It certainly works to avoid that choppy feeling that happens when words jump.

But if the highlighting isn’t 100% accurate, leaving the reader perhaps one word behind the ballgame, might this be worse than no highlighting at all? We don’t know this yet.

3. HIGHLIGHTING OF PHRASES

Cozmo’s Day Off

I prefer this method of word highlighting, where phrases are highlighted rather than individual words. This emulates the way we read as fluent readers – not just by taking in a single word at a time, but by encouraging us to take in several. This may aid reading fluency, and fluency aids comprehension.

I suspect this book has it right. If words are to be highlighted, this is how I’d like to see it done. I like that the words themselves don’t move. Instead, a blue outline appears around the words. This doesn’t prevent the reader from actually reading them.

Temporary Conclusion:

  • I suspect that the highlighting of individual words is useful in word games in which emergent literacy skills are the target.
  • I suspect story app developers should stay away from individual word highlighting, and consumers should be wary of expecting it by default.
  • Just because something is possible with the digital format doesn’t mean it’s an improvement on non-digital versions of a story.
  • For now, app developers who use word highlighting as a selling point are making money based on something which doesn’t have good research behind it.
  • The option to turn off word highlighting should be an option, just as it’s an option to turn off narration.

I’m prepared to change my mind on this. The issue of word highlighting in storybook apps desperately needs research. But we can’t assume that highlighting equals finger pointing. It may not.

50 Best iPad Apps for Reading Disabilities from Teachers With Apps