Our character Hilda is a deliberate subversion of the idea that female leads (especially those that happen to be royalty) need to be strong and ‘feisty’ and morally upright and I guess GWJ picked up on that.
I’ve noticed from my feed — due to the publishing professionals I follow — that editors are on the look out for ‘concept picture books’, and meta picture books are big right now — those such as Herve Tullet’s Press Here. I’ve been looking at those manuscript wish lists (#mswl) and wondering why certain critics are so skeptical of book apps while at the same time embracing the meta. I don’t have a solid answer for that, but it’s great to see Betsy Bird acknowledging that (even if book apps can’t yet take off and fly), at least developers are having an impact on the wider landscape.
Any writer will have heard the following advice from Anton Chekov:
If you say in the first chapter that there is a rifle hanging on the wall, in the second or third chapter it absolutely must go off. If it’s not going to be fired, it shouldn’t be hanging there.
I’m reminded of this quotation when reading a different kind of advice: Tips for what to do and what not to do, for those of us producing interactive content for children. In The Art and Science of the Children’s eBook, Dr. Warren Buckleitner advises something similar to Chekhov:
And of course, I think of a page in our own app, Midnight Feast, which features a balloon. Incidentally, the balloon cannot be popped. Nor can the guitar be strummed. The books on the shelf cannot be read. The rug cannot be vacuumed.
I do remember that ‘balloon pops on touch’ was a part of our initial plans for Midnight Feast. Why did we plan it that way? For exactly the reason Buckleitner explains above. But as the story took shape, it made more sense thematically and symbolically, that the interactive part of this page shouldn’t be about balloons popping — which more naturally symbolises the ‘bursting of hopes and dreams’ (this comes later in the story) — but rather this scene became about ‘enclosure’. Roya is locked inside a small house, and so the nesting of the dolls on top of the nested tables are something I wanted the reader to contemplate. So we made those interactive instead. If you touch the dolls they jump inside each other. I coloured them luridly to suggest they might be a hotspot.
The balloon remains as part of the interior decoration, because this is a father trying to create a party atmosphere for his daughter. I justified it at the time as an ironic counterpoint to the main character’s emotions — the balloon has a big smiley face on it, but is upside down, to match Roya’s lacklustre expression.
Should the balloon be pop-able, nonetheless?
The creation of interactive stories is rife with pitfalls, and full of contradictory advice. Counter advice from the very same document cautions against ‘sprinkling an app with hotspots’ that do not support the story:
Why does the reader think that by touching a balloon it must pop? This expectation does not come from real life, because a balloon cannot be burst simply by touching it. A balloon may burst if poked with something sharp, granted, but I would argue that the reader expectation for popping balloons on screen comes not from any real-world analogue but from prior touchscreen experience. Namely, from games.
It is true, however, that when the user expects functionality and doesn’t get it, there is a micro-disappointment. This leads to several more philosophical questions:
To what extent should creators of interactive books cater to the easily won thrills which follow the user from gaming-world?
To what extent should we expect readers to ‘work’ for meaning? (By ‘work’, I mean ‘think’ — ‘Why is this part of the screen responsive, but not this one?’)
When creating artwork for interactive stories, is it better to ‘leave out the balloon’ altogether, if the user expects it to pop? To what extent do we cater to this?
And which parts of a fictional environment will the user expect to be interactive anyhow? Might this change over time?
Must the user know exactly what to expect upon first reading, or second? Or should interactive books open themselves up slowly, upon subsequent readings? Will users even read something twice if they must tap without reward?
Are there ways artists can signal unobtrusively to the reader which elements are hotspots and which are not, perhaps with colour? If I had made the balloon the same hue as the wallpaper, perhaps no one would think to tap it? Then again, would they even notice it?
The user expects the balloon to pop because they have popped many digital balloons on touchscreens before. As end users continue to interact with touch screens, and as touch screens become ever more integrated into our daily lives, no doubt user expectations will evolve. The question is, in which direction? And who is driving the evolution? Developers, of course. The more pop-able balloons that arrive in the touchscreen world, the more balloons will exist to be popped.
Unfortunately we weren’t able to attend the Bookseller’s Children’s Conference held in London last month, mostly because we live on the other side of the world. Nevertheless, I read children’s book news with keen interest. Kate Wilson of Nosy Crow was indeed at this conference, and I was interested to read her response to something which was said by a respected critic of children’s literature. Here’s Kate’s summary, which is what I have to go on:
Nicolette [Jones] said that she had “reservations” about picture book apps, on the basis that the printed book “does it better”, and went on to say that the “technology of the app interferes with the story”. She worried that “interactivity in apps replaces the space in children’s imagination”, and that “the app doesn’t go through the adult”. She said that the only apps she’d found successful were apps like the Touchpress Warhorse app, and Hot Key’s Maggot Moon app which provided additional material around each book, which, in itself, remains unaffected by the surrounding multimedia or animation material.
I’m not the slightest bit surprised to read this, and a large part of me wants to ignore it. After all, if you’re not ‘in the ring’, your opinion as a critic ain’t worth all that much to me. On the other hand, this kind of thing directly affects our sales. And sales are, unfortunately, relevant in this discussion. As noted by Philip Jones at FutureBook:
[The] combination of huge abundance and the difficulties of commercialising the products should not be under-estimated when looking at book apps. We seem to spend an inordinate amount of time discussing whether particular apps work, or don’t, but not enough time figuring out how the market conditions may be impacting these developments.
I have suspected a dismissive attitude towards book apps from longtime children’s literature critics and established printbook authors for a while now, and I’m glad to see it voiced, as it gives those of us in App Land a chance to respond to something concrete.
ARE YOU THERE, CRITICS? IT’S WE, BOOK APPS.
As you may notice from the digital badge to the upper right of our website, our second picturebook app Midnight Feast won a mention in the BolognaRagazzi Digital Awards earlier this year. There are few literature prizes open to digital picturebooks — most we must pay to enter and are therefore not worth a damn — but the BolognaRagazzi is the one big exception. So we were thrilled when our book app was judged as one of the top three fiction apps of the year (alongside an app by Nosy Crow, as it happens). Of course, a literature prize is not a running race — individual tastes of the judges are relevant rather than a millisecond on a stopwatch, and so we must always take literature prizes for what they’re worth. The fact is, any of the shortlisted apps as judged by the BolognaRagazzi panel is a heavyweight. If you’re interested in checking out the jurors’ commentary, you’ll see this is not an effusive bunch. Each member of the panel is suitably critical and careful when it comes to literary use and abuse of new technologies.
So, what happens to your app after it wins a mention in a big prize these days? Less than you might think*. Organisers of the BolognaRagazzi Digital Prize asked us for fifteen promo codes which were to be distributed to reviewers and critics over the course of the book fair. The wonderful advantage about being in the business of apps (rather than in printed materials) is that Apple provides us with unambiguous statistics. We can tell you via our stats that of the fifteen promocodes requested by Bologna, only three of them were actually redeemed, and none of them was redeemed by a user with a UK iTunes account. It’s possible that Bologna did not get around to sending out the codes, or perhaps they sent them to the wrong people at the wrong time. But when influential critics publicly dismiss the entire shebang, I’m inclined to err on the side of, ‘critics weren’t interested in them’. Lest it be thought that I am focusing these thoughts on the single critic Nicolette Jones, this is obviously a bigger issue. I am asking the question: Which influential people in Children’s Literature world (not Tech World, not Teaching World, not Parent World) are seeking out the award winning picture book apps before dismissing them? Nicolette Jones is not personally responsible for evaluating our book apps in particular, especially since we never sent her any promocodes. I’m pointing out that Children’s Literature World, in general, is increasingly closed to creators of picturebook apps. I do wonder if the promoters of award-winning printed picture books have any trouble giving them away at book fairs?
*To be fair, I’m pretty sure — insofar as anyone can be sure of anything when it comes to the Charlie’s Chocolate Factory which is Apple — that the mention of Midnight Feast in the BolognaRagazzi led to our previous app The Artifacts being featured in the App Store several months later. (Midnight Feast itself is and always will be a hard sell. Nor does it fit neatly into any App Store age category in the Kids’ section.) It’s easy to criticise Apple for failing to help book developers. Apple prefers to promote games. Philip Jones sees ‘very little evidence that Apple is doing anything to help with this transition,’ and I can’t really argue, except to add that Apple has done a darn sight more for our sales figures than any critic of children’s literature. (And I say this knowing it’s easy to say, AFTER a book app of yours has been featured.)
Unredeemed promocodes aside, I would like to share with you Midnight Feast’s sales figures for the United Kingdom* over the year since our ‘award winning app’ has been released: 52. Fifty-two downloads from the UK.
The reason I share all of this is because it’s highly unlikely (though slimly possible, yes) that one of the fifty-two people with a UK iTunes account to download Midnight Feast was the children’s literature critic Nicolette Jones. Children’s book critics are busy. Most wouldn’t have much time to seek out stories they’re not sent. With the avalanche of children’s stories published these days, a children’s literature critic can make a more-than-full-time gig out of sticking to printed picture books alone. And it’s true — we did not send Nicolette Jones a promotional code on the off-chance she’d take a look. She has no history of being a book app enthusiast. We sent one to Stuart Dredge instead, whose attitude towards apps we respect, and who indeed gave us a review.
My point is this: Nicolette Jones of The Sunday Times is a wonderfully knowledgeable critic of children’s literature, but I don’t consider her an expert on picture book apps. At this point, it’s frustrating that she offers an opinion at all, other than to open up the discussion, which is important. Making a value judgement on the other hand, is inappropriate.
UPDATE: I thought it only fair to offer Nicolette James some promocodes. Here’s her response. Irony, indeed.
I wonder if Jones realises what the self-described ‘irony’ is? That she comments regardless upon an area she doesn’t review? I have since replied that I wasn’t actually asking for a review (though it was fair enough that she assumed I was, given how often she would be asked.) I hoped only to change her mind about book apps.
(On a more positive note, Nicolette Jones has since agreed to receive the promocodes I offered.)
A well-known critic of children’s literature would not consider herself widely read in her area of expertise were she unfamiliar with the winners of the Newbery and the Caldecott. Yet apparently it’s okay to make sweeping statements about children’s literature apps, even when you haven’t familiarised yourself with the award winning products from the previous year. Dredge urges developers to use “the considered criticism from experts like Nicolette Jones” as “an incentive for more developers to strain to reach those heights”. But really, what are ‘the heights’ exactly? Instinct tells me to listen to the enthusiasts anyhow, and not the dismissive critics. Rule of thumb for life, that. There are plenty of critics complaining about all those terrible apps out there, but who is willing to put into words what exactly they were looking for when they were at first optimistic about all those possibilities?
There’s only one thing worse than a children’s critic slamming an entire category, and that’s failing to mention it at all. Part of me is glad that critics recognise that book apps count as books. A book app’s lack of an ISBN is problematic to its credibility as a work of literature. But the difference between a critic and a reviewer is surely this: A ‘reviewer’ is welcome to pick and choose from a subcategory of books according to the reviewer’s own interest, whereas a ‘critic’ has a responsibility to read and seek out** award winning and starred reviewed examples of a category before speaking about the category in public as a figure of authority.
Stuart Dredge quotes Jones directly:
“I’ve never seen a picture book-app that does something that a book doesn’t do better.”
**I know that critics are not accustomed to ‘seeking things out’. They are sent an avalanche of material every week (50 printed picturebooks for Nicolette Jones, according to reports from Stuart Dredge) and for them it’s a matter of culling them. But we are now in a different place when it comes to publishing, and Ron Charles explains brusquely in the Washington Post ‘No, I don’t want to read your self-published book’ why critics and review sites simply cannot respond to everything. I understand, fully. Our apps are self-published, and largely ignored. My response to that: Nor may they comment on the quality of everything. And if they are interested in commenting on the quality of what’s being self-published, a time-efficient way of seeking out the best would be to seek out the prize winners.
I would prefer Nicolette Jones to add, ‘but I am not a particular enthusiast of picturebook apps and I certainly don’t pretend to have done a wide survey of them’. Note that Jones spoke of the most expertly marketed picture book apps:
“I can see some publishers like Nosy Crow doing fantastically well with very interesting apps, and trying to reproduce the quality of a book. [Aw, bless!] There’s a lot of energy and creativity and intelligence going into this, and I don’t want to be too dismissive,” she said [dismissively].
This should be a red flag. Would you listen to a games critic dismiss mobile apps if he offered the example of Angry Birds to make his main point? The Nosy Crow picture book apps are the Angry Birds of App Picturebook World, known to anyone who’s had even the most passing interest in the category.
BUT WHY THE HIERARCHY, ANYWAY?
So, ‘I’ve never seen a picture book-app that does something a book doesn’t do better,’ she said.
I’m hearing two distinct but conflicting messages from those of you who know children’s literature:
1. Apps should be simple. You’re encouraging us to think very hard and long about interaction and animation. This is good. I’m thinking. Hard.
2. Apps have to offer something more than a print book does. For less cost to the consumer, by the way.
But we didn’t go into this industry hoping to add something more than print books can achieve.
Printed picture books are an excellent medium. I can’t see a single way in which the print book fails. [There are fewer ways in which to cock it up.] The best of them do a great job of sparking imagination, transporting children to other worlds, offering the gift of story and creating a love of reading.
Can a digital medium possibly offer more oomph than that? And should the savvy consumer expect it to?
When Jones says that she’s never seen a picture book app that a book doesn’t ‘do better’, I am very suspicious: Has she actually seen and studied picture book apps (like ours) which were created for a touch screen? Our products do not exist alongside printed versions. They exist in their own right. A comparison, let alone a hierarchy, is therefore difficult. In order to even make a statement that includes the word ‘better’, a critic would have to compare a printed picture book alongside its appified version. If a critic were to approach Midnight Feast or The Artifacts in this way, she would have to imagine-up a printed version of the story — one which exists only inside her head. Here’s the thing about things that exist inside heads: They are always better than any real-world product. (Any creative’ll tell ya.)
THE DEVICE ISN’T NECESSARILY ‘OVER THERE’
From Stuart Dredge’s summary of Nicolette Jones’ comments:
“If you look at a book with a small child, it’s a hug,” [Jones] said, making a gesture to show a parent with a child sitting in their lap, and an open book in front of them.
“With a device over here, there isn’t that relationship, and it doesn’t go through the adult,” said Jones, motioning to an imaginary tablet by a child’s side.
Cutesy hug metaphors aside, this is frustrating. I mean, it’s ridiculous to blame app developers for adults who may (or may not be!) using our products as substitutes for (rather than as complements to) parent-child interactive reading. Although I sense Jones isn’t talking about the ‘point of purchase’ when she says that a picture book app ‘doesn’t go through the adult’, it’s worth pointing out that an adult with a credit card is required in order for a book app to appear in front of a child in the first place. We’re not just leaving them lying around in gutters, for instance, like discarded bottles of rum, ready for two-year-olds to clap their chubby hands upon. Is there hard evidence, even, that an adult who has just selected and installed an app is throwing the device into the lap of the child before briskly leaving the room? Our own anecdotal evidence suggests that any adults who buy and share our apps care very much about what their children are reading. I see it in the quality of their App Store reviews. Ours are some of the most coherent on the store. (I know that’s not saying much, but still.)
THERE ARE ACTUALLY A FEW THINGS THAT PICTURE BOOK APPS CAN DO BETTER.
It’s true that interactive books can be somewhat overhyped. Developers are quick to label something as ‘educational’ in the same way food companies are keen to label products as ‘natural’. (i.e. It’s meaningless.) I have my own reservations about certain taken-for-granted enhancements; in particular I’m waiting for more research about the benefits of text-highlighting, or for a study which will lead developers into best practice.
I’m not a big fan of heavily animated book apps, mainly for aesthetic reasons. I’m very wary about games which pull a reader out of the story. Perhaps my biggest reservation of all about e-reading in general is that when a book exists on a device, it exists alongside push-button access to games, TV and the Internet. I’ve noticed from my own experience of reading ebooks on a tablet that it’s harder to become immersed in a novel when you know you can dart here there and everywhere to look up a word, then dilly-dally back to the story-at-hand via a quick few games of Boggle. Still, I’m not going to blame publishers of my favourite ebooks for that.
So sure, let’s take a moment to indulge in the undying need for hierarchies. I have given much thought to what book apps can achieve and what printed books can’t, partly as a way of deciding the future for our own indie company: Is it worth it, to spend years creating picture book apps when the environment is basically hostile? I might include a number of other more obvious benefits of the touchscreens themselves, such as the ability for a teacher in a classroom to share and discuss a picturebook via the big screen, without needing to buy a slightly larger printed version of a popular book, seen best by those sitting at the front of the mat, or of publishing advantages, such as the ability to sell your stories in Saudi Arabia, and to keep a book app in the App Store for as long as it needs to be there, even if it doesn’t immediately take off. (The Artifacts took 2 and a half years to be featured by Apple. If it had been on a bookstore shelf it would’ve most likely have disappeared forever after 6 months.)
Happily, we’re not the only ones asking questions about how touch screens might be integrated in children’s literature. The academics care a lot and some of them are all over it. Junko Yokota spoke on this very topic for her keynote at the International Research Society Children’s Literature conference 2013 in Maastricht. You can watch her speak here. I highly recommend it, to developers, reviewers and critics alike. But when it comes to ‘interactivity’, as far as we’re concerned at Slap Happy Larry, the main ‘interaction’ happens between the screen and the reader’s brain.
While some picturebooks are in black and white for economic reasons, serious picture-book artists who choose to aavoid color in a medium noted for its use of color often have similar special points to make.
The obvious example is the work of Chris Van Allsburg. The black-and-white pictures in both The Garden of Abdul Gasazi and Jumanji evoke the feeling of black-and-white still photographs that have been slightly over-developed to emphasize their contrasts. They are uncompromisingly objective and detached–unlike the world we see subjectively with our own eyes simply because they are so much like photographs. Paradoxically, we commonly associate black and white with uncompromising truth, utter absence of subjective coloring: documentary. Van Allsburg’s pictures have the quality of documentary, of detached observation that shows exactly what there is to see without the frivolous intrusion of color, and they are unsettling simply because what we see so uncompromisingly is often magic and impossible.
– Perry Nodelman, Words About Pictures
2. BLACK AND WHITE MAKES THINGS LOOK OLD, AND THEREFORE CLASSIC
The technology of photography has influenced picturebooks — and art in general — in a number of different ways.
One standout convention is that greyscale images make the reader think of the days before colour film was invented. This works even if the artwork is an illustration, not a photo. It works if the illustration is not even close to photo-realistic. The effect is made very clear when looking at these old images which have been realistically colorised. The effect is really quite stunning: We’re used to looking at wartime photos in black and white, which lends a comfortable distance to horrific world events. Yet when the same photos are colorised, the events seem much more recent and therefore have more impact.
The inverse works too. I recently watched The Last Picture Show, which was filmed in 1971 and therefore could have been shot in full colour, but the black and whiteness of it makes the town seem older. The story is set in the 1950s.
What about recently produced films set in the past? Why are more not set in black and white? Possibly it’s because in the digital age of film, cinematographers have ready access to coloured filters. A yellow hue cast over the background can lend an old-style look (as seen in Delicatessen, 1991).
I mention these films for adults because I’m not so sure the bulk of consumers and critics feel that ‘black and white’ is of any artistic use at all, let alone ‘appropriate for children’. Do adult gatekeepers accept the convention of black and white pictures in the stories they choose to buy for children?
The opening of the app is gripping and quite dark (which makes sense given the inspiration), so it is unlikely to really capture the attention of pre-schoolers (I say this because as you’ll read later I have a hunch they are partly considered a key audience). It is in a black and white, slightly sepia tone and harks back to the type of animation that is aiming to appeal to both adults and children – though here it probably will be of greater interest to children of elementary school age and older.
I sense the reviewer believes the youngest readers cannot be drawn in to a black and white/dark image. Is this something to do with physiology and the way humans have evolved to learn, or is it because two and three year olds have already learned from the culture that anything in bright colours is more likely to have been produced with them in mind? Perhaps this is what put the reviewer of Mac News World off the scent:
I must admit, I felt a bit duped by the description of the Numberlys app after I bought, downloaded and launched the app. I was expecting something bigger and longer that would appeal more to adults.
Was it the black and white look of the app which lead the reviewer to assume that The Numberlys was intended for an older audience?
The story is told through beautiful black and white animated graphics which are clearly inspired by the classic sci-fi film Metropolis, but with a modern touch.
…thereby picking up that the black and white is influenced by work that has come before — in this case, the work of Fritz Lang. Will children appreciate any of this? The example of The Numberlys shows that regardless of what children themselves think, reviewers (and I guess adult consumers) are likely to assume that media produced in black and white will appeal to adult sensibilities. A black and white story for children, therefore, better make sure it lives up to the huge challenge of appealing to a dual audience of children and adults alike.
3. A BLACK AND WHITE IMAGE TYPICALLY CONVEYS EMOTIONS BETTER THAN A COLOUR ONE
In a portrait with strong lighting, a calm face can suddenly look menacing, or vice versa. In this case, the lack of colour means that colour can’t interfere with the tonal contrast. The best example of this is the entire art noir movement.
Perry Nodelman continues with his example of Chris Van Allsburg, and the contrast one can achieve via black and white:
Furthermore, the heavy contrasts of these pictures emphasize the patterns created by the various shapes and so do the black lines that outline each shape, so that the relationships o these shapes on the flat surface of the page are as significant as the relationships of the figures the shapes represent in the three-dimensional picture space. As a result, and as happens in photographs with high contrast, the often intense action the pictures depict is slowed down, held by the patterns; like still pictures of people caught in moments of fast action, the pictures depends to a great extent on these paradoxical relationships between what is depicted and the photographic techniques used to depict it–between our expectations of documentary truth and our perception of magic, between activity and stopped time.
MODERN ALTERNATIVES TO BLACK AND WHITE IN PICTUREBOOKS
There is nothing wrong with black and white as an artistic decision. But is there a ‘hybrid’ decision that can be made about colour, one which will satisfy the artistic goals of a limited palette as well as consumer expectation that children need colour?
In the digital era, illustrators can put a yellow tint on a picture and it immediately looks a bit old, but not too old (in which case black and white is good). I have no idea whether ‘yellow’ meant ‘aged’ before that crappy film did the rounds in the seventies, but there you have it. (See Delicatessen, above.)
It will be interesting to see how further developments in technology influence colour choice in art. With everyone sticking filters on things, the filters themselves are sure to come and go. Perhaps when we look back at the twenty-teens, we’ll see ‘iPhone filter’ stamped all over our family shots. So why do we do this? Perhaps photography got too good. Maybe we like the overexposed look, because one thing black and white early photography was very good at was adding a touch of glamour. And who needs every blemish magnified with a 50 megapixel camera?
If you want to create a retro-looking illustration, you can also limit your palette to the few colours that were available to printing houses way back when. Many older illustrations are red and black simply because the publishing houses couldn’t afford a wider range.
And here’s the article to which the poll is attached: Jury out on children’s book apps. As an overthinker myself, it’s good to know that pontification is alive and well. The article offers no evidence of the harm of picture book apps, just as it offers no comfort to those of us who have embraced the medium.
I’m reminded of this article from Charlotte’s Library, in which Charlotte wonders if certain literary sessions run at libraries might be doing more harm than good. I have similar concerns about seminars about interactive storytelling which almost seem to start with an assumption of harm. Or maybe it’s just the headlining of such events, because I’ve never attended one in person. Either way, might such discussion, in the absence of evidence, be influencing children’s literature itself?
Might our very best writers, illustrators and storyboarders be scared away from digital reading technologies if the kidlit subculture becomes infiltrated with doubt?
…the vast majority of picturebooks are created for children. If we wish to be clearer about the nature of the picturebook should we attend to what children make of them or will our own close reading of individual texts be sufficient? And how relevant is it to our attempts to understand picturebooks that they are often used for teaching children to read?
– from the introduction to Reading Contemporary Picturebooks: Picturing Text by David Lewis
As children we relate to our picture books in a holistic fashion, merging sensations of the eye and the ear (for first we are read to), which marries the image and the sound of the words, and later, as we learn to read, the look of the words.
– How Picturebooks Work, by Maria Nikolajeva and Carole Scott
It’s not surprising that research on a new medium happens only after the new medium comes into existence and gains a foothold in culture. Since interactive storybook apps are so new, there is still relatively little research that has been done, and when making development decisions, developers are instead reliant upon our own commonsense, and inevitably, our own experience of literature and reading.
One of the assumptions to have arisen about the nature of ‘good’ storybook apps is that they include word or phrase highlighting synchronous with narration. The assumption: that word highlighting is beneficial for emergent readers.
At this point, the beneficial nature of text highlighting is an assumption. It may be of benefit. It may not be. And it is also possible that word highlighting actually does more harm than good to an emergent reader.
Why this assumption in the first place? I think word highlighting is often considered the digital equivalent to pointing at words with a finger, and many are under the impression (rightly or wrongly) that when a caring adult co-reader points to words as they read, that the child will pick up reading — as part of a much wider program to teach reading skills, of course.
So before focusing on the topic of word highlighting, I would first like to look a little harder at the finger-pointing assumption.
Researchers claim this is the first time a study has shown a link between referencing during reading and literary achievement in later life.
So, if there have been many good studies on the effect of pointing to words on emergent readers, they haven’t been widely published.
Let’s go with that and trust our parental instincts: that occasionally pointing to words in books, and drawing children’s attention to various technical aspects of reading does improve literacy. I’m not going to argue with that because I intuit this is the case.
I don’t think I’m alone in this, because there seem to be many app developers who intuit that pointing to words by a trained or careful adult can be emulated electronically in a storybook app.
This, I’m not so confident about. Pointing to words may be really quite different from animating individual words in digital stories:
When pointing, the finger does not obscure the actual word. Instead, effective pointers would surely place their fingers BELOW the word in question, not over it.
Also when pointing, the fingers are not making those jerky movements reminiscent of colours flashing on a screen. The hand glides across the page unobtrusively. Emergent readers may well be less distracted by a hand than by digital animation of words.
Fluent readers do not read by looking at one word at a time. We take in three words at once. While it’s clear that early readers need to learn words one by one, when it comes to training the eyes to move across the page, is it really that helpful to highlight words individually, especially when the narrator is reading fluently themselves? I wonder about what we are modelling when app developers choose to individually highlight words.
It’s possible that some ways of highlighting words are better than others. We need more research into this. It’s not enough to simply assume that ‘apps with word highlighting are good’ while ‘apps without word highlighting are lacking’.
Here are some various ways of word highlighting that you’ll see in some popular storybook apps right now.
1. JUMPING WORDS
Sir Charly Stinkysocks and the really BIG Adventure
This is a storybook app produced by a large publishing house. The words ‘jump’ off the page as they are read. But when a word is jumping, it’s moving, and therefore not able to be read. All the emergent reader can see is where in the paragraph the narrator is up to; they can’t see the word itself. Not unless their own reading is actually out of sync with the highlighting.
Here is another app which makes use of the same technique:
Logan and the upside-down sea
2. FLOWING TEXT HIGHLIGHTING
Perhaps to avoid the choppiness which results from highlighting words individually, this app developer decided to make the word highlighting last slightly longer than the narration itself. The colour that appears around the words fades out slowly, so you end up with an ‘approximate’ highlighting of words. It certainly works to avoid that choppy feeling that happens when words jump.
But if the highlighting isn’t 100% accurate, leaving the reader perhaps one word behind the ballgame, might this be worse than no highlighting at all? We don’t know this yet.
3. HIGHLIGHTING OF PHRASES
Cozmo’s Day Off
I prefer this method of word highlighting, where phrases are highlighted rather than individual words. This emulates the way we read as fluent readers – not just by taking in a single word at a time, but by encouraging us to take in several. This may aid reading fluency, and fluency aids comprehension.
I suspect this book has it right. If words are to be highlighted, this is how I’d like to see it done. I like that the words themselves don’t move. Instead, a blue outline appears around the words. This doesn’t prevent the reader from actually reading them.
I suspect that the highlighting of individual words is useful in word games in which emergent literacy skills are the target.
I suspect story app developers should stay away from individual word highlighting, and consumers should be wary of expecting it by default.
Just because something is possible with the digital format doesn’t mean it’s an improvement on non-digital versions of a story.
For now, app developers who use word highlighting as a selling point are making money based on something which doesn’t have good research behind it.
The option to turn off word highlighting should be an option, just as it’s an option to turn off narration.
I’m prepared to change my mind on this. The issue of word highlighting in storybook apps desperately needs research. But we can’t assume that highlighting equals finger pointing. It may not.
David Lewis also argues a case for interactions in picturebooks being inherently metafictive in that they inevitably bring readers out of the story itself:
Books such as these … foreground the nature of the book as an object, an artefact to be handled and manipulated as wella s read. They are thus metafictive to the extent that they tempt readers to withdraw attention from the story (which, it must be said, is often pretty slender) in order to look at, play with and admire the paper engineering. One of the characteristics of a well-told tale is that as we read it our awareness of the book in which it is written tends to fade away, but when the material fabric of the book has been doctored in such a way as to draw attention to itself, it is less easy to withdraw into that fictive, secondary world.
Ultimately, Lewis considers interactive picturebooks as valid artifacts in their own right — a cross between books and toys:
Pop-ups and movables tend to produce a degree of unease amongst children’s book critics and scholars for they often do not seem to offer much in the way of a reading experience at all. For this reason they are sometimes considered to be more like toys than books, objects to play with rather than to read. There is some justice in this view, but it is far too simplistic for it tidies up too neatly something that, if we are honest, rather resists pigeonholing. We might better understand the world of the movable if we view it as a hybrid, a merging of two, otherwise incompatible artifacts: the toy and the picturebook.
I would argue instead that interactiions in picturebooks (whether printed or digital) come in various forms, and can be manipulated by careful developers to either pull readers out of the story or to draw them in deeper. Interactions are therefore not necessarily metafictive.