Apart from the vast resources and expensive software and the need for highly trained personnel there is another reason for avoiding too much animation in storybooks, regardless of ethical considerations regarding whether books should even be trying to emulate animated film anyhow, and it is demonstrated in the picture below:
As you can see, it’s a purely aesthetic one. The character in the top picture is beautifully rendered, depicting the light coming from above. The character below is what we generally see in even the high budget animated features, such as those from Studio Ghibli, because the time and resources required to have shadows fall naturally and beautifully upon a moving figure is vast, and impossible to achieve unless moving into the super-duper expensive realm of 3D — which has a distinct look all its own.
Here is a screenshot from Spirited Away:
Notice the backgrounds of Spirited Away are of a different, detailed style than the characters. This is a legitimate and successful style, with the simplified characters looking distinct against the backgrounds — overly detailed characters may well make the screen too cluttered. Characters are instead given form with solid fills of darker colour, with a maximum of two tones per object — Sen’s hair, for example, is one tone for the part of the hair which catches light, and a darker tone to depict the shadow.
A more photorealistic style would depict many shades of brown. When characters in apps are animated, a more ‘painterly’ or more photorealistic style becomes unattainable. This is something to consider whether developing a storybook app or judging a storybook app for its animation, or absence thereof.
But in many works, of course, these two different styles is not an unfortunate consequence of animation but a stylistic decision — as Perry Nodelman writes in Words About Pictures:
Illustrations do have a narrative purpose. They must show us not just beautiful patterns and evocative atmospheres but what people look like as stories happen to them; that is, as they move and talk and think and feel. So their faces and bodies usually have the simplicity, and consequently the expressiveness, of cartooning, a simplicity at variance from the frequent richness and detailed accuracy of their backgrounds, which give us a different sort of narrative information. When faces and bodies do have the same solidity and detail of shading and lines as their backgrounds, they may come to seem static and inexpressive…The extraordinary expressiveness of cartooning seems to make it a particularly appropriate means of communicating narrative information. To suggest that all picture-bo0k art is a sort of cartoon or caricature is no insult; it merely stresses the extent to which the purposes and pleasures of this art differ from those we assume of other kinds of visual art.
I remember asking a friend what his favourite music was. “Indie,” he replied.
He was expressing a political preference rather than a genre, because the fact of being independently produced is in itself a defiance of established norms, but at the time I was perplexed. How can anyone so broadly say “Indie” as a genre.
Then I knew someone who listened to a lot of different indie artists and sure enough, I knew what he’d meant: ‘indie music’ has a certain feel to it. My knowledge of music isn’t strong enough to encapsulate exactly what that is, but I’ll go with ‘complaining male voice almost drowned out by instrumentals’ and you might get a sense of it.
Have you seen Pixar’s short animated film The Blue Umbrella yet? I haven’t, but I’m looking forward to catching it sooner or later because unlike a lot of the most recent animated films, that sounds original and the screenshots look stunning.
Here’s what the creator says, remembering his pitch to the two guys at the head of Pixar:
“They are the strongest proponents of ‘animation is not a specific look.’ It feels like animated movies have a certain style, but there is no reason for that. Animation nowadays can be whatever it wants to be. With the first Toy Story, you could only do plastic, metal, stone, but you couldn’t do people. It took until The Incredibles–the first time there were humans in digital animation. Now you don’t really have that barrier anymore, it’s just that everyone got used to what animation looks like. Everyone [here at Pixar] really likes the idea of pushing animation into areas no one has thought of yet, at least in mainstream animation.”
And here are a couple of animated movies which have caught my eye lately. Unfortunately, they’re not always that easy to get a hold of here, but I have these on my to-watch list simply because they look animated in a completely original kind of way:
A CAT IN PARIS
THE TRIPLETS OF BELLEVILLE
Related: It’s not déjà vu. Summer movies are often described as formulaic. But what few people know is that there is actually a formula—one that lays out, on a page-by-page basis, exactly what should happen when in a screenplay. It’s as if a mad scientist has discovered a secret process for making a perfect, or at least perfectly conventional, summer blockbuster, from Hollywood and Blake Snyder at Slate
App developers would do well to remember that when it comes to providing a reading experience that is developmentally valuable for young children, it’s as much down to what the app doesn’t do, as what it does.
Many of the first digital picturebook apps (‘storyapps’ for short) produced for the tablet computer are created digital versions of real-book interactions (with the exception of scratch-and-sniff!), mimicking printed picturebooks in a retro kind of way. This is perhaps in a bid to win over reluctant audiences who are not yet convinced that digital picturebooks can create great memories in the same way physical, printed matter can. (See for example Popout! The Story Of Peter Rabbit, an iOS app which mimics a printed pop-out book.)
a skeuomorph is a design element of a product that imitates design elements that were functionally necessary in the original product design, but which have become ornamental in the new design.
Apple themselves make use of skeuomorphism, evident for example in their Garage Band app, in which the backgrounds look like grey leather, even though devices themselves are reflective plastic. iBooks looks to be set upon wood, and readers of iBooks place digital purchases upon something that looks like a wooden bookshelf. There is no real functional reason for the mimicry of real-world materials, except that real-world parallels help intuitive navigation through an app, and also imparts to consumers the feeling that they are not simply buying air when shelling out for bits and bytes, but are receiving an actual product, which costs real money to produce, market and update. It will be interesting to see how design principles evolve as smartphones and tablets become ubiquitous. Likewise, it will be interesting to see whether future generations feel a sense of value when spending money on digital media.
The illustrators of printed picturebooks have been creating skeuomorphs too*, since the emergence of digital art software. Few illustrators today produce illustrations without incorporating software at some part of the process, yet apart from a brief period in the 1990s when it was novel to make an illustration look digital (the art equivalent to pop-musical synthesizers), many digital illustrators now go out of our way to make sure the reader can see ‘the hand of the artist’. Hence, we might make an illustration look like a woodcut by making a ‘woodcut’ brush in Photoshop, or by erasing parts of a black layer with an eraser. (Also known as hedcuts.) And although it’s now possible to create photo-realism in illustration making use of powerful software, there seems to be still a general preference for artwork that looks as though it has been created using real-world media such as watercolours and oil paints. (Tradigital art.)
When it comes to storyapps, some of the major developers at present seem to be in limbo**, appeasing consumers who might still feel that physical, printed matter is superior to similar stories presented digitally. This applies not just to art style of course, despite the example above, but also to programming decisions.
We posit that when storyboarding for a storyapp, there is no real need to be limited by the constraints of printed matter. When stuck in limbo, storyapp developers end up compromising between what is digitally possible, and what consumers will accept in a definition of ‘picturebook’. For example, storyapps do not need to be either 32 or 24 pages to accommodate printing restraints. Yet readers have been primed to expect a standard picturebook of 32 pages***, so how far either way can storyapp developers push the boundaries before a reader feels a storyapp is either too long or too short? Does interactivity lengthen the reading event to the point where shorter picturebooks are preferred? Will readers happily accept a longer one? These are all questions currently being put to the test.
*There is another way of thinking about the different types of interactive stories, this time making use of film terminology: Native Interaction, to describe stories which have been created from scratch with the touch screen in mind; and Post-Processed Interactivity, to describe printed matter, often with a wide, adoring and attentive audience, which has then been adapted for the touch screen.
**It would be a mistake though, to assume that printed picturebooks are immune to evolution. Picturebooks have undergone consistent reinvention along with printing technologies, exploding to life after the introduction of offset lithography. Printed matter will continue to evolve along with digital storybooks.
***In principle, a storyapp can be as short or as long as it needs to be, with restrictions coming not from paper and printing but instead from memory allocation, in a rather nebulous guideline from Apple stating that apps need to play nicely with each other.
Moving The Story Forward
Knowledgeable folk of storyapp world — reviewers and developers — advise frequently that interactivity in a storybook app should move the story forward. I have spent the past few years unpacking this simple sounding phrase. ‘Meaningful interactivity’ is a ‘I know it when I see it’ kind of a term, but do we really know what it means? It pays for we as developers to unpack it fully.
Below, we make a case for various reasons for interactivity, where not all kinds of good interactivity ‘move a story forward’. Some interactions and animations exist purely for world-building; others for humorous effect. Some of the most sophisticated interactivity in storyapps may in fact contradict both words and pictures, providing an extra layer of meaning, and an added challenge for increasingly sophisticated readers. Hypothetically, storyapps can require more of a reader than printed matter can. (Digitisation of picturebooks certainly does not equal ‘dumbing down‘.)
The Various Purposes Of Touch Interactivity
For example, a reader touches an item on the screen and the written word appears. (Pop-out words.) Since the Mercer Meyer book apps became popular I see this from smaller publishers too. But developers need to be careful that there is genuine benefit here. If a reader touches a chair and the word ‘chair’ appears, the app is obviously aimed at either young readers or at NESB (non-English speaking background) readers. (Obviously I’m writing from an Anglocentric point of view — same is true for any language.)
If a reader cannot be expected to be fully familiar with the written word ‘chair’, however, it should also be assumed that the audience has very emergent language skills in general. Such a story should unfold mainly in pictures, with text pitched at the basic level expected of a reader who has yet to learn the word ‘chair’. But is it really adding anything for older, native readers? Pop-out words need to match the general level of the app. It’s easy for a developer to add this sort of faux-educational interactivity, but much thought should be given to which items of vocabulary the reader is likely to know. Otherwise, it’s all just distraction. I feel that kids recognise pedagogical intentions, even if it is just vocab and spelling.
That said, pop-out words may indeed result in an educational outcome, if done properly. In cases where text expands the picture (most often it’s the other way around), if parts of the picture are labelled on touch, the reader is encouraged to find all the words mentioned in the text, in a fun,Where’s Wally sort of a way. The pop-up words provide a sort of confirmation (‘You’ve found me!’), allowing the reader to move on.
Pop-out words can also be useful if the words are in fact new to the reader. For example, a storyapp adaptation of Elsa Beskow’s Children Of The Forest (1910) might teach the reader the names of the plants, which are botanically accurate. In order for this to be useful, I feel that the pop-out words must be at least one level of specialisation above the vocabulary used in the main text. So the reader would be expected to know ‘flower’ and ‘tree’ (main text) but is guided to learn more with a pop-out ‘snowdrop’ and ‘lingonberry’. I learned my hues from a 64 box set of Crayola crayons, so I know from experience that labeling can work to expand vocabulary, but only if the child reader has fallen in love with the story in the same way I fell in love with my crayons.
Another use for labeling parts of pictures is when making use of text within pictures (a.k.a. intraiconic text) without the intraiconic text competing for attention with the main text until the reader is good and ready, ie, on user touch. An example of intraiconic text is when, for example, an illustration depicts books on a bookshelf, which each have their own titles, or when illustrated shops display billboards. Ticket stubs, restaurant menus, letters, computer screens… there are many, many plot-advancing reasons for displaying intraiconic text within an illustration. The iPad screen does not afford the real estate of the larger printed picturebooks, so a developer might get around this limitation somewhat by having a magnified view of a sign or letter on user touch — too small to be seen beforehand, but read easily on iPhone sized screens on touch.
There is a place for humour and pure fun in story apps. Ideally, novelty interactivity should marry with interactivity that moves the story forward, or it should at least lead the story off into its own mini tangent. For example, the farting teddy bear in Teddy’s Night (by Bruno Hachler) is mainly for novelty value. I’ve noticed with my own toddler (and heard from other parents) that young children are inclined to skip straight to this page, then linger on it, presumably without getting into the story itself. So overly attractive novelty interactions are a double-edged sword: they encourage children to go back to your app over and over, but as far as brain candy goes, they aren’t necessarily any better than having them watch junk TV. It might be argued that these sorts of interactions inspire a love of reading. I’m inclined to think they inspire a love of novelty. But this is nothing parents can’t fix by guiding children through the story from start to finish.
As far as novelty interactions go, there is the full spectrum. The best seem to advance the setting of the story. An example is the floating swimmer in Heart And The Bottle, in which the swimmer shoots up a small fountain of water from her mouth on touch. The words on the page say, ‘With wonder at the sea’ — nothing about floating in the ocean, shooting seawater from your mouth. In this case, the shooting water is both fun for the reader to do and also adds to the atmosphere of the setting: lazily relaxing in the sea. This mood is reinforced with the sound of the ocean and cry of seagulls which autoplays in the background.
Another way in which novelty interactions can add to the setting is when, say, the user touches a bath tap which fills the bath with water. It can be fun to fill a virtual bath with virtual water, and perhaps draws readers into the story by helping them to feel that this storybook world is real. Novelty can add to the verisimilitude.
Some interactivity offers a surprise, reminiscent of that scene in Tim Burton’s 3D Alice In Wonderland movie, in which a ball flies toward the audience as the characters are playing a game. I remember ducking in my seat. Annoying as that was, it was memorable.
Interactivity can be used to create a ‘pageturner’ — a book that children must read to find out what happens. This sort of novelty, which encourages movement forward through the story, is a positive thing, though not when it comes at the expense of comprehension. This is a difficult balance to achieve, and depends somewhat on the age of the reader and all sorts of external things an app developer can’t control for (like the presence of adult co-readers, slowing the child down). The issue storyapp developers face is, where to put the interactive ‘pageturner’? That’s the conventional place to put something charming or interesting, to encourage the child to turn the page. Interactive story apps require that space for actual digital pageturning, be it push-button or swipe (which is even harder to incorporate). It’s likely that readers of digital stories need to learn a slightly altered story code. This will only work if it can happen intuitively and subconsciously, just as it happened when we all learnt to decode printed picturebooks.
One thing is clear: Developers should be able to rationalise every hotspot on a page. The ‘distraction issue’ isn’t limited to storyapp developers, though. The producers of printed picturebooks are equally capable of unconscious distractions in their work. From How Picturebooks Work:
In pictures…with many details, our reading is arbitrary. The artist may deliberately or unconsciously place a detail in the picture that will attract our attention and compel us to start reading the picture from this point.
While the developers of picturebooks are often great lovers of the form, and have spent so much time immersed in them that even the unconsciously placed details are meaningful, our choice to include interactions and animations in storybook apps is best if it’s the deliberate kind. The medium is perhaps too new, even for lovers of the form, to have developed a purely intuitive sense of what’s right. Besides, studying the form can never hurt.
“One of the things we love about the still image is the way in which it can stimulate the imagination to create a fiction around an image,” said Robin Kelsey, a professor of photography at Harvard. “The fact that we can commit a single image to memory in a way that we cannot with video is a big reason photography is still used so much today.”
You might call this ‘Paratextual Movement’. This kind of animation is not necessarily ‘interactive’, because animation can autoplay after the reader turns onto the page. Or it might be ‘semi-autoplay’, for example the reader taps anywhere on the page and something starts to move, regardless of whether the reader has engaged with that part of the picture.
This kind of animation can fulfill a limitation of picturebooks: animation to depict the flow of movement, long employed by film-makers. The words might say, ‘The boy jumped’. Then, when the reader touches the boy, he indeed jumps. This doesn’t advance the story but it does help with story comprehension in readers whose language skills are in the emerging stages. In other words, the reader doesn’t need to be able to read the text at all. This kind of interactivity is of limited value to older readers who are able to read the text for themselves and who remain unchallenged by paratextual movements.
For books aimed at competent readers though, I prefer animation to exist without the redundancy of words. If the boy on the screen jumps, then the words don’t need to tell us he jumped.
3.2 Animation To Advance The Plot
You might call this ‘Extratextual Movement’. The animation adds something to the picture which the words do not duplicate, thereby making it redundant.
This is the sort of interactivity to aim for in stories for older readers who are better-versed in reading pictures. At times this sort of interactivity creates a gap between the story and the text; at other times it helps the reader to fill it in. What results is a story in the reader’s head that is bigger than the sum of text/pictures and interactivity.
Printed picturebooks sometimes rely on a series of pictures to depict the passing of time and a series of events, which may or may not be dependent on each other. (Simultaneous succession.) But younger readers need time to decode this convention. A younger reader may look at five pictures of a dancing bear and mistakenly believe that there are five different bears rather than a single bear in five successive poses. Animation does away with this confusion, and therefore animation has its place in storyapps for younger readers.
On the issue of depicting the passage of time, this is one area which may be made easier for stories with animation and touch interactivity, as it is a difficult thing to convey in static pictures, without specific reference to a clock. But an interactive book can dim the lights or show hands moving round on a clock or have a cock crowing outside a window or something like that to show that time has passed.
5. Interactivity to Depict Internalised Thoughts/Emotions Of A Character
Something picturebooks and film must do differently from chapter books and novels is the depiction of a character’s inner life. This is not easy in visual media, and interactivity allows the possibility of helping with this task. I’m not seeing much of it, though.
For example, emotions can be depicted by changes in hue and saturation. This is a customary technique in visual media and sometimes in print picturebooks the reader turns the page to see a new colour palette. This change in colour could become part of the interactivity, with a slow fade out of saturation or with a rub-to-reveal portion, or with isolated areas of colour blending to create a new picture. The possibilities are endless, and these are just a few examples that spring to mind.
Or the user might ‘zoom in’ on a character to switch from omniscient to close-third-person point of view. Or developers might borrow from different graphic codes such as photography and camera work, making use of pull-focus, blur, motion lines and distortion of perspective.
Story app developers can also take advantage of comicbook conventions, with thought and speech bubbles, or with mimesis and onomatopoeia emerging from objects on touch. (See Scott McCloud’s Understanding Comics. TED Talk by author here.)
6. Pure Animation
Some interactivity exists purely to give the storybook an animated feel. Blinking eyes, nodding heads, walking legs can give a storyapp a polished feel that places the genre somewhere between printed book and TV. Sometimes these movements are not interactive but autoplay. This may be a better option. When planning interactivity, I do wrestle with where to draw the line. If you start out animating too much, the reader will expect the same level of animation on following pages. Do we really need to make eyes blink? Or should eyes blink only when there’s a reason to, for example because the scene has changed. Maybe the character’s eyeballs could move because the thing that’s drawing her attention has moved to the other side of the page. There is a certain amount of intuitiveness necessary when planning for animated interactivity. The blinking eyes are necessary in Teddy’s Night because the teddy and girl being awake when they’re not meant to be is an integral part of the story. Blinking eyes in other storyapps won’t necessarily move the story forward.
Other examples: clouds move across the sky, wind blows leaves on the trees, small animals peep out from bushes, birds fly across the sky.
Some storybook apps with higher budgets pull off entire animated scenes. Many of these are made from movies and short films. An example is The Amazing Flying Books Of Mr. Morris Lessmore. These sequences tend to have very high production values and turn a storybook into something between printed matter and a short film. Although these animated sequences tend to be set off by the reader, who turns a page or touches somewhere on the screen, I don’t consider them true interactions, because all that is asked of the reader is to sit before the screen and watch, immersed. As long as these sequences create immersion rather than impatience and distraction I don’t take issue. Here at Slap Happy Larry we both have a personal preference for story apps which function closer to printed matter than to film, but we are film lovers equally.
6. Extra-textual Interactivity
In some storyapps you find some dalliance that takes you out of the story for a while. Jigsaw puzzles are one common example. Heart And The Bottle has a page in which the reader draws on a piece of paper. The drawing appears on the wall in the next scene, which has some novelty value. It also shows that the main character is using her creativity by drawing pictures, and is therefore connected to the story. Similar interactivity exists in Teddy’s Day, in which the user touches the girl’s paper, flipping to another screen. Another page in the same app shows the girl building a block tower. On touch the user is taken to a different screen in which the physics engine has been employed to show the blocks floating midair. I have thought about how this advances the story, but concluded the developer enjoyed the novelty of the physics engine.
I do feel developers need to be careful though, because in being asked to draw a picture, apropos of nothing, the story turns temporarily into an art app, and the reader is first expected to work out how the mini-app works, and then to come up with something out of thin air before moving back into the story, which they’ve just been pulled out of. I liken this to a classroom experience in which a teacher gathers the class onto the mat for storytime, reads half the story, then asks the children to get out paper and pencils, draw a picture, then draws them back onto the mat to finish reading the story. Obviously this is not good teaching practice, and obviously it takes less time and organisation for a single child using a single iPad to move in and out of story/artistry. Nevertheless, I wonder if at the most basic level, we’re doing exactly the same by embedding fancy technologies into the middle of stories. At the very least, we’re changing the reading experience, turning it into metafiction by drawing attention to the fact that it’s a story.
I prefer to see puzzles, colouring activities and other kinds of gamification at the end of a story, or available only via the main menu, if at all. I hope extras do not become an expected part of story apps. I hope they don’t become a cheap drawcard to get children opening an app.
But while certainly a little peculiar, the toilet-paper-selling and jeans-hawking ultimately aren’t all that weird. In fact, they’re actually a reminder, kinda strange and kinda funny, that books are part of a commercial ecosystem that moves stories and steam cleaners in very similar ways. That may be especially true of the Kindle, considering Amazon’s totalizing ambitions and aggressive merchandising, but it applies more broadly, too. Books are, in many ways and in many contexts, simply commodities. They aren’t sacred, and they aren’t just disembodied ideas; they are things (whether of paper and ink or bits and bytes) first of all, and they move around like things do.
It’s worth pointing out from the get-go that advertising in children’s literature is far from new. (See Advertising In Children’s Literature, a paper by Afsana Kahn.) Unfortunately, interactivity, and the need to recoup funds spent on producing expensive storyapps lends an extra dimension to all things kidlit, not least in the advertising opportunities. Also, as pointed out by Nikolajeva and Scott in How Picturebooks Work, authors have long amused themselves by including references to their own former works, for example by drawing a picture of one of their own books lying on a bedroom floor.
Several developers have done a similar thing in their apps. In Teddy’s Day, for example, if the reader touches a bookshelf they are taken to a page advertising Teddy’s Night, the companion story. Yet unlike the print equivalent, the reader has touched the book expecting a different kind of interactivity: There is no warning that this is going to take the reader out of the story.
While this sort of promotion is common and almost expected in gaming apps, it’s unfortunate for story app developers that literature has a revered position in most people’s hearts, perhaps to an unreasonable degree. Nevertheless, marketing embedded within storyapps is something we hope consumers learn to avoid. Unless consumers boycott such practices they are likely to continue.
That’s not to say that there isn’t a place for non-story related activities embedded in the same app as the story itself. After all, teachers have always made use of extra-narrative activities when guiding students through the understanding of any given work of art. Also: Playing With Books Is Important Step in Path to Early Literacy. Might this apply to the digital literacy of eBook reading as well?