Words Academics Are Using To Describe Book Apps

Mostly from the papers:

Engineering stories? A narratological approach to children’s book apps

Multimedia book apps in a contemporary culture: commerce and innovation, continuity and rupture

 

Apps

Short for application software: Software designed to accomplish specific user tasks (in contrast to “system software”).

eBooks

While e-books are single files that require specific software (e-reader software), apps (being software) run by themselves.

Multimodal Narrative

A digital story is defined as a ‘multimodal narrative’ text comprising  pictures, music, speech, sound and script.

Remediation

Whereas books are ‘adapted’ for screen, books are ‘remediated’ as apps.

Continue reading “Words Academics Are Using To Describe Book Apps”

Slapping On Filters

The Horn Book has published a balanced and interesting article about digital and traditional artwork found in modern picturebooks, in which case some reviewers and enthusiasts are keen to know exactly how a piece of art has been created. The article wisely advises in its title: Just Enjoy The Pictures.

However, something from the following paragraph pulls me up short:

While many people embrace the notion that the computer is merely another tool in an artist’s toolbox, there also exists a disdain for art that tries to be something it isn’t, such as digitally created artwork that attempts to look like it was rendered in oils. Why go through the trouble to fake it when you can do the real thing? Why slap a filter on it to make it look like oils instead of taking the messy risk of working with actual paint?

I can’t work out from the article whether the writer realises that no (published) digital artist is simply ‘slapping on filters’.

So I’d like to make it clear here: Filters are about as useful as tits on a bull. There’s no ‘slapping one on’ to achieve any artwork with soul. Indeed, I don’t even know why Photoshop ships with filters. I haven’t yet noticed a picturebook artist who has found a use for a single one.

However, some artists may be making judicious use of filters. Judiciously. Maybe. With lots of mods.

Which leads me to my next point, a response to the clip from a new (niche) documentary called Making It, in which illustrators talk about their work. I haven’t seen the film. There’s a clip on video in which an illustrator called Anthony Francis Moorman talks about various things illustration related — how he enjoys drawing boobs and skulls and not buildings — then he goes on to  say that he takes great pride in the fact that his drawings are done by hand. He says that because of all the art software out there now, a lot of young illustrators are doing their work entirely on computer, and ‘it has no soul’.

As usual, I tend to question cause and effect.

1. ‘Soul’ and ‘spirit’ etc. are religious terms, which are trying to describe something else. ‘Soul’ is such a nebulous term that I wonder what it really refers to in relation to art. Does it mean, perhaps, that the slightly shaky lines, imperfectly coloured, are more attractive than the dead-straight lines which have been auto-filled in a program such as Adobe Illustrator? Does the ability to make perfect perspective with the guides in the latest versions of Adobe Photoshop take something away from the slightly skewed, morphed, unnatural perspective achieved by hand-illustrators?

2. Assuming that this is what is meant by ‘no soul’, is there no soul because the drawings have been done on a computer, or are the drawings soulless because they are being done by people who’ve had no real training in the art of perspective, shading and colour theory? In this case, the computer is not the problem; rather, computers may be standing in the way of a broad art education.

3. ‘By hand’ is an interesting term to those of us who paint digitally using a Wacom tablet. Because when I’m drawing on paper I’m using a pen-like tool. Similarly, when I’m drawing via computer, I’m using a pen-like tool. The degree to which I am able to replicate the experience of drawing on paper is determined by the amount of practice I’ve had: It takes several years to become really comfortable working with a Wacom pen, looking at the screen while drawing onto the desk. But I am still using my ‘hand’.

Moorman’s excerpt is interesting because his thoughts are representative of many attitudes towards the inferiority of digital art and the superiority of non-computer based art. It will be interesting to see if and how these attitudes evolve as illustrators move even further into an era in which it’s impossible to make a living without involving computers at some stage of the development process.

Text Highlighting In Storybook Apps

…the vast majority of picturebooks are created for children. If we wish to be clearer about the nature of the picturebook should we attend to what children make of them or will our own close reading of individual texts be sufficient? And how relevant is it to our attempts to understand picturebooks that they are often used for teaching children to read?

– from the introduction to Reading Contemporary Picturebooks: Picturing Text by David Lewis

As children we relate to our picture books in a holistic fashion, merging sensations of the eye and the ear (for first we are read to), which marries the image and the sound of the words, and later, as we learn to read, the look of the words.

How Picturebooks Work, by Maria Nikolajeva and Carole Scott

It’s not surprising that research on a new medium happens only after the new medium comes into existence and gains a foothold in culture. Since interactive storybook apps are so new, there is still relatively little research that has been done, and when making development decisions, developers are instead reliant upon our own commonsense, and inevitably, our own experience of literature and reading.

One of the assumptions to have arisen about the nature of ‘good’ storybook apps is that they include word or phrase highlighting synchronous with narration. The assumption: that word highlighting is beneficial for emergent readers.

At this point, the beneficial nature of text highlighting is an assumption. It may be of benefit. It may not be. And it is also possible that word highlighting actually does more harm than good to an emergent reader.

Why this assumption in the first place? I think word highlighting is often considered the digital equivalent to pointing at words with a finger, and many are under the impression (rightly or wrongly) that when a caring adult co-reader points to words as they read, that the child will pick up reading — as part of a much wider program to teach reading skills, of course.

So before focusing on the topic of word highlighting, I would first like to look a little harder at the finger-pointing assumption.

From an article in the Telegraph titled ‘Pointing to words helps children read in later years‘:

Researchers claim this is the first time a study has shown a link between referencing during reading and literary achievement in later life.

So, if there have been many good studies on the effect of pointing to words on emergent readers, they haven’t been widely published.

Let’s go with that and trust our parental instincts: that occasionally pointing to words in books, and drawing children’s attention to various technical aspects of reading does improve literacy. I’m not going to argue with that because I intuit this is the case.

I don’t think I’m alone in this, because there seem to be many app developers who intuit that pointing to words by a trained or careful adult can be emulated electronically in a storybook app.

This, I’m not so confident about. Pointing to words may be really quite different from animating individual words in digital stories:

  1. When pointing, the finger does not obscure the actual word. Instead, effective pointers would  surely place their fingers BELOW the word in question, not over it.
  2. Also when pointing, the fingers are not making those jerky movements reminiscent of colours flashing on a screen. The hand glides across the page unobtrusively. Emergent readers may well be less distracted by a hand than by digital animation of words.
  3. Fluent readers do not read by looking at one word at a time. We take in three words at once. While it’s clear that early readers need to learn words one by one, when it comes to training the eyes to move across the page, is it really that helpful to highlight words individually, especially when the narrator is reading fluently themselves? I wonder about what we are modelling when app developers choose to individually highlight words.
  4. It’s possible that some ways of highlighting words are better than others. We need more research into this. It’s not enough to simply assume that ‘apps with word highlighting are good’ while ‘apps without word highlighting are lacking’.
Here are some various ways of word highlighting that you’ll see in some popular storybook apps right now.

1. JUMPING WORDS

Sir Charly Stinkysocks and the really BIG Adventure

This is a storybook app produced by a large publishing house. The words ‘jump’ off the page as they are read. But when a word is jumping, it’s moving, and therefore not able to be read. All the emergent reader can see is where in the paragraph the narrator is up to; they can’t see the word itself. Not unless their own reading is actually out of sync with the highlighting.

Here is another app which makes use of the same technique:

Logan and the upside-down sea

*

2. FLOWING TEXT HIGHLIGHTING

Perhaps to avoid the choppiness which results from highlighting words individually, this app developer decided to make the word highlighting last slightly longer than the narration itself. The colour that appears around the words fades out slowly, so you end up with an ‘approximate’ highlighting of words. It certainly works to avoid that choppy feeling that happens when words jump.

But if the highlighting isn’t 100% accurate, leaving the reader perhaps one word behind the ballgame, might this be worse than no highlighting at all? We don’t know this yet.

3. HIGHLIGHTING OF PHRASES

Cozmo’s Day Off

I prefer this method of word highlighting, where phrases are highlighted rather than individual words. This emulates the way we read as fluent readers – not just by taking in a single word at a time, but by encouraging us to take in several. This may aid reading fluency, and fluency aids comprehension.

I suspect this book has it right. If words are to be highlighted, this is how I’d like to see it done. I like that the words themselves don’t move. Instead, a blue outline appears around the words. This doesn’t prevent the reader from actually reading them.

Temporary Conclusion:

  • I suspect that the highlighting of individual words is useful in word games in which emergent literacy skills are the target.
  • I suspect story app developers should stay away from individual word highlighting, and consumers should be wary of expecting it by default.
  • Just because something is possible with the digital format doesn’t mean it’s an improvement on non-digital versions of a story.
  • For now, app developers who use word highlighting as a selling point are making money based on something which doesn’t have good research behind it.
  • The option to turn off word highlighting should be an option, just as it’s an option to turn off narration.

I’m prepared to change my mind on this. The issue of word highlighting in storybook apps desperately needs research. But we can’t assume that highlighting equals finger pointing. It may not.

50 Best iPad Apps for Reading Disabilities from Teachers With Apps