R2: Investigating AAC technologies to support the transition from graphic symbols to literacy

Figure R2-1 Pict and text v2 jpgTeam Leaders: J. Light, D. McNaughton, T. Jakobs (InvoTek), Hershberger (Saltillo)

Ben is 15 years old and has severe autism. He has no functional speech and relies on a speech generating device (SGD) with picture symbols to express his needs and wants; he is very frustrated when he can’t communicate effectively, and he has many challenging behaviors. He is currently in a transition program preparing for employment. His transition team has had trouble identifying potential employment options because of his limited literacy skills. He was introduced to literacy instruction recently and is making progress. However, his current SGD does not support his transition to literacy.

Challenge: Literacy skills are essential to positive outcomes in education, employment, participation and community living. Literacy skills also support the development of generative language and communicative competence for individuals with CCN, and are essential for access to the tools of 21st century communication (e.g., Internet, social media). Unfortunately, current research demonstrates that up to 90% of individuals with CCN enter adulthood without functional literacy skills.  At present, there are no AAC apps that effectively support the transition from the use of graphic AAC symbols  to the use of orthographic text for communication.

Goals: This project will investigate the effects of AAC apps that dynamically display text upon selection of graphic symbols in order to support pre-literate individuals with CCN in the transition from using graphic AAC symbols to using literacy skills to communicate in daily activities at home, at school, at work and in the community. This improvement in AAC technology design is not intended to replace formal literacy instruction, but rather to complement instruction and infuse literacy into daily communication.

It is expected that dynamically displaying text in graphics- based AAC software will result in increases in the literacy skills of individuals with CCN who were previously nonliterate, thereby providing a powerful means to complement formal literacy instruction and support increased communication and participation. The data from this study will be used to guide tech transfer of this innovation to the market place.

Summary Statement

Developing literacy skills is a complex process that rests on the fluent integration of knowledge and skills across a wide range of domains, including, for example, language skills, background knowledge, phonological awareness skills, letter sound correspondences, decoding, and sight word recognition.

Learning to read written text is facilitated by the integration of:

• orthographic processing (i.e., knowledge of letters and letter patterns);
• phonological processing (i.e., identification, manipulation, and memory of the sound structure of speech);
• meaning processing (i.e., knowledge of words and their meanings); and
• contextual processing (i.e., use of background knowledge to derive meaning from text) (e.g., Adams, 1994).

This project investigates the effects of redesigning AAC apps to incorporate transition to literacy (T2L) features to support these processes. The T2L features investigated in this project focus specifically on supports for sight word reading and include the following:

(a) the dynamic smooth animation of text (upon selection of a graphic symbol from the AAC display), using motion to draw visual attention to the text (cf. Jagaroo & Wilkinson, 2008) and to support orthographic processing;
(b) origination of the text from the graphic symbol to support the association of the symbol and the text and thereby support understanding of the meaning of the text;
(c) replacement of the graphic symbol by the text to make the word salient and mitigate the difficulties that may arise from static pairing of graphic symbols and text (cf. Erikson et al., 2010);
(d) pairing of the speech output with the appearance of the written word on the screen to support phonological processing of the text; and,
(e) targeting of sight words for the symbols within the learner’s AAC system to ensure that concepts are known thus supporting the association of meaning with the text.

The broad context provided by the AAC display and the communication situation may also support learning. The exposure to text is infused into the individual’s AAC system, thus ensuring that literacy learning is driven by the individual’s interests and needs (Light & McNaughton, 2009).


Adams, M. J. (1994). Beginning to read: Thinking and learning about print. MIT press.

Erickson, K. A., Hatch, P., & Clendon, S. (2010). Literacy, assistive technology, and students with significant disabilities. Focus on Exceptional Children42(5), 1-16.

Jagaroo, V., & Wilkinson, K. (2008). Further considerations of visual cognitive neuroscience in aided AAC: The potential role of motion perception systems in maximizing design display. Augmentative and Alternative Communication24, 29-42.

Light, J., & McNaughton, D. (2009). Addressing the literacy demands of the curriculum for conventional and more advanced readers and writers who require AAC. In Zangari, C., & Soto, G. (Eds) Practically speaking: Language, literacy, and academic development for students with AAC needs, 217-246. Brookes Publishing.

Cite as: Light, J., McNaughton, D., Jakobs, T., & Hershberger, D. (2014). Investigating AAC technologies to support the transition from graphic symbols to literacy. RERC on AAC: Rehabilitation Engineering Research Center on Augmentative and Alternative Communication. Retrieved from https://rerc-aac.psu.edu/research/r2-investigating-aac-technologies-to-support-the-transition-from-graphic-symbols-to-literacy/

Example of Transition to Literacy (T2L) feature with a grid display app

Powered by WordPress. Designed by WooThemes