R1: Investigating use of a brain-computer interface (BCI) with enhanced language modeling

Person using a BCI interface to operate computerTeam Leaders: Melanie Fried-Oken, Deniz Erdogmus, Steven Bedrick, 
Consumer Team: Greg Bieker, John Simpson

  • Study 1: Communication effectiveness and efficiency improvements with modified natural language processing models;
  • Study 2: Effects of training protocols on learning to use a BCI

In 1995, Jean-Dominique Bauby, the editor of French Elle magazine, experienced a stroke. He lapsed into a coma and awoke days later with locked-in syndrome (LIS), his entire body paralyzed aside from some limited movement in his head and eyes. Unable to speak, Bauby learned to communicate by blinking: he could blink to answer yes/no questions, or to spell words using partner-assisted scanning (someone recited the letters of the alphabet and Bauby blinked to indicate his desired letter). Using this painstaking method, Bauby dictated his memoir, The Diving Bell and the Butterfly (adapted into a film of the same name), relating his experiences living with LIS. He died in 1997, three days after the book’s publication. Bauby’s story illustrates the need for a safe, reliable, and innovative access method for people who have minimal movement. As the population of adults with locked-in syndrome continues to increase, so will the demand for research based training tools and technologies that enhance communication performance and participation in medical decisions and management.

In recent years, brain-computer interface (BCI) has provided a potential means for individuals with minimal movement to control a computer (for spelling, Internet access, or other functions) using only their brain waves, with no motor output required. There are limited data about BCI usability with clinical populations and we do not yet have research-based training protocols to help individuals learn to use this challenging access method. Most BCI research has been accomplished in engineering labs, with non-disabled participants and investigators. Most reports of BCI use do not include training protocols or evidence-based treatment goals. Unfortunately, people with minimal movement have tested very few BCI devices, and their input has been largely absent from the efforts guiding the development and use of this technology.


Currently under development at OHSU and Northeastern University is The RSVP Keyboard™ – a noninvasive BCI communication system. It acquires brain signals via electroencephalography (EEG), with the user wearing an electrode cap connected to a portable computer. Letters are displayed one at a time on the screen in large font, following a rapid serial visual presentation (RSVP) paradigm. When the user sees a desired letter appear in this rapid sequence, the brain produces a response called a P300 evoked potential, which is translated into a keystroke to type the letter on the screen. The system incorporates a predictive language model, combining knowledge of letter frequencies with EEG evidence to improve typing accuracy.

Using The RSVP Keyboard™, our goal for this project is to provide clinical data to improve the functionality of a BCI spelling system for individuals with minimal movement, and to create and test training protocols that will support implementation as this technology appears in the rehabilitation marketplace. The research and development activities proposed for this project will continue to follow the principles of participatory action research and will extend our prior BCI work significantly.

The proposed research will have significant implications for enhancing BCI performance and may also have implications for improving AAC training paradigms in general. The research will impact future development by identifying language model features to improve BCI performance and reliability and by incorporating evidence-based training protocols into BCI implementation.


Fried-Oken, M. (2015, November 1) Brain-Computer Interfaces [Audio podcast]. Assistive Technology Update, Indiana Assistive Technology Tech Act Project. Retrieved from http://www.eastersealstech.com/2015/10/30/atu331-brain-computer-interfaces-bci-with-dr-melanie-fried-oken/

Fried-Oken, M. (2015, July 15) Brain-Computer Interfaces and ALS [Audio podcast]. ALS Association of Greater Philadelphia. Retrieved from http://alsphiladelphia.podbean.com/e/episode-25-bci-with-melanie-fried-oken/

Fried-Oken, M., Mooney, A., & Peters, B. (2015). Supporting communication for patients with neurodegenerative disease. NeuroRehabilitation, 37, 69-87.

Peters, B., Mooney, A., Oken, B., & Fried-Oken, M. (2016). Soliciting BCI user experience feedback from people with severe speech and physical impairmentsBrain-Computer Interfaces, 3, 47-58.

Powered by WordPress. Designed by WooThemes