Assistive Context-Aware Toolkit (The open source app used by Professor Stephen Hawking)

As soon as I read Mansib Rahman's post yesterday (as I write this) I knew I found the perfect project to highlight. I mean, come on it's Professor Stephen Hawking, Intel, .NET, WinForms (got to show some WinForm love now and then), open source and it's just cool!

Intel just open sourced Stephen Hawking’s speech system and it’s a .NET 4.5 WinForms app that you can try for yourself

I’m typing this sentence with my face. And no, I didn’t somehow smash my face onto a keyboard with laser-like precision. I used Intel’s ACAT, or Assistive Context-Aware Toolkit, an open source platform developed in C# using .NET 4.5 and Visual Studio 2012 at Intel Labs to allow people with disabilities to communicate with ease, even in very constrained situations, like Stephen Hawking’s,

As we all know, the venerable physicist Professor Stephen Hawking is unable to talk as he is afflicted with ALS and thus relies on a computer system to communicate. In 2011, his condition was deteriorating so badly that he could best communicate at a rate of only 2 words per minute. He reached out to Alan Moore at Intel and asked if Intel could come up with new technology to help his plight.

Intel was happy to oblige. For 3 years, Intel Labs worked in close collaboration with Hawking to acutely address his needs and in January of 2014, they announced the first stable release that Hawking would use to replace his decades-old speech system. There was a near instant 10x improvement to all common tasks such as conducting a web search or opening a Word document. Fast-forward a few months and now Intel has open sourced the whole platform to allow researchers and hackers to broaden its use for people with disabilities of all backgrounds.

Exploring ACAT

...

The principle in which ACAT operates is very simple. There is essentially only one ‘switch’ and you can trigger a sequence of these switches to do anything. Hawking operates this switch with his cheek. An infrared beam from Hawking’s glasses tracks his cheek and a twitch from it breaks the beam and triggers the switch. With a webcam, we can emulate Hawking’s method of communication as well.

cheekintel

...

While the primary intention of ACAT was to allow Hawking to communicate, it does more than just that. It’s actually a comprehensive system that allows Hawking to control the entire operating system. If he selects the ‘M’ character on the bottom right of his menu, he access the menu, from where he can choose to access many options such as the ‘New File’ dialog or the ‘Launch Application’ dialog. From the ‘Launch Application’ dialog he can start a Google search or use Internet Explorer (though Intel should really set him up to use Edge). The principle of navigation is still the same, the yellow box cursor alternates on the visual keyboard and awaits cheek twitches.

...

Well, what are you waiting for? You can try all this out for yourself. Visit the software's Github release page to get the installer. Make sure to check out the user guide, where you can learn advanced navigation techniques like the radial laser scanner, To make a fork and incorporate your own contributions, check out the Github repository.

... [Click through to read the whole post]

Assistive Context-Aware Toolkit (ACAT)

Introduction

Assistive Context-Aware Toolkit (ACAT) is an open source platform developed at Intel Labs to enable people with motor neuron diseases and other disabilities to have full access to the capabilities and applications of their computers through very constrained interfaces suitable for their condition.  More specifically, ACAT enables users to easily communicate with others through keyboard simulation, word prediction and speech synthesis. Users can perform a range of tasks such as editing, managing documents, navigating the Web and accessing emails. 

ACAT was originally developed by researchers at Intel Labs for Professor Stephen Hawking, through a very iterative design process over the course of three years. Professor Hawking was instrumental to the design process and was a key contributor to the project design and validation. After Intel deployed the system to Professor Hawking, we turned our attention to the larger community and continued to make ACAT more configurable to support a larger set of users with different conditions.

Our hope is that, by open sourcing this configurable platform, developers will continue to expand on this system by adding new user interfaces, new sensing modalities, word prediction and many other features. ACAT is designed to run on Microsoft Windows* machines and can interface to different sensor inputs such as infrared switches, camera, push buttons, and more. 

Who It's For

ACAT is useful for Microsoft Windows developers who are interested in developing assistive technologies to people with ALS or similar disabilities. Also for researchers who are working on new user interfaces, new sensing modalities or word prediction and wanting to explore these innovations in the this community.

Project Specifics

This ACAT toolkit was developed in C# using Microsoft Visual Studio* 2012 and .NET 4.5.  It runs on Microsoft Windows 7 or newer and is distributed under the Apache License, Version 2.0.  

About Intel Involvement

Intel has developed this ACAT system "from scratch" except for the word prediction software. Predictive text functionality is powered by Presage* (http://presage.sourceforge.net/), an intelligent predictive text engine created by Matteo Vescovi. Integration with Presage is through the Windows Communication Framework.

End Users

You can download the application here....

01org/acat

Introduction

...

ACAT was originally developed by researchers at Intel Labs for Professor Stephen Hawking, through a very iterative design process over the course of three years. Professor Hawking was instrumental to the design process and was a key contributor to the project design and validation. After Intel deployed the system to Professor Hawking, we turned our attention to the larger community and continued to make ACAT more configurable to support a larger set of users with different conditions.

Our hope is that, by open sourcing this configurable platform, developers will continue to expand on this system by adding new user interfaces, new sensing modalities, word prediction and many other features. ACAT is designed to run on Microsoft Windows* machines and can interface to different sensor inputs such as infrared switches, camera, push buttons, and more.

Intel has developed ACAT "from scratch". It is written in C# using Microsoft Visual Studio 2012 and .NET 4.5 and runs on Microsoft Windows 7 or later. Word prediction functionality is powered by Presage, an intelligent predictive text engine created by Matteo Vescovi.

Project Website

Click here for the ACAT project website. Documentation can be found here

Licensing

ACAT is distributed under the Apache License, Version 2.0.

Who It's For

ACAT is useful for Microsoft Windows developers who are interested in developing assistive technologies to people with ALS or similar disabilities. Also for researchers who are working on new user interfaces, new sensing modalities or word prediction and wanting to explore these innovations in the this community.

After cloning the repo, it loaded, compiled and ran the first time with no problems. I love it when that happens. :)

This is not some simple, little app. The Solution has 33 Projects...

image



Tags:

Follow the discussion

  • Oops, something didn't work.

    Getting subscription
    Subscribe to this conversation
    Unsubscribing
    Subscribing

Comments closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums, or Contact Us and let us know.