Eye Tracking for Cognitive Computing

Background image

The KOGNIT project led by the German DFKI is concerned with fundamental research on augmented cognition scenarios which can assist dementia patients in everyday situations.

Challenge

What are the possibilities for using eye tracking technology in cognitive computing, specifically in research on applications for persons with dementia? This is one of the central questions which Dr. Daniel Sonntag and his team at the German Research Center for Artificial Intelligence pursue in their KOGNIT research project.

To do so, they use the SMI Eye Tracking Upgrade for the Oculus Rift DK2 HMD – complemented with the SMI Unity plug-in – for integrating a person’s gaze with real and virtual environments.

Solution

Different scenarios are being set up within the KOGNIT research project financed by the German Ministry for Education and Research.

With the help of eye tracking, eye and gaze data of patients is to be recorded in virtual representations of every-day situations, such as crossing the street or getting breakfast ready.

SMI Eye Tracking

Besides the SMI Eye Tracking Upgrade for the Oculus Rift DK2, DFKI researchers are using SMI Eye Tracking Glasses combined with propietary DFKI object recognition software.

Benefit

The DFKI research program explores how cognitive computing systems can interact with people in real time, in order to extend what neither person or machine could do on their own, for example fill in the gaps caused by memory loss.

One purpose is to learn more about the gaze behavior of persons with certain cognitive deficits like dementia. Another is to find out how eye tracking can assist dementia patients with memory loss in every-day situations.

Background

In one particular approach, the team has created an application where episodic memory visualizations from a digital data base are shown to patients to help them with every-day tasks, such as

– Recalling names of persons and objects
– Navigation, orientation
– Searching for objects (e.g. pair of glasses, book)
– Reminding (to drink, take pills, etc.)
– Games for entertainment and memory training

Benefit

The DFKI research program explores how cognitive computing systems can interact with people in real time, in order to extend what neither person or machine could do on their own, for example fill in the gaps caused by memory loss.

One purpose is to learn more about the gaze behavior of persons with certain cognitive deficits like dementia. Another is to find out how eye tracking can assist dementia patients with memory loss in every-day situations.

Customer institution Kognit project @DFKI
Customer name Dr. Daniel Sonntag
Customer website http://kognit.dfki.de
Related publication