Abstract: Eyetracking has recently gained a lot of attention and has also found a way into consumer products like modern laptops 1. The aim of this explorative research was to use an eyetracker to analyze different behavioral patterns over longer periods for two participants without creating laboratory conditions. This setup was consciously chosen so that we could test the robustness of our assertions under natural conditions. Here, we analyzed the pupil diameter, the position on the screen, and the position of the head relative to the display while recording which software applications the participants used and event data for the respective applications.

Setting: The project described in the following was conducted as part of a seminar called “Neurodesign Seminar – How to Measure and Analyse Physiological and Psychological Data?” as part of my master’s degree at Hasso Plattner Institute together with Philipp Bode.

According to the Stanford NeuroDesign Research Lab, Neurodesign describes the intersection between Psychology, Cognitive Science, Neuroscience, Anthropology, and Design2.

For the explorative research, we used a Tobii Pro Nano with a sampling frequency of 60Hz3. The eyetracker was mounted on the bottom of the laptop screen or a computer display and needed to be configured to identify the relative screen size and the screen-to-eyetracker angle and position.

The analysis was conducted over multiple days for the entire day while working on non-experimental regular work/university tasks. Beyond the analysis, we created a toolset available on GitHub to conduct experiments & allow for analysis such as a heatmap for gaze analysis

Eyetracking-Tooling: Heatmap

Furthermore, by making use of REMoDNaV4, we classified the different eye movements into the types: fixation (maintaining gaze on a single point/location), saccades (rapid but smooth ballistic movements of the eye that change the point of fixation abruptly) and smooth pursuit movements (eye focus on a moving target).

Classification of eye-movement-types

For further analysis, we also needed to track the applications in parallel to split the gaze-data per application currently in use. The following picture shows the data obtained by the application tracker across all currently open applications.

Application Tracking

Finding 1: We found that we tend to blink more as the day progresses by analyzing multiple working days of two participants performing various tasks over a day. Furthermore, external influences like weather, lowering of roller shutters in the respective rooms, or other factors are reflected very accurately in the data. Besides, we tend to move closer to the screen over the course of the day.

Mean distance to the monitor

Finding 2: When analyzing the working behavior which manifests in the eyetracking data, we found that there are clear interaction patterns for the different applications we use. For example, a messenger service is different from interaction with pdf-readers, browsers, or development environments for coding. In the analysis, we could clearly distinguish between different activities (reading, coding, scrolling, etc.). In a longer analysis, we might have the opportunity to differentiate between more activities and analyze more subtleties within the different activities (including differentiating between different reading types), for which various research results already exist.

Pattern of gaze in Jupyter and Slack

In this picture, the difference in the interaction between the messenger service Slack and the use of the development environment Jupyter (Jupyter Notebook in split-screen-mode with two notebooks side-by-side) can be seen.

Proposal: Encouraged by previous research in this field, we have thought about an experimental framework to elaborate and analyze whether blinks, micro-saccades, and pupil-diameter qualify for determining the cognitive load of users. This can either be used to develop user interfaces that only impose little ground cognitive load to the user or adapt the cognitive load to the user’s situation. We came up with a research proposal considering blinks, micro-saccades, and pupil-diameter in their role and the interplay as an indicator for cognitive load via the n-back-task.

Proposal: Towards Behaviorally Adaptive Software Systems: As part of another seminar work, I worked on a research proposal how to use such data sources in software systems of the future to ultimately create an experience completely tailored to the user’s knowledge level, emotions, interest level and overall well-being. The research proposal with the title >Towards Behaviorally Adaptive Software Systems< has the goal to enable software systems to adapt towards users behavior and interaction patterns by providing a generally applicable architecture making use of physiological data (among others).

Architecture as part of the research proposal

The explorative research we conducted was ultimately also featured in the Book Design Thinking Research - Interrogating the Doing by Meinel and Leifer published 2021.

GitHub Project of the Neurogaze Tooling & Research

  1. Gaminig Laptop featuring Tobii Eyetracking Technology ↩︎

  2. Neurodesign Research at Stanford ↩︎

  3. Tobii Pro Nano Eyetracker ↩︎

  4. Dar, A., Wagner, A., & Hanke, M. (2020). REMoDNaV: Robust Eye-Movement Classification for Dynamic Stimulation. ↩︎