Apple Vision Pro ’s eye-tracking technology introduces a new way to interact while typing. However, hackers are reportedly exploiting this technology to steal sensitive data. This new technology available in Apple ’s first-ever mixed reality headset comes with a vulnerability called GAZEploit . This flaw developed by researchers from the University of Florida , CertiK Skyfall Team , and Texas Tech University puts users at risk of privacy breaches during Apple Vision Pro FaceTime calls.
How this vulnerability can affect users
In a blog post, the researchers describe that GAZEploit leverages eye-tracking data in virtual reality to predict what a user is typing. When using virtual or mixed reality devices like the Apple Vision Pro, users type by looking at keys on a virtual keyboard. Instead of pressing physical keys, the device tracks their eye movements to identify the selected letters or numbers.
GAZEploit targets the virtual keyboard by analysing eye movement data to guess what the user is typing. It works by recording the movements of the user's virtual avatar's eyes, focusing on two key factors: eye aspect ratio (EAR), which measures how wide the eyes are open, and eye gaze estimation, which tracks where the user is looking on the screen.
By evaluating these factors, hackers can detect when a user is typing and even identify the specific keys being selected. In virtual reality, users' eye movements follow distinct patterns and blinking decreases while typing. GAZEploit leverages this by using a machine learning model called a recurrent neural network (RNN) to analyse and interpret these eye movement patterns.
The researchers noted that they trained the RNN using data from 30 individuals, achieving a 98% accuracy rate in identifying typing sessions.
Once a typing session is detected, GAZEploit predicts keystrokes by analysing rapid eye movements, known as saccades, followed by pauses, or fixations, when the eyes focus on a key. The attack maps these eye movements to the virtual keyboard layout, identifying the letters or numbers being typed.
GAZEploit determines the selected keys by assessing the stability of the gaze during fixations. In tests, researchers achieved 85.9% accuracy in predicting individual keystrokes and an impressive 96.8% recall in detecting typing activity.
Since the attack can be executed remotely, attackers only need video footage of the user's avatar to analyse eye movements and infer what is being typed. This remote vulnerability means that during virtual meetings, video calls, or live streams, sensitive information like passwords or private messages could be stolen without the user's awareness.
How users can protect themselves
To safeguard against potential attacks like GAZEploit, users should take a few key precautions. First, they should avoid entering sensitive information, such as passwords or personal data, through eye-tracking in virtual reality (VR) environments.
Using physical keyboards or other secure input methods is a safer alternative. It's also important to keep software up to date, as Apple frequently releases security patches to address vulnerabilities.
Lastly, adjusting privacy settings on VR/MR devices to limit or disable eye-tracking when it's not necessary can further minimise the risk of exposure.
How this vulnerability can affect users
In a blog post, the researchers describe that GAZEploit leverages eye-tracking data in virtual reality to predict what a user is typing. When using virtual or mixed reality devices like the Apple Vision Pro, users type by looking at keys on a virtual keyboard. Instead of pressing physical keys, the device tracks their eye movements to identify the selected letters or numbers.
GAZEploit targets the virtual keyboard by analysing eye movement data to guess what the user is typing. It works by recording the movements of the user's virtual avatar's eyes, focusing on two key factors: eye aspect ratio (EAR), which measures how wide the eyes are open, and eye gaze estimation, which tracks where the user is looking on the screen.
By evaluating these factors, hackers can detect when a user is typing and even identify the specific keys being selected. In virtual reality, users' eye movements follow distinct patterns and blinking decreases while typing. GAZEploit leverages this by using a machine learning model called a recurrent neural network (RNN) to analyse and interpret these eye movement patterns.
The researchers noted that they trained the RNN using data from 30 individuals, achieving a 98% accuracy rate in identifying typing sessions.
Once a typing session is detected, GAZEploit predicts keystrokes by analysing rapid eye movements, known as saccades, followed by pauses, or fixations, when the eyes focus on a key. The attack maps these eye movements to the virtual keyboard layout, identifying the letters or numbers being typed.
GAZEploit determines the selected keys by assessing the stability of the gaze during fixations. In tests, researchers achieved 85.9% accuracy in predicting individual keystrokes and an impressive 96.8% recall in detecting typing activity.
Since the attack can be executed remotely, attackers only need video footage of the user's avatar to analyse eye movements and infer what is being typed. This remote vulnerability means that during virtual meetings, video calls, or live streams, sensitive information like passwords or private messages could be stolen without the user's awareness.
How users can protect themselves
To safeguard against potential attacks like GAZEploit, users should take a few key precautions. First, they should avoid entering sensitive information, such as passwords or personal data, through eye-tracking in virtual reality (VR) environments.
Using physical keyboards or other secure input methods is a safer alternative. It's also important to keep software up to date, as Apple frequently releases security patches to address vulnerabilities.
Lastly, adjusting privacy settings on VR/MR devices to limit or disable eye-tracking when it's not necessary can further minimise the risk of exposure.
