VertiGo – a pilot project in nystagmus detection via webcam

Introduction

Dizziness is one of the most common, non-specific complaints in medicine [1]. Approximately every third person is complaining of dizziness at some point in their lives, affecting daily activities in 20% of patients older than 60 years and increasing up to 50% for over 80-year-olds [2]. The balance disorder can result in absence from work, social isolation, drop-attacks causing fractures which can lead to immobility as well as loss of self-determination.

In case of an acute vestibular attack, patients complain of experiencing a rapid onset of vertigo with illusory sensation of movement, nausea or vomiting, and gait unsteadiness. The symptoms can last days to weeks and are often self-limited. In U.S. emergency departments it is estimated that more than two million visits annually for dizziness or vertigo are expected [3]. It is difficult for emergency physicians or general practitioners to determine whether the vertigo is of peripheral or central origin because the presence of central disorders such as cerebellar infarction or haemorrhage, basilar artery occlusion, vertebral artery require immediate management.

In the differentiation of peripheral or vestibular origin of the vertigo, the history as well as clinical examination with detection of a nystagmus is essential [2]. A nystagmus is an involuntary rhythmic oscillation of the eyes induced by a dynamic, patterned visual stimulus or malfunction in the vestibular system.

Elementary in the detection of a nystagmus are Frenzel spectacles or video-nystagmography [4]. Depending on their existence, direction and movement pattern, the nystagmus can be distinguished between primarily vestibular or neurological pathogenesis.

Further nystagmus detection tools and vestibular tests are available which could only provide a snapshot of vestibular function [5]. Additionally, these traditional and contemporary testing tools need expertise in handling, experience in evaluation of test results, high acquisition costs and are just available in specialized clinical settings, e.g., clinic or hospital [5], [6], [7], [8], [9]. Especially in rural areas these testing tools or an expert are often not available.

The sporadic onset and shortly self-limited episode of dizziness like Ménière’s disease and Vestibular Migraine or benign paroxysmal positional vertigo (BPPV) make it unlikely that this episode can be examined and seen by a physician at an appointment. Therefore, the aim of this study was the feasibility of nystagmus detection with a low-cost and easy to handle method using a commercially available webcam under less than ideal lighting conditions. The necessary pupil motion profiles should be recorded using computer vision techniques while a self-developed algorithm should evaluate the existence or absence of nystagmus from recorded motion profiles.

Material and methods

In the feasibility study, 30 healthy volunteers participated in caloric vestibular testing with water at 44 °C for 30 s with a 30° head inclination position [10]. All probands denied ear problems or diseases as well as dizziness at the time of testing and in the past. Microscopic ear inspection was unsuspicious. Video-nystagmography was utilized with all probands to assure an absence of nystagmus without vestibular stimulation. The detection of nystagmus was recorded conventionally via video-nystagmography (Diatec, Germany) of one ear (Figure 1). After a recovery period of 5–7 min the other ear was tested in order to make sure that no vestibular hypofunction or dysfunction existed.

Figure 1:

Setup for nystagmus detection via webcam.

Followed by an additional recovery break of 20 min, a further caloric testing was performed in the same manner for each ear. At this time, the detection of nystagmus was recorded using a FullHD webcam (n=57) for 60 s with an average amount of around 20–25 frames per second of the facial region. Video recording was only accepted if the face and two eyes have been identified frontally in almost every frame within the first 20 s of the detection.

Videos were consecutively numbered and pseudonymized. Data was saved as a video sequence with a small rectangle cut out of the eyes which limited the recognition and assured anonymous data evaluation (Figure 2). Only the principal investigator was able to identify each proband.

Figure 2:

Pseudonymized video recording with a rectangle cutout bounding the eye regions.

The recorded data was analyzed with a self-developed software which has been implemented using the open source programming library OpenCV (Open Source Computer Vision Library) [11]. The software detected faces and eyes by “Facial Landmark Detection” which is integrated in the OpenCV library [11], [12]. Given the bounding rectangle of a single eye (Figure 2), the convex hull of the facial landmarks describing the eye was computed in order to minimize the region of interest as much as possible. Afterwards, the cropped eye region was blurred to remove any noise, and eroded to reduce the impact of backlights, i.e., a pixel is set to a local minimum with respect to its neighbouring region. The processed image was then binarized in order to have only black and white pixels. This image binarization aimed at separating the iris from the eyeball. After that, the contour of the iris was detected which was separated by the color contrast. Next, the pupil position was derived by calculating the centroid of the detected iris contour. Thus, the applied method for pupil detection assumed that the center of the iris is the center of the pupil. The aforementioned binarization of the eye region used a threshold to set pixels to either black or white. As this threshold was sensitive to the quality of the image, e.g., image resolution and light conditions, the threshold was calibrated using the first 20 images of a given video. Empirical results have shown that the size of the iris usually took around 50% of the size of an eye region which has been detected using Facial Landmark Detection. Accordingly, iteration over a set of possible threshold candidates was performed and the value which provided a contour of the iris that is closest to 50% of the overall eye region’s size was kept for each image. The most common threshold obtained by the images was used for the following pupil detection. The pupil detection provided a sequence of pupil positions with timestamps for each eye (Figure 3), i.e., a time series of data. For horizontal nystagmus detection, vertical pupil positions were removed. An algorithm to detect nystagmus in a sequence of horizontal pupil positions was designed according to the nystagmus definition describing a fast and slow phase. The software thus detected a nystagmus if pupil movement was faster than a predefined threshold value between two frames and the velocity of pupil movement was consistent until a maximum peak has been reached. This fast phase then had to be followed by a slow phase where the pupil moved considerably slower back to the primary point of pupil movement (Figure 3). The self-developed algorithm used predefined threshold values, e.g., defining the minimum velocity of a nystagmus fast phase and the minimum distance between the primary point and peak of a nystagmus. The threshold values have been examined empirically using the data gathered during the ongoing research study. For each dataset, a threshold of at least two nystagmus in the same direction was used to distinguish between the presence or absence of nystagmus.

Figure 3:

Graph of recorded ocular movements with arrows at detected nystagmus.

An evaluation of the software was performed by three experienced ear, nose, and throat (ENT) specialists who analyzed the existence and possible direction of all nystagmus in all videos without Frenzel spectacles. A video sequence was defined as positive for nystagmus if at least one ENT specialist evaluated their existence. An additional comparison was performed where the direction of a nystagmus was determined by the software and had to match to the evaluation of at least one ENT specialist.

Descriptive analyses were carried out using Microsoft Excel (Microsoft Corporation, Redmond, USA) to compare the evaluation of the ENT specialists and the software.

Results

All probands have shown objective vestibular response with the first caloric testing on both ears.

Webcam-based nystagmus detection had identified a single nystagmus in 73% of cases, while the ENT specialists evaluated 68.42% of data to contain nystagmus. When using the threshold of two nystagmus in the same direction to determine the presence of nystagmus, the software determined 57.90% of data to contain nystagmus.

The software achieved an accuracy of 50.88%, which is the proportion of data which were correctly classified including true positives and true negatives when using the criteria of two nystagmus in the same direction. Furthermore, 63.64% of the data which the software evaluated to contain nystagmus are correctly classified compared to the evaluation of the ENT specialists (precision), while 56.76% of the data which contained nystagmus according to the ENT specialists have been identified correctly by the software (sensitivity) (Table 1).

Besides the presence of nystagmus, the expected direction of nystagmus was recorded during the caloric vestibular testing. In 83.33% of cases where the ENT specialists detected nystagmus, at least one ENT specialist assigned a nystagmus to the expected direction, while the software assigned the expected direction of nystagmus in 60.60% of cases where it detected nystagmus using the threshold of two nystagmus in the same direction.

Discussion

Besides history taking, vital signs and gait function, the detection of nystagmus is one of the key components in examination of dizziness [13]. In dependence of their existence, direction and movement patterns are important to distinguish between primarily vestibular or neurological pathogenesis. It could be shown that examination of eye movements is more sensitive and less costly for the diagnosis of acute vestibular syndromes and the differentiation between peripheral and central lesions than magnetic resonance imaging (MRI) [14], [15].

In the present feasibility study, it could be shown that horizontal vestibular nystagmus detection with a commercially available webcam is possible. The software algorithms were also able to distinguish between left and right-beating horizontal nystagmus.

In comparison to other traditional or contemporary nystagmography devices the inexpensive and easy to handle equipment has many advantages: it does not require an additional google or other bulky gadgets and no detailed calibration or professional expertise in application is mandatory which is time-, budget- and personal-saving [5], [6], [7], [8], [9]. Turuwhenua et al. presented webcam-based nystagmography with tracking limbus edge of eyes with the aim to investigate optokinetic nystagmus [7]. Yet, the study used a limited patient cohort, similar lighting conditions, and a device fixating the head. Similar to other nystagmography methods, this precise technique is limited in future emergency application for vestibular testing due to the head fixation and it is unclear how it will perform under varying lighting conditions as well as for different patients.

With the visualization of a video sequence a consultation of a specialized doctor at another location or clinic is possible. Therefore, patients could record sporadic onset of vertigo episodes showing the video footage at the following doctor’s appointment or unspecialized doctors in rural areas which are then able to consult specialized doctors [5].

In the experimental setting the vestibular nystagmus detection could be shown. Yet, further improvements are required to develop the software from a research tool into a reliable investigation tool in primary or secondary care. So far, the results are affected by some limitations. All probands have shown vestibular response with first caloric testing. Limited nystagmus detection could be affected in individual cases by an insufficient recovery period of 20 min after first caloric testing. Further, we assume that in the feasibility study the major limitation was a suppressed nystagmus due to fixating the webcam. Therefore, the ENT-specialists and the software could not always detect any nystagmus. With the use of a simple and low-cost penlight the fixation of the eye could be prevented [16]. Furthermore, the inexpensive investment of a commercial webcam with a maximum rate of 30 frames per second could limit the fundamental frequency of nystagmus detection. Similar to the contemporary continuous ambulatory vestibular assessment (CAVA) system, the slow phase of velocity of a nystagmus which quantifies the ocular movement was not possible to calculate [5]. Additional interferences are experienced with fatigue and resulting nictitating of eyes, dark coloured iris, reflection of extensive lashes or eye-liner as well as shady lighting [5], [7].

In future, further improvements and tests of the software are necessary to increase its accuracy and reduce interference in order to develop a low-budget and easy to handle examination tool for patients in primary and secondary care. In further trials, it should be evaluated whether a machine learning algorithm can be trained to detect horizontal nystagmus from a recorded time series of data. In this context, a larger amount of gathered data is essential to evaluate whether a machine learning algorithm can be trained. Further, we want to extend the study to evaluate whether vertical or rotating eye movements can be detected as well.