Modeling Human Pupil Dilation to Decouple the Pupillary Light Reflex

1 Introduction

The human eye is a complex biological structure, sensitive to light and pressure. Thanks to its structure, this organ provides one of the most important senses for humans, which is vision. For million years of evolution, the mammal eye has been able to differentiate between about 10 million colors and it is possibly capable of detecting a single photon. For the purpose of this research, the anatomy of the eye human has been simplified (see Figure 1).

Figure 1

Anatomy of human eye https://goo.gl/k5qap4

Light passes through the cornea, the pupil and the lens to finally fall on the light-sensitive cells of the retina. When the amount of light coming through the iris exceeds the normal requisition, or there is a shortage of light for the eye to achieve normal vision, the iris sphincter muscleand iris dilator muscle react, creating what is known as the pupillary light reflex. The first muscle constricts the pupil when light is too bright, while the other expands it in poor light conditions. In a heightened sympathetic activity, these muscles can follow the same response pattern as in the case of various light conditions. The varying size of the pupil is a physiological response to many external factors (the pupillary response). The change in pupil size can be prompted by an involuntary reflex reaction to light (exposure to light) [1]. An increase in the size of the pupil of the eye has been found to accompany interest understood in the context of attention, and even sexual arousal [2, 3]. Other factors are memory operations [4] or a cognitive effort to distinguish between automated and controlled cognitive processing [5, 6] In normal light conditions, the pupil the pupil diameter is around 3.09mm, and 4.93mm for darkness [7]. In Figure 2, the effect of bright light and dim light is presented. As far as the latency of the light reflex, the shortest one was recorded in [8] and it was 220 ms. Some algorithms aiming to create a virtual pupil [14] can be found in the literature,which can be used to conduct experiments without the need for human subjects.

Figure 2

Pupil dilatation based on given light [9]

The pupillary light reflex is modeled as a physiologically-based non-linear delay differential equation. Subsequent pupillary measurements will be made with a different physiological index. It is observed that the pupil diameter changes each fixation cross location in the different emotional state [16]. The magnitude of the pupil is strongly correlated with visual attention and task performance [28]. The numerical accuracy of different variants of integral image computation algorithms allow for the possibility of multiple evaluations of different statistical characteristics of the image [27] to extract the differences from different parameters of the pupil size. The other solution is based on artificial neural networks [17, 18] used to predict the pupillary response and factor localization. [16] propose a model based on the fact that the pupil’s response to brightness changes is linear and they try to cover the range of pupillary behavior. They assume that the response of the pupil, at each time instant, is independent of the previous light intensity that it was exposed to. Models of pupil changes under the influence of light may allow us to separate factors related to the emotional state of the human study subject from the rate of these emotional cues. We propose to collect information about pupil dilation and then decouple the pupillary light reflex from this data. It is important to know the behavior of the eye due to changes in the light environment. Currently, studies on emotional states based on pupil size are performed in isolated laboratory environments. Accordingly, knowing and modeling pupil changes based on light conditions will allow us to isolate these changes. To ensure the most accurate simulation of pupil behavior, we implemented and the most popular mathematical models of the pupillary light reflex

2 Pupillary light reflex in equilibrium state

The pupillary light reflex is the main mechanism that regulates the pupillary diameter.Most models so far have been developed against the background of empirical research. Data gathered in those experiment was obtained after the pupil reached the equilibrium state, when the pupil stabilized for each illumination level. Those models are not suitable for describing dynamic changes of the pupil, but are used as a base for some models of dynamic changes of the pupil. The literature describes several dynamic behavior models built around experiments designed to measure the values of some parameters as a function of incident light intensity [10], described in Dynamic model of pupillary light reflex section. Calculation of pupil size in am equilibrium state based on given illumination can be performed using multiple formulas, which are presented at below. Based on [15], the entire equation was transformed to use nit(cd/m−2) as the main unit of luminance.

The first study was aimed at extrapolating pupil size based on given luminance was conducted by Holladay [11]. Data was collected for 3 participants of unknown age and binocular viewing.

Equation 1 plotted at Figure 3 fails for high luminance beyond ≈ 600cd/m−2 for 2mm asymptote. As in the previous study, a study by Crawford [12] also fails to reveal the age of its subjects, but their adapting field diameter is 55^∘. In this examination, the author took samples from 10 people. It can be observed that the asymptote of 2mm given in Holladay is not present (see Figure 4). Moon and Spencer [19],

Figure 3

Pupil size for Equation 1 of Holladay

Figure 4

Pupil size for Equation 2 of Crawford

based on the equation and data approximation provided by Crawford, Blanchard [25] and Reeves [26], developed a slightly different formula, plotted in Figure 5.

Figure 5

Pupil size for Equation 3 of Moon and Spencer

De Groot and Gebhard [20] formulated an equation based on the averaged data from previous work, taking into consideration the number of people examined in the experiment. As can be seen in Figure 6, the asymptote of 2mm is not present in this formula either.According to the authors, it is not necessarily demanded by the data of the physiological restrictions.

Figure 6

Pupil size for Equation 4 of De Groot and Gebhard

Figure 7

Pupil size for Equation 5 of Stanley and Davies. Field diameter 0.4^∘(first line) and 25.4^∘(second line)

Figure 8

Pupil size for Equation 6 Barten plotted against Stanley and Davies

Figure 9

Pupil size for Equation 7 Blackie and Howland

Important changes in the formulas were introduced by Stanley and Davies [13]. In contrast to previous work which assumed that dilatation depend only on a given luminance, they went on to conclude, based on Crawford, that the pupil diameter is directly correlated with the product of luminance L and the adapting field size, expressed

by a in deq2,which gives corneal flux density(cdm−2deq2). To arrive at this formula, data was gathered from 6 volunteers whose adapting field size ranged from 0.4^∘ to 25.4^∘. These two values were plotted at Figure 7 and used as thresholds for this experiment.

Barten [21] developed his own formula based on Moon and Spencer and Stanley Davies consideration of field size.

Blackie and Howland [22], based on a big project of modeling the emmetropization experiment, designed a formula expressed by:

[23] Winn carried out an experiment for 91 subjects aged from 17 to 83, including students, staff and the clinical population of Aston University. Subjects maintained a steady fixation at the center of an evenly illuminated circular field with a 10^∘ diameter. Pupil diameter was measured at five luminance levels (9, 44, 220, 1100, and 4400 cdm−2). The mean pupil size of 91 subjects was plotted as a function of age for five different levels of luminance. Thanks to this data, the slope and intercept were estimated. These values are given in Table 1.

Luminance was restricted to illumination in Winn et al.’s experiment. To use values from Table 1 in pupil prediction equation, the slope needs to be presented as a function of log luminance with a cubic polynomial (see Equation 8). The same pattern was followed in the case of the intercept (see Equation 9).

hen, to estimate the data of Winn et al., both functions were combined:

where y denotes the age of a participant.

Wattson and Yellot [15] created unified formula, which combines the previously described equation as well as considering new parameters.

Taking into account previously conducted experiments in which only the dilatation of one eye was measured where the result differs from the experiments with

Figure 10

Pupil size for Equation 10 off Winn.

both, given in the same adapting light (as in the Figure 11), the monocular effect was created. Based on a comparison of mono and binocular data the conclusion was that the modeling of the monocular effect can be superimposed by a horizontal shift on a log axis. In which the average optimal shift of monocular data to match binocular data amounted to 0.1015. With this information, a the factor of 0.1 can be used to shift data when in an experiment only one eye is examined, and 1 when the data is collected on the binocular (Equation 11)

Figure 11

Binocular and monocular data from Doesschate and Alpem [24]

In addition to using the monocular effect, the effective corneal flux density formula was also developed, which extends the previous corneal flux density by:

The result is the product of luminance, adapting area and monocular effect.

In Winn et al. the age of participant was given as a crucial parameter in pupil size approximation, although data was limited to 4400 cdm2 and the adapting field size of 10^∘. Assuming a linear relation between the slope function from Winnet al. and the pupil diameter calculated in Stanley Davies for the same luminance, the new general slope function could be designed.

The rate of change of the pupil diameter with age was estimated based on the function of luminance, area flux density, age, referenced age and eye function.

Age must be in the range of that of the subjects in Winn et al. Reference age is the age of observers in the experiment. As far as reference age in the equation of Wattson Yellot, it is equal to 28.58, which is the average age of the subjects in the Stanley Davies study.

Figure 12 and 13 show all the described methods and they demonstrate that these methods can differ depending on the parameters. Figure 12 presents monocular e=2 data from a small adapting area a=10. It can be seen that the Unified formula, based on Stanley and Davies, gives similar results to the Barten formula. Meanwhile In Figure 13, it can be observed that the Unified, Barten, Stanley Davies and Moon-Spencer formulas give similar results, which shows that the Watson and Yellot model can be used to predict pupil size more precisely than previous models

Figure 12

Pupil size for Equation 15 of Watson and Yellot, compared to all previous models. a=10°, monocular.

Figure 13

Pupil size for Equation 15 of Watson and Yellot, compared to all previous models. a=60°, binocular.

3 Dynamic model of pupillary light reflex

As described in previous chapter, models only give information about pupil size after adaptation to a given light luminosity. Meaning that it does not describe changes of the pupil in a time interval. This poses a significant problem in research which includes constant changes of light or in monitoring the changes on psychological impulses. Models of such changes need to take into consideration the specific physiological behavior of the pupil. In the case of PLR, iridial reaction is caused by light striking through the retina. This action sends a neural signal, but it needs time to cause the iridial reaction, which is why it is modelled by a time delayed equation. Another parameter that affects the pupil change is the threshold value of light flux. Luminosity higher than that given threshold will not cause the pupil to react. As the final visual effect of PLR is the movement of dilator and sphincter muscles, there are PLR models that are designed to simulate behavior of those muscles based on autonomic nervous system input. In [29] the model is targeted to simulate only short light flashes, which does not cover all possible cases of PLR behavior. The model presented at [30] is over complicated with multiple second-order delay differential equations, and thus it is impractical for designing a robust inverse algorithm to obtain model parameters from experimental data. The model used in this experiment is based on the Moon and Spencer (Equation 3), for equilibrium pupil state, and Longtin and Milton’s time dependent pupil size which is described by a time delayed equation. The Equation 16 describes the behavior of dynamic changes of the pupil in response to light where the pupillary latency – the time delay between the instant in which the light pulse reaches the retina and the beginning of iridial reaction – is given by τ and time t. Parameter D describes the pupil diameter in millimeters, which is the output of the solved equation. This model is able to describe changes of the individual pupil in specific light conditions. Despite the fact that Pamplona is based on Moon and Spencer’s function and not the newest Wattson and Yellot, the literature shows that this method gives very high correlation with real life scenarios.

4 Description of the research environment

In order to record pupil changes caused by either internal or external factors, a device called an eye-tracker can be used. The main task of this device is to track and record gaze, but in addition, vital oculography parameters can be recorded. Blink occurrence frequency, saccades detection and real time pupil size monitoring, are the most common ones. In order to record those changes multiple cameras, either InfraRed or RGB, must be used. The arrangement of those cameras can be used to distinguish two main types of eye trackers.

Head-mounted eye trackers, mostly in the shape of glasses, have multiple cameras aimed directly onto the subject’s eyes, and one so-called global camera to record the scene in front of the subject. The greatest advantage of these devices is their mobility, which allows the subject to move their head freely and change locations. The biggest disadvantage is that they have lower precision and accuracy compared with stationary ones, because of the subject’s head movement.

The second type of eye-trackers are so-called remoteeye trackers. This is a nonintrusive peripheral device that has no contact with human body. Cameras that are placed inside this device are aimed in front of the scene, where the face of subject must be present. These are stationary devices with no possibility of changing its location, thus the test must take place in same area. The biggest advantage over the wearable eye tracker is that this kind of eye tracker is nonintrusive, which is more suitable to study psychological reflexes. Thus, the device is calibrated in a way that has a certain window where the eye of the subject must be visible and there is no head movement of subject. Accuracy is higher in comparison with a wearable one.

Taking into consideration that experiment is aimed at real life scenarios like shopping, walking in the street, or sport event activity, the most important property of the device must be its mobility. Based on other research and solutions available on the market, the wearable eye tracker was chosen for this experiment. The second attribute of the device must be the possibility to develop and implement new algorithms for measuring pupil changes over time. There are multiple DYI wearable eye-trackers, but they lack precision and accuracy in comparison to devices provided by commercial solutions. Based on devices available for our use, the Tobii Pro Glasses 2, and Pupil Lab, both were tested in accuracy and precision. The conclusion of the preliminary conducted experiment was that the Tobii Pro Glasses 2 have slightly better results than the other device. However, the lack of possibility to extend usage of Tobii by adding new algorithms or another peripheral device resulted in the Pupil Lab being chosen as the best choice.

This device is capable of gaze estimation with 0.6 degree of visual angle (0.08 degree precision) with a processing pipeline latency of only 0.045 seconds, blink detection, pupil size monitoring and more. The glasses-like headset eye tracker, presented at Figure 14a, consists of two RGB cameras with an IR filter and one global directed RGB camera at the front of the user. The frontal camera is able to capture an image at a max resolution of 1920x1080 pixels at 30Hz with a 90-degree diagonal field of view. Two IR cameras, which can operate up to 200Hz, are located at glasses frames and aimed directly at the eyes of the subject. Software used in Pupil Lab uses the “dark pupil” detection method,presented at Figure 15, thus, IR cameras must be able to capture video at a specific range of the IR spectrum, in this device it is a 860nm wavelength. Each image captured by the IR camera is analyzed by open source Pupil Lab software. Thanks to the flexibility and extensibility of the source code, additional sensors can be added to this eye tracker. Additionally, in the case of this research, a simple luxometer was added to eye tracker as an external sensor.

Figure 14

Pupil Lab eye tracker and BH1603FVC Rohm sensor.

Source https://bit.ly/2TH3hby https://bit.ly/2VbrV1d

Figure 15

Pupil recognition algorithm in PupilLab device

The BH1603FVC Rohm ^[1](Figure 14b) analog current output ambient light sensor was used to determine the amount of light cast at the pupil. It is a device with a compacted surface of 3.0 x 1.6 mm and a spectral sensitivity close to human eye sensitivity. The sensor is capable of measuring ambient light in a range of 0 - 1 000 lux on H-Gain Mode, 0 – 10 000 in M-Gain and 0 – 100 000 on L-Gain. To measure which range will be most suitable for the conducted experiment the sensor was placed with 50cm range from LED lamp. This experiment gave values from 6 000 to 9 000 lux,which remains in the H-Gain range of the sensor and in that mode the sensor was calibrated. This sensor was placed near the IR camera of the left eye, and only these eye responses are recorded during the experiment.

To recognize and measure the changes of the pupil over time, the Pupil Lab method was used, as a default for the hardware. The detection method used in the software provided by Pupil Lab consists of 7 steps, presented at Figure 15. First the image of eye is converted to grayscale.

Using a Haar-like feature, the region of interest – the estimated region where pupil can be – is calculated, and significantly cut the unwanted pixels from source image (Figure 15a). The region obtained in the previous step is filtered by a Canny edge detector (Figure 15b). For the region of interest, the histogram is calculated (Figure 15c), and the “dark” region is defined by the offset from the lowest spike in the histogram. All edges with spectral reflections and not in the “dark” region are excluded(Figure 15d). Remaining edges are filtered and split into sub contours, based on the criteria of curvature continuity (Figure 15e). Finally, the ellipse is fitted using the least square method and additional criteria like ellipse circumference (Figure 15f).

5 Experiment

In the experiment, various scenarios were tested. In each of them, the person on whom the experiment is carried out was asked to always look forward, with the freedom to move their head, without squinting eyes to ensure that both th eye-tracker and light detector will give the best results. The figures presented below show the deviation that can be treated as non-light triggered changes. Data from luxometer and eye-tracker gathered along the experiment was used to generate the artificial signal using Equation 16, plotted as solid line. The real data, points, was plotted against the generated one to find how much those two signals differ at, example at Figure 16. All not convergence is treated as pupil change caused by non-light factors, such as emotion, accommodation, etc.

Figure 16

Plot of data generated by [10] (solid line), and data gathered by PupilLab (points)

6 Comparison of data and model

To compare data gathered by device and data calculated by Pamplona (Equation 16) the difference in pupil size have been calculated. For experiment presented at Figure 16a the difference was plotted at Figure 17a where the mean difference in distance is −0.215916mm. For second experiment presented at Figure 16b the difference was plotted at Figure 17b where the mean difference in distance is 0.041678mm. For third experiment presented at Figure 16c the difference was plotted at Figure 17c where the mean difference in distance is −0.278634mm.

Figure 17

Plot of difference between pupil size gathered by device and calculated by Equation 16

That information shows how much prediction from Pamplona differ from data gathered by device. Those values can give information how much change in pupil size was cause by specific type of stimulus, either internal or external one.

7 Conclusion

Based on the existing models, the method for measuring a light-adapted pupil diameter was implemented for pupil in equilibrium and dynamic case. The solution to predict pupil size changes under visible light stimuli was presented in this article. The change in pupil size under the influence of light has been modeled and tested against the existing popular models and real life data. Those data has been gathered using constructed device that consist of eye-tracker and additional ambient light sensor. This device has gave the opportunity to verify implemented methods and to find light and non-light factors that can affect human pupillary light reflex. Conducted experiment have shown the lack of possibility to distinguish, analyze and classify the root cause of each change in size of the pupil.

8 Future work

Next conducted experiments will be focused on finding and classifying the causes of each deviation in scenarios which the light can change its intensity. Additional diagnostic methods like EEG will be use, because of the connectivity between human sympathetic nervous system and the eye.