Eyes are the most effective means of communication. Humans share considerable information through their eyes. Internet and other digital technologies enhances the human experience of communication with others. One such thing is eye typing, which would be a tremendous technology for disabled persons. Locked- in syndrome (LIS) and cerebral palsy (CP) are severe disabilities that lead to the complete loss of control over voluntary muscles making the individual paralyzed and mute. The above disabilities do not show any effect on the eye muscles and for individuals with such disabilities communication is very difficult and hence eyes are the main means of communication. The eyes are hence the important input modality for people with severe motor impairment and offer the means of communication with the outer world. This is all achieved by the text entry system using eye blinks.
1.1 Tracking and Typing through eyes
Eye movements are commonly captured using video cameras with infrared light to produce the reflections of the cornea. The most suitable technique for capturing interactive systems is video based capturing.
The interfaces of most eyes tracking video-based system, when used for input, are designed to simulate a pointing device. Text entry is the common application. So called "eye typing" allows to enter text by the user looking at the virtual keys on an on-screen keyboard (Majaranta and Raiha 2002). These keys are selected by holding the point of gaze on a key for a predetermined time period.
The existing eye typing systems provide means of communication for the disabled individuals. However, they fail to include the severely motor impaired. Considering the case of an individual with locked-in syndrome, the eye movements are limited in this case and so there can be inaccuracy in the input and so the use of eye trackers make the system unreliable (Laureys et al. 2005). In such case, eye blinking is the only reliable method for communication with eyes.
Most of the eyes tracking systems have the capability to detect the rapid actions of eyelids. The actions of the eyelids are opening and closing. The changes in the interface can make use of the blinks for the entry of text as input. In this paper, we discuss the text entry system using eye blinks.
1.2 Text Entry using Eye Blinks
Consider the case of a person who is in the state of locked-in syndrome after suffering a massive stroke is completely paralyzed and the only means of communication is through the blink of the left eye. In this case, a person to write a typical word using blinks was two minutes. The method is slow and required two people.
The available system with text entry using blinks consists of task-based menu selections like delivering voice messages, typing and controlling home appliances. The item selection rate was increased using row-column scanning, as with other methods of single-switch input (Baljko and Tam 2006). Sub-tasks and the related options were displayed when the task was selected. For the task of typing, a scanning keyboard with scanning intervals of two seconds and three-level selection was evaluated (Bhattacharya et al.2008). The result was reported as 0.6 words per minute.
The automated scanning method for selection-traversal which is popular is replaced by BlinkWriter that is based on blinkpatterns (Krolak and Strumillo 2008). Advancing to the next column and row is done by long blinks. If there is no blink detected in two seconds then there will be advancement to the next row or column. Therefore, typing a letter needs a blink and delaying a move to the other row or column. This has an entry time of 0.7 words per minute. An Integrated text correction mechanism is not provided by these systems. BlinkWrite is the first system that met the entry speed of 4.8 words per minute and further entry speed was increased in BlinkWrite2.
2. BlinkWrite
BlinkWrite is a single key text entry system that uses scanning ambiguous keyboard (SAK) (MacKenzie and Ashtiani (in press)). BlinkWrite combines scanning keyboard (one switch input) with ambiguous keyboard (one keypress per character). Here, BlinkWrite uses blinks to select keys that are scanned or highlighted on the interface.
BlinkWrite has two regions on its interface, one region for letter selection and the other for word selection. The letter selection region contains four keys. Three keys among the four contain all the 26 letters of English alphabets that are distributed across them and the fourth key is a space key. The keys are cyclically scanned and the desired selection is done by blinking when the key is highlighted. When a key is selected, a word list is displayed in the region for word selection. Scanning switches to the word region by selecting the space key when all the letters in the word are selected or the desired word is displayed earlier. This process of selection is similar to the mobile phone keypad.
Figure 1. BlinkWrite interface
For text to be entered in English, the key part of the design is to reduce the average number of scan steps per character (SPC). In this design, a frequency ordered list of 9025 most common words in English are used, similar to the number in the research on previous text entry systems (Silfverberg et al. 2000). This design has SPC=1.713.
3. BlinkWrite2
The modified and improved version of BlinkWrite is BlinkWrite2, with changes in the layout and operation. In this section, the modifications to BlinkWrite are summarized.
3.1 Implementation
An eye tracking system from EyeTech Digital Systems by name TM3 eye tracker (http://www.eyetechds.com/) is used BlinkWrite2 to capture eye blinks. QuickGlance software acts as a hardware driver and also provides control over data capture, eye tracking, system setup and calibration. QuickLink, is an additional application programming interface that further customizes the interface and allows direct access to raw image data. In BlinkWrite, TrackerSetup is a background application developed that modified control variables of QuickGlance to enable blink clicks and disable tracking of eye movement.
QuickGlance provides various features, those for the development of BlinkWrite are sufficient, impose limitations for implementation of BlinkWrite2. One such limitation is that QuickGlance supports only two action intervals whereas, BlinkWrite2 needs three action intervals.
In BlinkWrite2, BlinkCapture replaces TrackerSetup of BlinkWrite. BlinkCapture is an application developed, that gains direct access to the image data of the eyes. This permitted flexibility for blink detection, interval calculations, and auditory feedback. BlinkCapture continually accesses the image data of the eyes and then reports visibility. A timer starts recording the duration when a blink is detected and stops when at least one eye is opened.
3.2 Design Improvements
The BlinkWrite system was reviewed for improvements and those flaws are implemented in designing BlinkWrite2 to make it more efficient and faster. Minimizing SPC improved the upper limit text entry speed of the system. The letter selection region of the BlinkWrite system has a space key that is used for ending letter selection and moving to the word selection region. The system had to scan the space key in every cycle that resulted in unnecessary scanning. Another flaw in the system was to scan extra letters to reach space key when letter selection was finished.
BlinkWrite2 introduced a new command, "jump" that jumps to the word selection region whenever the word needs to be chosen. The command can be accessed by implementing a new blink interval. The word selection region introduced column-row scanning of the words that reduces the number of scans to reach the desired word. The column- row scanning reduces the SPC of the system and the overall text entry speed by increasing the selections per scan step (SPS).
Figure 2.
3.3 Blink Intervals
The blinks in BlinkWrite2 are classified based on the duration into four intervals. A new interval for "jump" command is added to the existing three intervals of the BlinkWrite. All the upper bounds and lower bounds are adjusted as required.
Blinks with an interval less than 140ms are considered involuntary and attributed as false positives in blink detection. BlinkWrite used a 200ms threshold in accordance with previous research (Grauman et al. 2003; Jacob 1990); however this value was found to be too conservative and often voluntary blinks, causing frustration. The upper and lower bounds of the blink intervals are based on the pilot study. Blinks of duration between 140 ms and 540 ms are classified as "selection blinks". Blinks of duration between 540 ms and 1200 ms are classified as "jump blinks". Blinks with duration longer than 1200 ms are classified as "deletion blinks".
3.4 Error correction
The delete operation of BlinkWrite2 is same as that in BlinkWrite that supports both single letter and word deletion.
3.5 Auditory Feedback
Feedback is necessary for a user to know whether the intended action was performed or not. In BlinkWrite2, the eye are closed for a period of time for input, thus there is a chance of missing visual feedback. Auditory feedback is required in BlinkWrite2 as there is chances of problem arise when system commands are performed. This provides feedback to the user by informing the exact time of the operation.
Several sounds were created and checked for BlinkWrite2. Audio feedback is generated whenever a command begins. Feedback helps the users to learn accurately and perform the action commands available.
4. Methodology
The EyeTech Digital System that is eye tracker is connected to our personal pc with Windows Xp and 2.0 GHz processor. Now the blink write 2 software application is displayed on the screen and then the user perform the eye blinks. The three scanning intervals such as 700 ms, 800 ms, and 1000ms are considered to scan the data. The optimal scanning interval for BlinkWrite2 is determined by using these intervals as the marginal time for participants in scanning the data.
The eye tracker is adjusted in such a way it captures the eye blinks and perform the operations.
Figure 3. Experimental apparatus used for BlinkWrite2 evaluation. The EyeTech Digital Systems TM3 is identified by the block arrow.
The arrow is pointing towards the eye reader and it is adjusted in such a way that captures the eye blinks accurately.
Prior to the data collection the software is completely demonstrated and participants can go for trials in using the QuickGlance software. So, that the user gets familiar with the application as a result data access, word selection and error correction and feedback can be easily determined. The eye tracker is adjusted with the appropriate distance and the appropriate angle so that eyes will not get strained and get into a lock in syndrome.
5. Results and Discussion
As per the analysis the BlinkWrite2 technique has shown improvements in speed, Accuracy and scanning interval of the text while entering into the electronic device such as PC.
The speed of text entry has increased by 16% by using BlinkWrite2 when compared with that of BlinkWrite. Apart from BlinkWrite when we compared with the other techniques which use eye blinks as the input mode the speed of text entry by using BlinkWrite2 has drastically increased by 657%.
The accuracy while entering the text also improved. The former were computed using the minimum string distance (MSD) method of comparing the presented text string with the transcribed text string (MacKenzie and Soukoreff 2002).
The Scanning interval was optimized by analyzing the user behavior by which the text entry speed has improved and errors have decreased and the deletion of the text entered has reduced.
6. Conclusion
The improvements to the text entry system BlinkWrite and the scanning ambiguous keyboard were implemented and an improved text entry system BlinkWrite2 was obtained that achieved a text entry speed that is 16% faster than BlinkWrite and a 657% increase over the fastest eye blink text entry system that has been reported. Thus, BlinkWrite2 is a reliable text entry method for users with severe motor disabilities.
