Analysis of visually impaired people's static tactile graphics perception based on computer vision system using colour markers detection method

Abstract

Accessibility to visual information has always been a challenge for people with visual impairment and blindness. Therefore, tactile graphics became one of the alternatives to help them access maps, graphs, painting, illustration and other graphical materials. However, there are still challenges to use tactile graphics. This is because the blind and visually impaired people need good exploratory procedures and strategies to gather information tactually. Several studies in psychology have been conducted to clarify how these group of people perform tactile graphics exploration. However, there are limitations to the methods used and the information is still insufficient. Most of the studies conducted rely on video observation which is time-consuming to analyse and to interpret the captured video. To provide more information on how these affected people explore and read tactile graphics, a fingertip tracking system using Logitech Webcam and MATLAB software has been developed. The system can track the fingertips by using colour markers detection method. It is also able to record the position of the fingertips which will then be used for analysing the reading and exploration behaviour. After being tested under various conditions, the system had worked well to detect and track all colour markers. The system also had been used for the experimentation with four totally blind participants from Malaysian Association for The Blind (MAB). They have read five different raised line tactile graphics that have been prepared there. Overall, the average elapsed, maximum and minimum time for the system to detect and track all colour markers is at 0.22 second, 1.00 second and 0.19 second respectively. For the experiment results, all four participants responded that the most difficult diagram to read was the apple diagram and it had the longest reading time with an average time of 141 seconds. The distributions and concentrations of fingertips locations on each tactile graphics were also being analysed. The method that had been used was two-dimensional heat map spatial distribution with marginal histogram. It is identified that every diagram has salient regions that have been focused on by every participant for recognition and interpretation. The final analysis was the reading strategies used by each participant focusing only on contour following and lateral motions. The methods used for the analysis were path segmentation based on the range of frames and trajectory plotting. It is identified that every participant had performed those two movements, but they were not systematic and strategize. Their movements of reading were also too random and no discernible pattern. In conclusion, it is aimed that the system used can be a tool to visualize the cognitive process of people visual impairment and blindness explore tactile graphics materials. The results presented through this dissertation can be a preliminary insight into the cognitive process that involves during tactile graphics reading and exploration by people with visual impairment and blindness.