A machine-learning-based fingertip recognition towards assisting hand rehabilitation

Abstract

For human beings, hands play a very important role in performing normal daily tasks. Therefore, when a person loses his/her hand’s functionality, completely or partially, because of suffering from stroke for example, treatment to regain their motor skills is crucial. One of the widely practiced is by asking the patient to squeeze a flexible therapy ball in his/her hands repetitively. This post-stroke hand rehabilitation helps patients to improve dexterity, strength and fine motor skills that have deteriorated after a stroke. In order to improve the effectiveness of the therapy, the ability to measure objectively the progress that has been made without having to make any contact is deemed to be beneficial. The first step for achieving this is the ability to recognize the fingertips, which has been the aim of this work. This research developed algorithms that allow to recognize fingertips using commercial webcams and machine learning approach when a hand is holding a therapy ball. Two proposed methods were considered using the idea of extracting features from the image and use a trained classifier to identify the object of interest. The first algorithm is using Histogram of Oriented Gradient (HOG) as feature extractor and Support Vector Machine (SVM) as classifiers while the second algorithm is using Bag-of-Features (BoF) as a feature extractor and SVM as a classifier. Feature extractors like HOG and BoF extracts distinctive features from the input image and uses the information to train the SVM classifier. The trained SVM produces a classifier that distinguishes whether the feature belongs to a fingertip or not. Our results show that the success rates for the second method has an accuracy of 96% which is higher than the first algorithm that has an accuracy of 77%. This demonstrates that both BoF and SVM are promising techniques for the recognition of fingertip in therapy-ball-holding hands.