Biometric handwriting analysis to support Parkinson’s Disease assessment and grading

Background Handwriting represents one of the major symptom in Parkinson’s Disease (PD) patients. The computer-aided analysis of the handwriting allows for the identification of promising patterns that might be useful in PD detection and rating. In this study, we propose an innovative set of features extracted by geometrical, dynamical and muscle activation signals acquired during handwriting tasks, and evaluate the contribution of such features in detecting and rating PD by means of artificial neural networks. Methods Eleven healthy subjects and twenty-one PD patients were enrolled in this study. Each involved subject was asked to write three different patterns on a graphic tablet while wearing the Myo Armband used to collect the muscle activation signals of the main forearm muscles. We have then extracted several features related to the written pattern, the movement of the pen and the pressure exerted with the pen and the muscle activations. The computed features have been used to classify healthy subjects versus PD patients and to discriminate mild PD patients from moderate PD patients by using an artificial neural network (ANN). Results After the training and evaluation of different ANN topologies, the obtained results showed that the proposed features have high relevance in PD detection and rating. In particular, we found that our approach both detect and rate (mild and moderate PD) with a classification accuracy higher than 90%. Conclusions In this paper we have investigated the representativeness of a set of proposed features related to handwriting tasks in PD detection and rating. In particular, we used an ANN to classify healthy subjects and PD patients (PD detection), and to classify mild and moderate PD patients (PD rating). The implemented and tested methods showed promising results proven by the high level of accuracy, sensitivity and specificity. Such results suggest the usability of the proposed setup in clinical settings to support the medical decision about Parkinson’s Disease.


Background
Parkinson's Disease (PD) is the second most common neurodegenerative disorder after Alzheimer's disease that leads to several neuro-motor deficits. It is well known that PD patients exhibit problems when the perform movements that are executed sequentially due to the loss of coordination among the motor sequence components [1][2][3][4][5]. As a result, sequential movements are more segmented and characterized by pauses between submovements [6].
The handwriting is a task composed of sequential movements that involves fine and complex manual skills relying on a sophisticated mix of cognitive, sensory and motor components [7]. This explains the manifestation of abnormal features in the handwriting of PD patients. The difficulties in the handwriting process affecting PD patients are mainly two: • difficulties related to the control of the movement amplitude, e.g. decreasing the size of the characters (micrographia) and failing in keeping the stroke width of the characters constant as the writing progresses [8][9][10][11][12][13][14][15]; • not regular and bradykinetic movements that lead to an increased movement duration, decreased speed and accelerations, and unstable velocity and acceleration [16][17][18][19][20][21].
Several research groups have investigated the use of handwriting's features to classify PD patients and healthy subjects.
Helsper et al. published a study that investigated the handwriting differences between preclinical PD patients and healthy controls [22]. The authors analysed two lines of the handwritten text (sampled from a longer written text) and proposed an approach that considers (1) the extraction of 10 features from text segments written by test subjects as a first step, and then (2) the computation of a single resulting feature set based on the mean, the standard deviation and the frequency of the occurrences. The authors statistically proved the existence of features characterizing many years before the diagnosis.
Longstaff and collegues studied the relation between the inclination of PD patients to scale the character size and reduce the speed of drawing movements and the movement variability [23]. The experiment is based on the analysis of several geometrical writing patterns with different shape and size drawn with a pen on a graphics tablet. By analysing the extracted features the authors stated that there is a substantial divergence in the quality of movements between PD patients and healthy people A different recording set-up has been used by Ünlü et al. [24] that recorded the pressure and the inclination of an electronic pen during writing tasks. Their proposed approach considered the extraction of 8 different features and the use the Receiver Operating Characteristic (ROC) to analyse the diagnostic possibilities both in term of sensitivity and specificity. Their results showed that the most representative feature is based on the difference between the writing pressure and the tremor of the pen tilt angle.
Electronic pen and tablet have been also used by Rosenblum et al. to collect the position, the pressure and the angle of the pen tip during the writing of two main patterns (i.e. the name and fixed address) [25]. The average values of the pressure and velocity acquired during the entire task and other spatial and temporal characteristics of each stroke allowed them to differentiate PD patients from control subjects with a sensitivity of 95.0%.
In our preliminary previous works [26,27], we proposed a promising method to classify PD patients from healthy subjects by using only 4 features extracted by scanned text (through image processing techniques) and surface ElectroMyoGraphy (sEMG) signals. Recently, we used a graphic tablet and the Myo armband to extract biometric signals related to pen movements (pen tip position, inclination and pressure) and muscle activation [28,29]. These signals were processed to extract a number of features used as input to two different classifiers.
In this work, we improve our recent studies by proposing a larger feature set and testing our classifiers on bigger cohort of subjects. Furthermore, we focused on the selection of the most representative features that better highlight the handwriting differences between (1) mild and moderate PD patients, and (2) PD patients and healthy subjects.

Methods
The proposed model-free technique for the analysis of handwriting is computer-assisted and based on the extraction of features from biometrical signals [30] (i.e., sEMG signals, pen tilt, etc.) related to hand movements during handwriting tasks.
In the following sections, the features selected and used in our technique and the algorithms used for feature selection and classification are presented and described.

Handwriting feature extraction
Handwriting features have been derived from biometric signals obtained during handwriting tasks. In general, the proposed features can be grouped into two groups based on sEMG and pen tip signals: • Features derived from sEMG signals -these features are related to the subject's muscle activity and are derived from the sEMG signals obtained from the forearm of the subject: -RMS features: for every sEMG channel, Root Mean Square (RMS) is computed. It is , where x i is the sample value at the discrete-time i, and n is the total number of samples acquired.
-ZC features: Zero Crossing (ZC) is a variance-related index. In detail, it is the number of sign variations between two consecutive samples, and it is only increased if the difference exceeds a predefined tolerance value. Considering two consecutive samples x k and x k+1 , the ZC value is increased if and only if the following condition is satisfied: x k > 0 and x k+1 < 0, or x k < 0 and x k+1 > 0 and |x k − x k+1 | ≥ tol. Due to the presence of noisy signals, the tolerance value (tol ) is used to prevent ZC increment; it is measured as the average of the standard deviations of all the sEMG channels considered. In addition, its value is divided by the length of the signal to normalize the features among the subjects.  • Pen tip related features -these features are extracted during the handwriting task from the signals produced by a graphic tablet: -Cartesian and XY features: these features refer to the pen tip writing kinematics on the graphic tablet and are derived from the orientation of the XY axes. This group of features includes the features extracted from the following signals: Cartesian and XY-velocity, Cartesian and XY-acceleration, Cartesian and XY-jerk. The features are determined as first, second and third derivatives, respectively, starting from the X-Y position. It leads to nine output signals. -Pen tip pressure feature: this is a scalar feature which refers to the pressure applied to the tablet surface by the pen tip. -Azimuth and altitude feature: the azimuth feature is the angle value between a reference direction (e.g., the Y axes of the tablet) and the pen direction projected on the horizontal  plane. The altitude feature is the angle value between the pen direction and the horizontal plane. -Pattern specific features: these features are related to the writing size of letter-based patterns and the writing precision of spiral writing ones. In detail, for the letter-based patterns, the pen tip Y coordinate has been processed, and the upper and lower peaks are then determined. The two groups of peak points are then independently used to determine the upper regression line R up and the lower regression line R low using linear regression. Finally, the α angle between the regression lines is computed. The graphical representation of the procedure is shown in Fig. 1. A further feature taken into account is the coefficient of determination (R 2 ) computed according to Eqs. (2).
The three resulting patterns chosen as descriptors of the variability of the writing size are the α angle between the regression lines and the two coefficients of determination (R 2 ). For the spiral patterns, instead, the feature is based on the variability of the strokes. We computed the vector − → r originating in P with respect to the spiral centroid point C for each  The output layer configuration was preliminarily fixed with two neurons and softmax as activation function point P of the X-Y pen tip position. Then, we calculated the angle β between − → r and the direction vector − → d tangent to the spiral in P. The visual explanation of the process is depicted in Fig. 2. Lastly, we calculated the standard deviation of the β angles computed for each point P. The standard deviation value is the feature chosen to describe the precision of the spiral.

Feature selection and classification
A feature selection procedure has been used to understand the main representative for the subject's status [31]; we used an approach similar to the entropy criterion (i.e., information gain) and based on a classification decision tree technique with Gini's diversity index [32].
The classification procedure is based on Artificial Neural Network (ANN) classifier [33][34][35]. Since designing the topology of neural classifiers is challenging, several works have been proposed dealing with this task [36][37][38]. In this work, the topology of the classifier was optimised by the evolutionary approach proposed in [39].
We used a Multi-Objective Genetic Algorithm (MOGA) to find the optimal network topology. A genetic algorithm is a powerful optimization technique that reflects the process of natural selection where the fittest individuals are selected for reproduction in order to produce offspring of the next generation, thus explaining its feasibility in several research domains [40,41]. The MOGA algorithm was configured to find the optimum varying the following network parameters: (i) number of hidden layers (integer interval: 1 to 3), (ii) number of neurons per layer (integer interval: 1 to 256 for the first hidden layer and 0 to 255 for other ones), and (iii) activation functions for each layer (one among: log-sigmoid -logsig, hyperbolic tangent sigmoid -tansig, pure linear -purelin and symmetric saturating linear -satlins).
The overall performances of MOGA algorithm and classification network were evaluated using the confusion matrix reported in Table 1 and the performance indexes reported in Eqs. 3, 4 and 5.

System set-up
The data acquisition is based on the system set-up (Fig. 3) presented in a previous work [28]. The set-up is based on two sensors: the Myo™Gesture Control Armband [42] for sEMG signal acquisition and the graphics tablet WACOM Cintiq 13" HD [43] for acquiring handwriting signals and data. In detail, the Myo™Armband allowed us to acquire 8 different sEMG signals of the forearm, whereas the tablet permitted us to acquire the pen tip coordinates and pressure, and the tilt of the pen with respect to the writing surface.

Experimental description
We used three writing patterns for the experiments leading to as many writing tasks. The patterns selected were as follows: 1 Pattern 1 -a five-turn spiral drawn in anticlockwise direction; 2 Pattern 2 -a sequence of 8 Latin letter "l" with a size of 2.5cm; 3 Pattern 3 -a sequence of 8 Latin letter "l" with a size of 5cm.
It is possible to observe that only two writing patterns (Patterns 2 and 3) were size-constrained, and for those, a visual marker has been provided as a size reference.
In the experiment, we asked each subject to perform four repetitions for each writing tasks: the first one was used to familiarise with the task and was discarded, the remaining three were recorded as data for the following processing.
Each handwriting tasks was interleaved with a rest period of three seconds, and the first pressure point on the tablet has been used as a trigger for the tasks begin.
Each subject generated 41 features for writing task 1 and to 43 features for writing task 2 and 3: • Root Mean Square (RMS) of each sEMG signal (8 RMS features for each subject and for each task); • Zero Crossing (ZC) of each sEMG signal (8 RMS features for each subject and for each task); • mean and standard deviation for each subject and for each task of the following signals: -pen tip Cartesian velocity (two features); -X and Y velocity components of the pen tip (four features); -pen tip Cartesian acceleration (two features);  -X and Y acceleration components of the pen tip (four features); -pen tip Cartesian jerk (two features); -X and Y jerk components of the pen tip (four features); -pen tip pressure (two features); -pen azimuth (two features); -pen altitude (two features); • pattern specific features: -index of precision of the spiral drawing (one feature); -size related features for the constrained patterns (three features).

Experimental data processing description
The objectives of the conducted experiments were mainly two: 1 the first was to separate PD patients from healthy subjects; 2 the second was to correctly classify mild and moderate Parkinson patients.
For each objective, the features extracted during the experiments were grouped according to the following scheme: • creation of three different feature datasets: -dataset A including only the features extracted from writing pattern 1 (41 features); -dataset B including only the features extracted from writing pattern 2 (43 features); -dataset C including only the features extracted from writing pattern 3 (43 features);  To assess both objectives of the experiments, an Artificial Neural Network (ANN) classifier featuring an optimised topology provided a Multi-Objective Genetic Algorithm (MOGA) was used.
In detail, for each objective and each of the six cases, we estimated the performance indexes of the ANN optimal topology approach, reported in Eqs. 3, 4 and 5, in terms of percentage and standard deviation. Due to the dependence of the ANN performances from both net initialisation and permutation of training-validation datasets, we iterated the assessment over 250 repetitions of the net training procedure. Figure 4 depicts the scheme of the experiments.

Results
Two samples of one repetition of the writing tasks performed respectively by healthy and PD subject are reported (not in real scale) in Figs. 5 (task no.1), 6 (task no.2) and 7 (task no.3).
Based on the two objectives the presentation and the discussion of the results obtained from the experiments have been subdivided. In particular, for each objective, we have reported the results obtained from both the feature selection and the classification for each of the six cases.
For each case the training procedure was iterated 250 times to assess the stability of the learning process; hence, the confusion matrices and the related results are presented in percentage with the standard deviation reported in brackets.

Objective 1 -separating PD patients and healthy subjects:
• Feature Selection results: The application of the feature selection algorithm previously reported, led to a significant reduction of the number of considered features for all three datasets of features extracted from the writing patterns. In particular: • for dataset A including the 41 features extracted from writing pattern 1, the sEMG RMS value, three sEMG ZC values, the mean Cartesian velocity and the mean acceleration on X axes were the six selected features to be classified in Case 4; • for dataset B including the 43 features extracted from writing pattern 2, the mean jerk on Y axes, three sEMG ZC values, the mean Cartesian acceleration and the mean velocity on X axes were the six selected features to be classified in Case 5; • for dataset C including the 43 features extracted from writing pattern 3, two sEMG RMS values, a sEMG ZC value, the mean cartesian velocity, the altitude STD, the azimuth RMS and the mean velocity on X axes were the seven selected features to be classified in Case 6. • Classification results: for each of the six different feature datasets, the MOGA algorithm is applied to provide the optimal ANN topology. The optimal topology results and the confusion matrices are reported in Table 2 and in Tables 3, 4, 5, 6, 7 and 8, respectively; the performances expressed in terms of accuracy, specificity and sensitivity have been summarised in Table 9.

Objective 2 -separating mild and moderate PD patients:
• Feature Selection results: the application of the feature selection algorithm previously reported, led to a significant reduction of the number of considered features for all three datasets of features extracted from writing patterns. In particular: -for dataset A including the 41 features extracted from writing pattern 1, two sEMG • Classification results: for each of the six different feature datasets, the MOGA algorithm is applied to provide the optimal ANN topology. The optimal topology results and the confusion matrices are reported in Table 10 and in Tables 11, 12, 13, 14, 15 and 16, respectively; the performances expressed in terms of accuracy, specificity and sensitivity have been summarised in Table 17.

Discussion
For the sake of clarity, we summarised the accuracy of all cases for both objectives in Table 18. As reported in the table, the proposed procedure leads to high accuracy performances; the results for both objectives present accuracy in the range 86 < x < 97, with a standard deviation lower than 0.09. The low value of the standard deviation allows us to assess the stability of the learning process of the optimal ANN. Similar observations can be stated for both objectives for the classification of the selected features. In detail: • the classification between PD patients and healthy subjects (objective 1) achieves the best accuracy (96.85%) in Case 6 (seven features selected from the dataset of 43 features extracted from writing pattern 3). The feature selection stated that three out of seven features were related to sEMG signals (RMS and ZC), whereas the others to pen tilt and velocity; • the classification between mild and moderate PD patients (objective 2) achieves the best accuracy (96.00%) in Case 4 (six features selected from the dataset of 41 features extracted from writing pattern 1). The feature selection stated that four out of six features were related to sEMG signals (RMS and ZC), whereas the others to pen tilt and velocity;

Conclusions
In this work, we proposed an innovative methodology for computer-assisted handwriting analysis with the main goal of PD detection (healthy subjects vs. PD patients) and rating (mild vs moderate PD patients). The proposed approach is based on extracting different features from biometric signals collected during handwriting tasks and using such features to detect and rate PD. The developed decision support system (DSS) is based on an artificial neural network whose topology has been optimized with a MOGA. The results showed that the proposed DSS is able to classify healthy subjects vs PD patients and mild vs moderate PD patients with a high classification accuracy (more than 90.0%). Furthermore, we proved that a limited set of representative feature selected by means of a classification decision tree technique, that uses the Gini's diversity index, improves the overall accuracy (more than 96.0%).
Future works are needed to investigate the DSS performance with a larger cohort of subjects that includes severe PD patients too. This will allow us to classify PD patients by using more than two PD status classes and to monitor        the progress of the disease over time. Furthermore, due to the time-consuming acquisition steps, it is desirable to reduce the required number of pattern tasks; this will be achieved through a proper writing pattern selection among the proposed ones.