跳到正文
原文
Nature 健康护理主题·· 11 天前AI 评分40

使用错误相关脑活动识别康复设备辅助下运动错误

Using error-related brain activity to identify mismatches during virtually simulated and electrically stimulated upper-limb movements

AI 导读

本研究通过虚拟模拟和电刺激上肢运动,探索了康复设备(如功能性电刺激FES)辅助下运动错误是否能引发错误相关电位(ErrPs)。研究发现,即使在外部设备辅助下,参与者也能产生ErrPs,表明错误感知与设备无关。研究还开发了一种新的实验范式,用于测量连续运动中的ErrPs,为改善神经假肢控制提供了新方法。

正文

Introduction

We often encounter situations in daily life when we perform an action and realize that we have committed an error, based on the outcome of our action. For example, taking a wrong turn while driving, knocking over a bottle when reaching for it, etc. Neurophysiological studies involving electroencephalography (EEG) have established the existence of a neural correlate for error awareness, which occurs when there is mismatch or discrepancy between a person’s expected and observed outcomes1,2. These are known as error-related potentials (ErrPs), which can be recorded over the scalp of an individual, above the medial frontocentral and parietal lobes, approximately 200 milliseconds after a person becomes aware that an error has occurred3. The presence of ErrPs has been documented in stroke and spinal cord injury patients undergoing rehabilitation as well4,5,6. Generally, ErrP waveforms comprise of two signal components - an initial negativity that corresponds to error processing and reward prediction, followed by a positive component that is associated with error perception2. Based on existing evidence, ErrPs are known to occur not only when individuals themselves commit an error, but also when they observe someone else commits a mistake, as well as when they receive erroneous feedback during a task. However, it is still unclear whether the error potential arises when the outcome mismatch is caused by an external device, such as a rehabilitation device for assisting movement. Patients recovering from a neurological injury such as stroke or spinal cord injury often participate in rehabilitation therapy that promotes movement of their impaired arm using a neuroprosthesis such as functional electrical stimulation (FES)7. FES operates by delivering brief pulses of low intensity electrical currents on the skin surface of a paralyzed limb, to contract the underlying muscles and restore movement in that limb8. However, several biomechanical challenges such as rapid onset of muscle fatigue, nonlinear muscle recruitment, and low specificity of surface stimulation electrodes, have undermined the efficacy and reliability of conventional FES devices, leading to weak and ineffective limb movements9. To overcome these challenges, several closed-loop algorithms have been proposed that modulate the stimulation intensity in response to errors produced during generation of movement10,11. However, all the existing strategies rely on the availability of contact or non-contact based sensors which measure the current limb position or force and compare it against a desired final position or force12. Besides man-made sensors, neural action potentials occurring within the body (e.g., from mechanoreceptors) have also been utilized as feedback signals for controlling prosthetics. But this approach requires surgically implanted electrodes, which can be risky13.

Commercially available sensors are designed to measure only a single parameter, for example, either position/acceleration or force and it is left up to the engineer to integrate multiple such sensors if they wish to simultaneously control multiple output variables (e.g., simultaneously controlling finger position and grip force to lift a cup without slipping). The use of additional sensors not only increases the device’s weight and cost, but it is also difficult to deploy in a clinical environment. Further, in real-world scenarios, it is often difficult to know beforehand what a person’s desired or final objective is. For example, crushing a bottle while holding it with a prosthetic hand is generally undesirable, but becomes highly desirable if you want to discard the bottle into a recycling bin. A neuroprosthesis devised to avoid exerting excessive grip force would fail to adapt in such scenarios, where the user’s priority changes with time. Hence, we need a versatile sensing approach that can dynamically determine when an objective is being met or when a mismatch in the user’s expected outcome occurs.

Various experimental paradigms such as choice reaction tasks, speed typing, or playing a game have been proposed earlier for measuring ErrPs2,14. Recent studies have also demonstrated the use of ErrPs for reinforcement learning, in order to improve the performance of brain-computer interfaces (BCI)3,4. However, in these studies the role of ErrP in BCIs was limited to that of a critic which helped in fine-tuning the BCI classifier for accurate detection of user intentions. The goal of this study is to investigate whether humans perceive errors in movement while receiving assistance from a rehabilitation device such as FES and whether this gives rise to error-related potentials. We also wish to understand whether the application FES pulses can interfere with simultaneously recorded brain activity via EEG, since it could hinder our ability to measure ErrPs from EEG signals. Our hypothesis is that ErrPs will occur even when the movement error is caused by an externally powered rehabilitation device, just as it occurs during the generation of self-powered or volitional movement. Previous studies on ErrPs have only utilized tasks wherein the end effector (e.g., computer cursor or robotic arm) moved in discrete steps towards the target position3. In Experiment 1, we recreated this paradigm with the help of prosthetic hand (Fig. 1A) and verified that error potentials can be measured during discrete or stepwise movements of the prosthetic. But real-world movements are seldom discrete and often occur in continuous movement sweeps, e.g., sweeping motion of one’s hand to grasp a cup, or elbow flexion to lift an object. Therefore, in this study we developed a novel experimental paradigm (Fig.1B, C) for measuring ErrPs, wherein a participant observed and reported errors generated during continuous (or sweeping) movements by FES. While previous studies2,15 have measured ErrPs when participants were continuously moving a computer gaming object to avoid obstacles, the cognitive processes involved during observation of erroneous movements are entirely different and hence, the relevance of the current study.

Fig. 1: Experimental setups for measuring error-related potentials (ErrPs) from EEG signals.

A Participants observed discrete or stepwise movements of a prosthetic hand and responded if the hand moved away from the goal position. B Participants either observed continuous (i.e., sweeping) motion of an Avatar’s arm or received electrical stimulation on their dominant arm to generate elbow flexion.

An overview of the experiment protocol and EEG processing pipeline is presented in Figs. 2 & 3, while in-depth details are included in the methods (Section 4). Section 2 presents the results of ErrPs measurements using both experimental paradigms, as well as the performance of a pattern recognition algorithm to identify ErrP in EEG signals. In Section 3, we discuss our findings from the current study and advocate a potential use case of ErrPs in deploying closed-loop control of FES-based rehabilitation devices.

Fig. 2: Timeline for Experiment 2 trials.

In case of mismatch between target and test movements, participants were asked to press the response button as soon as possible.

Fig. 3: Flowchart of EEG Signal Processing and Data Analysis Pipeline.

Top inset figure shows sample data for electrical stimulation artefacts (in black) captured at channel Oz, which have been suppressed by bandpass filtering and common average referencing.

Results

Our experimental strategy is based on the oddball paradigm, wherein frequently occurring trials without movement errors are randomly interspersed by trials with movement errors and participants are asked to respond when an error occurs. To ensure trials with error were randomly generated, we used a binomial distribution with 25% probability of generating an erroneous movement (see Methods). For experiment 1, five able-bodied participants (age 24.4 ± 7.8 y, 3 males and 2 females) were recruited. Experiment 2 attempted to measure ErrPs during continuous (or sweeping) movements by an external medium such as an animated Avatar or an electrical stimulation (Estim) device, and 16 able-bodied volunteers (age 28.7 ± 11.7 y, 13 males and 3 females) participated for this experiment.

Task performance & behavioral analysis

Response time was taken as the duration between when an error occurred and when the participant notified it by pressing the response button. As shown in Fig. 4A, participants in experiment 1 (i.e., error during discrete movements) responded in half the time than participants in experiment 2 (i.e., continuous movement errors). While the response times between the Avatar and Estim conditions showed significant differences (1.5 ± 0.38 s v/s 1.57 ± 0.5 s), the differences were much smaller than in comparison with discrete error condition (0.71 ± 0.1 s). Also, the consistency in response times was higher for discrete movement errors versus continuous movement errors. Figure 4B shows that participants were able to accurately identify trials with error from those without errors. Specifically, they were able to identify almost all the Avatar trials with error, while for the Estim trials their precision slightly dropped.

Fig. 4: Behavioral Performance Analysis.

Comparison of participants’ responses times (A) and task accuracy (B) for discrete and continuous movement errors.

Effect of Discrete versus Continuous Movement Errors/Mismatch on ErrPs

The grand-averaged EEG potentials for a few select channels under all the experiment conditions are shown in Fig. 5. Several components of ErrP signals can be identified in Fig. 5A, which depicts the effect of errors generated during discrete movements on brain signals. Firstly, an initial negativity can be observed in error potentials measured over channels CP1 and Pz, which peaks about 200 ms after an error occurs. This is followed by a strong positive component that lasts from 200 to 500 ms in channels FC1 and Cz. After the positive component, another negative component with higher amplitude occurs and lasts from 500 to 1000 ms. Note that for trials without error (shown in red), this additional negativity is absent from all the channels and could be the result of error processing. Interestingly, for channels CP1 and Pz the positive component is slightly delayed by about 200 ms and is much longer in duration ~500 ms.

Fig. 5: Grand-averaged ErrPs for different experiments and task conditions.

Only EEG channels used for training the machine learning classifier are shown here and their corresponding locations over the human scalp are shown in the inset figure. Time t = 0 s corresponds to the instant when either discrete or continuous movement error occurs. A ErrPs for a sample participant of experiment 1. The averaged traces (solid lines) and std. error (shown in shaded color) were computed for 178 trials with mismatch or error and 546 trials without mismatch. B, C ErrPs for two different participants from experiment 2, under Avatar and Estim conditions. Averaged traces are computed from approx. 40 error trials & 80 trials without error, respectively.

For the continuous movement error conditions shown in Fig. 5B, C, in general we see that ErrPs have weaker amplitude and longer latencies than for the discrete movement condition seen above. Specifically, for the Avatar trials the positive components appeared 1 sec after the error had occurred, which is followed by a negative component that peaks around 1.75 sec. For the Estim condition, an initial negative deflection appeared immediately, ~300 ms after stimulation onset for both trials with and without errors. This is likely caused by the electrotactile sensation when the stimulator turns ON. Following this, a stronger and longer positive ErrP component is observed, for example in channels Cz and CPz. It is important to note that in experiment 2 we had far fewer number of trials to compute the grand average (max. 40 for experiment 2 versus 135 ± 51 trials in experiment 1). This is the reason why the grand averages in Fig. 5B, C, are not as smooth as compared to Fig. 5A.

Effect of Error Magnitude on ErrPs

Since the end position for target and test movements varied randomly between three angles (45°, 90°, and 150°, details in Section 4.B), we further divided the error trials into 2 sub-groups, based on the magnitude of errors. Trials with large errors, comprised of only those movements where the target and test position were separated by 105° (e.g. goal was 150° and test movement reached only 45°, and vice versa). Likewise, trials with small error included all the trials where the difference between target and test position was either 45° or 60° (i.e., goal was 90° and test movement stopped at 45° or 150°, and similar combinations). We wanted to determine whether the magnitude of the error (large v/s small) influenced ErrPs. As seen in Fig. 6, trials with large errors lead to stronger ErrPs as compared to trials with smaller errors. Moreover, as seen from the traces for channels Cz and CPz, the initial response for both conditions is identical, and only after the mismatch has been processed by the brain, we see higher magnitude of ErrP for the large error condition.

Fig. 6: Changes in ErrP for trials with large and small magnitude errors between the target and test positions.

Data from a sample participant (P006), for channels Cz and CPz, during Estim condition only. Combined  grand-averages from all error trials included here, can be seen in Fig. 5C.

Since the magnitude of the ErrPs was diminished for small errors and comparable to that of trials without errors (shown in red in Fig. 5), for training our machine learning algorithm we used only the trials with large errors and measured its discriminability against trials without errors. As a result, our training dataset for the Avatar and Estim conditions had about 96 samples, which comprised of only 16 trials with error and 80 trials without error.

Single-trial classifier performance

In the previous sections, we presented results for measuring ErrPs by averaging EEG signals recorded over multiple trials. For any real-time application involving ErrPs, specifically in brain-computer interfaces, it is essential to detect them in individual EEG trials with minimal processing delays. To explore this feasibility, we trained a supervised machine learning algorithm to identify trials with continuous movement errors from those without errors (details in Methods, Section 4.D). Figure 7 plots the mean ± std. dev of classification accuracy across all participants, for both experimental paradigms and for different feature extraction windows aligned with respect to when the error occurs. As seen in the figure, for all conditions the classifiers perform above chance level (50%). In the case of discrete error condition, the accuracy stayed consistently high and above 90% and the highest accuracy obtained was 96 ± 3%. In case of continuous movements, for Avatar condition the highest accuracy hovered around 90% and for the Estim condition the highest mean accuracy for all participants was 91 ± 13%. Also, the variability in performance of classifier was much higher for continuous movement error than for discrete condition.

Fig. 7: Mean +/− SD of classification accuracies on for both discrete (N = 5 participants) and continuous movement (N = 16 participants) paradigms.

The starting point of windows used for extracting features are marked on the x-axis.

Discussions

Historically, studies on error-related potentials have investigated the neurocognitive underpinnings of our brain’s perception and compensation of error16,17. More recent work have employed ErrPs to primarily evaluate the performance of a brain-machine/computer interfaces (BCI/BMI) and improve or adapt a machine learning algorithm via reinforcement learning1,4. In other words, the role of ErrPs in BCIs was limited to that of a critic, which helped in fine-tuning the BCI classifier for detecting user intentions. In this work, we extend the role of ErrPs for improving the performance and robustness of neuroprosthetic devices, against movement errors without the need for physical sensors. We hypothesized that ErrPs occur even when observing movement errors caused by a rehabilitation device, just as it occurs when a person receives erroneous feedback from a computer or when a BCI makes an incorrect prediction.

To test this hypothesis, a novel experimental paradigm was developed for measuring ErrPs, wherein a participant observed and reported errors generated during continuous (or sweeping) movements of a rehabilitation device based on functional electrical stimulation or FES. While previous studies on ErrPs have only considered discrete movements of the end effector (e.g. computer cursor, robotic arm), it is not applicable in real-world scenarios where movement occurs in continuous movement sweeps. To better elucidate these differences, we conducted two experiments in able-bodied participants while observing errors generated during discrete (or stepping) and continuous (or sweeping) movements of a neuroprosthetic device and compared their effects on ErrPs. As seen in Section 2.A, for both experiments, participants were able to identify trials with movement errors with high precision. However, their response times were significantly altered, and it took twice as long to respond for the continuous movement task than the discrete movement task. These differences are more likely due to the differences in design of the experimental paradigms than in a participant’s ability to detect errors. For the discrete movement task, trials were much shorter in duration (i.e., 2 s) and as a result, it is likely participants were more attentive and responded quickly. Whereas for the continuous movement task, a trial typically lasted for 10–15 sec (c.f. Figure 2), which could have led participants to lose focus.

Lack of focus or attention could also have contributed to weaker amplitudes of ErrP signals observed in Fig. 5B, C, as compared to those in Fig. 5A for the discrete error condition. In fact, lack of task engagement is already known to modulate ErrPs3. While lack of engagement could be one of the contributing factors that resulted in weaker ErrPs observed in this study, there are other potential contributing factors too. One factor could be that we had too few trials (N = 40 (max) for trials with error) for computing grand averages (as compared to the 100 s of samples available in the discrete error case) and hence, the background neural activity couldn’t be entirely eliminated by averaging. While this may be true, given the constraints of time taken per trial (10–15 s), the number of trials required to ensure a reasonable level of unpredictability, and fatigue level of participants, a total of 120 trials per condition (Avatar & Estim) was a judicious choice.

Another contributing factor that could explain the results is perhaps; the complexity and attentional demand of the continuous error task was too strenuous for the participants. This is apparent from Fig. 4A, where we see that on average, participants not only took longer to respond when an error occurred during continuous movement, but the inter-trial variability was also quite high (380–500 ms). This suggests that during continuous movement task, for few trials participants must have realized it late that an error had occurred, which led to higher variability in response times. Hence, while in the case of discrete movement errors, the ErrPs are assumed to be time-locked to the instant when the error occurred, the same assumption does not hold true for continuous movement tasks. This could have potentially affected our ERP analysis for measuring ErrPs, where we followed the conventional assumption that the neural activity of interest is time-locked to when the error occurred, whereas in fact, it is time-locked to when the participants realize that an error has occurred.

In the current experiment design, participants pressed a button only when they detected an error. Therefore, it is possible that instead of ErrPs, we are instead measuring brain activity associated with pressing the button. To rule out the possibility of mistaking brain activity associated with motor preparation and execution as ErrPs, we conducted an additional response-locked analysis. We took the trials from Estim condition and epoch them with respect to when the button was pressed. In case of correct trials when the button was not pressed, we aligned the epochs with respect to when the trial ended. As shown in the figure below (Fig. 8), we observed that instead of ErrPs, Bereitschaftspotential18 or movement-related cortical potentials (MRCPs) occurred prior to the button being pressed. These signals have very different characteristics as compared to ErrPs observed in Fig. 5 (in particular, part C for Estim trials). The MRCPs are very brief and exactly aligned to the button press event, while the ErrPs are much wider. This confirms that the signals we are measuring are indeed generated by error and not from motor-related activity of pressing the button.

Interestingly, while traditional ERP analysis has some limitations in measuring ErrP, our pattern recognition classifier was able to accurately distinguish between trials with and without errors for all the conditions. The classifier performed consistently high for the discrete movement errors, and its performance wasn’t far behind for the continuous movement errors too. This suggests the potential usability of less computationally expensive machine learning models such as decision trees for real-time detection of ErrPs, when operating a neuroprosthetic. However, few important shortcomings of the current study need to be addressed before this approach can be translated to online prediction of ErrPs and are discussed next. Firstly, in this study we used independent component analysis (ICA) for removing artefacts arising from eye blinks, muscle movements, etc. and currently this can be only effectively applied for offline data processing. Fortunately, as shown in Fig. 3, the effect of electrical stimulation on brain activity could be nullified using bandpass filtering. Secondly, as seen from Fig. 5, there is a latency of about 200–500 ms between when an error occurs, and when it becomes distinguishable from EEG signals. This delay is unfavorable for implementing closed-loop control for a rehabilitation device, where poor device performance can be demotivating for the patient and adversely affect therapy outcomes. Furthermore, factors such as fatigue, attention, and learning effects are not systematically analyzed, despite their known influence on ErrP amplitude and latency. These limitations may confound the interpretation of the results and should be addressed in future studies. Finally, this study was conducted using only able-bodied participants and it remains to be seen whether the results generalize to the intended clinical population. i.e., stroke and spinal cord injury patients. Nonetheless, the study does confirm that ErrPs arise while receiving assistance from a rehabilitation device or neuroprosthetic such as FES and that the application FES does not adversely affect simultaneously recorded brain activity via EEG. This will be instrumental in future studies, for developing a closed-loop BCI-FES system that can decode ErrPs in real-time and use it to compensate for any mismatches or errors during FES-induced movement. In summary, several practical challenges do remain for closed-loop implementation such as offline ICA processing, ErrP detection latencies, etc. and hence the feasibility of this approach needs to be substantiated in future.

Fig. 8: Grand-averaged epoch EEG signals for the Estim condition at the time of button press or end of trial.

Bereitschaftspotential or MRCPs can be seen in the case of button press (black trace), which are significantly different than ErrPs observed in Fig. 5 for the error condition.

We offer a couple of directions to address above mentioned limitations. Firstly, with additional data from participants and neurofeedback provided to participants over few sessions, it is possible to adapt the classifier’s weights, such that its ErrPs detection latency is minimized. Secondly, ErrPs-based feedback can be employed with an Iterative Learning Control scheme19, wherein ErrPs measured in previous trials can be used to improve the FES’s performance in the subsequent trials. The later solution is applicable for rehabilitation, since patients typically perform several repetitions of the same task during a therapy session. The effect of above mitigation strategies on the overall performance of the system will be explored in a future study.

Methods

Two experimental setups (c.f. Figure 1) were developed to record ErrPs from able-bodied participants while observing movements generated by an external device. The goal of Experiment 1 was to verify whether error potentials can be measured during discrete or stepwise movements, as has been reported by previous studies20,21. The study procedures for both experiments were approved by the Institutional Ethics Committees (IECs) of IIT Kanpur, India (Application No. IITK/IEC/2024-25/I/17) and IISc Bengaluru, India (Application No. 08/12.10.2023), respectively.

  1. A.

    Experiment 1 Setup & Paradigm Description

    In the first experiment (Fig. 1A), participants observed a prosthetic hand move towards a set goal position in discrete steps. During each trial, the goal position was randomly set at an angle between 40° and 150° with the help of a digital servo motor (Pro-Range OT5330M, 35kg-cm, 180°) that had a pole attached to it. Although the servo motor and hence the goal position could be set anywhere between 0 and 180°, we found that limiting the range to 40°–150° was a good balance between having too few versus too many movements, during an experimental run. A second servo motor was attached to the prosthetic hand’s base, which together with a 3D printed exoskeleton (EduExo, Auxivo AG, Switzerland), formed a rotational elbow joint. This servo motor allowed the hand to rotate from 0-180° with respect to the horizontal plane. A positional encoder was also attached to the servo motor of the hand, which allowed us to measure the angular position of the hand. The prosthetic hand itself was passive and unactuated during the experiment and only used here as a dummy for a human hand.

    During the experiment, the prosthetic hand rotated either towards or away from the goal, in discrete steps of 10 degrees. For most of the trials, the hand moved towards the goal position. However, for a few random trials, the hand moved away from the goal position. The participants were informed that if their hand moves toward the goal position, then it was a correct movement, and they don’t have to do anything. However, if the hand moved in the opposite direction away from the goal, then it was an erroneous movement, and they must respond by pressing a button as soon as possible. To ensure trials with error were randomly generated, we used a binomial distribution with 25% probability of generating an erroneous movement. While each trial was 25% likely of being an error trial, it did not guarantee that in a run of 25 trials (25% or 8 trials) will always be error trials thereby minimizing any expectancy effects or influencing participant responses and neural signals. In every block, there were 10 runs for which the goal position was randomly set. Every run had up to 25 trials or movements after which the goal position and hand would reset to initial position. After 25 trials, the current run would end, even if the hand didn’t reach the goal position. But if the goal position was reached before 25 movements were completed, then the ongoing run would end, and the program moved on to the next one. Between 6 and 8 blocks were recorded for each of the participants, and a short break was given in between blocks.

  2. B.

    Experiment 2 Setup & Paradigm Description

    In the second experiment (Fig. 1B), we tested two conditions for measuring error-related potentials under continuous limb movements. In the first condition, hereafter called Avatar condition, participants watched an animated video of a virtual Avatar flexing its arm to reach a goal position. The target or goal position for the elbow joint was randomly chosen between 45°, 90°, or 150° with respect to the vertical plane, and during each trial the Avatar’s arm flexed from 0 degrees (i.e. completely extended) till it reached the target angle. This was followed by a blank screen for 2 seconds. Subsequently, the Avatar animation was shown again, which was called the test movement. In case of test movement, the arm could move to either of the three goal positions. The participants were instructed that if the target and test movements coincided, then it was a correct trial. Otherwise, if the two movements were different, then it was an erroneous or false trial, and they had to press a button as soon as they realized that a mismatch occurred.

    In condition II of the second experiment, hereafter referred to as Estim condition, participants first watched the avatar perform a target movement. Subsequently, instead of watching the test movement animation, participants received functional electrical stimulation on their bicep muscles of their dominant arm, resulting in an elbow flexion. The stimulation intensity was set at one of three predefined current levels within the range (7–24 mA), which produced either low, medium, or complete elbow flexion. Initially during calibration for each participant, the current levels were set such that their arm movement coincided with either 45°, 90°, or 150° flexion of the Avatar’s arm. However, due to the nonlinearity in muscle response, at times the electrically stimulated elbow movement did not exactly match that of the Avatar. Once a test movement began by activating stimulation, participants were asked to observe their own arm’s movement and determine if it matched the target movement. In case the target and stimulation evoked movements had a mismatch, then they had to press a button to report that there was an error.

    To deliver electrical stimulation, we used a custom-built muscle stimulator that generated biphasic current pulses with following specifications: frequency (25–35 Hz), pulse duration (100–1000 µsec), and amplitude (7–24 mA), 4 output channels. The stimulation parameters could be adjusted using a Touchscreen interface. Several on-board safety features had been built into the device, including optical isolation between the analog and digital circuits, fast blow fuses, and solid-state relays that diverted current to a low-resistance load, in case of any faults. The stimulator also included a Bluetooth LE receiver, which allowed it to be remotely turned ON/Off using a computer program.

    Figure 2 shows the timelines for sample trials under the two conditions for experiment 2. Each trial consisted of 3 phases: (i) a fixation cross would appear for 3–5 sec during which the participants remained idle and fixated on the cross, (ii) a target or desired movement animation is played for 3 sec. that shows an avatar flexing its arm, which the participants observe and are expected to remember, (iii) after a recall interval of 2 sec, a test movement occurs, which the participants compare with the target movement and respond in case of a mismatch. In the Avatar condition, during test movement phase participants only watched the Avatar arm animation being played. Whereas in the Estim condition, participants received electrical stimulation on their dominant arm, and the Avatar animation was not shown. To create a mismatch error between the target and test movements, the program intentionally chose a different final position of the Avatar in condition I. While in condition II, it selected one of the 3 intensity levels such that the participant’s arm would either overshoot or undershoot the target position. Each block consisted of 30 trials, and about 4 blocks per condition were recorded for each participant. To minimize fatigue and ensure attention, we gave sufficient breaks to the participants in between runs. We randomized the participants recruited for this experiment into 2 groups of 8 participants each and the order of experiments for Avatar and Estim conditions, were swapped for each group.

  3. C.

    Data Collection & Analysis

    For both the experiments described above, we recorded the participants’ brain activity using a 32-channel active electrode EEG system (Brain Products GmBH, Germany). EEG electrodes were placed according to the International 10–20 standard, and the reference electrode was placed above the mastoid bone, on the dominant hand side. In front of the participants, a buzzer-type response button was placed which marked an event trigger in the EEG stream, every time it was pressed. The design of experimental paradigms, presentation of task cues, as well as, generating triggers for synchronization with EEG, was achieved using Lab Streaming Layer22 and PsychoPy23. PsychoPy was also used to record a participant’s response and response time into an excel sheet. For experiment 1, on average we collected 135 ± 51 trials per participant with mismatch error, as well as 461 ± 162 trials without mismatch. In experiment 2, we collected 120 trials/condition, i.e., 240 trials per participant and one-third of these trials included mismatch or error.

    Data analysis comprised of behavioral analysis as well as processing EEG signals to identify ErrPs. Under behavioral analysis, we quantified the participants’ ability to correctly identify trials with mismatch error under different conditions and quantified their response times for each error condition. For EEG analysis, we preprocessed the multi-channel signals and eliminated non-neural artefacts using the steps shown in Fig. 3. All data analysis was performed offline in MATLAB (R2024b), and additionally, we used EEGLAB24 and ERPLAB25 toolboxes. First, we preprocessed the EEG signals by applying a bandpass filter (4th order Butterworth, 0.5–30 Hz), followed by common average referencing. As shown in Fig. 3 (inset figure), this step was sufficient to remove any artefacts introduced in the EEG signals during electrical stimulation and ensure that the signal quality has not been compromised. Next, we segmented the data into epochs with duration (−0.2 s to 2 s) and aligned the epochs with respect to when a correct or incorrect movement occurred. For the discrete movement experiment, the epochs were aligned to when the prosthetic hand moved by a step. While in case of the Avatar experiment, the epochs were aligned to when the Avatar committed an error. However, for trials with electrical stimulation, participants were able to figure out whether it was a correct or incorrect stimulation level, simply from the electrotactile feedback provided when the stimulation turned ON. As a result, we noticed that participants pressed the response button earlier and didn’t wait for the stimulator to overshoot or undershoot the target position. Hence, for trials with the Estim condition, we aligned the epochs to the start of the test trial, i.e., the time point when the stimulator was turned ON.

    To eliminate artefacts corrupting EEG data such as eye blinks, muscle artefacts, channel noise, etc. we applied the Independent Component Analysis (ICA) technique available in EEGLAB and used the extended Infomax algorithm with default settings. ICA is a statistical technique that can separate linearly mixed independent sources by maximizing their non-Gaussianity24. Further, using the ICLabel plugin available within EEGLAB, we identified and rejected between 4 and 11 artefactual ICA components. The threshold for rejecting an ICA component was set at 20% probability based on ICLabel’s output. We then performed an event-related potential (ERP) analysis, which is a well-established method in EEG literature to eliminate background noise and identify underlying trends in EEG data. We used the ERPLAB toolbox to group individual trials into bins that corresponded to trials with and without movement errors. We computed a bin-wise average for each EEG channel, and the resulting grand-average traces were examined to determine the presence of ErrPs during both discrete and continuous movement errors. Subsequently, we used the EEG signals from selected channels (FCz/FC1, Cz, CPz/CP1, and Pz) to train a pattern recognition classifier that can detect ErrPs in individual trials. These channels were selected because they predominantly showed the presence of ErrPs from the grand-averaged ERP plots for many of the participants.

  4. D.

    Training a Pattern Recognition Classifier to Identify Discrete and Continuous Movement Errors

In our training and validation EEG dataset, an imbalance in class representation exists as there are about twice the number of trials without error than those with error for each condition. To overcome class imbalance, we used a classifier based on RUSBoost (Random Under-Sampling with Adaptive Boosting) algorithm, which randomly removes examples from the majority class and iteratively builds an ensemble of classifiers using decision trees26. To classify any unseen trial, a weighted voting of all the decisions from within the ensemble is performed. A 5-fold cross validation with hyperparameter optimization was undertaken to validate generalizability of the classifier’s performance. To create a feature matrix, we calculated 4 features per selected EEG channel: slope, peak-peak amplitude, area under the envelope, and Mahalanobis distance. These features where selected for their suitability in detecting ERP-like signals from background noise, as has been previously demonstrated27. The features were calculated over 500 ms window length, starting from 0.5 s and shifting every 100 ms until 1.0 s after error occurs (for Avatar condition) or after the stimulation starts (for Estim condition). For the discrete movement errors condition, since the ErrPs occurred earlier, the window’s starting point was kept at 0 s instead of 0.5 s.

Data availability

The datasets generated and/or analyzed during the current study are not publicly available due to size constraints, but are available from the corresponding author on reasonable request.

References

  1. Buttfield, A., Ferrez, P. W. & Millan, J. D. R. Towards a robust BCI: error potentials and online learning. IEEE Trans. Neural Syst. Rehabil. Eng. 14, 164–168 (2006).

    Article  PubMed  Google Scholar 

  2. Spuler, M. & Niethammer, C. Error-related potentials during continuous feedback: using EEG to detect errors of different type and severity. Front. Hum. Neurosci. 9, 1–10 (2015).

    Google Scholar 

  3. Chavarriaga, R., Sobolewski, A., Millán, J. & del, R. Errare machinale est: the use of error-related potentials in brain-machine interfaces. Front. Neurosci. 8, 1–13 (2014).

    Article  Google Scholar 

  4. Roset, S. A., Gant, K., Prasad, A., Sanchez, J. C. & Jackson, A. An adaptive brain actuated system for augmenting rehabilitation. Front. Neurosci. 8, 1–8 (2014).

    Article  Google Scholar 

  5. Kumar, A., Pirogova, E., Mahmoud, S. S. & Fang, Q. Classification of error-related potentials evoked during stroke rehabilitation training. J. Neural Eng. 18, 056022 (2021).

    Article  Google Scholar 

  6. Usama, N., Niazi, I. K., Dremstrup, K. & Jochumsen, M. Detection of error-related potentials in stroke patients from EEG using an artificial neural network. Sensors 21, 6274 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  7. Thrasher, T. A., Zivanovic, V., McIlroy, W. & Popovic, M. R. Rehabilitation of reaching and grasping function in severe hemiplegic patients using functional electrical stimulation therapy. Neurorehabil. Neural Repair 22, 706–714 (2008).

    Article  PubMed  Google Scholar 

  8. Popović, D. B. Advances in functional electrical stimulation (FES). J. Electromyogr. Kinesiol. Off. J. Int. Soc. Electrophysiol. Kinesiol. 24, 795–802 (2014).

  9. Lynch, C. L. & Popovic, M. R. Functional electrical stimulation. IEEE Control Syst. Mag. 28, 40–50 (2008).

    Article  Google Scholar 

  10. Lynch, C. L., Sayenko, D. & Popovic, M. R. Co-contraction of antagonist muscles during knee extension against gravity: Insights for functional electrical stimulation control design. Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS 1843–1846 (2012) https://doi.org/10.1109/EMBC.2012.6346310.

  11. Bhagat, N. A., Francisco, G. E. & Contreras-Vidal, J. L. A state-space control approach for tracking isometric grip force during BMI enabled neuromuscular stimulation. IEEE Trans. Hum.-Mach. Syst. 1–10 https://doi.org/10.1109/THMS.2023.3316185 (2023).

  12. Crago, P. E., Lan, N., Veltink, P. H., Abbas, J. J. & Kantor, C. New control strategies for neuroprosthetic systems. J. Rehabil. Res. Dev. 33, 158–172 (1996).

    CAS  PubMed  Google Scholar 

  13. Scott, T. R. D. & Haugland, M. Command and control interfaces for advanced neuroprosthetic applications. Neuromodulation 4, 165–174 (2001).

    Article  CAS  PubMed  Google Scholar 

  14. Van Schie, H. T., Mars, R. B., Coles, M. G. H. & Bekkering, H. Modulation of activity in medial frontal and motor cortices during error observation. Nat. Neurosci. 7, 549–554 (2004).

    Article  PubMed  Google Scholar 

  15. Milekovic, T., Ball, T., Schulze-Bonhage, A., Aertsen, A. & Mehring, C. Detection of error related neuronal responses recorded by electrocorticography in humans during continuous movements. PLoS ONE 8, e55235 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Falkenstein, M., Hohnsbein, J., Hoormann, J. & Blanke, L. Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. Electroencephalogr. Clin. Neurophysiol. 78, 447–455 (1991).

    Article  CAS  PubMed  Google Scholar 

  17. Gehring, W. J., Goss, B., Coles, M. G. H., Meyer, D. E. & Donchin, E. A neural system for error detection and compensation. Psychol. Sci. 4, 385–390 (1993).

    Article  Google Scholar 

  18. Shibasaki, H. & Hallett, M. What is the Bereitschaftspotential? Clin. Neurophysiol. 117, 2341–2356 (2006).

    Article  PubMed  Google Scholar 

  19. Freeman, C. T., Rogers, E., Hughes, A., Burridge, J. H. & Meadmore, K. L. Iterative learning control in health care: electrical stimulation and robotic-assisted upper-limb stroke rehabilitation. IEEE Control Syst. 32, 18–43 (2012).

    Article  Google Scholar 

  20. Chavarriaga, R. & Millán, J. D. R. Learning from EEG error-related potentials in noninvasive brain-computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 18, 381–388 (2010).

    Article  PubMed  Google Scholar 

  21. Iturrate, I., Montesano, L. & Minguez, J. Task-dependent signal variations in EEG error-related potentials for brain – computer interfaces. J. Neural Eng. 10, 026024 (2013).

    Article  CAS  PubMed  Google Scholar 

  22. Kothe, C. et al. The lab streaming layer for synchronized multimodal recording. Imaging Neurosci. 3, IMAG.a.136 (2025).

    Article  Google Scholar 

  23. Peirce, J. et al. PsychoPy2: experiments in behavior made easy. Behav. Res. Methods 51, 195–203 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  24. Delorme, A. & Makeig, S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21 (2004).

    Article  PubMed  Google Scholar 

  25. Lopez-Calderon, J. & Luck, S. J. ERPLAB: an open-source toolbox for the analysis of event-related potentials. Front. Hum. Neurosci. 8, 213 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  26. Seiffert, C., Khoshgoftaar, T. M., Van Hulse, J. & Napolitano, A. RUSBoost: Improving classification performance when training data is skewed. in 2008 19th International Conference on Pattern Recognition 1–4 (IEEE, Tampa, FL, USA, 2008) https://doi.org/10.1109/ICPR.2008.4761297.

  27. Bhagat, N. A. et al. Design and optimization of an EEG-based brain machine interface (BMI) to an upper-limb exoskeleton for stroke survivors. Front. Neurosci. 10, 122 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank Prof. Niraj Sinha, Dept. of Mechanical Engineering, IIT Kanpur for providing the prosthetic hand that was used in part of this research, and Prof. Chandrashekhar Seelamantula, Dept. of Electrical Engineering, Indian Institute of Science Bengaluru, for the insightful discussions related to machine learning.

Funding

This research was funded by the Ramalingaswami Fellowship, Dept of Biotechnology, Ministry of Science and Technology, India (Project No. BT/RLF/Re-entry/30/2021).

Author information

Authors and Affiliations

  1. Assistant Professor, Dept. of Electrical Engineering & Dept. of Biological Sciences and Bioengineering, Indian Institute of Technology, Kanpur, U.P, India

    Nikunj Arunkumar Bhagat

Authors

  1. Nikunj Arunkumar Bhagat

Contributions

N.B. designed the study, collected the experimental data, performed the analysis, and prepared the manuscript.

Corresponding author

Correspondence to Nikunj Arunkumar Bhagat.

Ethics declarations

Competing interests

The author declares no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bhagat, N.A. Using error-related brain activity to identify mismatches during virtually simulated and electrically stimulated upper-limb movements. npj Biomed. Innov. 3, 66 (2026). https://doi.org/10.1038/s44385-026-00123-4

Download citation

  • Received: 12 February 2026

  • Accepted: 13 October 2026

  • Published: 09 November 2026

  • Version of record: 09 November 2026

  • DOI: https://doi.org/10.1038/s44385-026-00123-4

来源:Nature 健康护理主题 · nature.com