WHITE PAPERS 7276

Validation and Performance of a Wearable Heart-Rate Monitoring Algorithm

By: Husnu Hanayli

Abstract:

Biosensing wearables rely on sophisticated algorithms that process signals collected by sensor ICs, turning the data into actionable insights. Read about a study validating the performance of Maxim Integrated's wearable heart-rate monitoring (WHRM) algorithm, which utilizes synchronized three-axis accelerometer and PPG data to provide several end-user-friendly fitness tracking outputs.


Table of Contents

  1. Introduction

    1.1 Maxim’s WHRM Solution

  2. Validation and Performance

    2.1 Data Collection
    2.2 Performance Metrics
    2.2.1 +/- 5 BPM Error Band Accuracy
    2.2.2 Mean Absolute Error (MAE)
    2.2.3 %95 Confidence Interval
    2.2.4 Time-Alignment
    2.3 Performance Results
    2.4 Daily Data Sets Validation
    2.5 Measurement Quality Metric
    2.6 Maxim HRM Algorithm Signal Requirements
    2.6.1 Signal Requirements
    2.6.2 Reference Device
    2.6.3 Signal Quality
    2.7 Activity Classification and Steps Counting Validation

  3. Summary
  4. References

1 Introduction

By continuously tracking parameters such as heart rate, wearable health and fitness monitors are providing valuable insights into well-being. Heart rate is the frequency of heart contractions that is measured in beats per minute (BPM). The pulse rate is the number of times the arteries create a measurable pulse due to the blood-pressure change created by heart contraction. Normally, the PPG (photoplethysmographic) signal provides information about the pulse rate rather than heart rate. However, in this paper, we will use “heart rate,” which is a standard in the market.

Biosensing wearables rely on sensor ICs that measure biometric signals and sophisticated algorithms that process these signals into actionable data. This paper presents a study validating the performance of Maxim’s wearable heart-rate monitoring (WHRM) algorithm. The WHRM algorithm utilizes synchronized three-axis accelerometer and PPG data to provide several end-user-friendly fitness tracking outputs like heart rate, as well as activity-related statistics such as step count, burned calories, and activity class of the user. The PPG signal can be gathered from contractions of ventricles in the finger, wrist, ear, toe, chest, etc.

1.1 Maxim’s WHRM Solution

Maxim’s WHRM solution includes heart rate and activity-tracking algorithms, especially for heart-rate detection, cardiac-beat detection, step measurement, and activity classification. The envisioned product targets health and wellness applications for continuous 24/7 monitoring with optimized power management and the provision of accurate heart rate and interbeat interval (IBI) detection, step detection, and sample-by-sample activity classification.

There are, however, critical factors which make PPG-based heart-rate monitoring from the wrist extremely challenging:

  • Periodic/non-periodic motion
  • Electrical noise
  • Mechanical/strap design
  • Ambient light
  • Melanin/skin color
  • Low blood perfusion

The WHRM algorithm overcomes these challenges via:

  • Motion compensation and suppression
    • Detects the activity/motion type and estimates the effect of this motion on the PPG signal
    • Clears the effect of the motion from the PPG signal for reliable heart-rate monitoring
  • Multi-channel fusion
    • Combining several channels with signal-processing techniques to get one improved PPG signal
  • Control of the AFE settings:
    • To optimize signal quality (in the scope of electrical noise) and power consumption
    • To overcome challenging cases which come from low blood perfusion or melanin/skin color

2 Validation and Performance

To validate the performance of the WHRM algorithm, we conducted an evaluation on 25 subjects. The participants had varying ages, skin color, blood perfusion, and physical conditions. The protocol was designed to experience different heart-rate levels and heart-rate variability behaviors to assess the algorithm on all possible conditions.

2.1 Data Collection

In the evaluation, we utilized a dataset consisting of 82 data sequences collected from 25 different subjects. Each sequence of data was about 36 minutes long and, during this period, each subject performed several activities like lying, sitting, walking, running, cycling, etc. The order and duration of these activities were defined via the “Maxim sports protocol,” and each subject applied this protocol. Table 1 shows the complete protocol used to collect this dataset.

Table 1. Maxim 36-Minute Sports Protocol

Activity Time Duration Location Comments
Resting (Lying Down) 0:00 3 min In Gym  
Resting (Sitting) 3:00 3 min In Gym  
Transition (Standing) 6:00 1 min In Gym  
Walking 7:00 5 min In Gym 2.5 mph
Transition (Standing) 12:00 1 min In Gym   
Running 13:00 5 min In Gym Comfortable running speed Ambient light @ 3000 lux
Transition (Standing) 18:00 2 min In Gym  
Walking (Fast) 20:00 3 min In Gym  3.5 mph
Walking (Fast with Incline) 23:00 3 min In Gym 3.5 mph, 5% incline
Transition (Standing) 26:00 2 min In Gym  
Cycling 28:00 3 min In Gym  Comfortable pace
Resting (Standing) 31:00 5 min In Gym  

The subjects had varying ages, skin color, perfusion, physical conditions, etc. The ages and Fitzpatrick (FP) scale distribution of the participants can be found in Figure 1.

Fitzpatrick scale distribution of 25 subjects.Figure 1. Fitzpatrick scale distribution of 25 subjects.

Age distribution of 25 subjects.Figure 2. Age distribution of 25 subjects.

The data was collected with a Maxim reference design watch, which consists of one LED and two multi-channel photodiodes (PDs), and, as the reference measurement device, an ECG base chest strap. The data was collected in a laboratory environment using a treadmill and gym/exercise bike for the activities. Sample heart-rate plots can be found in Figures 4 through 8.

2.2 Performance Metrics

Three different performance measures are used in this study. All metrics are given in 1Hz due to the limitations of the reference device.

2.2.1 +/- 5 BPM Error-Band Accuracy

This is the percentage of the valid data points which have absolute percent error of ≤ 5BPM with respect to reference device output (ECG chest strap). This metric is calculated in 1Hz.

2.2.2 Mean Absolute Error (MAE)

Mean Absolute Error is the average of the absolute difference of each Maxim HRM algorithm and the reference device outputs (ECG chest strap). This metric is calculated in 1Hz.

2.2.3 95% Confidence Interval

This is an interval that the 5BPM error band accuracy numbers will fall within (with 95% probability), if the tests are performed in a similar way again. For example, let us take the 95% confidence interval of the walking activity which is [95.15, 98.38]. If we perform the tests again in a similar way, the 5BPM error-band accuracy will fall within 95.15% and 98.38% with 95% probability.

2.2.4 Time Alignment

Due to real-time data collection conditions, there might be a time difference between the Maxim heart-rate output and the reference device (ECG chest strap) output due to:

  • The possibility of the reference device starting with some time delay originating from the device electronics or apps
  • The possibility of some time delay, originating from the user, in the start of the data collection

The evaluation was applied after the data sets were time-aligned by removing that time difference. The time alignment sample can be found in Figure 3.

Time alignment between maxim heart rate and reference device (ECG Chest Strap) outputs example.Figure 3. Time alignment between maxim heart rate and reference device (ECG chest strap) outputs example.

2.3 Performance Results

The Maxim sports protocol data sets (Table 1) were processed with the WHRM algorithm. Performance results were calculated with the metrics defined in Section 2.2. The results were calculated using multi-channel fusion (as shown in Table 2).

Table 2. Maxim WHRM Algorithm Performance Results Based on Performance Metrics Defined in Section 2.2

Activity +/- 5BPM Error-Band Accuracy [%] MAE [BPM] 95% Confidence Interval [%, %]
Resting (Lying Down) 99.13 0.82 [98.56 - 99.69]
Resting (Sitting) 97.55 1.12 [96.69 - 98.39]
Walking 86.31 2.74 [82.33 - 90.29]
Running 86.57 3.04 [82.57 - 90.55]
Walking (Fast) 91.42 1.92 [88.48 - 94.36]
Walking (Fast with Incline) 93.86 1.79 [90.97- 96.74]
Cycling 89.71 2.17 [86.23 - 93.18]
Resting (Standing) 96.13 1.56 [94.56- 97.70]
Overall 92.00 2.01 [90.17 - 93.83]

The WHRM algorithm and ECG chest strap outputs were highly correlated within the data set with a wide range of melanin/skin color, perfusion, ages, physical conditions, etc. The results show that the algorithm reports reliable heart rate in all activities. The most challenging activities are walking and cycling, as can be seen in Table 2. The challenging part in walking is that the heart-rate frequency is generally close to the motion frequency, and the motion artifact component is much stronger than the heart-rate component in the PPG signal. On the other hand, in indoor cycling (at gym/exercise bike), the accelerometer signal can weakly be used for motion compensation.

Multi-channel fusion is another strong approach in Maxim’s WHRM solution. The end-user device might generate more than one PPG signal with multiple PDs/LEDs. The Maxim solution has a special multi-channel fusion method which yields a performance increase, especially in running and walking activities. The method’s enhancement results can be viewed in Table 3.

Table 3. The Maxim WHRM Solution Multi-Channel Fusion Method Enhancement on Performance

Activity Single-Channel +/- 5BPM Error-Band Accuracy [%] Multi-Channel +/- 5BPM Error-Band Accuracy [%]
Resting (Lying Down) 99.13 99.43
Resting (Sitting) 97.55 97.55
Walking 86.31 89.76
Running 86.57 90.65
Walking (Fast) 91.42 94.26
Walking (Fast with Incline) 93.86 95.47
Cycling 89.71 91.99
Resting (Standing) 96.13 97.36
Overall 92.00 94.17

Figure 4. Maxim WHRM solution versus ECG chest strap (reference device) outputs – standard protocol, example 1.

Figure 5. Maxim WHRM solution versus ECG chest strap (reference device) outputs – standard protocol, example 2.

Figure 6. Maxim WHRM solution versus ECG chest strap (reference device) outputs – standard protocol, example 3.

Figure 7. Maxim WHRM solution versus ECG chest strap (reference device) outputs – standard protocol, example 4.

Figure 8. Maxim WHRM solution versus ECG chest strap (reference device) outputs – standard protocol, example 5.

2.4 Daily Data Sets Validation

The WHRM solution was also validated in daily life conditions. ~120 separate data sequences of about three to four hours were collected from 50 different subjects. The data was collected with the Maxim reference design watch. After the data collection was triggered, the participants continued their daily life activities such as walking around in an office, typing, eating, etc. The most challenging part in collecting daily life data is the impact of non-periodic/irregular motion. The performance results are shown in Table 4.

Table 4. Maxim WHRM Algorithm – Daily Data Performance Results

Activity +/- 5BPM Error-Band Accuracy [%] MAE [BPM] 95% Confidence Interval [%, %]
Daily Life 91.1 1.82 [89.75, 92.45]

Figures 9 and 10 show sample daily life outputs from the WHRM solution with respect to the ECG chest strap. It can be seen in both the table and the figure that the WHRM algorithm’s results are highly correlated with the ECG chest strap.

Figure 9. Maxim WHRM solution versus ECG chest strap (reference device) outputs – standard protocol, xxample 1.

Figure 10. Maxim WHRM solution versus ECG chest strap (reference device) outputs – standard protocol, example 2.

2.5 Measurement Quality Metric

As explained in Section 1.1, there are critical factors which affect the performance of PPG-based heart-rate monitoring. Therefore, to reliably report on heart rate, the quality of the measurement is the key factor. In the validation phase of our study, we compared the report of the algorithm with the data collected by the ECG-based device and, thus, were able to calculate the quality/performance of the measurement. However, in real-life conditions, there will not be a reference device with which to evaluate the quality of the measurement. The WHRM solution provides a measurement quality metric which enables evaluation of the heart-rate reports sample-by-sample in real time.

Maxim’s heart-rate measurement quality metric scores the measurement between 0 and 100:

  • 100%: Perfect measurement quality
  • 75%: Good enough measurement quality
  • 50%: Mostly good enough measurement quality
  • 25%: Poor measurement quality
  • 0%: Unreliable measurement

Each heart-rate value is reported with a measurement quality value in the algorithm, so the heart-rate report can be evaluated in real time. Table 5 provides the results of the performance of the measurement quality metric with the data sets explained above.

Table 5. Example of the Performance of the Measurement Quality Metric and Heart-Rate Measurement.

Rest (Lying, Sitting, Standing, Sleeping)
Data Points with Measurement Quality Metric> ≥ 100 ≥ 75 ≥ 50 ≥ 25 ≥ 0
+/- 5BPM Accuracy [%] 98.77 98.28 97.75 97.42 97.34
MAE [BPM] 0.99 1.08 1.18 1.24 1.24
Reporting Coverage [%] 82.80 94.07 97.61 99.36 100.00
Walking
Data Points with Measurement Quality Metric ≥ 100 ≥ 75 ≥ 50 ≥ 25 ≥ 0
+/- 5BPM Accuracy [%] 94.48 92.49 91.24 90.21 89.77
MAE [BPM] 1.67 1.86 1.98 2.18 2.26
Reporting Coverage [%] 64.04 84.07 93.34 97.99 100.00
Running
Data Points with Measurement Quality Metric ≥ 100 ≥ 75 ≥ 50 ≥ 25 ≥ 0
+/- 5BPM Accuracy [%] 96.39 95.03 92.11 88.61 86.57
MAE [BPM] 1.36 1.67 2.07 2.63 3.04
Reporting Coverage [%] 55.12 72.79 82.46 91.15 100.00
Biking
Data Points with Measurement Quality Metric ≥ 100 ≥ 75 ≥ 50 ≥ 25 ≥ 0
+/- 5BPM Accuracy [%] 95.81 94.22 92.27 90.41 89.71
MAE [BPM] 1.55 1.78 1.94 2.11 2.17
Reporting Coverage [%] 60.20 80.17 89.97 96.83 100.00
Overall
Data Points with Measurement Quality Metric ≥ 100 ≥ 75 ≥ 50 ≥ 25 ≥ 0
+/- 5BPM Accuracy [%] 96.62 95.28 94.02 92.78 92.07
MAE [BPM] 1.34 1.53 1.68 1.88 1.99
Reporting Coverage [%] 67.13 84.45 92.16 96.94 100.00
Daily
Data Points with Measurement Quality Metric ≥ 100 ≥ 75 ≥ 50 ≥ 25 ≥ 0
+/- 5BPM Accuracy [%] 97.40 95.20 93.60 92.90 91.10
MAE [BPM] 1.34 1.41 1.47 1.61 1.82
Reporting Coverage [%] 53.00 75.00 88.00 96.00 100.00

2.6 WHRM Algorithm Signal Requirements

2.6.1 Signal Requirements

  • Proper sensor-to-skin coupling shall be satisfied. The watch shall have a snug and comfortable fit.
  • Accelerometer and PPG signal sampling rate shall be 25Hz, with sampling interval between 38 to 43ms.
  • Accelerometer to PPG synchronization shall be within 25ms, in reference to the sample time stamps.
  • There shall not be accelerometer and PPG data point drops, for no more than one sample per minute.

2.6.2 Reference Device

  • Heart rate – ECG chest strap with minimum of 250Hz sampling rate shall be used, such as Polar H10

2.6.3 Signal Quality

Maxim’s algorithm expects signal quality with pSNR* > 15dB at rest and 6dB during exercise. The signals are to be measured on each green PPG channel to achieve these accuracy numbers, and HR frequency is the dominant frequency in the PPG spectrogram.

*For pSNR calculation (also known as AC signal SNR), signal power is defined to be the power of the signal at heartbeat frequency, in addition to its second and third harmonics. The noise power is defined to be the power of all other signal components, including motion artifacts.

2.7 Activity Classification and Step-Counting Validation

The activity classification and step-counting performance are validated by a data set consisting of 72 data sequences collected from 36 subjects. The indoor performance is validated by a data set consisting of 40 data sequences collected from 20 subjects. The indoor protocol is shown on Table 6.

Table 6. Activity Classification and Step-Counting Indoor Performance Validation Protocol

Activity Duration Location Comments
Resting 2 min In Gym  
Walking 500 steps In Gym 2.5mph
Transition 3 min In Gym  
Running 500 steps In Gym Comfortable running speed
Transition 1 min In Gym  
Office Walking ~100 steps In Gym  
Transition 2 min In Gym  
Hiking ~100 steps In Gym  

The outdoor performance is validated by a data set consisting of 31 data sequences collected from 16 subjects. The outdoor protocol is shown on Table 7.

Table 7. Activity Classification and Step-Counting Outdoor Performance Validation Protocol

Activity Duration Location Comments
Resting 1 min Outdoor -
Walking 500 steps Outdoor -
Transition 1 min Outdoor -
Strolling with Baby 500 steps Outdoor -
Transition 1 min Outdoor -
Cycling 3 min Outdoor ;-

This data was collected with the Maxim reference design watch and manual clickers were used as reference measurement. The activity classification output of the algorithm was compared with the protocol (the activity transition time stamps were recorded during data collection) and step-counting results were compared with manual clickers. Indoor and outdoor performance results can be found in Table 8.

Table 8. Activity Classification and Step-Counting Performance Results

Activity Classification Step Count
Activity Accuracy Recall Precision Absolute Percent Error
Resting 0.99 0.93 1.00 -
Walking - Indoor 0.96 0.97 0.91 0.90
Running - Indoor 0.98 0.93 0.96 4.18
Walking - Outdoor 0.95 0.95 0.99 6.45
Biking - Outdoor 0.98 0.85 0.95 -

3 Summary

The results of our study show that heart-rate monitoring with the Maxim WHRM solution is highly correlated with heart-rate monitoring performed by the ECG chest strap (reference device). The WHRM algorithm complies with the ANSI/CTA standards for physical heart-rate measurement and perform even better than these standards1. The algorithm uses accelerometer information to suppress motion artifacts from PPG in periodic motion activities (walking, running, cycling, etc.), motion compensation, and multi-channel fusion to enhance overall performance, especially in daily life activities (non-periodic motion). The WHRM solution also controls the AFE settings to optimize signal quality (in the presence of electrical noise) and power consumption, which is highly critical for continuous heart-rate monitoring from wearable devices. The reliability of the algorithm in monitoring heart rate can be beneficial for other new use cases, such as sleep, stress, VO2-MAX, EPOCH monitoring, etc. The algorithm is also quite successful in activity classification (resting, walking, running, biking) and step counting.

References

  1. Consumer Technology Association (2018). Physical Activity Monitoring for Heart Rate. In ANSI/CTA Standard (Vol. CTA-2065, pp. 1-11). Arlington VA: Consumer Technology Association: Technology & Standards Department.