Wearable sensors are becoming central to how clinical researchers and drug developers measure how well a patient functions in daily life.
But a number on a screen is not a clinical endpoint. The distance between the two is validation — conducted in clinical populations, under real-world conditions that reflect how patients actually live
Validation was the connecting thread across two studies our team presented at this year’s International Conference on Ambulatory Monitoring of Physical Activity and Movement (ICAMPAM) The first examined step counting in chronic heart failure (CHF); the second examined heart rate captured by wrist-worn photoplethysmography (PPG). Different signals, different populations — but the underlying objective remains the same: prove a measure holds up where it will be used, not only where it is easy to validate.
Most performance claims for wearable-derived measures are generated under favorable conditions: young, healthy cohorts in controlled lab settings performing steady-state movement. Those claims rarely transfer cleanly to the people and the movements that define real studies — older adults, patients with cardiac disease, a range of skin tones, and challenging movement patterns. Two failure modes recur. Step counts look deceptively simple, yet they hinge on definitions and on how an algorithm behaves when a patient is moving but not walking. Optical heart rate is fragile during movement and has a documented history of uneven accuracy across skin tones.
Both studies were built to confront those failure modes directly rather than design around them.
"What counts as a step" can differ between how an algorithm is designed and how a validation reference is defined. In a population that moves slowly and intermittently, that ambiguity is not academic; it drives the result.
Rakesh Pilkar, PhD, Lead of DHT Solutions at Ametris, presented this research, conducted with Nicole Freene, PhD, from the University of Canberra and supported through Ametris’ Digital Endpoint Accelerator Research (DEAR) grant program, focused primarily on accelerometer cut-points for physical activity intensities in coronary heart disease. The CHF dataset (a subsample from the original study) comprised 20 participants (mean age 66 years), with video-based manual step counts serving as ground truth at both one-minute and five-minute granularity.
Crucially, the activity set went beyond overground walking to include activities that not only represent real activities of daily living, but also pressure-test the algorithms for step detection (watching television, stretching, marching, and sweeping). Three algorithms were compared: CSEM, a motion-frequency approach that classifies movement as rhythmic or non-rhythmic and labels walk, run, and other; and two Ametris CPIW machine-learning algorithms (versions one and two), with the second trained on a wider age range and tuned for improved step detection.
During actual walking, all three algorithms tracked ground truth closely — roughly eight steps per minute of error and about 9% mean absolute percent error at the one-minute level. The algorithms diverged on everything else. CSEM reliably separated walking from non-walking movement and stayed near zero during stationary tasks such as watching TV and stretching, while the CPIW machine-learning algorithms accumulated slightly more false positives. Sweeping and marching were the most challenging cases for every algorithm.
Two methodological points proved as important as the headline numbers: the choice of performance metric matters (mean absolute percent error suits active tasks; mean absolute error better suits stationary ones), and the underlying definition of a step for an annotator and an algorithm can itself shift the interpretation.
Wrist PPG is convenient and continuous, but its weaknesses — degradation under motion and inconsistency across skin tones — are exactly the conditions a sponsor cannot afford to leave unexamined. Both are data-quality and equity concerns at once.
Matthew Patterson, PhD, Senior Data Scientist at Ametris,presented an analytical validation of the PPG signal on the ActiGraph LEAP, with a heart rate algorithm developed in collaboration with CSEM. LEAP heart rate data, collected on the non-dominant wrist, was compared against a four-lead electrocardiogram (ECG) reference in 35 participants at the Institute for Human and Machine Cognition (IHMC), across a graded protocol that ran from simulated desk work and seated rest through comfortable walking, brisk walking, jogging, and running. Heart rate from both systems was averaged into 15-second segments and time-synchronized for comparison. The Monk Skin Tone scale was used to stratify participants, and an onboard signal-quality algorithm gated which values were reported — segments below a 30% quality threshold were rejected rather than published as low-confidence noise.
Overall agreement was strong: a mean absolute error of 2.42 beats per minute (bpm) and a correlation of 0.990 against ECG was found. A linear mixed-effects model was used to examine factors associated with heart rate (HR) error. The estimated intercept was 2.876 (95% CI: 2.295 to 3.458, p < 0.001), indicating the baseline level of HR error.
There was no evidence of differences in HR error by skin tone, with the comparison of light versus dark skin tone yielding a negligible and non-significant effect (β = 0.006, 95% CI: −0.403 to 0.414, p = 0.978). Similarly, sex was not associated with HR error, as the effect for males compared to the reference group was small and not statistically significant (β = −0.017, 95% CI: −0.379 to 0.345, p = 0.927).
Age was statistically significantly associated with HR error, with each additional year corresponding to a small decrease in error (β = −0.016 per year, 95% CI: −0.026 to −0.005, p = 0.004). However, the magnitude of this effect is minimal and not practically meaningful, indicating that HR accuracy is effectively consistent across ages in real-world terms.
Overall, these findings indicate that HR error does not vary meaningfully by skin tone or sex, and although age shows a statistically detectable association, it does not translate into a practically significant difference.
Accuracy under motion remained high (MAE = 2.91 bpm); what changed under vigorous movement was coverage, with the acceptance rate falling from 97.9% during sedentary segments to 53.9% during motion. That trade-off is a feature, not a flaw: the device withholds values it cannot stand behind. Peer-reviewed publication is the next step.
Put together, the two studies point to a shared set of principles for anyone selecting or qualifying a wearable-derived measure:
A sensor reading becomes an endpoint only when it has been carefully characterized across the right people, the right movements, and the right definitions.
These findings demonstrate the rigorous scientific work that goes into every digital measure we deliver and the standards we hold to achieve precision at every phase of a clinical study.
Reach out to the Ametris team to discuss fit-for-purpose validation for your program. Schedule a meeting for your study’s needs.