Is the ‘Lifelight’ app adequately validated for blood pressure measurement?
May 4, 2020
Professor Lionel Tarassenko1
Professor Richard McManus2
Professor Trisha Greenhalgh2
On behalf of the Oxford COVID-19 Evidence Service Team
1Institute of Biomedical Engineering, Department of Engineering Science
University of Oxford
2Nuffield Department of Primary Care Health Sciences,
University of Oxford
Correspondence to lionel.tarassenko@eng.ox.ac.uk
VERDICT
‘Lifelight First’ is a smartphone app that attempts to measure pulse, respiratory rate and blood pressure without making physical contact with the patient. In relation to the measurement of blood pressure, the app has not yet been validated to the standards needed and hard evidence of its accuracy and safety is currently lacking. We believe that a Medtech Innovation Briefing (MIB213) by the National Institute for Health and Clinical Excellence (NICE) is premature and suggest that it should be withdrawn pending further studies.
BACKGROUND
The current Covid-19 pandemic means that new technologies are being developed rapidly and fast-tracked into clinical practice. That is particularly true of technologies designed to support remote and non-contact care. In this context, an app, Lifelight First, designed to measure vital signs (pulse, respiratory rate and blood pressure) without touching the patient, is being promoted as a partial solution. NICE has already published a Medtech Innovation Briefing (MIB213)1 which links to other NICE guidance on three care pathways: management of hypertension, managing acutely ill patients with covid-19, and national early warning scores which aim to identify deteriorating patients.
The NICE MIB213 report includes a summary which states:
- “The main points from the evidence summarised in this briefing are from 2 unpublished studies, a single-arm observational study and a validation study. These studies included a mixed population of 8,712 inpatients, outpatients and healthy volunteers in both clinical and non-clinical settings. These studies show that the accuracy of Lifelight First is similar to some clinical-grade monitoring devices when compared with the Welch Allyn Connex Monitor for measuring heart rate, blood pressure and manual counting for measuring respiratory rate.
- “Key uncertainties around the evidence or technology are that the evidence base does not evaluate the technology in a clinically relevant population, including people that need routine monitoring of vital signs. The evidence is unpublished, meaning there are uncertainties about the reliability of the findings. The evidence only reports direct statistical comparison of the accuracy data with the Welsh Allyn Connex monitor. Comparisons with other similar technologies are indirect. The evidence does not report diagnostic accuracy data or clinical outcomes, such as how the technology would be used to inform care decisions.”
In this rapid review, we focus on blood pressure measurement and consider three questions. First, how does a non-contact approach to measuring blood pressure work? Second, what is a validation study and to what extent are the claims to have “validated” the Lifelight First app supported? Third, to what extent does current evidence support the use of the Lifelight First app in the management of patients with possible Covid-19?
1.How does a non-contact approach to measuring blood pressure work?
The Lifelight website gives a very short explanation of how systolic and diastolic blood pressures are estimated:
“Lifelight’s data source is derived from the signal that is picked up from the patient’s face. A signal is detected by Lifelight First from analysing tiny colour changes of the face with each pulse beat. This is turned into a stream of red, green and blue (RGB) average numbers.
The data processed is a combination of the waveform signal as derived from the mobile device’s camera plus anonymous biometric data (age, height and sex). The patient’s age, height and sex are captured to help the Lifelight First algorithm to accurately calculate systolic and diastolic blood pressure.”
There is no peer-reviewed paper or even a white paper from the company describing their methods, data analysis and results. The only evidence offered, other than the two short paragraphs above, is the block diagram shown below and a link to a paper from a research group in Slovenia,2 but the latter is concerned with analysis of waveform data from pulse oximeters attached to the patient’s finger (not remote measurement of ambient light reflected from the face), so is not directly relevant to the claims made.
Diagram of how Lifelight First app measures blood pressure, taken from Lifelight website
The above block diagram indicates that the Lifelight First “app” uses Machine Learning to estimate blood pressure. Blood pressure does vary with age, height and sex, and with heart rate (“Pulse beat count” is another input to the blood pressure machine learning algorithm in the block diagram above), but there is no physiological reason why the red, green and blue outputs from the video camera should vary consistently with a person’s blood pressure.
There is evidence in the scientific literature (some of it presented in the paper by Slapničar, Mlakar and Luštrek2) that the shape of the Photoplethysmogram (PPG) waveform recorded by a conventional transmission-mode pulse oximeter attached to a patient’s finger shows a weak dependence on blood pressure. However, the relevant waveform features are not present in all PPG recordings, and they are affected by the presence of cardiovascular disease. It is even less likely that these features can be extracted reliably, for all types of patients, in reflectance mode some distance away from the patient’s face.
The Lifelight First blood pressure estimation algorithm therefore appears to depend mostly on the age, height, sex, and heart rate of the patient, with the contributions from the remote PPG waveforms extracted from the Red (R), Blue (B) and Green (G) channels of the video camera likely to be “noise”. Even if there was some blood pressure information in the PPG waveforms recorded in each of the three channels, the relative contributions of the R, G and B channels are likely to be strongly affected by the levels of ambient light and the patient’s skin tone.
We therefore conclude that there is no scientific basis for the claims by Lifelight that blood pressure can be estimated by processing the ambient light reflected from a patient’s face using their algorithms.
This conclusion is supported by the following individual data, collected by one of us (TG), a 61-year-old female with physiologically low blood pressure for her age. TG took a series of readings of her own blood pressure, some of which were at rest and some after vigorous exercise. Simultaneous blood pressure readings were taken from the Lifelight First app and an Omron M4 electronic home blood pressure monitor using an inflatable arm cuff. The results are shown in the table below (note: the app did not give the second digit in the diastolic reading):
2.What is a validation study?In this small intra-individual study (which is not being presented as a formal validation study, but which calls for explanation), systolic blood pressure as measured by the Lifelight First app appeared almost constant, whereas that measured by a standard validated home monitor varied by 30 mm Hg as would be expected physiologically. Furthermore, the lowest readings on the Lifelight First app occurred after vigorous exercise and along with the highest readings from the Omron monitor. It would appear that the main data influencing the reading from the Lifelight First app comprise the subject’s age, height and sex (which were of course constant across all the readings taken). Importantly, the app did not distinguish between a blood pressure within the normal range and one that could be dangerously low in people of this age.
Conventional blood pressure monitor validation protocols have been used since the 1990s.3 Such protocols are important because they ensure that blood pressure monitoring equipment is accurate throughout the range of clinical values of both systolic and diastolic pressure and that it has been tested in a relevant population. The current internationally agreed standard requires a sample of at least 85 individuals providing at least 255 paired comparisons between the reference auscultatory standard and the test device.4 5 Choice of subjects must be without bias including both males and females. Larger samples can give spurious precision to estimates. Results are expressed both in terms of mean differences and proportions with clinically meaningful differences, notably 5, 10 and 15mmHg. Where particular patient groups are concerned, including children and pregnant women, additional samples are needed. These protocols are designed for use in assessing traditional sphygmomanometers (typically auscultatory or oscillometric) rather than apps such as this but the basic premise of comparing similtaneous or serial measurement with multiple comparisons to a recognised reference standard holds. This does not appear to have been done here.
Blood pressure measurement can vary with characteritics related to measurement such as size of arm (and hence cuff size) for traditional measurement but age and sex are only relevant in choosing a sample or if one is trying to estimate the blood pressure on the basis of population norms as opposed to measuring an absolute clinical parameter. Given the importance of blood pressure as a clinical sign driving acute decision making, it seems surprising that any measurement method would take into account parameters not directly relevant to the blood pressure at the precise time an individual was undergoing measurement. Including population norms as part of an algorithm seems likely to lead to spurious results and may explain the findings from the n=1 study presented above.
The validation study on the Lifelight First app has not been published. However, on the company website, the following information is given:
Lifelight is the result of one of the largest digital physiological studies of its kind – an 8,500 patient clinical study at Portsmouth Hospitals Trust, recording over 1 million heartbeats.
It would appear that whilst the sample size in this study was large, only one reading was taken per patient (or healthy volunteer), and no attempt was made to assess the app’s ability to detect intra-individual variation. The use of age, sex and height data to predict blood pressure will increase the accuracy of the app on average – but this will also surely make the app inherently insensitive to deviations from the population average for any particular age-sex cohort which would be especially important in acute illness. It would be interesting to compare this validation study with a study in which the age-sex-height data were used to predict blood pressure, without the PPG reading.
3.What is the relevance of our findings for BP monitoring in the context of covid-19?
Because Covid-19 is highly contagious, there are three main situations in which a non-contact method of taking blood pressure may be useful: self-monitoring by the patient in the context of an acute illness that could be Covid-19; self-monitoring of chronic disease or during pregnancy due to social isolation; and non-touch blood pressure measurement in the clinic.
In the patient with acute illness being assessed remotely, the measurement of blood pressure is aimed at detecting or excluding signs of shock. The ability accurately to detect an abnormally low blood pressure would be crucial. Blood pressure, for example, is a key element of the NEWS2 early warning score used to pick up deterioration in vital signs.6
In chronic blood pressure monitoring, the purpose of monitoring is to a) detect elevations of blood pressure indicating inadequate treatment; b) detect falls in blood pressure indicating overtreatment; and c) provide the patient with feedback and reassurance that their blood pressure is in the target range. In pregnancy, an elevation of blood pressure may indicate pre-eclampsia and needs urgent assessment. If the main factor influencing the blood pressure reading is the patient’s own age and sex, it will not be possible to achieve any of these goals. Poor control of hypertension is likely to result, and pre-eclampsia could be missed.
In the in-person examination situation, the imagined scenario is that the patient (suspected of Covid-19) and the clinician are in the same room. Instead of physically touching the patient (and contaminating a cuff), the clinician would use the app on his or her mobile phone, getting the patient to look into it from a short distance away. The plausiblity of this scenario in relation to other necessary physical measurements does not appear to have been thought through in detail. We have already shown that smartphone apps for measuring oximetry are not accurate.7
The company website states the following:
“Our ground-breaking solution, Lifelight First, available to clinicians today, has the potential to deliver benefits across a wide range of use cases. Our starting point is in primary and secondary care, where pressure on workload and staff shortages are critical. Lifelight reduces workload pressure by:
Improving throughput of consultations. Blood Pressure measurements are taken in 31% of England’s 372 million annual GP appointments.
Reducing the time taken for these could free up more than 4 million hours of GP and Practice Nurse time.
Reducing care home and home visits. The average surgery makes 27 home and 13 care home visits weekly, which consumes considerable clinical time, whereas Lifelight can be provided to frequent service users to make better informed remote consultations.
Easier capture of NEWS2 across all care settings leading to improved detection of sepsis and other forms of health deterioration.
As well as staff productivity, the contactless nature of Lifelight has been reported to improve patient experience, and reduce anxiety as no cuff is needed. This may also help to reduce ‘white coat’ syndrome which affects over 15%-30% of the population and leads to £10.5 million of unnecessary costs pa.”
The references given do not appear to support the claims made. No data are provided on the amount of time taken to use the Lifelight First app, but our own experience is that it is no quicker than using an electronic cuff. No data have been presented on the use of this app in care homes or in patients’ homes, but our own experience is that the app requires the patient to remain very still and look directly at a smartphone for at least 30 seconds.
The claim that the Lifelight First app can be used specifically for assessment of the deteriorating patient is concerning. The NEWS2 score was developed as an in-hospital test for looking at a change from the patient’s own baseline using a ‘track-and-trigger’ principle.6 No evidence has been presented that the Lifelight First app is sensitive to this kind of change in a patient’s serial blood pressure readings. The fact that the algorithm becomes less sensitive the more the actual blood pressure deviates from the population average suggests that it could provide false reassurance in life-threatening situations.
CONCLUSION
Despite a statement on the NICE website that the Lifelight Plus app has been subject to a “validation study” for the measurement of blood pressure, the data presented are not comparisons to a relevant reference standard using a recognised validation methodology and there are basic scientific reasons why it is unlikely to provide an accurate estimate of blood pressure. Inclusion of the patient’s age, height and sex in the algorithm for estimating blood pressure means that much of the data influencing the reading is the average blood pressure for others of the same age, height and sex. This is likely to make the instrument particularly inaccurate at extremes of blood pressure which are relevant in the safe management of patients, and also relatively unresponsive to change. We could find no evidence that blood pressure monitoring using the Lifelight First app will contribute to clinical decision-making either in the acute situation or in ongoing monitoring of long-term conditions or pregnancy. We are concerned that the NICE MIB213 report, even though it expresses some caution, will be interpreted as an endorsement of the product.
Disclaimer: the article has not been peer-reviewed; it should not replace individual clinical judgement and the sources cited should be checked. The views expressed in this commentary represent the views of the authors and not necessarily those of the host institution, the NHS, the NIHR, or the Department of Health and Social Care. The views are not a substitute for professional medical advice.
AUTHORS
Professor Tarassenko is Professor of Electrical Engineering at the University of Oxford.
Professor McManus and Professor Greenhalgh are Professors of Primary Care Health Sciences at the University of Oxford.
SEARCH
This review did not include a database search. Rather, the expertise of a Professor of Electrical Engineering (LT) was applied to consider the technical claims made about the app, and the expertise of a blood pressure researcher (RMcM) was applied to consider the validation claims.
REFERENCES
- National Institute for Health and Clincial Excellence. Lifelight First for monitoring vital signs. London: NICE 2020 (8th Apil). Accessed 28th April 2020 at https://www.nice.org.uk/advice/mib213/chapter/The-technology.
- Slapničar G, Mlakar N, Luštrek M. Blood Pressure Estimation from Photoplethysmogram Using a Spectro-Temporal Deep Neural Network. Sensors 2019;19(15):3420.
- O’Brien E, Petrie J, Littler W, et al. The British Hypertension Society protocol for the evaluation of automated and semi-automated blood pressure measuring devices with special reference to ambulatory systems. J Hypertens 1990;8(7):607-19. doi: 10.1097/00004872-199007000-00004 [published Online First: 1990/07/01]
- Stergiou GS, Alpert B, Mieke S, et al. A Universal Standard for the Validation of Blood Pressure Measuring Devices: Association for the Advancement of Medical Instrumentation/European Society of Hypertension/International Organization for Standardization (AAMI/ESH/ISO) Collaboration Statement. Hypertension 2018;71(3):368-74. doi: 10.1161/HYPERTENSIONAHA.117.10237 [published Online First: 2018/02/02]
- Stergiou GS, Palatini P, Asmar R, et al. Recommendations and Practical Guidance for performing and reporting validation studies according to the Universal Standard for the validation of blood pressure measuring devices by the Association for the Advancement of Medical Instrumentation/European Society of Hypertension/International Organization for Standardization (AAMI/ESH/ISO). J Hypertens 2019;37(3):459-66. doi: 10.1097/HJH.0000000000002039 [published Online First: 2019/02/01]
- Royal College of Physicians (UK). National Early Warning Score (NEWS) 2. London: RCP 2017. Accessed 5th April 2020 at https://www.rcplondon.ac.uk/projects/outputs/national-early-warning-score-news-2.
- Tarassenko L, Greenhalgh T. Should smartphone apps be used as oximeters? Oxford COVID-19 Evidence rapid review service: University of Oxford 2020. Accessed 30th April 2020 at https://www.cebm.net/covid-19/question-should-smartphone-apps-be-used-as-oximeters-answer-no/
Conflict of Interest
Professor Tarassenko CBE FREng FMedSci is a non-executive Director of Oxehealth Ltd (https://www.oxehealth.com), who have a CE-marked product which measures heart rate and breathing rate using a video camera positioned several metres away from the patient. He has carried out this review in his capacity as Professor of Electrical Engineering at the University of Oxford, where he has been an academic for the past 32 years.
