Levine D, Co Z , Newmark L, et al. (2020). Design and testing of a mobile health application rating tool. npj Digital Medicine. 3:74. doi: 10.1038/s41746-020-0268-9
Researchers developed an open-source rubric rating tool called THESIS to measure the risks and benefits of mobile health applications (apps). THESIS improves on tools such as the Mobile App Rating Scale (MARS), which fails to assess app privacy, security, and interoperability. Informed by a panel of digital health experts, THESIS addresses 27 individual criteria in 6 domains of mobile app content: transparency, health, technical, security/privacy, usability, and subjective ratings. The THESIS acronym draws from the 6 domain titles. Three raters (a medical student, a pre-medical student, and a business graduate student) used THESIS to score 211 chronic disease-focused apps from the Apple and Google app stores. Two raters independently scored each app on each of the 27 THESIS criteria (about 12 minutes per app). The overall score for each app was the average of its 6 mean domain scores. Overall performance of the 211 apps was mediocre (3.02 out of 5, on average). Mean domain scores were lowest in privacy/security (2.21 out of 5) and highest in transparency (3.54 out of 5). Apps focused on HIV and schizophrenia received the lowest overall ratings (HIV mean: 2.43, schizophrenia mean: 2.54). Individual criteria with the lowest average scores among apps included interoperability with other apps (1.75 out of 5) and user consent (1.86 out of 5). The tool demonstrated adequate interrater reliability and excellent scale reliability. THESIS merits further testing and may be useful to policymakers and software developers who seek to improve app performance.