Skip to main content

Checking the validity and reliability of the Japanese version of the Mini-Cog using a smartphone application

Abstract

Objective

Cognitive decline is an important and well-documented health problem. The Mini-Cog, a simple cognitive function test, is recommended as a potential early cognitive screening tool. Kanagawa Prefecture, one of the largest prefectures in Japan, developed this self-testing application on a smartphone to enable a large number of residents to assess their cognitive function. This study aimed to verify the validity and reliability of the Mini-Cog.

Results

Twenty men and 20 women aged 65–85 years were enrolled. Criterion-related validity of the method tested by professional staff was found to have an acceptable positive association. The test–retest reliability was lower than the clinically expected intraclass correlation coefficient value because of the inclusion of learning and order effects. If the Mini-Cog score of this application is low, the system is equipped with a function that advises the users on preventing cognitive decline, directing them to the appropriate services, and recommending visits to a medical institution. Therefore, the system can be used continuously as a tool for health behaviors and promotions.

Introduction

Cognitive decline is an important and well-documented health problem. Identifying older adults with dementia is important for planning their care needs and treatments. Screening tests to identify dementia, including the Mini-Mental State Examination (MMSE) [1], Clinical Dementia Rating [2], Montreal Cognitive Assessment [3], and Mini-Cog [4, 5], are widely used. Among them, the Mini-Cog is the simplest screening test that consists of a three-word recall task and a clock drawing test. Therefore, it is useful when clinicians have limited time and resources to administer the test [6].

The Mini-Cog was originally developed in a study of ethnolinguistically diverse community-aged Americans to screen for dementia in primary care settings. This original version of the study could be administered in an average of 3.2 min and showed high sensitivity (99%) and specificity (96%) [4]. In a population-based post hoc examination with participants from a random sample of 1119 older adults, the Mini-Cog and the MMSE had similar sensitivity (76% vs. 79%) and specificity (89% vs. 88%) [5].

Recently, the Mini-Cog has been translated into various languages and is recommended as a potential early cognitive screening tool for memory impairment screening and general practitioner evaluation [7,8,9].

In March 2020, Kanagawa Prefecture, one of the biggest local governments in Japan developed the ME-BYO index, to measure the ME-BYO in the health management application, My ME-BYO Record [10]. ME-BYO is a concept that does not consider health and sickness as two separable conditions; it covers the entire process of continuous change in the physical and mental conditions between health and sickness [10]. The ME-BYO index visualizes an individuals’ current state of ME-BYO and future disease risk in numerical form and comprehensively quantifies the domains of lifestyle, physical function, cognitive function, and mental health and stress. The ME-BYO index was developed based on evidence and discussion among experts. The specific evaluation items included sex, age, body mass index, systolic blood pressure (lifestyle), the Mini-Cog (cognitive function) [4, 5], locomotive function [11], walking speed (physical function), and mind-monitoring system (mental health and stress) [12].

For the evaluation of cognitive function using the ME-BYO index, we adopted the Mini-Cog, which is a simple and recommended screening test. Therefore, we developed a smartphone application to enable residents to perform the test. The widespread use of an application that allows residents to measure their cognitive function easily in their daily lives could speed up the diagnostic process, improve accessibility, and lead to early prevention measures. Moreover, it could help improve the health of the individuals, their families, and the community. Although various applications for assessing cognitive function have been developed [13, 14], to our knowledge, no free application of the Mini-Cog has been validated and reliably self-administered by the residents. Therefore, this study aimed to verify the validity and reliability of the Mini-Cog measurement method used by the general public through an application on their smartphones.

Main text

Methods

Study participants

The participants were healthy older adults aged 65–85 years who attended a health promotion facility. The target number of participants was 40 (20 men and 20 women). In the sample size design, the required number of participants was 29, assuming a significance level, power, and effect size of 5%, 80%, and 0.5, respectively. The final number of research subjects was set at 40, assuming dropouts and multiple applicants signing up simultaneously, as well as the burden on the study participants.

The Research Ethics Review Committee of the Graduate School of Health Innovation, Kanagawa Prefectural University of Health and Welfare approved this study (Approval No. Hodai No. 30-011). The purpose and content of the study were explained to the study participants in advance, and written consent was obtained before the study was conducted.

Measurement items

Mini-Cog assessment by the examiner and smartphone application

The Mini-Cog assessment consisted of a three-word recall task and clock drawing test. In the word recall task, a score of 0–3 points was assigned for each correct recall of 0, 1, 2, or 3 words. The word recall was allowed in any order. In the clock-drawing test, the participants were scored as “normal” or “abnormal” based on the placement of the clock hands. The Mini-Cog test was scored positive (dementia) if the delayed recall was 0 out of 3, or if the delayed recall was 1 or 2, and the clock drawing test was abnormal. It scored negative (not dementia) if the delayed recall is 3 points or if the delayed recall is 1 or 2 points and the clock drawing is normal [4]. Mini-Cog has developed currently 21 language versions, including Japanese, which is available for download from the Mini-Cog website [15].

The basic procedure and scoring criteria for the Mini-Cog using the smartphone application were similar to those used by the examiner. In the initial three-word recall task, one of the six versions was played randomly over the audio, and the participants were instructed to memorize the three words. Next, the participants were instructed to drag and place numbers 1–12 in a circle representing a clock and move the hands of the clock to 11:10. The last three-word recall was performed using voice or text input (see Additional file 1: Fig. S1).

Mini-Cog evaluation using the application

The research participants assessed their Mini-Cog judgment using the ME-BYO index application on a research smartphone. Mini-Cog assessment was performed once, following voice instructions, and evaluated using a score (0–5 points) and judgments of normal and abnormal. Mini-Cog self-assessments by the study participants using the application were conducted at the beginning of the study and after 8 weeks to verify the test–retest reliability.

Mini-Cog evaluation by examiners

A public health nurse was trained in the Mini-Cog conducted assessment. To verify criterion-related validity, a similarly trained public health nurse conducted a face-to-face assessment 4 weeks after the self-assessment by the study participants, referring to a previous study [16]. The 3-word recall task was conducted using a different version of the first application of the ME-BYO index.

Statistical analysis

Mini-Cog scores and judgments (normal and abnormal) were used to assess criterion-related validity and test–retest reliability. For criterion-related validity, Spearman's rank correlation coefficient was used to evaluate the results of the self-administered application by the study participants at the beginning of the study and the results administered by the public health nurses after 4 weeks. The kappa statistic was used to evaluate judgment (normal or abnormal).

Test–retest reliability was assessed using intraclass correlation coefficients (ICCs) (1,1) and 95% confidence intervals for the results of the self-administered application by the study participants and the retesting after 8 weeks. Statistical analyses were performed using SPSS version 27 (IBM Corp., Armonk, NY, USA) software. The significance level was set at 5%.

Results

The characteristics of the participants during their participation in the study and the Mini-Cog values measured using the application are shown in Table 1. This study included 40 participants (20 men and 20 women). The mean age (standard deviation) was 74.9 (± 5.2) years. The mean Mini-Cog score (standard deviation) was 4.2 (± 1.0) points. Thirty-eight participants (95.7%) had a normal Mini-Cog judgment (3–5 points) and two (5.0%) had an abnormal judgment (0–2 points).

Table 1 Characteristics of the study participants

The results of the validation and reliability of the Mini-Cog are shown in Table 2. The Spearman's rank correlation coefficient was ρ = 0.442 based on the criterion-related validation of the ME-BYO index application scores at the beginning of the study and after 4 weeks. This indicates a positive correlation and a 100% agreement rate with the Mini-Cog judgment.

Table 2 Validity and reliability of the Mini-Cog by study participants and medical staff

The results of the test–retest reliability of the ME-BYO index application scores at the beginning of the study and the reassessment after 8 weeks showed that the ICC (95% confidence interval) was 0.381 (0.085–0.616) and the Mini-Cog judgment was 0.362 (− 0.201–0.924), indicating poor reliability.

Discussion

In this study, we developed a Mini-Cog application that can be self-administered by residents using a smartphone and tested its criterion-related validity and test–retest reliability.

Criterion-related validity was evaluated using the self-administered Mini-Cog results at the beginning of the study and those administered by medical professionals after 4 weeks. A positive correlation (ρ = 0.44) was observed between the scores, and the agreement rate of the Mini-Cog judgment was 100%. The evaluation of this application depends on the proficiency level of the smartphone, especially in the clock drawing test. The correlation coefficient of a previous study that validated the Thai version of the Mini-Cog face-to-face survey with the MMSE was 0.47 [17], which is comparable to the results of this study. Therefore, the method of measuring the Mini-Cog using a smartphone application was acceptable.

When the individuals repeated the Mini-Cog using the ME-BYO index application after 8 weeks, the test–retest reliability was lower compared to the clinically expected ICC value (≥ 0.6). In this study, the same test was repeated three times over a certain period, similarly to previous studies. Therefore, the inclusion of learning and order effects may have affected the results [mean (standard deviation)] of the Mini-Cog scores at the time of participation in the study: in the beginning 4.2 (± 1.0); 4 weeks later: 4.8 (± 0.5); 8 weeks later: 4.6 (± 1.0) [18].

One issue with the original version of the Mini-Cog is that it overestimates the positive predictive value, especially for mild cognitive impairment [6]. However, it can be administered in a short period and is recommended as a convenient screening test in areas where time and resources are limited [7,8,9]. If the score of this application is low, the system is equipped with a function that provides advice on preventing cognitive decline, directing users to appropriate services, and recommending visits to a medical institution. Therefore, the system can be used continuously as a tool for health behaviors and promotions. It can also measure in a remote capacity during an infectious disease epidemic, such as coronavirus disease 2019. In the future, it is expected that the Mini-Cog application will be equipped with technology that will alert the user when the score starts to decrease. Moreover, only a few large-scale studies have been reported in the region using the Mini-Cog [6]. Utilizing the data from the Kanagawa Prefecture will contribute to the accumulation of evidence towards this issue.

To conclude, this application has sufficient scientific validity to test a large population. Therefore, it can be used as a tool to maintain and improve health behaviors.

Limitations

This study did not use random sampling, and the participants were older adults attending health promotion facilities. Therefore, caution should be applied while applying the results to older adults with diverse backgrounds. Another limitation is that comparisons with other mobile applications were not conducted.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Abbreviations

ICC:

Intraclass correlation coefficient

CIs:

Confidence intervals

References

  1. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–98.

    Article  CAS  Google Scholar 

  2. Morris JC. The clinical dementia rating (CDR): current version and scoring rules. Neurology. 1993;43:2412–4.

    Article  CAS  Google Scholar 

  3. Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53:695–9.

    Article  Google Scholar 

  4. Borson S, Scanlan J, Brush M, Vitaliano P, Dokmak A. The mini-cog: a cognitive ‘vital signs’ measure for dementia screening in multi-lingual elderly. Int J Geriatr Psychiatry. 2000;15:1021–7.

    Article  CAS  Google Scholar 

  5. Borson S, Scanlan JM, Chen P, Ganguli M. The Mini-Cog as a screen for dementia: validation in a population-based sample. J Am Geriatr Soc. 2003;51:1451–4.

    Article  Google Scholar 

  6. Fage BA, Chan CC, Gill SS, Noel-Storr AH, Herrmann N, Smailagic N, et al. Mini-Cog for the detection of dementia within a community setting. Cochrane Database Syst Rev. 2021;7:CD010860.

    PubMed  Google Scholar 

  7. Brodaty H, Low LF, Gibson L, Burns K. What is the best dementia screening instrument for general practitioners to use? Am J Geriatr Psychiatry. 2006;14:391–400.

    Article  Google Scholar 

  8. Cordell CB, Borson S, Boustani M, Chodosh J, Reuben D, Verghese J, et al. Medicare detection of cognitive impairment W: Alzheimer’s association recommendations for operationalizing the detection of cognitive impairment during the medicare annual wellness visit in a primary care setting. Alzheimers Dement. 2013;9:141–50.

    Article  Google Scholar 

  9. Lin JS, O’Connor E, Rossom RC, Perdue LA, Eckstrom E. Screening for cognitive impairment in older adults: a systematic review for the US Preventive Services Task Force. Ann Intern Med. 2013;159:601–12.

    PubMed  Google Scholar 

  10. World Health Organization. Decade of healthy ageing: baseline report. 2020.

  11. Seichi A, Hoshino Y, Doi T, Akai M, Tobimatsu Y, Iwaya T. Development of a screening tool for risk of locomotive syndrome in the elderly: the 25-question geriatric locomotive function scale. J Orthop Sci. 2012;17:163–72.

    Article  Google Scholar 

  12. Higuchi M, Nakamura M, Shinohara S, Omiya Y, Takano T, Mitsuyoshi S, et al. Effectiveness of a voice-based mental health evaluation system for mobile devices: prospective study. JMIR Form Res. 2020;4: e16455.

    Article  Google Scholar 

  13. Anthony Berauk VL, Murugiah MK, Soh YC, Chuan Sheng Y, Wong TW, Ming LC. Mobile health applications for caring of older people: review and comparison. Ther Innov Regul Sci. 2018;52:374–82.

    Article  Google Scholar 

  14. Thabtah F, Peebles D, Retzler J, Hathurusingha C. Dementia medical screening using mobile applications: a systematic review with a new mapping model. J Biomed Inform. 2020;111: 103573.

    Article  Google Scholar 

  15. Mini-Cog©. https://mini-cog.com/. Accessed 6 May 2022.

  16. Makizako H, Shimada H, Park H, Doi T, Yoshida D, Uemura K, et al. Evaluation of multidimensional neurocognitive function using a tablet personal computer: test–retest reliability and validity in community-dwelling older adults. Geriatr Gerontol Int. 2013;13:860–6.

    Article  Google Scholar 

  17. Trongsakul S, Lambert R, Clark A, Wongpakaran N, Cross J. Development of the Thai version of Mini-Cog, a brief cognitive screening test. Geriatr Gerontol Int. 2015;15:594–600.

    Article  Google Scholar 

  18. Lee YC, Lee SC, Chiu EC. Practice effect and test-retest reliability of the mini-mental state examination-2 in people with dementia. BMC Geriatr. 2022;22:67.

    Article  Google Scholar 

Download references

Acknowledgements

We are grateful to the residents who participated in this study. The authors would like to acknowledge Hygia Communication for their support in managing the research. We would like to thank Editage (www.editage.jp) for English language editing.

Funding

This study was supported by the ME-BYO index project of the Kanagawa Prefecture and Grants-in-Aid for the Japan Society for the Promotion of Science (JSPS) KAKENHI Grant (No. 16H06277 [CoBiA]) from the Japanese Ministry of Education, Culture, Sports, Science, and Technology.

Author information

Authors and Affiliations

Authors

Contributions

YS: conceptualization, methodology, formal analysis, investigation, data curation, writing—original draft, writing—review and editing, and project administration. SN: conceptualization, methodology, writing—review and editing. AT: investigation, and writing—review and editing. RW: conceptualization, and writing—review and editing. HN: conceptualization, methodology, writing—review and editing, supervision, project administration, and funding acquisition. UC: conceptualization, writing—review and editing, supervision, and funding acquisition. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Yoshinobu Saito.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the Research Ethics Committee of the Graduate School of Health Innovation, Kanagawa University of Human Services (Approval No. Hodai 30-011). The purpose and content of the study were explained to the participants, and written informed consent was obtained before the study was conducted.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Figure S1.

Screenshot of the Mini-Cog application.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saito, Y., Nakamura, S., Tanaka, A. et al. Checking the validity and reliability of the Japanese version of the Mini-Cog using a smartphone application. BMC Res Notes 15, 222 (2022). https://0-doi-org.brum.beds.ac.uk/10.1186/s13104-022-06101-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13104-022-06101-4

Keywords