Skip to main content

Personal identification with artificial intelligence under COVID-19 crisis: a scoping review

Abstract

Background

Artificial intelligence is useful for building objective and rapid personal identification systems. It is important to research and develop personal identification methods as social and institutional infrastructure. A critical consideration during the coronavirus disease 2019 pandemic is that there is no contact between the subjects and personal identification systems. The aim of this study was to organize the recent 5-year development of contactless personal identification methods that use artificial intelligence.

Methods

This study used a scoping review approach to map the progression of contactless personal identification systems using artificial intelligence over the past 5 years. An electronic systematic literature search was conducted using the PubMed, Web of Science, Cochrane Library, CINAHL, and IEEE Xplore databases. Studies published between January 2016 and December 2020 were included in the study.

Results

By performing an electronic literature search, 83 articles were extracted. Based on the PRISMA flow diagram, 8 eligible articles were included in this study. These eligible articles were divided based on the analysis targets as follows: (1) face and/or body, (2) eye, and (3) forearm and/or hand. Artificial intelligence, including convolutional neural networks, contributed to the progress of research on contactless personal identification methods.

Conclusions

This study clarified that contactless personal identification methods using artificial intelligence have progressed and that they have used information obtained from the face and/or body, eyes, and forearm and/or hand.

Peer Review reports

Background

Personal identification (PI) indicates techniques and social systems of identification of unidentified persons and identity resolution regardless of whether the persons are alive or dead. PI has been primarily useful in the areas of forensic science and criminal justice [1]. Additionally, PI has become more important in relation to the aging of developed countries and large-scale disasters related to global warming in recent years [1,2,3,4]. During the coronavirus disease 2019 (COVID-19) pandemic in 2020, ties between local communities and households have been tested, and households have separated from the safety nets provided by social networks [5]. The importance of PI methods and social systems has increased during this crisis. For timely identification of unidentified persons and bodies, the cooperation of community residents and related experts, such as police officers, rescue teams, medical doctors, and forensic scientists, is critical [1,2,3,4]. Further, there is an urgent need to develop objective methods based on digital technologies used for PI and to build a database of the use of these methods as social and institutional infrastructure [6].

Artificial intelligence (AI) is a complex information processing system that has its roots in some aspects of biological information processing [7]. In recent years, AI has made remarkable progress and has been applied in various fields, including the medical field [8]. In the future, not only will it be possible to detect all organic lesions but AI technology will also be helpful in diagnosing psychiatric diseases [9, 10]. In addition, its usefulness in the forensic field has also been reported [11]. AI enables objective PI by analyzing various aspects of personal information of the human body and may contribute to building a personal information database as a component of social infrastructure [12].

COVID-19 is transmitted by respiratory droplets and contact routes [13]. Therefore, during the COVID-19 pandemic, it is an important consideration that there is no contact between the subjects of the PI and the PI systems. In 2014, Sun et al. proposed a remote infection screening system based on multiple vital signs and multivariable logistic regression model and reported that the sensitivity and specificity of this screening method was approximately 80% [14]. Since then, what progress has been made in contactless bioanalytical technologies that could be applied to PI based on AI and could be useful even in the face of widespread infectious diseases such as COVID-19? To the best of our knowledge, the answer to this question has not been investigated in a progress review in the past 5 years.

The aim of this scoping review was to organize the progression of contactless PI methods using AI over the past 5 years and discuss methods for their use during the COVID-19 crisis.

Methods

The authors conducted a scoping review, which is used to clarify working definitions and conceptual boundaries of a topic or field, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews (PRISMA-ScR) and adopting The Joanna Briggs Institute methodology framework [15,16,17,18,19] (Additional file 1). The authors have registered the protocol in the Open Science Framework (https://osf.io/wns3e/).

Eligibility, inclusion, and exclusion criteria

Eligibility criteria were established using the Participant-Concept-Context (PCC) model outlined by The Joanna Briggs Institute guidelines as follows [15,16,17,18,19]: P: unidentified persons and bodies; C: the development status of AI technology applied for PI under infectious disease pandemic; that is, only the technologies that did not require contact between the unidentified persons and bodies and the analytical instrument were extracted; and C: contactless PI methods using AI technology applicable under infectious disease pandemics.

Contactless vital sign monitoring system was reported in 2014 [14]. The recent development of digital technology has been remarkable, and the COVID-19 pandemic has reaffirmed the importance of contactless PI methods. Therefore, only studies published between January 2016 and December 2020 were included in this review to identify recent advances in the field of AI.

The inclusion criteria were as follows: (1) studies on contactless PI methods using AI.

The exclusion criteria were as follows: (1) case reports, case series, reviews, conference papers or proceedings, and letters to the editor; (2) animal experiment trials; (3) unavailability of full text; and (4) articles in languages other than English.

Information sources and search strategy

An electronic systematic literature search was conducted using the PubMed, Web of Science, Cochrane Library, CINAHL, and IEEE Xplore databases. The literature search strategy is presented in Table 1. The electronic searches were performed on December 31, 2020.

Table 1 Electronic literature search strategy

Study selection

S.M. and H.Y. independently performed the literature evaluation and selection. Based on the PRISMA flow diagram for the scoping review process, the study selection process consisted of the removal of duplicates, screening of titles and abstracts, and reviewing of full texts [15,16,17,18,19]. Disagreements between reviewers were resolved through discussion and consensus.

Critical appraisal

The reviewers performed critical quality appraisal of included articles using the mixed method appraisal tool version 2018 [20].

Data charting process and synthesis of results

According to the guidance for scoping review of Joanna Briggs Institute [15,16,17,18,19], both reviewers (S.M. and H.Y.) independently extracted the following data: author(s), year of publication, PI methods using AI, and the biometric information targeted by those methods. After the data extraction, both reviewers (S.M. and H.Y.) independently categorized and synthesized the data. At each step, disagreements between reviewers were resolved through discussion and consensus.

Results

Study selection

Based on the PRISMA flow diagram, the process of search and selection of studies was performed (Fig. 1). An electronic literature search was performed using the PubMed, Web of Science, Cochrane Library, CINAHL, and IEEE Xplore databases, and 83 articles were extracted. There were no duplicate articles. Based on the PCC eligibility criteria described above, 72 articles were excluded on the basis of their title and abstract. Eleven full articles [21,22,23,24,25,26,27,28,29,30,31] were assessed for eligibility, and the critical appraisal step was performed with each. Finally, 8 eligible articles [21, 22, 24, 26,27,28, 30, 31] were included in this study. These eligible articles were divided based on the analysis target as follows: (1) face and/or body [21, 22, 26, 27], (2) eye [24, 28], and (3) forearm and/or hand [30, 31]. The details of these studies are described below.

Fig. 1.
figure 1

Flow diagram based on the PRISMA flowchart template of the search and selection process

Face and/or body

The facial region is the most individually recognizable region and has special characteristics and landmarks. The progress of cameras that can capture facial and bodily features accurately has contributed to the development of PI methods [21, 22, 26, 27]. Nguyen and Park reported a gender recognition method using a support vector machine and a local binary pattern [21]. Much conventional supervised machine learning requires a “gold standard” dataset annotated by human experts [22]. Therefore, Huang et al. reported a user-dependent approach for identifying affective states from spontaneous facial expressions based on multiple-instance learning without the need for expert annotation [22]. In 2017, Ranjan et al. reported an algorithm based on a deep convolutional neural network (CNN) for face detection, landmark localization, pose estimation, and gender recognition [26]. Furthermore, Sang et al. reported a person identification method based on a multi-information flow CNN model and attribute recognition for improving the accuracy of the technology [27]. CNN-based methods have shown high accuracy and have become an indispensable technology in this field.

Eye

The eye and facial area containing the eye have been used in PI [24, 28]. PI based on iris recognition has been applied in border security control, banking, law enforcement, public welfare distribution, and accounting because each individual has a unique iris pattern [28]. Improvements in accuracy are constantly required since these data are treated as important information for PI. Wang and Kumar proposed an iris recognition method based on a deep CNN with dilated and residual learning [28]. In 2018, Chen et al. reported an end-to-end network based on conditional generative adversarial networks designed to generate face information based only on the eye region and suggested that the method could provide potential for PI even in cases of face occlusion [24].

Forearm and/or hand

Methods of PI using hand information, including finger/palmprints and blood vessels, are widely accepted in law enforcement and security [30]. However, most of these methods require contact between the body parts and the devices to obtain the information for PI. This may not be applicable in the face of widespread infectious diseases, including COVID-19. Liu and Kumar reported an identification method designed using a contactless palm detector and trained CNN that may have the potential for online PI [30]. Nabulsi et al. reported a noncontact forearm measurement method using microwaves [31]. They suggested that the noncontact microwave biometric information scan technique was useful because of its robustness in the presence of environmental lighting and its unobtrusiveness [31].

Discussion

This scoping review clarified the recent 5-year progression of contactless PI systems using AI. The analysis targets were divided into three body regions: face and/or body, eye, and forearm and/or hand. These results show that there is no significant change in analysis targets from conventional PI methods. Moreover, this review clarified that PI methods using AI, including CNN architectures, are enabling progress in this area. Based on the collected evidence, scientists continue to recognize the importance of contactless PI during the COVID-19 pandemic and will continue to do so after the pandemic is over. Thus, the progress and commercialization of contactless PI methods will be accelerated. Furthermore, these technologies might build social and institutional infrastructure.

Advances in this digital area constructed with body and facial image acquisition devices and AI have the potential to replace fingerprint-based methods, which have long been used for PI. However, most PI systems based on AI related to information from faces and/or body parts must be trained with images of humans, and selection of images requires involvement of human trainers [29]. Therefore, at this time, it is difficult to completely eliminate planners’ and trainers’ sociocultural biases in facial recognition systems with AI [29]. Moreover, the problem of overfitting is inevitable with the small-scale databases used in these systems [25]. Therefore, the scale of the situation and conditions in which the technology will be applied should be carefully considered.

Recently, research and development of wireless and contactless methods of PI have been actively pursued [21,22,23,24,25,26,27,28,29,30,31]. However, as a limitation of contactless PI methods using cameras to capture body part information, Wu et al. mentioned their sensitivity to illumination changes and visual occlusion [32]. Additionally, accuracy and security need to be considered in signal processing of biometric personal information and in information management, including the steps required to send and receive the information. Therefore, researchers and politicians involved in the development of contactless PI systems for social infrastructure must work to improve the accuracy of PI methods and determine how to handle biometric information, register it in databases, and use it. It is also necessary to reach a consensus on how to obtain informed consent from individuals (from parents or caregivers in case of minors) to be registered in the database and the use of their information.

The World Health Organization (WHO) described COVID-19 as the disease caused by a new coronavirus called SARS-CoV-2. WHO first learned of this new virus on December 31, 2019 [13, 33]. Therefore, man is not fully equipped with COVID-19 countermeasures, though responses are being updated daily. The global spread of infectious diseases could hinder the achievement of the 17 Sustainable Development Goals announced by the United Nations in 2015 in the targets of health, poverty, communities, and others [34]. Medical experts need to make efforts to collect appropriate information on infectious diseases, including COVID-19, and scientists need to make efforts to develop PI methods based on such information. The contactless system for the collection of PI information will be able to play a central role in this process and in the achievement of the Sustainable Development Goals.

Conclusions

This scoping review clarified the recent 5-year progression of contactless PI systems using AI, which have used information from the face and/or body, eyes, and forearm and/or hand. During and after the COVID-19 pandemic, scientists should recognize the importance of contactless PI and strive for the advancement of these technologies to build social and institutional infrastructure.

Availability of data and materials

The data used to support the findings of this study are available from the corresponding author upon request.

Abbreviations

PI:

Personal identification

COVID-19:

The coronavirus disease 2019

AI:

Artificial intelligence

PRISMA-ScR:

The preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews

PCC:

Participant-Concept-Context

WHO:

The World Health Organization

CNN:

Convolutional neural network

References

  1. 1.

    de Boer HH, Obertová Z, Cunha E, Adalian P, Baccino E, Fracasso T, et al. Strengthening the role of forensic anthropology in personal identification: position statement by the Board of the Forensic Anthropology Society of Europe (FASE). Forensic Sci Int. 2020;315:110456.

    Article  Google Scholar 

  2. 2.

    Sweet D. INTERPOL DVI best-practice standards–an overview. Forensic Sci Int. 2010;201:18–21.

    Article  Google Scholar 

  3. 3.

    Schuliar Y, Knudsen PJ. Role of forensic pathologists in mass disasters. Forensic Sci Med Pathol. 2012;8:164–73.

    Article  Google Scholar 

  4. 4.

    Matsuda S, Yoshimura H. Personal identification systems for persons with dementia: a systematic review. Psychogeriatrics. 2021;21:832–8.

    Article  Google Scholar 

  5. 5.

    Fiorella KJ, Coffin-Schmitt J, Gaynor KM, Gregory GH, Rasolofoson R, Seto KL. Feedbacks from human health to household reliance on natural resources during the COVID-19 pandemic. Lancet Planet Health. 2020;4:e441–2.

    Article  Google Scholar 

  6. 6.

    Matsuda S, Yoshida H, Ebata K, Shimada I, Yoshimura H. Forensic odontology with digital technologies: a systematic review. J Forensic Leg Med. 2020;74:102004.

    Article  Google Scholar 

  7. 7.

    Marr D. Artificial intelligence–a personal view. Artif Intell. 1977;9:37–48.

    Article  Google Scholar 

  8. 8.

    Wallis C. How artificial intelligence will change medicine. Nature. 2019;576:S48.

    CAS  Article  Google Scholar 

  9. 9.

    Santin M, Brama C, Théro H, Ketheeswaran E, El-Karoui I, Bidault F, et al. Detecting abnormal thyroid cartilages on CT using deep learning. Diagn Interv Imaging. 2019;100:251–7.

    CAS  Article  Google Scholar 

  10. 10.

    Qiu S, Joshi PS, Miller MI, Xue C, Zhou X, Karjadi C, et al. Development and validation of an interpretable deep learning framework for Alzheimer's disease classification. Brain. 2020;143:1920–33.

    Article  Google Scholar 

  11. 11.

    Dobay A, Ford J, Decker S, Ampanozi G, Franckenberg S, Affolter R, et al. Potential use of deep learning techniques for postmortem imaging. Forensic Sci Med Pathol. 2020;16:671–9.

    Article  Google Scholar 

  12. 12.

    Matsuda S, Miyamoto Y, Yoshimura H, Hasegawa T. Personal identification with orthopantomography using simple convolutional neural networks: a preliminary study. Sci Rep. 2020;10:13559.

    CAS  Article  Google Scholar 

  13. 13.

    World Health Organization. Coronavirus disease (COVID-19) pandemic. https://www.who.int/emergencies/diseases/novel-coronavirus-2019. Accessed 11 Nov 2021.

  14. 14.

    Sun G, Vinh NQ, Matsuoka A, Miyata K, Chen C, Ueda A, et al. Design an easy-to-use infection screening system for non-contact monitoring of vital-signs to prevent the spread of pandemic diseases. Annu Int Conf IEEE Eng Med Biol Soc. 2014;2014:4811–4.

    PubMed  Google Scholar 

  15. 15.

    Arksey H, O'Malley LS. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:19–32.

    Article  Google Scholar 

  16. 16.

    Peters MD, Godfrey CM, McInerney P, Soares CB, Khalil H, Parker D. Methodology for JBI Scoping Reviews. The Joanna Briggs Institute Reviewers’ Manual 2015. Adelaide: The Joanna Briggs Institute; 2015.

  17. 17.

    Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13:141–6.

    Article  Google Scholar 

  18. 18.

    Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evidence Synthesis. 2020;18:2119–26.

    Article  Google Scholar 

  19. 19.

    Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169:467–73.

    Article  Google Scholar 

  20. 20.

    Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al. Mixed Methods Appraisal Tool (MMAT) Version 2018: User guide. McGill; 2018. http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127916259/MMAT_2018_criteria-manual_2018-08-01_ENG.pdf%0A. Accessed 11 Nov 2021.

  21. 21.

    Nguyen DT, Park KR. Body-based gender recognition using images from visible and thermal cameras. Sensors (Basel). 2016;16:156.

    Article  Google Scholar 

  22. 22.

    Huang MX, Ngai G, Hua KA, Chan SCF, Leong HV. Identifying user-specific facial affects from spontaneous expressions with minimal annotation. IEEE Trans Affect Comput. 2016;7:360–73. https://doi.org/10.1109/TAFFC.2015.2495222.

    Article  Google Scholar 

  23. 23.

    Diederichs K, Qiu A, Shaker G. Wireless biometric individual identification utilizing millimeter waves. IEEE Sens Lett. 2017;1:1–4. https://doi.org/10.1109/LSENS.2017.2673551.

  24. 24.

    Chen X, Qing L, He X, Su J, Peng Y. From eyes to face synthesis: a new approach for human-centered smart surveillance. IEEE. 2018;6:14567–75. https://doi.org/10.1109/ACCESS.2018.2803787 Access.

    Article  Google Scholar 

  25. 25.

    Wang J, Wang G, Zhou M. Bimodal vein data mining via cross-selected-domain knowledge transfer. IEEE Trans Inform Forens Secur. 2018;13:733–44. https://doi.org/10.1109/TIFS.2017.2766039.

    Article  Google Scholar 

  26. 26.

    Ranjan R, Patel VM, Chellappa R. HyperFace: A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans Pattern Anal Mach Intell. 2019;41:121–35.

    Article  Google Scholar 

  27. 27.

    Sang H, Wang C, He D, Liu Q. Multi-information flow CNN and attribute-aided reranking for person reidentification. Comput Intell Neurosci. 2019;2019:7028107.

    Article  Google Scholar 

  28. 28.

    Wang K, Kumar A. Toward more accurate iris recognition using dilated residual features. IEEE Trans Inform Forens Secur. 2019;14:3233–2345. https://doi.org/10.1109/TIFS.2019.2913234.

    Article  Google Scholar 

  29. 29.

    Massimo L. From fingers to faces: visual semiotics and digital forensics. Int J Semiot Law. 2021;34:579–99.

    Article  Google Scholar 

  30. 30.

    Liu Y, Kumar A. Contactless palmprint identification using deeply learned residual features. IEEE Trans Biom Behav Identity Sci. 2020;2:172–81. https://doi.org/10.1109/TBIOM.2020.2967073.

    Article  Google Scholar 

  31. 31.

    Nabulsi A, Al-Shaikhli W, Kettlewell C, Hejtmanek K, Hassan AM, Derakhshani R. Machine learning classification of S-Band microwave scattering measurements from the forearm as a novel biometric technique. IEEE Open J Antenn Propagat. 2020;1:118–25.

  32. 32.

    Wu S, Sakamoto T, Oishi K, Sato T, Inoue K, Fukuda T, et al. Person-specific heart rate estimation with ultra-wideband radar using convolutional neural networks. IEEE Access. 2019;7:168484–94. https://doi.org/10.1109/ACCESS.2019.2954294.

    Article  Google Scholar 

  33. 33.

    Matsuda S, Yoshida H, Yoshimura H, Goi T, Yoshida Y, Iwasaki H. Has the COVID-19 pandemic influenced the oral health and nutritional status of elderly patients with digestive cancer? A retrospective study in Fukui, Japan. Medicine (Baltimore). 2021;100:e27500.

    CAS  Article  Google Scholar 

  34. 34.

    United Nations. Department of Economic and Social Affairs, Sustainable Development, The 17 goals. https://sdgs.un.org/goals. Accessed 11 Nov 2021

Download references

Acknowledgements

None.

Funding

This study was supported by Life Science Innovation Center of University of Fukui (Grant No. LSI20202).

Author information

Affiliations

Authors

Contributions

S.M. and H.Y. contributed to the conception, design of the work, analyzed data, and wrote original draft. All authors edited and substantively revised the manuscript and read and approved the final version of the manuscript.

Corresponding author

Correspondence to Shinpei Matsuda.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. 

Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Matsuda, S., Yoshimura, H. Personal identification with artificial intelligence under COVID-19 crisis: a scoping review. Syst Rev 11, 7 (2022). https://doi.org/10.1186/s13643-021-01879-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13643-021-01879-z

Keywords

  • Personal identification
  • Artificial intelligence
  • Convolutional neural network
  • COVID-19 pandemic
  • Contactless methods
  • Scoping review