Human digital twin: a survey

Abstract

The concept of the Human Digital Twin (HDT) has recently emerged as a new research area within the domain of digital twin technology. HDT refers to the replica of a physical-world human in the digital world. Currently, research on HDT is still in its early stages, with a lack of comprehensive and in-depth analysis from the perspectives of universal frameworks, core technologies, and applications. Therefore, this paper conducts an extensive literature review on HDT research, analyzing the underlying technologies and establishing typical frameworks in which the core HDT functions or components are organized. Based on the findings from the aforementioned work, the paper proposes a generic architecture for the HDT system and describes the core function blocks and corresponding technologies. Subsequently, the paper presents the state of the art of HDT technologies and their applications in the healthcare, industry, and daily life domains. Finally, the paper discusses various issues related to the development of HDT and points out the trends and challenges of future HDT research and development.

Introduction

The concept of digital twin (DT) has its origins in the NASA Apollo program in the 1970s, where the idea of constructing identical space vehicles on Earth to simulate and predict the behavior of those in space was first explored [1]. Although not a complete digital twin system, this program marked the realization of the importance of physical twins [2]. In 2003, Professor Grieves introduced the concept of a “Virtual Digital Expression Equivalent to Physical Product” as part of Product Lifecycle Management at the University of Michigan [3]. This concept, initially referred to as “Mirrored Space Model” [4] and later “Information Mirroring Model” [5], laid the foundation for the core blocks of digital twin, including physical space, cyberspace, and their interface. Compared with the “twin” put forward by the NASA Apollo program, Professor Grieves has completed the transformation from entity to digital model.

Under Industry 4.0, DT has attracted increasing attention [6] and was defined as a technology that creates a digital replica of a physical entity using real-time data and models. This technology can help physical objects achieve intelligence, automation, and optimized management. DT is wildly used in industry [7, 8], healthcare [9, 10], intelligent transportation [11, 12], and other domains. With the rapid development of digital twin related technologies (e.g., Internet of Things (IoT), 5G \6G, and artificial intelligence (AI)), researchers are trying to reconstruct the digital twin world in virtual cyberspace, extending from the application of atoms and devices to cells, hearts, and human bodies. Along with this, a novel research area, dubbed as “human digital twin” (HDT), has emerged. HDT is referred to as the replica of a physical human in the digital world.

HDT is an incarnation in the digital world of a human in the physical world. Shengli et al. [13] defined it as a model or database which records human current and historical data. Firstly, human information obtained continuously through various smart sensors (e.g., wearable devices, smartphones, and GPS), is transferred to the digital world consecutively. Secondly, the database is updated according to the recorded information. Thirdly, the HDT model analyzes the current and historical data to extract meaningful insights. Finally, it provides feedback information (e.g., diagnoses, predictions, and other suggestions) to the human. Miller et al. [14] further refined the definition of HDT. They pointed out that a human in the physical world may be an individual, who could be characterized as a model, or a group of humans with common attributes, which could be summarized into eight categories of human attributes, including physical, physiological perceptual performance, cognitive performance, personality characteristics, emotional state, ethical stance, and behavior. The HDT they defined includes both first principles models which are based on fundamental understanding and statistical models, which are both modeled using various attributes from an individual or a human class [15]. Recently, HDT has been widely used in various domains, including precision medicine [10, 16,17,18,19], fitness management [20], industrial production [6, 21,22,23,24,25,26,27].

HDT has shown considerable potential and has attracted extensive attention from industry and academia since it was proposed. However, the development of HDT is still in its infancy. Despite the growing number of publications discussing HDT, it can still be observed that the existing research lacks an in-depth analysis of HDT from the perspective of generic frameworks, technologies, applications, and challenges. Thus, we intend to analyze the status of HDT research, propose a comprehensive architecture for organizing the HDT, present the state of the art of core technologies and applications of HDT, and discuss various potential trends and challenges related to the development of HDT. The main contributions of this paper can be summarized as follows:

• 1. Present a comprehensive literature review on the current state-of-the-art in HDT, comparative analysis of the research focus, technological advantages, and existing shortcomings to provide a typical research framework for the field.

• 2. Propose a novel framework for HDT, focusing on the core technology of utilizing multi-modal and multi-source data to model the human body and behavior.

• 3. Discuss the future trends and challenges of HDT in various domains, including technology, social and human thinking and cognition issues.

The overall structure of the paper takes the form of seven sections, including this Introductory Section. “ The state-of-art of human digital twin” section reviews the state of the art in HDT. “ A generic architecture of the human digital twin” section proposes a generic HDT architecture. “ Core technology review of human digital twin” section reviews the core technologies required for the application of HDT. Applications of HDT in healthcare, industry, and daily life are presented in “ Human digital twin application” section. “ Human digital twin trends and challenges” section discusses various HDT trends and challenges from the perspectives of technology, society, and human thinking and cognition issues, and “ Conclusions” section draws conclusions. For convenience, a summary of all relevant abbreviations is shown in Table 1.

Table 1 Summary of abbreviations

The state-of-art of human digital twin

Review of the literature

With the increasing importance of digitalization and the trends of IoT, big data, cloud computing, edge computing and AI, more and more researchers started to focus on HDT. The popularity of HDT continues to grow, and the application of HDT has great potential in engineering, aviation, education, medicine and a wide variety of other domains.

To make a comprehensive review of HDT, we searched “Human digital twin” in Springer, IEEE, ACM, Elsevier, from January 2018 to July 2024. In order to establish a clear structure of existing literature we decide to clarify publications based on the nature of the papers, namely the concept, technology, framework or application of HDT. After excluding irrelevant publications, 327 highly relevant literatures were screened. Figure 1 shows a steady increase in the number of HDT publications. The growth of the number of HDT-related publications began recently, after 2018. Before 2021, the development of HDT in the academic field is relatively slow. From 2021 to 2024, the number of HDT publications in the academic field increased rapidly. It shows that HDT is gradually coming out of the embryonic stage and entering the rapid development stage. Researchers have shifted from exploring the concept of HDT to exploring the generic framework of HDT and the practical application and related technologies.

Fig. 1

The human digital twin implementation architecture

HDT originated from the addition of human resources in the industry domain. The first publication did not mention the term “human digital twin”, instead using other terms like “digital twin for workers” or “digital athlete”. Various human attributes have been used to develop digital twin models for workers, which is helpful for coordination between workers and the system. Graessler et al. [28] developed the digital twin by combining the integration of human resources with the task assumption of computer systems, communication and coordination between workers and production systems, and acting as the representative of worker in the digital world. The authors optimized the digital twin for workers [29], applied it to the task of assembly station, and demonstrated the effects of human integration in the production system and production control system. HDT appeared not only in the industry domain, but also in the sports domain. Garon et al. [30] reviewed the virtual world and the legal framework regulating content ownership, and discussed the legal issues of avatar actors and digital athletes in the sports domain. Baskaran et al. recognized assembly operations from real-world vehicle assembly factories and created digital twin models of the human body in the Siemens Tecnomatix suite, and this is where the term “human digital twin” first appeared in the literature [27]. This research started by simulating different human body models to find the limitations of performing assembly tasks based on gender, weight and height. Furthermore, the research also introduced a mobile robot digital twin, which provided assistance to humans in the physical world to perform assembly operations. Finally, the research evaluated the results from process time and joint ergonomics, and revealed the limitations of the combined digital twin modeling of human-robot cooperation.

With the full onset of the Industry 5.0 era, HDT has entered a rapid development stage, presenting high real-time capabilities and accuracy through highly integrated sensor networks and advanced data processing techniques. This technology captures and reflects dynamic changes of individuals or systems almost instantly, bringing revolutionary changes to fields such as medical diagnosis, safety alerts, and resource optimization [31]. Simultaneously, the personalized and customized capabilities of this technology, supported by big data and AI, accurately match the unique needs of each individual, enabling personalized customization in areas such as health management, education and training, and career planning, which greatly enhances user experiences and quality of life [32]. Furthermore, the interactivity and collaborative nature of HDT promote interdisciplinary and cross-domain cooperation, driving the development of new work models such as human-machine collaboration and collective intelligence. This accelerates the pace of social innovation and progress [33]. Moreover, the intelligence and adaptability exhibited by HDT enable them to flexibly respond to complex and changing environments and challenges, providing intelligent decision support for emergency scenarios such as medical emergencies and disaster responses [34]. Finally, the interdisciplinary integration and innovative capabilities of HDT technology are gradually permeating and reshaping multiple fields including science and technology, medicine, sociology, and economics. This gives rise to numerous emerging research directions and application scenarios, injecting strong power and vitality into the sustainable development of human society [35].

Table 2 Comparison of review research content

Categorization of the literature

Through a detailed analysis of 327 related literatures, the research content of thses literatures can be broadly 4 categorized as follows:

Comprehensive review: a total of 109 literatures. These literatures provide a comprehensive and systematic review of the concept, development process, technological basis, application scenarios, and future trends of HDT. They not only offer readers a macroscopic view of HDT but also reveal the potential value and challenges of this technology in different fields.

Technology field: a total of 78 literatures. These literaturesfocus on the specific implementation and optimization of HDT technology, including data collection and processing techniques, high-precision modeling techniques, real-time simulation techniques, and the application of artificial intelligence and machine learning in digital twins. Researchers continuously improve the accuracy, real-time performance, and intelligence level of HDT through technological innovation, providing technical support for a wider range of application scenarios.

Application field: a total of 96 literatures. These literatures explore the specific applications of HDT in various fields such as healthcare, urban planning, smart manufacturing, and education and training. Through case studies and model validations, researchers demonstrate the unique advantages and practical effectiveness of HDT in different fields, providing valuable references and insights for the digital transformation of related sectors.

Challenge field: a total of 44 literatures. Faced with various challenges in the development of HDT, such as data security and privacy protection, technical standards and interoperability, ethical issues, these literatures offer corresponding solutions. Researchers call for joint efforts from all sectors of society to strengthen cooperation and communication, collectively promoting the healthy and sustainable development of HDT.

In the following “ Typical comprehensive review of HDT” section, we will conduct an analysis of the comprehensive review literatures, present its key role and important contributions in HDT research. Meanwhile, we will reveal the advantages and potential shortcomings of these literatures, and then make a detailed comparison with the research content of this paper. Moreover, subsequent Section on technology, applications, and challenges will showcase and introduce the corresponding literatures in each category.

Typical comprehensive review of HDT

We selected 8 literatures from 109 comprehensive review literatures for in-depth analysis based on criteria such as the novelty of publication time, the breadth of academic influence (i.e. the number of citations), and the relevance to the research topic of this paper. These literatures extensively cover multiple key areas such as industrial innovation and healthcare, of their research content through typical and latest research in this field, to ensure the timeliness and cutting-edge nature. The comparison results are shown in the Table 2.

Fig. 2

The generic human digital twin framework

Earlier research such as Wei Shengli [13] in 2021 and Miller [14] in 2022 did not discuss the open issue of HDT. With the passage of time, more recent research have started to focus on challenges beyond technological aspects, such as societal challenges and human behavior and cognition. This shift signifies an evolving focus from initial technical applications to the exploration of the societal impacts and human cognitive aspects of HDT. Despite making certain contributions, these research primarily focus on its application in a specific field, especially in healthcare and industrial domain, and did not deeply consider societal issues and human behavior cognition.

Therefore, this paper demonstrates certain advantages in terms of a broader scope of technological applications, a comprehensive technological architecture, and considerate various challenges. Our study not only covers applications in healthcare and industrial domains but also expands to daily life. The technological architecture includes various aspects such as perception, data processing, and model construction, with the addition of human behavior modeling. Furthermore, it also discuss societal issues and challenges related to human behavior and cognition.

Table 3 Detailed descriptions and examples of HDT data

A generic architecture of the human digital twin

Previous studies have mostly demonstrated the idea of dividing the conceptual model of HDT into three core blocks: the physical world block, the digital world block, and the connection and interface block to connect the two worlds. Therefore, we designed the HTD framework based on this idea, and explored the main components and related technologies of each module. Figure 2 shows the HDT designed by us, the division of the sub-blocks, as well as a correlation between each block.

The first block represents a physical-world human representation, including a data collection sub-block and a sensing and perception sub-block, which realizes the data collection function in the physical world. The selection of appropriate human-related data for HDT modeling is critical. Such data falls into eight categories: external human data, physiological data, human-to-human social interaction data, and human-to-environment data. Detailed descriptions and examples of these data are shown in Table 3.

Utilizing this data contributes to a more comprehensive description of the HDT. Each individual and their corresponding HDT would change synchronously. Changes in human-related data in the physical world are sensed and transferred to the digital world, leading to corresponding changes in the HDT. To enable cognitive sharing and real-time interaction between the physical world and digital world, sensors are responsible for gathering data from humans in the physical world, connecting to the environmental system to provide environmental cognition (e.g., humidity and gas concentration). By thoroughly capturing the attributes and states of the physical world to link it to the digital world, the sensing component transmits the data to the ingestion program of the digital world. As such, at least one sensor is required in the sensing and perception process, while the arithmetic unit and the controller serve as optional components in HDT, potentially absent in certain instances.

The second block explores the construction of the HDT in the digital world, and mainly contains data processing, modeling, simulation and analysis engine. Data cleaning is using to solve data discontinuity and deletion. In terms of data storage, the physical world has a corresponding HDT stored in the digital world. Each of the HDTs has a unique index, which can be used as its ID and can also be used as an account to log in to the HDT. In addition, as for data fusion and integration, HDT aims to fuse information from these sensors associated with the physical world human representation and to integrate new real-time data with historical data [46]. Modeling and simulation is the key sub-block of building HDT in the digital world. Human modeling and simulation creates a virtual model by defining and extracting the key features of the human to reflect the key characteristics and dynamic motion of the human representation. The simulation further explores the system performance based on different modeling techniques [14]. The modeling methods we explored are divided into organ and body modeling at the physical level, human activity modeling, social interaction modeling and lifestyle modeling at the behavioral level. As the core sub-block of HDT, modeling technology will be introduced in detail in “ Core technology review of human digital twin” section. The data analysis and model optimization sub-block aims to further predict, evaluate and optimize the HDT through the analysis engine, makes a distortion for suggestions and decision-making through the prediction engine, and compares the prediction results with the data in the database, evaluates the performance of the HDT through the evaluation engine, determines whether the prediction results can be used as feedback to the physical-world human representation. The optimization objective is determined according to the evaluation results to further optimize the HDT and improve its accuracy [14].

The third block focuses on the human-computer interface. The intelligent interface is the bridge between the physical world and the digital world, and is a critical sub-block for realizing the real-time interaction between the human representation and the HDT. Therefore, the two sub-modules of intelligent interface and communication tools are necessary for the human-computer interface block. The visualization engine sub-block could enhance the interaction between the users and the HDT, and increase the users’ understanding and trust of the HDT in the virtual world, which is realized through VR \AR.

In order to clarify the relationship between the three blocks (physical-world human representation block, digital-world human digital twin block and human-computer interface block) and the information flow process in HDT, we present a further explanation for this HDT framework to make it more relevant, interactive and transparent. The working process of HDT is as follows:

• 1. Determine the application purpose of the HDT and select appropriate attributes. A different application of HDTs will determine different data used in HDT modeling.

• 2. The sensors of the physical world perceive the relevant attributes and state of the human and the surrounding environment in the physical world. The physical-world human representation is built using human-related data.

• 3. The intelligent human-computer interaction interface connects the physical and digital worlds. It transfers the sensed multi-source and multi-modal data from the physical world to the digital world, providing support for HDT modeling in the digital world. Moreover, when the state of people (e.g., emotion, psychology, and behavior) in the physical world changes, the real-time data is also fed back to the digital world to optimize and modify the HDT model.

• 4. After receiving the data, the virtual world processes the data, selects features, and stores the processed current and historical data in the HDT database.

• 5. HDT uses modeling technology (e.g., ML, DL and continuous learning) and the data in the database to build the HDT model corresponding to the human in the physical world. HDT model can be divided into physical model and behavior model according to its state. Physical models are used to model organs or the human body. Among them, the organ model is mainly used for precision medical treatment, and the body model can be used for identity authentication, virtual shopping, and others. The behavior model is more widely used in industry, healthcare, daily life and other domains.

• 6. HDT model simulates and evolves with data collected in the physical world to provide the future prediction of corresponding individuals, conduct data analysis and evaluation, judge the accuracy of HDT model prediction, and adjust and optimize the HDT model.

• 7. The human-computer interaction interface module could provide a user-friendly HDT interaction interface through visualization technology, and provide feedback.

Compared with other latest HCPS frameworks (e.g., HDT designed by [14], and HCPS presented by [36]), our proposed HDT surpasses the existing research in two aspects:

• With the development of various technologies (e.g., 5 \6G, edge computing, cloud computing and IoT), multi-modal and multi-source data can be applied to this framework, considering not only human data (external and physiological), but also the data generated by human social interaction, and also considers environment data which could affect the HDT modeling. These data provide more features for the construction of HDT through mutual support, supplement and correction, and avoid the problem of single data affecting the prediction accuracy of HDT.

• We consider the modeling of HDT from two aspects: physical (organs and body) and human behavior (human activities, social interaction and lifestyle). Compared with the framework proposed to apply in specific domains (e.g., industry, healthcare and sports), this framework increases the flexibility of HDT application and improves the universality of the HDT framework.

Fig. 3

The human digital twin implementation architecture

From the short review above, we proposed a HDT implementation architecture, shown in the Fig. 3, which included related technologies (e.g., HDT model, physical objects, and digital infrastructure), the application and open issues of HDT. Base on , this paper will introduce the core technologies, applications, and open issues of HDT in sequence.

Core technology review of human digital twin

The generic HDT framework we propose includes sensing (perception) technology and two modeling technologies of organs body and behavior. Therefore, we review the state of the art of these three core HDT-related technologies, and present each technology’s straightforward methods, development status, advantages and disadvantages in the functional block of HDT in this section.

Sensing \perception

Sensing is the foundation of HDT. The sensor is used to detect the properties, parameters and actions of the entities and their surrounding in the physical world [13], and efficiently provides data to the HDT of the design object [47]. The ubiquitous data acquisition technology has dramatically improved the perception ability of the physical world. The development of advances in hardware (e.g., GPS, smartphone, wearable devices) has made the modeling and analysis of HDT possible [36].

The implementation of HDT necessitates the integration of data from multiple sources, including, but not limited to, sensors, databases, and social media. Wearable sensors serve as vital tools for collecting signals from humans, obtaining valuable physiological data such as galvanic skin response, skin conductivity, skin temperature, and skin blood flow volume, and capturing human living environment data such as temperature, motion, humidity, and distance. Additionally, wearable sensors have the capability to automatically collect images or videos capturing human facial expressions, body language, and social behaviors. Social media data collection involves gathering social text, pictures, user comments, likes, and other information through the public application programming interface of social networking websites. Moreover, medical databases contribute clinical monitoring data, early warning indicators for chronic diseases, daily activity records, and vital signs detection data.

The integration of multi-source and multi-modal data provides more comprehensive information compared to single data sources [48]. These data, through mutual support, supplements, and correction, facilitate the provision of more accurate information, thereby meeting the varied requirements of HDT modeling. Advanced sensing technology has the potential to accurately predict the physiological and psychological status of individuals using features extracted from collected data [36], providing insights into and predictions of the physical world.

Furthermore, the sensing or perception technology used to collect data for HDT modeling varies according to different domains. HDT has been introduced in healthcare to enable thorough and continuous examination of individuals’ health and personal history, facilitating disease diagnosis, treatment, and prediction [49]. For instance, Liu et al. [10] constructed an HDT using data collected from IoT-connected wearable devices and achieved success in predicting athletes’ performance by leveraging fitness-related data such as exercise time, sleep quality, and diet. Similarly, Romero et al. [50] and May et al. [51] designed HDTs in the industrial domain to utilize physical, sensory, cognitive performance, anthropometric data, functional capabilities, knowledge, skills, and expertise. Moreover, kinematic data collected by sensors play a crucial role in industrial production, and studies by Bilberg et al. [52], Peruzzini et al. [53], Shen et al. [54], and Greco et al. [22] showcased the use of sensors to monitor human motion, evaluate workload severity, and assess muscle fatigue. These technologies and methodologies hold significant potential for monitoring and improving human health in various domains, underscoring the broad applications and contributions of multi-source data integration for HDT modeling.

Table 4 The analysis of human digital twin modeling for organs

Human body \organs modeling

Human organs modeling

The tumor domain is the main application area of organ modeling (e.g., heart, cardiovascular and liver) of HDT. For tumor research, traditional technology can only cultivate specific tumors in vitro in the physical world, but this method cannot represent the patient’s actual situation. Each modelled organ can represent the original tumor patient, which makes it possible to simulate an appropriate treatment plan on the HDT modelled organ. This is the main advantage of HDT organ modelling technology.

Martine et al. [41] proposed the Cardio Twin, a human heart HDT. Various technologies (e.g., VR, AR, and robotics) were used to demonstrate the heart model in this system. Edge computing gave the subjects full control and continuous monitoring of the information collected and processed. Mazumder et al. [42] described a cardiovascular HDT model, including blood vessels, cardiac chambers, and the central nervous system. Chakshu et al. [16] proposed an inverse analysis of the cardiovascular system using recurrent neural networks, to detect abdominal aortic aneurysms. Crea et al. [43] presented a cardiovascular HDT model, augmented by computational inductive (using statistical models learned from data) and deductive (incorporating multiscale knowledge a synergistic combination of mechanical modeling and simulation) to provide precision medicine for cardiovascular disease. These three papers developed HDT models to address cardiovascular issues, but with various focuses. The methods or techniques and model description are shown in Table 4. Subramanian et al. [55] designed a liver HDT model using methods that allow nonlinear, feedback, and dynamic analyses such as ordinary differential equations. Golse et al. [56] presented a liver HDT model to reflect the patient’s status by setting the model parameters, so that the model variables were close to the measured data of each patient. The liver HDT model helped physicians in decision-making strategies Table 4 also analyzes the difference in liver HDT between these two literatures.

Human body modeling

HDT is a digital avatar of a physical human being and can describe the physical human body in a comprehensive way in the digital world. Physical body modeling vividly represents the human body shape and appearance in HDT.

Several studies have focused on modeling the basic information of the human body, such as height, weight, and physical proportions, offering innovative approaches for body shape modeling in HDT. The most commonly employed paradigm for constructing human body shape in HDT is parametric modeling [57]. For instance, Cheng et al. [57] devised a parametric 3D modeling approach based on background differentiation and human contours separation. They enhanced the GoogleNet network model to capture both front and side views of the human body, ultimately reconstructing the human body in 3D based on actual human contours.RGB-D sequences data is second method for human body modeling in HDT. Hesse et al. [58] proposed a statistical 3D Skinned Multi-Infant Linear body model, offering a new method to accurately construct infant body shape and motion, while reducing data acquisition costs and accommodating incomplete, low-quality RGB-D sequence data. Zuo et al. [59] developed a texture mapping method for reconstructing 3D human models based on sparse RGB-D images, successfully applying it to personalized customization of human body models with an error margin of several millimeters. 3D scans and 2D images is the third menthod to model human body shapes in HDT. Tong et al. [60] introduced a scanning system based on multiple Kinects for constructing human body shape, demonstrating applicability in 3D human animation and virtual try-on. Xu et al. [61] proposed a critical point-based method for estimating high-fidelity human body models, constructing the human body based on raw human body scans and human body 2D images. Furthermore, facial modeling technologies have been employed to map the human face onto the HDT. Liu et al. [62] developed a novel 3D face modeling method combining face alignment with 3D face reconstruction, whereas Lin et al. [63] presented a reliable scheme for facial editing in video construction, reducing identity loss and semantic entanglement. Chu et al. [64] developed a parametric face modeling framework for characterizing facial parameters and 3D facial synthesis using the Kriging method.

Human behavior modeling

HDT behavior modeling aims to build a human behavior model in the digital world to simulate human behavior in all aspects, so that HDT behavior is close to human behavior. Researchers collect human behaviors in the physical world, and endow HDT models with corresponding behaviors in the virtual world [65]. These behaviors include not only HDT behaviors but also interactive behaviors. According to the different modeling angles of behavioral models, we investigate behavioral models from the aspects of activity modeling, social interaction modeling, and lifestyle modeling, and study how different behaviors are modeled in HDT behavior modeling from different modeling perspectives.

Activity modeling

Activity modeling refers to the modeling of human physical activities by HDT, such as facial expression changes, walking, running, and jumping. It is the basis of HDT behavior modeling, and the other behaviors below are extensions or combinations of these behaviors. At the same time, this is also the most basic behavioral feature of “humans” in HDT.

Wang et al. [66] presented an approach to extract a 3D facial model from an image using a shape regression network. This method obtained high-quality 3D model pictures at minimal cost, and achieved high-accuracy facial modeling. Although this method is not strictly HDT, it is also the theoretical basis for constituting HDT. Razzaq et al. [67] used Convolutional Neural Networks (CNN) to build HDT models of students’ and teachers’ facial expressions and behaviors for remote teaching and classroom supervision. This method was born out of face modeling, and on this basis, the recognition of facial expressions was added, which had higher practicability. However, the limitation of this method was that using only CNN as a modeling tool may reduce generalization, and the deployment cost was raised by the requirement of the participation of a large amount of facial expression data and the requirements on sample quality. Trobinger et al. [68] collected information from doctors and patients using multi-modal sensors, and established a digital twin of patients using motion equations. The patient’s activities and facial expressions were identified through motion analysis and support vector machines to help doctors diagnose patients remotely. This method can quickly and accurately establish a patient’s HDT model, but it cannot further learn from the patient’s historical records and medical conditions, which has certain limitations. Similarly, in the medical domain, Chen et al. [45] used a 3D visualization system to build a model of knee joint motion and supported the analysis of motion trajectories to achieve real-time motion tracking. The above two methods model human motion, which is more complicated than a single part. Therefore, these two studies established a complete set of HDT models, from data collection to model presentation, which has higher usability and sets a reference standard for future HDT tasks.

Social interaction modeling

The social interaction model serves as a means to facilitate interaction between HDT models, aiming to simulate classical modes of human interaction in the physical world. It necessitates the participation of two or more HDTs in the virtual realm, each equipped with interfaces enabling communication with one another [14]. This model can be categorized into traditional social interaction models and online social interaction models.

Traditional social interaction entails interaction between HDTs through limited means, such as face-to-face and telephone communication. Assadi et al. [24] established a lightweight Message Queuing Telemetry Transport protocol based on the “JSON” format, modeling the interaction between workers in the factory. Their approach realized intelligently assigning tasks based on specific conditions, treating social interaction as a distinct module facilitating qualitative analysis and utilization of inter-HDT interactions. Additionally, Rashik Parmar et al. [69] proposed principles for building organizational digital twin, optimizing these twins by simulating human transactions and competition, thereby achieving dynamic decision-making and evolution. Although they scarcely delved into the construction of social interaction models, their principles provide a valuable reference template for HDT’s social interaction behavior modeling.

On the other hand, online social interaction modeling offers an alternative approach for social interaction between HDTs, abstracted from traditional social networking. Online social networking involves communication through online social platforms and facilitates more accessible interaction. Jianshan Sun et al. [70] observed users’ online published content and preferences, applying the pretrained Glove-vector model to transform text into word vectors. By constructing a user-likes matrix through singular value decomposition (SVD) techniques, they formalized personality as an HDT model [71]. Utilizing a multitask learning depth neural network model, they predicted user personality through these data representations, thereby improving prediction accuracy. In this study, Deep Neural Networks (DNN) outperformed standard ML algorithms, with the best performance arising from the combination of document and similarity vectors obtained through SVD.

Lifestyle modeling

Lifestyle modelling incorporates all of the human behaviours that take place throughout the course of a day into the HDT model by translating how people live in the physical world. This modeling method integrates the above two behaviors and has a higher level of abstraction and universality.

Barricelli et al. [20] established the SmartFit model, which established the HDT after observing the athlete’s behavior for a while to detect the athlete’s physical condition. They treated the prediction of the athlete’s behaviour as a classification problem and used the error-correcting output code method to address it. To impute the data and increase data reliability, they used a K-Nearest Neighbor (KNN) based method. Herrgard et al. [72] proposed a hybrid model to simulate ischemic stroke, combining formula-based modeling with ML to simulate known physiological risk factors and calculate the risk of developing the disease following intervention in real-time. This model was different from the conventional HDT model, because researchers gave the operator a model with disease risk when it was created, and asked the operator to optimize it with different interaction methods to make it unique. It was highly innovative and had high use value. In addition, R. Ferdousi et al. [44] proposed an HDT of the Mental Stress model, and achieved 98% accuracy by using ML algorithms. Data were collected from 20 sources (e.g., activity, social, lifestyle, physical and psychological data). It solved the problem that the existing stress prediction system was not suitable for dealing with various changing stress sources, and the incomplete characteristics and static prediction technology that traditional methods usually use only a single source of data (such as only wearable sensors or user devices).

Human digital twin application

Healthcare domain

Healthcare is defined as a broad concept covering precision medicine [73], smart medicine, sports, health, nursing and rehabilitation [74] etc. Healthcare refers to the health of people in all aspects (e.g., physical, psychological, social levels), with the goal of “putting the individual in a state of wellbeing”. One healthcare domain problem is the lack of real-time information interaction [73]. Another problem is that it is difficult to extract accurate molecular data due to the complex structure of the human body and the constant changes in the human condition [9]. A further issue is that different chronic diseases have a prolonged course, are frequently recurrent, progress slowly, and are expensive and difficult to detect and manage. In addition, the world is facing the problem of an aging population, so it is necessary to consider more intelligent life monitoring for the elderly [10, 75,76,77]. HDT technology offers new opportunities for healthcare. It has been widely used in precision medicine and sports medicine domain [47]. It has seamlessly connected the physical world and digital world, effectively integrated information [78], provided real-time monitoring of patient body data [79], enabled more real-time relevant medical responses [74], and improved resource utilization rate [73]. Researchers also achieved predictive maintenance for the elderly [77] by combining HDT with healthcare, providing more accurate and faster services for elderly healthcare [10]. Furthermore, HDT is expected to achieve better disease prediction, disease diagnosis, disease treatment, continuous monitoring, medical assessment, and risk assessment without interfering with patients’ daily activities [9, 73].

In the precision medicine domain, the concept of a precision HDT model tailored specifically to each individual stands as a pivotal advancement. This model encapsulates a vast array of individual characteristics, encompassing behavior habits, lifestyle preferences, as well as physiological and psychological states. By leveraging this personalized HDT model, which treats an individual’s unique information as its cornerstone, to establish a robust foundation for crafting highly individualized treatment plans for patients. For instance, Shamanna et al. [17] developed the Twin Precision Treatment Program, which utilized body sensors and a mobile phone application to collect, track, and analyze bodily health signals, ultimately providing personalized treatment for each patient. Furthermore, Barbiero et al. [80] introduced a groundbreaking framework that integrates advanced AI methodologies with sophisticated mathematical modeling. This framework offers a comprehensive, panoramic view of both present and anticipated pathophysiological conditions, further enhancing the precision and effectiveness of medical interventions. In addition, HDT is poised to revolutionize tumor management and surgical procedures. For instance, Mourtzis et al. [19] developed an HDT specifically designed to analyze Magnetic Resonance Imaging (MRI) scans. This technology enables oncologists to meticulously monitor patients’ status and anticipate tumor progression based on intricate biometric data, thereby facilitating the formulation of the most appropriate treatment strategies. Moreover, Ahmed et al. [18] have explored the exciting potential of HDT models in surgical education and training. Their work, while primarily focused on the theoretical construction of models that mimic human self-behavior, lays a solid foundation for future applications in this domain, underscoring the versatility and impact of HDT across various medical specialties.

In the sports medicine domain, the HDT model not only digitizes the information related to the human body, but also constructs the human body structure with high fidelity. Various sports-related options, such as sports plans and diet, can be tested and taught on the HDT model, which dramatically reduces the risk of sports training. As a result, HDT offers a highly integrated simulation system for use in sports medicine. By creating appropriate datasets and developing appropriate simulation models, the HDT model can simulate the behavior of the human body and its subsystems, enabling continuous monitoring of the human and prediction of its future state. Barricelli et al. [20] established the SmartFit framework to monitor the physical condition of athletes. It built a digital model of the athletes’ bodies after observing the behavior of athletes for several consecutive days, to analyze athletes’ states and judge if their future exercise and diet plans are optimal. This HDT model concentrated on the behaviour of athletes and their professional conduct; social behaviour or grouping of athletes for greater accuracy was not considered. However, it gave the HDT model the capacity to learn and had a high usage value. Alekseyev et al. [81] introduced a method to research mobile information and measurement systems for constructing personal dynamic portraits of the human body reflecting human movement in different domains of activity study portraits, and proposed a personalized system for assessing the recovery of the patient’s walking skills. This HDT system tracked patients’ tendencies based on assessments of temporal characteristics of walking technique, to improve motor skills during rehabilitation.

Industry domain

Under the trend of Industry 4.0, HDT has been given more significance. Researchers started using HDT in intelligent factories to improve productivity and production efficiency. Pires et al. [7] discussed the significance and challenges of HDT. Locklin et al. [21] analyzed the feasibility of HDT in the industry. They argued that HDT could adjust its relationship with the physical world at a lower cost, as natural as worker job transfers.

HDT is assigned various forms and tasks within the manufacturing industry. Because the majority of human self-behavior expansion in the manufacturing domain is the expansion of some occupational knowledge and behaviour, namely professional behaviour, researchers focus more on the expansion of human social behaviour and human self-behavior. Montini et al. [26] proposed a meta-model of factory HDT, including human psychological, physiological and behavioral aspects, which provided a reference for building a unique HDT model. Greco et al. [22] employed HDT to accurately capture actual error operations in human-computer interactions within manufacturing settings, increased equipment reliability and enabled objective and repeatable ergonomic screenings, providing a more efficient and accurate evaluation compared to traditional methods. The ability to continuously monitor workstations and propose solutions based on real data contributes to reducing risk indexes and improving overall ergonomics. Furthermore, the procedure offers the potential for immediate ergonomic analysis through devices capable of real-time data transfer. Sun et al. [23] also adopted a similar method and improved assembly quality and efficiency by constructing an HDT model to debug high-precision models. Baskaran et al. [27] designed HDT in automotive assembly domain to test the possible impact and labor feasibility of various solutions. Ariansyah et al. [25] built HDT models by collecting human fatigue data to better allocate production operations work and decision-making. Al Assadi et al. [24] implemented a personnel management system for the automatic training and placement of workers by modeling worker’ behavior and professional behavior. Sharotry et al. [6] modeled the behavior of people carrying heavy objects and assessed the risk of injury by the degree of muscle fatigue and the standard of carrying movements. They built the model by collecting the data of each joint of the person during handling and evaluating the person’s physical state, and added a digital twin module to establish the HDT model through the learning of various fatigue data to ensure the correctness of the operator’s handling work. This method also focused on human self-behavior and professional behavior. However, it did not realize the steps from “general” to “specialized”, because it just learned the average value of human through a large number of samples and made judgments with more remarkable universality. Nevertheless, it may not play a role in applying extreme cases.

Daily life domain

The indicators of the HDT model in the digital world should be as close as possible to human characteristics to help and guide people to work and learn better by simulating the components of the physical world [34]. The social attributes of HDT are expected to be high. Toshima et al. [82] took HDT as a simulation model similar to human’s internal and external self to assist in completing daily tasks. In order to predict the user’s weight and combine it with BMI indices, Caballero et al. [83] created an HDT model, modelled the data obtained from the body weight sensors of people, described an algorithm for filtering and predicting human body weight, proposed various thresholds based on gender to filter possible error values. When the user was predicted to be underweight, overweight or heavier, the HDT model can provide coping suggestions on time. Moreover, Magic Weaver Inc. is emerging as a leading example of HDT in the realm of clothing customization. Their implementation of AI-driven body digitization and measurement solutions, including intelligent size recommendations and personalized customization services, underscores the practicality of such advancements for the apparel industry [84]. By leveraging 3D human modeling technology, Magic Weaver Inc. accurately calculates optimal clothing sizes for individual customers based solely on their photographs. This approach has resulted in a notable 70% reduction in customer returns and exchanges, achieved through the creation of highly realistic digital avatars that optimize inventory management and enhance customer satisfaction.

HDT is also considered to be applied to the metaverse to solve problems caused by the interaction between digital HDT model and human in daily life. Stacchio et al. [85] created the HDT model in the metaverse by scanning the human body, and asking specific questions for the HDT model to test its performance. The store clerk in the physical world accepted the request on the screen as if the HDT was visiting the store. This project studied the effect of HDT and human interaction in the context of clothing stores. In addition, preserving the identity details of real human faces plays an important role in accurately confirming the identity of the HDT model in the metaverse. Lin et al. [63] implemented controllable face editing in video reconstruction, realized high-resolution and realistic face generation, and ensured the operability and high fidelity of HDT. This method achieved prominent identity preservation and semantic unwrapping in controllable face editing, which was superior to the most advanced method.

With the development of IoT technology, HDT is expected to be widely used in human smart life (e.g., smart home, smart office and smart travel) in the future. The use of personal and contextual data stored and processed in HDT will bring better, more personalized, safer and more efficient applications for the intelligent environment driven by IoT devices. Zibuschka et al. [86] envisioned HDT applications in the domains of home, architec- ture and office automation. For example, the HDT could help observe and simulate residents’ living or working behaviors in the smart home environment within a limited time.

Human digital twin trends and challenges

In the above research work, technology issues related to the development of HDT are the challenges that have received the most attention in the literature. With the rapid development of advanced technologies (e.g., AI, IoT, 5G \6G, Cloud Computing), significant progress has been made in the development HDT. Nevertheless, various challenges remain, such as data heterogeneity, data transmission and real-time interaction. After constructing an HDT, various social issues with the application of HDT in the physical world cannot be ignored. We mainly discuss three social issues, one is HDT regulation, another is ethics and privacy, the third one is trust and transparency. Finally, due to the unique human thinking and cognition, machine intelligence and cognitive simulation could be considered to optimize HDT.

Technology issues

 

• Artificial Intelligence AI is one of the underlying core technologies of HDT, mainly used for data processing and system optimization [13]. It is the central brain of HDT. Wan et al. [87] reviewed the feature detection, diagnosis, and prediction performance of semi supervised SVM in brain image fusion HDT, and proposed a semi supervised SVM to deal with the large amount of unlabeled data in brain images, utilizing both unlabeled and labeled data. Meanwhile, the study also described how to enhance the AlexNet model and utilize HDT models to map brain images from physical world to digital world. Xiong et al. [88] proposed Digital Twin Brain, which is a transformative platform to builds a bridge between human intelligence and machine intelligence in the form of brain networks. They pointed out that artificial neural networks based on network nodes and connections share basic units with brain network groups, and human intelligence knowledge can be transferred to machine intelligence in the form of networks.

• Data Heterogeneity HDT involves the storage and analysis of various information related to humans, such as social attributes, physical behavior, physiological signals, and so on [89]. This information is not only represented in different forms (e.g., images, text, audio, etc.), but also stored in different systems, including Windows, Unix, Linux, and others. Multi-modal and multi-source data are used to construct an HDT. Therefore, data heterogeneity arises, which causes difficulties in data integration and leads to the “isolated information island”. Data heterogeneity hinders the further exploration and application of the inherent knowledge of the data. Although previous studies on ontology technology have not dealt with HDT, it has been widely applied in the digital twin to solve the problem of data heterogeneity. Singh et al. [90] analyzed the challenges of data management, including data diversity, data mining, and dynamic analysis. They applied ontology technology to digital twin modeling to address the above challenges and proposed a digital twin Ontology Model. Steinmetz et al. [91] used an ontology to formalize the digital twin in the CPS. Based on the ontology, they conducted the simulation experiment in industrial case studies and preliminarily verified that the ontology can effectively integrate multiple data sources. In short, the ontology technology is not only suitable for digital twin, but also hopes to be used for HDT in the future. An HDT ontology model can be constructed based on ontology technology to provide a unified paradigm for the human body information-related data, which is expected to be an effective solution for data heterogeneity [92, 93].

• Real-time Interaction 5G technology helps HDT to address the problem of real-time interaction because the development of 5G \6G networks has added the additional potential for precise real-time data collection and access [94]. Due to its highly reliable connection, high speed and low latency, 5G systems have been widely applied and deployed in various countries [95]. 5G is the critical technology for realizing real-time remote interaction. Various experimental verifications show that the 5G video transmission delay is 49ms, which well meets the needs of real-time interaction for low delay. Although it is not widely used in the early stage of HDT, similarly, it can draw on the experience of the video transmission protocol based on 5G communication [96] in the domain of real-time unmanned aerial vehicle video transmission. 5G has realized the gigabit speed, greater capacity and ultra-low latency, while 6G will use more advanced radio equipment and various radio waves, and can provide ultra-high speed and huge capacity in a short distance. Holographic telepresence is an essential application of 6G [95]. It will allow people to communicate with the authentic 3D model of objects, bring people with an immersive real-time 3D experience, and elevate human communication to a new level. With the support of holographic communication technology, the real-time and high mobility characteristics of HDT can be fully realized in the future

• Privacy and Ethics Data privacy and ethics issues and HDT ownership should be given priority in the construction and implementation of HDT [97]. Multi-modal and multi-source data used in HDT modeling, such as patient’s medical data, are compassionate and involve security and privacy issues. Theft or misuse of these data may cause serious damage to users. For traditional ML, privacy protection is supported by digital signatures and data encryption. It guarantees the protection of personal data, but causes an additional burden on data processing and increases the processing delay [98]. To overcome this problem, federated learning can be considered to apply to HDT to reduce the processing delay while privacy is protected. Through federated learning technology, the data does not need to be transmitted and the model training is carried out at the local client and the parameters obtained by training are uploaded to the central server finally, which reduces the risk of data leakage [99, 100]. Blockchain technology, to a large extent, can ensure the security of data transmission of HDT. And it has been widely used in digital twin, and has achieved excellent efficiency. A blockchain-based digital twin framework, named Trusted Twins for Securing Cyber-Physical Systems, was proposed by Sabah Suhail et al. [101], which demonstrated the feasibility and operability for a production line in the auto- motive industry domain. W. Dong et al. [102] proposed a dual-blockchain framework to improve data cred- ibility integrated into the digital twin network, including an authorization blockchain for control operations security and a data blockchain for data content security Blockchain has a prior potential to solve the security lacunae to put HDT on track. With the decentralization and immutability characteristics of blockchain, HDT can achieve excellent innovation [103]. For another, concerning the growing importance of data privacy and HDT ownership, ethical issues arose. The first problem is how to deal with the HDT and the associated data if an (unexpected) termination of therapy arises. The majority of respondents in De Maeyer and Markopoulos [104], stated that their HDT should be deleted when they die, while others imagined it could be continuously used to promote further optimization and upgrading of the HDT. The second problem is whether a person could be represented by his HDT. The French philosopher Baudellard suggested that the fundamental ambiguity of representation is further exacerbated when an HDT, rather than a natural person, acts on its behalf. More importantly, the HDT threatens the true and false difference. It directly affects the person who is rep- resented by an HDT [105]. Additionally, the ethical issues encountered by virtual digital humans in the metaverse are also reminiscent of the HDT. The reproduced stage of a deceased artist in the form of a virtual human attracted widespread attention: Is the artist remodeling authorized? If there are interests, how should they be distributed? From a moral and ethical point of view, digital resurrection violates the deceased’s dignity and may cause a series of ethical issues. It is necessary to build a metaverse with a worldview and ethical consciousness in which various HDT can live [106].

Social issues

At the initial stage of HDT, researchers pay more attention to the study of HDT core technology and application, however, far too little attention has been paid to the open social issues in the physical world. Thus, we notice the limitation and discuss the open issues related to the development of HDT from three perspectives: HDT social regulation, privacy and ethics, and trust and dependence.

• HDT Social Regulation Due to the widespread application of HDT in industry, healthcare, and other domains, the issue of HDT regulation requires increased attention. Robust governance mechanisms and policies are needed to oversee the ethical control and management of these digital assets [107]. At the same time, some ethical principles in digital cloning may also apply to HDT. HDT continues to raise social regulation issues contrary to human ethics and dignity, as embodied in the United Nations Declaration on Human Cloning [94]. It is notifiable that digital cloning challenges human ethics and social regulation. The European Patent Convention under Article 53(a) denies to grant a patent to inventions that breach public order or morality. The European Court of Justice has confirmed the need to protect human dignity and integrity, which it deems as fundamental rights [108].

• Trust and Dependence We investigate the existing modes of human attitudes toward AI, which may be available for reference to study HDT. On the one hand, the European Commission’s High Level Expert Group adopted the position that human should establish trust relations with AI and cultivate trustworthy AI [109]. On the other hand, Ryan et al. [110] proposed that AI cannot be trusted, because it does not have an emotional state or can be responsible for its behavior with the emotional and normative account requirements of trust. AI meets all the requirements of rational trust, but it is not human trust, but a form of dependence. From the above two opposite views on AI trust, we summarize three points of HDT trust. (i) Multi-modal and multi-source data are used in the construction of HDT, which involves various users’ privacy information. Only by protecting the ethics and privacy rights of users from infringement can make it possible to establish users’ trust in HDT. (ii) Improving computing power can significantly improve the HDT model accuracy. The HDT compute capacity requirements can be approached through High-Performance Computing [111] and Quantum Computing [112]. Cloud/Edge computing can further improve real-time computing by combining distributed computing with central computing [113]. Furthermore, the evaluation method also should be considered, and it is crucial to evaluate the prediction results to judge the classifier error and provide the reliability of the calculated recommendations [20]. (iii) Establish a simple, operable and intelligible interactive interface. Maeyer et al. [114] emphasized the design of human-computer interface should be based on the important premise that users can easily understand and operate. Braun et al. [105] advised to pay more attention to human-computer interface between human and HDT, to strengthen human control over HDT, and to protect people’s freedom from being violated. Fuller et al. [115] recommended that the HDT framework should be explained from the basic level to help users gain a deeper understanding.

Human thinking and cognition issues

Human thinking and cognition are uniquely human characteristics, which greatly help people solve information-processing problems. Most of human information processing process is sub- conscious. Thus, it is not easy to understand people’s thoughts clearly. Aiming to make machines serve people better and improve their work efficiency, researchers have devoted themselves to the study of machine intelligence. Originally, Turing et al. [116] proposed imitating human thinking through computer programs. Turing machine was first used for mathematicians processing information. It imitated human thinking, neither bionic artifacts nor imitating biological structures. After Turing, various researchers began to study whether programs could ever be truly intelligent, creative or even conscious [117]. These studies have identified it is necessary to consider the cognitive simulation block when designing HDT system, to analyze the results of human behavior and performance in the process of processing information and improve prediction accuracy.

Cognitive simulation studies human thought and cognitive processes, both shared and individual, and the psychological content, representations, and constraints that determine the limits and forms of these processes. It analyzes how people perform intelligent tasks and uses this information to design novel technical solutions. In order to describe the interaction between people and machines, Saariluoma et al. [118] proposed a cognitive simulation method that uses human information processes to design intelligent systems. The system analyzed human information processes, such as perception and reasoning, to imitate how human process information, and designed intelligent HDT. Subsequently, they proposed a detailed HDT process model designed by using the cognitive modeling method [119]. They aimed to propose a practical method to design intelligent information processes from a cognitive simulation approach [120].

Conclusions

This paper has shed more light on a cutting-edge field of study known as HDT. In this paper, we briefly introduce digital twin and HDT, and explore their similarities and differences. We conduct a thorough review of the literature on HDT research. Along with analyzing underlying technologies, we propose the HDT architecture with the core functional blocks based on the literature. Additionally, we review recent developments in sensing, human organ modelling, and modelling human behaviour for HTD core technologies. We also present the application of HDT in the domains of healthcare, daily life and industry. Finally, we discuss various trends and challenges of the future HDT development.

A practical approach for generic HDT modelling technologies build HDT models that are appropriate for anyone regardless of location, age, gender, or other characteristics. The design of generic model is intriguing and valuable to be usefully explored in further research. Study certainly contributes to our understanding of the HDT concept, state of the art technologies, applications and challenges. The rapid development of HDT will promote the further development of its core technologies, and its application will be extended to more innovative domains. The open issues and challenges covered in this study show that HDT is a promising area for future research.