Internet of things and ensemble learning-based mental and physical health monitoring for elderly care

The Evolving Landscape of Healthcare 4.0

The transition of conventional healthcare systems to a data-driven and patient-centric Healthcare 4.0 paradigm has initiated a pragmatic change in health statistics. The adaption of cutting-edge technologies like the internet of things (IoT) and body area networks, driven by sophisticated data-driven algorithms like machine learning and deep learning, has supported this transformative shift in healthcare systems equipped with smart devices such as wearable sensors, medical gadgets, and other connected technologies.

This modern healthcare ecosystem relies heavily on cross-organizational services that promote personalized assistance and support using big data analytics. The enabling technologies of data analytics and recommender systems hold vast research potential in the context of healthcare systems. Due to this transformation, personalized recommendations can now be provided to patients suffering from various diseases, considering different verticals like staging, severity, risks involved, and feasible assistance.

Healthcare can be further bifurcated into physical and mental healthcare. The past decade has witnessed a shift from physical to mental healthcare, with Alzheimer’s disease becoming one of the most prevalent mental health concerns among the elderly population. This is supported by the Alzheimer’s Association report (2018), which depicts a 123% growth in Alzheimer’s-related deaths compared to a decline in deaths from heart disease and stroke. By 2050, the number of Alzheimer’s patients in the USA alone is projected to surpass 15 million, with nearly two new cases expected to develop every minute.

The Need for Reliable Technological Solutions

The increase in Alzheimer’s-related deaths is attributed to the late detection of the disease, making it difficult to reverse the degradation, and the insufficient support mechanisms for such patients. These facts necessitate the need for reliable technological solutions to detect, track, and provide assistance to people suffering from Alzheimer’s disease, improving their quality of life.

Various existing proposals have tried to devise different solutions related to the prediction or detection of Alzheimer’s disease. These approaches use either medical data like MRI or sensor data like gait sensors and motion sensors. However, the artificial intelligence-driven algorithms based on deep learning and cognitive intelligence can help recognize, detect, or predict the early symptoms of Alzheimer’s disease more efficiently.

To overcome the limitations of existing proposals, we propose a deep learning-based Internet of Health (IoH) framework, called DeTrAs, for assisting Alzheimer’s patients. DeTrAs works in three phases:

1. A recurrent neural network (RNN)-based Alzheimer’s prediction scheme using sensory movement data.
2. An ensemble approach for abnormality tracking for Alzheimer’s patients, comprising a convolutional neural network (CNN)-based emotion detection scheme and a timestamp window-based natural language processing scheme.
3. An IoT-based assistance mechanism for Alzheimer’s patients.

The DeTrAs Framework: A Layered IoH Ecosystem

The proposed DeTrAs framework is built upon a layered architecture for the IoH ecosystem, as shown in Figure 1. The architecture comprises five layers:

1. User Layer: Consists of human beings, both Alzheimer’s and non-Alzheimer’s, at different locations in a smart environment.

2. IoT/Sensor Layer: Consists of various IoT sensors used to sense heterogeneous data, including auditory, visual, motion, and medical data.

3. Fog Layer: The data sensed by the sensors are forwarded to the fog layer for computational tasks like data processing and analysis, closer to the users’ locations.

4. Core Network Layer: Comprises core network infrastructure responsible for data transmission from the fog layer to the cloud layer.

5. Cloud Layer: The sensed data are finally forwarded to the cloud-based storage for future analysis. If the fog devices lack sufficient computing power, the computation is passed to the cloud servers.

The deployed sensors are divided into two categories: master (α) and slave (β) sensors. A trigger-based activation mechanism is used to handle these sensors for improved energy efficiency. The master sensors, such as gait sensors, track the patient’s movement and trigger the activation of slave sensors when required, such as zenith sensors, audio sensors, and medical sensors.

Phase 1: Alzheimer’s Prediction using RNN

The first phase of DeTrAs involves identifying whether the subject under supervision is an Alzheimer’s patient or not. For this purpose, a recurrent neural network (RNN) model is used. RNN is a popular deep learning approach that can learn representations of data using abstraction.

The RNN model is trained using the input data sensed by the sensory system to predict whether the patient has Alzheimer’s disease. The model assigns different weights to the parameters in multiple hidden layers while performing supervised learning to train the model. The trained model is then installed on the computational fog devices, which receive the input from the master sensors (gait sensors) for movement monitoring.

If a patient is detected as positive, the next phase of DeTrAs is triggered for behavior abnormality detection using multiple sensors.

Phase 2: Abnormality Tracking using Ensemble Approach

The second phase of DeTrAs involves an ensemble approach for tracking abnormality in Alzheimer’s patients. This approach considers two cases: video data (from zenith sensors) and audio data (from auditory sensors).

Video Data Analysis:
The video data from the zenith sensors are converted to time-variant images, which are fed into a pre-trained CNN model to detect the patient’s emotion. The CNN model comprises an input layer, hidden layers (convolutional, pooling, and fully connected), and an output layer. The output of this phase is represented as SA.

Audio Data Analysis:
The audio data collected from the auditory sensors are used to validate the partial alarm generated in the video data analysis. A timestamp window mechanism is used to filter the audio stream, and a speech-to-text synthesizer is used to convert the speech to text. Then, preprocessing steps like tokenization, stop words removal, and lemmatization are performed on the textual data. A naive Bayes classifier is used to identify sentences representing questions triggered to extract information about the patient’s identity. The analytical result of this phase is represented as SB.

The ensemble approach combines the outputs of the video and audio data analysis (SA and SB) to reduce the number of incorrect detections and reach a firm conclusion about the patient’s abnormal behavior.

Phase 3: IoT-Based Assistance Mechanism

The third phase of DeTrAs involves the use of IoT devices to provide different types of assistance to the Alzheimer’s patient when a trigger is generated by Phase 2. The assistance can be classified into three categories:

1. Corrective Assistance: Informing the patient about incorrect decisions that need to be corrected or reverted.
2. Reinforcing Assistance: Reiterating the process for an incorrectly performed action in the past to help the patient perform it correctly.
3. Supportive Assistance: Providing assistance based on the requirement triggered by the Alzheimer’s patient using an application.

The IoT-based assistance mechanism is triggered based on inputs from wearable and zenith sensors to detect the patient’s activity and provide the appropriate assistance, such as meal tracking, bath activity, medicine intake, hydration monitoring, visitor recognition, task scheduling, and cognitive stimulation therapy.

Evaluating the DeTrAs Framework

The DeTrAs framework has been evaluated using different datasets for the three phases:

1. Phase 1 – Alzheimer’s Prediction: The Daphnet dataset was used to train the RNN model, and its performance was compared with other classification algorithms like Bayes Net, naive Bayes, logistic regression, and decision tree. The RNN model achieved the highest precision, recall, and F-score, outperforming the other methods.

2. Phase 2 – Abnormality Tracking: For video data analysis, the CNN model was evaluated using the MMI dataset and compared with an SVM classifier, showing superior emotion detection accuracy. For audio data analysis, the naive Bayes classifier performed better than the decision tree approach in detecting abnormality from the text data.

3. Phase 3 – IoT-Based Assistance: The proposed IoT-based assistance mechanism was evaluated qualitatively based on the different verticals and use cases to demonstrate its effectiveness in improving the quality of life for Alzheimer’s patients.

The evaluation results showcase the superiority of the DeTrAs framework in accurately predicting Alzheimer’s disease, tracking abnormal behavior, and providing personalized IoT-based assistance to improve the patient’s quality of life.

Conclusion

This paper presented the DeTrAs framework, a deep learning-based IoH ecosystem for assisting Alzheimer’s patients. DeTrAs works in three phases: Alzheimer’s prediction using RNN, abnormality tracking using an ensemble approach, and IoT-based assistance mechanism. The evaluation results demonstrate the effectiveness of DeTrAs in accurately detecting Alzheimer’s, tracking abnormal behavior, and providing personalized assistance to improve the patient’s quality of life.

The DeTrAs framework amalgamates different existing techniques to provide an ecosystem that can assist Alzheimer’s patients in their day-to-day activities. The layered architecture of the IoH ecosystem, the trigger-based sensor activation model, and the ensemble approach for abnormality tracking are the key innovations of the DeTrAs framework. The integration of deep learning and cognitive intelligence with IoT-based assistance mechanism enhances the framework’s ability to recognize, detect, and predict the early symptoms of Alzheimer’s disease efficiently, addressing the limitations of existing proposals.

The DeTrAs framework can be further enhanced using ambient intelligence and game-theoretic approaches to achieve Nash equilibrium and increase its performance. The framework’s ability to provide personalized assistance and ensure privacy and security can play a crucial role in the widespread adoption of IoT-based solutions for elderly care, especially in the context of Alzheimer’s disease management.