Improving the performance of human–computer interaction systems is an essential indicator of aircraft intelligence. To address the limitations of single-modal interaction methods, a multimodal interaction model based on gaze and EEG target selection is proposed using deep learning technology. This model consists of two parts: target classification and intention recognition. The target classification model based on long short-term memory networks is established and trained by combining the eye movement information of the operator. The intention recognition model based on transformers is constructed and trained by combining the operator’s EEG information. In the application scenario of the aircraft radar page system, the highest accuracy of the target classification model is 98%. The intention recognition rate obtained by training the 32-channel EEG information in the intention recognition model is 98.5%, which is higher than other compared models. In addition, we validated the model on a simulated flight platform, and the experimental results show that the proposed multimodal interaction framework outperforms the single gaze interaction in terms of performance.
Information-centric networking (ICN) has gained significant attention due to its in-network caching and named-based routing capabilities. Caching plays a crucial role in managing the increasing network traffic and improving the content delivery efficiency. However, caching faces…
Malware authors apply different techniques of control flow obfuscation, in order to create new malware variants to avoid detection. Existing Siamese neural network (SNN)-based malware detection methods fail to correctly classify different malware families when such obfuscated mal…
Machine learning techniques for network-based intrusion detection systems (NIDS) have advanced considerably over the past decade. Still, improvements are inhibited by handcrafted feature pipelines, isolated public benchmark data, and evaluation procedures that do not reflect real…
The development of robust anomaly-based network detection systems, which are preferred over static signal-based network intrusion, is vital for cybersecurity. The development of a flexible and dynamic security system is required to tackle the new attacks. Current intrusion detect…
Extracting 3D skeletons from point clouds is a challenging task in computer vision. Most existing deep learning methods rely heavily on supervised data requiring extensive manual annotation. Consequently, re-labeling is often necessary for cross-category applications, while the p…
In the digital twin system of mechatronics engineering, the scale and accuracy of models are continually improving. Nevertheless, this growth can hinder real-time interaction and decision-making accuracy within digital twins. The resulting delay impacts the entire system’s reliab…