Recently, Prof. Shang Yuanyuan’s team from the Information Engineering College, CNU has achieved a range of important research results in affective computing. Based on artificial intelligence technology, the team has made significant progress in multi-modal depression recognition, facial visual representation and speech feature research. These research results have been published in authoritative international journals by the CNU, the first execution unit, together with West Virginia University in the United States.

For the main purpose of solving key issues in the field of depression recognition such as facial privacy protection and multi-modal fusion, the research team has successfully proposed a new artificial intelligence model for multi-modal cross-attention information interaction. The paper titled Integrating Deep Facial Priors into Landmarks for Privacy Preserving Multimodal Depression Recognition was published in IEEE's top journal IEEE Transactions on Affective Computing. With an impact factor of 11.9, this journal has remarkable influence in the field of affective computing. It is also another breakthrough after the team published research results in this journal in 2020 and 2022.

In addition, the team also proposed a spatial-temporal attention mechanism fusion method to address core research issues such as facial visual representation related to depressive affective disorders. In doing so, the team has deeply analyzed the visual features of depression and achieved satisfactory results in feature visualization, promoting the research on the interpretability of artificial intelligent models. Furthermore, the team has published a paper titled Spatial-Temporal Attention Network for Depression Recognition from Facial Videos in the Expert Systems With Applications (a top journal in the Q1, CAS). With an impact factor of 8.5, this journal of Elsevier is considered one of the important international journals in the fields of artificial intelligence, operations research, etc.

Focusing on the study of depression-related speech features, the team proposed an information fusion model based on phonetics and multi-acoustic features. This model can be used to deeply explore the temporal dependence and depression acoustic features in speech through multi-scale audio feature decomposition. This achievement not only successfully decouples depression information from identity information, but also significantly improves recognition performance. Hence, the team published papers titled Multi-Feature Deep Supervised Voiceprint Adversarial Network for Depression Recognition from Speech and Spatial-Temporal Feature Network for Speech-Based Depression Recognition respectively in Biomedical Signal Processing and Controland IEEE Transactions on Cognitive and Developmental Systems, two cross-research international authoritative journals.

The above papers were completed by graduate students Pan Yuchen, Han Zhuojin, et al. under the guidance of Shang Yuanyuan, the corresponding author of these papers. As a cutting-edge subject in the field of affective computing, AI-based depression identification has received widespread attention from domestic and foreign scholars. It deserves interdisciplinary research involving artificial intelligence, mathematics, statistics, psychology, etc. The team has kept going deep into this field in recent years. So far, some of their papers have been selected as highly-cited ESI papers, with a certain impact both at home and abroad.

Original text link: https://doi.org/10.1016/j.eswa.2023.121410

Original text link: https://doi.org/10.1109/TCDS.2023.3273614