Emotion detection with data fusion
| dc.contributor.author | Khuzhaniyazova, Maida | |
| dc.contributor.supervisor | Li, Kin Fun | |
| dc.date.accessioned | 2024-11-22T17:09:18Z | |
| dc.date.available | 2024-11-22T17:09:18Z | |
| dc.date.issued | 2024 | |
| dc.degree.department | Department of Electrical and Computer Engineering | |
| dc.degree.level | Master of Engineering MEng | |
| dc.description.abstract | This report explores the performance of three machine learning models — SVM with SGDClassifier, Gradient Boosting, and XGBoost — in detecting emotions using data fusion techniques. Early Fusion was chosen for integrating features due to its simplicity and reliable performance. The study employs the MELD dataset, which combines text, audio, and visual data from over 1,300 dialogues and 13,000 utterances in the “Friends” TV show. This dataset provides a unique multimodal approach to understanding emotions in conversational contexts, making it ideal for emotion recognition tasks. Evaluation metrics for the models included accuracy, F1-score, precision, recall, and AUCROC, calculated over multiple training iterations. By comparing the performance of these models on a comprehensive, multimodal dataset, this study meets the growing demand for accurate emotion detection in conversational AI. XGBoost demonstrated high and consistent performance on the MELD dataset; however, its effectiveness may vary under different conditions or datasets. SVM with SGDClassifier achieved the widest accuracy range, though less stable on nuanced emotions. Gradient Boosting delivered consistently strong AUC-ROC values but required full retraining with each data update, affecting its adaptability. Overall, while XGBoost and SVM delivered good performance, their accuracy was subject to fluctuations across iterations. Gradient Boosting consistently showed strong AUC-ROC values, but its disadvantage is the need to completely retrain the model when new data is added, which reduces efficiency. | |
| dc.description.scholarlevel | Graduate | |
| dc.identifier.uri | https://hdl.handle.net/1828/20799 | |
| dc.language.iso | en | |
| dc.rights | Available to the World Wide Web | |
| dc.subject | emotion detection | |
| dc.subject | data fusion | |
| dc.subject | multimodal emotion recognition | |
| dc.subject | machine learning | |
| dc.subject | neural networks | |
| dc.subject | emotion recognition systems | |
| dc.subject | multimodal fusion | |
| dc.subject | XGBoost | |
| dc.subject | Support Vector Machine (SVM) | |
| dc.subject | Gradient Boosting | |
| dc.subject | emotion classification | |
| dc.subject | incremental learning | |
| dc.subject | performance metrics | |
| dc.title | Emotion detection with data fusion | |
| dc.type | project |