Trpkoski, Mitko and Nedelkovski, Igor and Bocevska, Andrijana (2026) Speech and Facial Expressions in Digital Avatars for Macedonian using Multimodal Emotion Detection. In: 2026 61st International Scientific Conference on Information, Communication and Energy Systems and Technologies (ICEST), 01-03 July 2026, Nis, Serbia.
Full text not available from this repository.Abstract
This paper presents a multimodal system for automatic emotion detection from Macedonian speech and text, which is designed to generate expressive speech and facial behaviors in digital avatars. Due to the lack of annotated emotional resources for the Macedonian language, digital avatars that are "aware" of emotion representation for languages with small corpora such as Macedonian are less advanced compared to solutions developed for widely used languages such as English and Chinese. The proposed architecture combines an audio-based model using Wav2Vec2 with a text-based classifier that serves as a support and backup mechanism when audio signals are not good enough or are unavailable. The emotional states displayed by the developed system are integrated into the speech synthesis using SSML-based parameter control, resulting in a more natural and expressive Macedonian speech output. In order to assess the reliability of the system itself, three classification measures are proposed to evaluate speech and temporal consistency. The results obtained provide reliable detection of expressed emotions and stable representations, thereby establishing a good foundation for expressive speech synthesis and emotion-driven facial animation in Macedonian.
| Item Type: | Conference or Workshop Item (Paper) |
|---|---|
| Subjects: | Scientific Fields (Frascati) > Natural sciences > Computer and information sciences |
| Divisions: | Faculty of Information and Communication Technologies |
| Depositing User: | Prof. d-r. Andrijana Bocevska |
| Date Deposited: | 22 Aug 2026 19:38 |
| Last Modified: | 22 Aug 2026 19:38 |
| URI: | https://eprints.uklo.edu.mk/id/eprint/11703 |
Actions (login required)
 |
View Item |