Speech emotion recognition
Date Issued
May 2026
Author(s)
Shimitras, Andreas
Advisor
Abstract
Speech Emotion Recognition is an important research area that focuses on identifying the emotional state
of a speaker from speech audio. Emotion in speech appears through acoustic cues such as pitch, inten-
sity, rhythm, articulation, and vocal tone. However, automatic emotion recognition remains challenging
because emotional expression varies between speakers, datasets, and recording conditions.
This thesis investigates the use of a pretrained Wav2Vec2 speech model for audio-based Speech Emotion
Recognition. The proposed system classifies speech recordings into six emotional categories: angry,
disgust, fearful, happy, neutral, and sad. The experiments use three publicly available emotional speech
datasets, RAVDESS, SAVEE, and TESS, which are harmonized into a common six-class label structure.
Audio preprocessing standardizes the recordings to a 16 kHz sampling rate and prepares them as mono
waveform inputs.
The model uses Wav2Vec2ForSequenceClassification as the main architecture and applies Low-Rank
Adaptation for parameter-efficient fine-tuning. This approach allows the model to adapt to the emotion
recognition task while reducing the number of trainable parameters compared with full fine-tuning. The
final model achieves approximately 70% to 71% accuracy and 68% to 69% macro F1-score on the test
set. The results show stronger performance on fearful, neutral, and sad speech, while some confusion
remains between acoustically similar emotions such as angry, disgust, and happy. Additional dataset-
level analysis shows that many angry-class errors are concentrated in the TESS subset, highlighting the
importance of per-dataset evaluation in Speech Emotion Recognition.
Overall, this thesis demonstrates that Wav2Vec2 combined with LoRA provides a practical and repro-
ducible approach for speech-based emotion classification, while also revealing the limitations caused by
dataset variation and acoustic overlap between emotion classes
of a speaker from speech audio. Emotion in speech appears through acoustic cues such as pitch, inten-
sity, rhythm, articulation, and vocal tone. However, automatic emotion recognition remains challenging
because emotional expression varies between speakers, datasets, and recording conditions.
This thesis investigates the use of a pretrained Wav2Vec2 speech model for audio-based Speech Emotion
Recognition. The proposed system classifies speech recordings into six emotional categories: angry,
disgust, fearful, happy, neutral, and sad. The experiments use three publicly available emotional speech
datasets, RAVDESS, SAVEE, and TESS, which are harmonized into a common six-class label structure.
Audio preprocessing standardizes the recordings to a 16 kHz sampling rate and prepares them as mono
waveform inputs.
The model uses Wav2Vec2ForSequenceClassification as the main architecture and applies Low-Rank
Adaptation for parameter-efficient fine-tuning. This approach allows the model to adapt to the emotion
recognition task while reducing the number of trainable parameters compared with full fine-tuning. The
final model achieves approximately 70% to 71% accuracy and 68% to 69% macro F1-score on the test
set. The results show stronger performance on fearful, neutral, and sad speech, while some confusion
remains between acoustically similar emotions such as angry, disgust, and happy. Additional dataset-
level analysis shows that many angry-class errors are concentrated in the TESS subset, highlighting the
importance of per-dataset evaluation in Speech Emotion Recognition.
Overall, this thesis demonstrates that Wav2Vec2 combined with LoRA provides a practical and repro-
ducible approach for speech-based emotion classification, while also revealing the limitations caused by
dataset variation and acoustic overlap between emotion classes
Affiliation
File(s)![Thumbnail Image]()
Name
Andreas_Shimitras.bsc.2026.abstract.pdf
Size
153.07 KB
Format
Adobe PDF
Checksum (MD5)
dc6148a37363f2ef025e1c07e5cefbf3