Thesis Proposal by BAO Chunhui | Generating Music with Emotions

About The Talk

In this thesis, I focus on the music generation conditional on human emotions such as sad and happy. As there are no existing music datasets annotated with emotion labels, conditional generation is impossible. I thus plan to build a new dataset consisting of the triplets of lyric, melody and emotion, without requiring any manual annotations. I plan to utilize an automated emotion recognition model (based on the BERT trained on GoEmotions) to ``label'' the music according to the emotions recognized from its lyrics. I then train the model of generating emotional music and call my method Emotional Lyric and Melody Generator (ELMG).
Specifically, ELMG is consisted of three modules:
1) an encoder-decoder model trained end-to-end for generating lyric and melody;
2) a music emotion classifier trained on labelled data (our dataset); and
3) a modified beam search algorithm that guides the music generation process by incorporating the music emotion classifier.
I plan to conduct subjective and objective evaluations on the generated music to demonstrate that ELMG is capable of generating tuneful lyric and melody with specific emotions.

Speaker Biography

Chunhui Bao received the B.S. degree in Computer Science and Technology from Sichuan University, Chengdu, China, in 2019. He is currently working toward the MSc degree in Information Systems with the School of Computing and Information Systems, Singapore Management University, Singapore. His research interests include natural language processing, music generation and deep learning.