Music style transfer
| Title | Music style transfer |
|---|---|
| Summary | Develop a system that receives a piece of music in one genre and changes/transfers its style into another genre, using machine learning algorithms. |
| Keywords | Deep Learning, Neural Networks, music style, genre, domain, transfer |
| TimeFrame | Fall 2021 |
| References | [[References::[1] Dai, Shuqi, Zheng Zhang, and Gus G. Xia. "Music style transfer: A position paper." arXiv preprint arXiv:1803.06841(2018).
[2] Gatys, Leon A., Alexander S. Ecker, and Matthias Bethge. "Image style transfer using convolutional neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. [3] Fazıl Say, Alla Turca Jazz. https://youtu.be/WWftABQV4Wk [4] Taro Hakase et al., BWV 1043 Jazz. https://youtu.be/2OiBX07ImA0 [5] Hung, Yun-Ning, et al. "Musical composition style transfer via disentangled timbre representations." arXiv preprint arXiv:1905.13567 (2019). [6] Brunner, Gino, et al. "Symbolic music genre transfer with CycleGAN." 2018 IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI). IEEE, 2018. [7] Brunner, Gino, et al. "MIDI-VAE: Modeling dynamics and instrumentation of music with applications to style transfer." arXiv preprint arXiv:1809.07600 (2018).]] |
| Prerequisites | Artificial Intelligence, Data Mining, and Learning Systems courses; good knowledge of machine learning and neural networks; programming skills for implementing machine learning algorithms; interests in music (of many genres) |
| Author | |
| Supervisor | Peyman Mashhadi, Yuantao Fan |
| Level | Master |
| Status | Open |
Music style transfer [1, 5, 6, 7] can be considered as the counterpart of image style transfer [2]. The aim of this thesis project is to develop a system that, given a piece of music in one genre, changes its style into another genre. For example, this transition can be from classical to jazz, e.g. Alla Turca Jazz by Fazıl Say [3], and Bach Jazz such as BWV.1043 by Taro Hakase [4]. The specific type of music style transfer, in this work, is Composition Style Transfer [1, 5], i.e. preserving the identifiable melody contour of the input pieces, while altering some other score features in a meaningful way, i.e. interpretation in other music style/genre. One of the challenges when it comes to study/research music from a scientific perspective is that music is, by nature, very subjective and it is difficult to evaluate the results objectively. In this work, the genre of the music pieces will be evaluated using a trained genre classifier, which discriminates different genres from each other.
One approach to address music style transfer is using adversarial deep networks [6]. A generator takes a piece of music in a specific genre as input and tries to generate the transferred version of the same piece in another genre. A discriminator then tries to discern between generated music and real music. This way through adversarial training, the generator will hopefully end up generating a genre-transferred version of the inputs. The generated genre-transferred music can be evaluated using a genre classifier. The mentioned architecture is one way of doing a style transfer.
