Computer evaluation of musical timbre transfer on drum tracks




Lee, Keon Ju

Journal Title

Journal ISSN

Volume Title



Musical timbre transfer is the task of re-rendering the musical content of a given source using the rendering style of a target sound. The source keeps its musical content, e.g., pitch, microtiming, orchestration, and syncopation. I specifically focus on the task of transferring the style of percussive patterns extracted from polyphonic audio using a MelGAN-VC model [57] by training acoustic properties for each genre. Evaluating audio style transfer is challenging and typically requires user studies. An analytical methodology based on supervised and unsupervised learning including visualization for evaluating musical timbre transfer is proposed. The proposed methodology is used to evaluate the MelGAN-VC model for musical timbre transfer of drum tracks. The method uses audio features to analyze results of the timbre transfer based on classification probability from Random Forest classifier. And K-means algorithm can classify unlabeled instances using audio features and style-transformed results are visualized by t-SNE dimensionality reduction technique, which is helpful for interpreting relations between musical genres and comparing results from the Random Forest classifier.



Audio Style Transfer, Musical Timbre Transfer, GANs, Computer Evaluation of Audio Style Transfer, Methodology, Machine Learning Evaluation Pipelines, AI-assisted Music Analysis, Audio Feature Engineering, Computer Evaluation of Autonomously Creative and Co-creative Music Systems