Music curation with Machine learning

Go Through the Project with Zhi in 3 Minutes

Digital Music Curation, the Workflows and Conflicts

With the emerging of digital music streaming and distribution services, a new profession comes into being - Digital Music Curators - those who create “Romantic Dinner” playlist on Spotify or the music experience for McDonald's all over the world.

Conflict 1

Sheer Amount of Music vs. Limited Human Power

“Facing the sheer amount of music makes me not feeling close to music anymore.
It just feels very industrial - Where's the human behind this? Am I just a machine doing this?”
—— Jenny, Music Curator

Conflict 2

The Fluidity of Music vs. The Solidity of Music Decisions

“Sometimes we organized music in a certain way and then we changed our mind.
But since we have already tagged 70,000 songs, it is just impossible for us to go back and clean up all the music.”
—— Amanda, Music Curator

Conflict 3

Subjective vs. Objective

“It is hard to bring the team on the same page because every person has his own way of understanding music.
In stead of simply saying ‘80s Rock’, there will be tons of meetings to define what exactly we are talking about.”
—— Stefan, Music Curator

Project Positioning in Intelligence Levels and Learning Types

With the goals of “Finding new ways of understanding music” and “Detailing the current music classification model” in mind, me and the machine learning engineers from Soundtrack Your Brand identified Unsupervised Learning as the suitable learning type of the project. Based on our understanding of current progress of Artificial Intelligence, we positioned the project as Augmented Intelligence.
In short, Machine Learning would act like an exoskeleton to human for music curators in this project.

The Mechanism - How it works

This project will be using Clustering technique within the unsupervised machine learning field.
Clustering is a technique of grouping data into clusters based on their similarity. Since it will figure out relationships of different attributes in a data set without any predefined input, it would be able to “Find new dimensions of looking at music” and “Detail the current music classification model”.
However, the decision making process of the algorithm is in a black box, we, as human, can not really understand the true reasons why the algorithm thinks these songs sound similar. That is why we need human music expert to review, valid, identify and fine-tune the clusters, for example, deciding if the cluster has one or more universal themes, e.g., Christmas music, happy songs or R&B. 

2 Challenges in the New Way of Working

Challenge 1

HMW Identify Meanings in the Unknown?

The current workflow is straight forward and the music curators always have a clear goal of what type music to look for in their mind.

However,  with the new work model , the music curators will be presented clusters of music without any idea of what this cluster is.
It is like shooting in the dark or traveling without a direction, which leads to the first challenge of the new work model: How might we identify meaning in the unknown clusters?

Challenge 2

HMW Manage the Music Clusters with Different Objective/Subjective Levels?

The notion of “Making Sense” can have very different subjective/objective levels in music.
With the clustering algorithm, there could be clusters of all kinds.
Theoretically, there can be a music cluster with cat meowing in the background.
How are we going to manage, communicate and make use of all the clusters and tags we are getting?

Co-Creation, Testing and Iteration

With challenges I found from the research on music curators and machine learning, I did a co-creation workshop and multiple early concept testings with music curators, machine learning engineers and the playlist consumers. I was able to gather ideas and validate them through the real users.

A Full Workflow of Applying Machine Learning in Music Curation

The final outcome of this project is a full workflow of applying Machine Learning in music curation, which are communicated in the format of wireframes and a click through prototype. Click HERE to get the high resolution of the wireframes.

Exhibition

As part of the final deliverables, the project was presented in the UID 19 Degree Exhibition. The Exhibition was separated into two stations. 

The first station was to communicate the background of the project, including the four design principles of designing for human raters of machine learning algorithms and how these were realized in the context of music curation. Two posters and a click through video were presented in this part.

The second station was to invite the audience to experience life as a music curator.
In this station, the audience could participate in crowd-curating the music experience for UID19 by adding one to two songs in a collaborative Spotify playlist. By going through how diverse music was added to the playlist, we can again see how different people understand music as well as a certain event. Click HERE to join the Crowd-Curated Experience.
The audience could also go through the simple version of the tagging flow by listening to a machine learning generated playlist and answer a printed questionnaire. The questionnaires were collected and used as the research material of how we understand music. Click HERE to experience a music cluster generated by Machine Learning.

Read More…