AI and music festival brings together musicians, artists and scientists

by Universitat Politècnica de Catalunya

04 November 2021

Laboratory of applied investigation

One of the main initiatives of the AI and Music S+T+ARTS Festival, and one which transcended the event, was the creation of Thinking Lab, an applied research lab where participants can explore the interrelation between AI and musical creativity, and, ultimately, the interaction between people and machines. The Thinking Lab is led by the Intelligent Data Science and Artificial Intelligence Research Centre (IDEAI-UPC) and the Centre for Image and Multimedia Technology (CITM), both at UPC, and coordinated together with Sónar.

In this space of debate and co-creation, which has been in place since April, scientists, artists and other professionals exchange experiences and knowledge that has contributed to some of the presentations and performances at the festival. The conclusions will be published at a later date in a white paper about music and AI. During the festival, the Thinking Lab continued to be a space for debate and experimentation, where participants discussed the principal challenges and practices of the application of AI in the context of musical activity.

Co-creation groups

Sónar and UPC have facilitated three co-creation groups between artists and AI specialists. These co-creations are spaces of interchange between musicians and AI experts dedicated to the development of this field through the use of AI techniques. Each co-creation works on a specific line of investigation.

Of the festival performances, some highlighted developments made specifically by the co-creation teams. These collaborations included:

The inaugural concert of the festival, held in the Barcelona auditorium, led by the pianist and composer Marco Mezquida, who conceived his performance “Piano + AI” together with IDEAI-UPC researchers Ivan Paz and Philippe Salembier, in collaboration with the Escola Superior de Música de Catalunya (ESMUC). This is a live conversation between piano and AI, using audio analysis in real time and digital sound synthesis.

An audiovisual performance “Engendered otherness. A symbiotic AI dance ensemble”, by Hamill Industries and choreographed by Kiani del Valle. This performance applied AI to the world of dance, and was created in collaboration with the singer-songwriter Stefano Rosso and the UPC researchers Martí de Castro and Javier Ruiz. The show used computer vision and movement capture to transform choreography interpreted in real time by a dance ensemble formed by Kiani and her synthetic dancers.

An interactive performance by the DJ and producer Awwz, with audience participation, that was made possible by the collaboration of IDEAI researchers Ioannis Tsiamas, Mireia de Gracia, Casimiro Pio and Marta Ruiz-Costajussà. The performance brought machine learning to the world of DJing and invited the public to directly participate. The AI technology was trained with musical databases based on emojis and texts, and can to pick up trends in the general audience mood and translate them into song suggestions. At the event, the comments from the audience on social media were fed into the system, which then adapted the songs that were played by the DJ in response to the public mood.

Two hackathons

The festival began with two hackathons that took place in parallel at the North campus of UPC. These were designed to share knowledge, experiences, ideas and inspiration, around education (EducationalHack) and live coding (LiveCoding Hack). You can watch a presentation of the outcomes of the hackathons below: