People of ACM - Cynthia Liem
October 18, 2016
How does your background as a concert pianist inform your approach to music information retrieval research?
Clearly, it was my passion about music that led me to the research field. Problems I started wondering about as a musician have strongly been shaping my interests and research questions.
During my PhD, I worked on digital analysis techniques applied to interpretational aspects of music (comparative performance characteristics, and connections between music and non-musical media). This came forth from my dissatisfaction about digital services not being capable of taking such aspects into account. The research community is getting increasingly better at automatically describing and annotating music items, but this very much still is geared at library-style knowledge and physical properties (e.g., the artist playing, or the notes that are being played).
At the same time, what makes music interesting or useful to us often comes forth from extra interpretation layers on top of these basic properties. This requires us to look beyond single audio files, and consider music in the context of larger corpora and varying use contexts. While it may sound strange for a computer scientist to be interested in artistic and connotative aspects, they will strongly influence how music items ultimately will be consumed and used. The nice thing is that you can look at this from various disciplinary perspectives. As computer scientists, we can employ data analysis approaches and consider both the signal and its consumption information. But music also is studied under very different disciplinary angles in the humanities and social sciences. It's extremely interesting to see how the various disciplines approach it, and to think of how to use that when considering digital consumption.
Even outside of the academic realm, somehow, music is a phenomenon intriguing us all, with which we willingly engage daily under various situations. That is why I'd like to help enabling better and more comprehensive access to digital music. There is so much around to discover! But we really need algorithms to help us with that, and as a picky musician, I'd like those algorithms to truly help me to discover new repertoire and perspectives.
You have mentioned that "music is multimedia." Could you explain this?
The concept of music is projected into the world in different formats. As a classical musician, I work a lot with sheet music: scores that contain symbolic information. Yet I need to convert that information into audio when conveying it to an audience. In case of live performance settings, my audience may be especially interested in seeing me on the stage while I'm performing this conversion procedure. Music is a very multimodal phenomenon, as much as (or even more so than) videos. I believe this makes it a very interesting topic to feature and discuss at multimedia venues.
Did your computer science research background also influence your musicianship?
Surprisingly, it did. From 2013 to 2016, I was a main initiator and co-coordinator of the European collaborative PHENICX project, which focused on digital enrichment opportunities for symphonic concert performances. Again, my drive behind this project came forth from my daily ponderings on how performed music could find its way to broader audiences in the best way.
During PHENICX, we realized that the idea of "strategies for reaching out to new audiences" still needed a lot of investigation, also from music institution perspectives. Out of curiosity, in our project, I joined the user-centered design and development initiatives considering various types of potential audiences for our technology, from complete outsiders to seasoned classical music lovers. This made me realize how much we tend to follow basic success formulas in music marketing, that are targeted at a rather narrow, "safe" audience.
With my Magma Duo, I then tried to refute this from the stage. Many people believe contemporary classical music is inaccessible, but my duo believes it actually may be key to engaging new audiences. The "contemporary" sound actually is much closer to what many outsider listeners are familiar with, while an evergreen classic from Mozart or Beethoven may actually seem more foreign and old-fashioned to these ears. I would never have thought of that as strongly if I hadn't joined the PHENICX user studies. And following insights from these, as the Magma Duo, we actively are putting effort in presenting music in engaging ways: we invest a lot of time in personally connecting to our audiences, and sometimes chip in some extra theatrical techniques while doing this.
Your presentation at the upcoming ACM SIGMM Rising Stars Symposium is titled "Purpose and Repurpose: Lessons from the Concert Stage." Will you provide us with a brief primer of your talk?
I will try to cover a lot of what I said in the questions above. What makes music interesting to me is the way in which it is purposed and repurposed. I believe that finding technical solutions to my thoughts on reaching out to broader audiences will be useful to multimedia search and recommendation in a much broader sense, particularly against the backdrop of information overload and filter bubble concerns.
Finally, I also want to briefly cover the various activities I undertook as the dissemination coordinator of PHENICX, in which I tried improving the visibility of our work towards the general public. Regardless of our backgrounds, as we all care in some form about music (and multimedia), and we increasingly access it online, this can be a very powerful application domain to explain the impact of ICT research in a recognizable and concrete way to novices. Now that PHENICX has ended, I still care deeply about dissemination and outreach, and believe this visibility is key to truly making ICT empower and inspire the world in a diverse, inclusive and comprehensive way.
Why is it an exciting time to be in the multimedia field?
Music and multimedia are everywhere around us. We engage with multiple pieces of it daily. And now that a digital-physical blur is increasingly happening, these are shaping our attitudes, opinions, tastes, communications, and world perspectives. This can be very empowering when done in a proper way, yet very limiting when done in a bad way. As the people behind digital organization and filtering—and as a consequence search and recommendation—we as computer scientists carry an enormous responsibility. Can we avoid polarizing and trivializing the world? How do we make sure that digital information can empower people, rather than overpower them? Multimedia has the potential to engage us in impactful, immersive ways. I'd be excited to be helping to make that happen.
Cynthia Liem is an Assistant Professor in the Multimedia Computing Group at the Delft University of Technology in the Netherlands. Her research focuses on music and multimedia information retrieval. Digital cultural resources are among her special interests, and one of her long-term goals has been to improve the accessibility of digital music recordings in multimedia archives. In 2011, she initiated and co-organized the International Workshop on Information Retrieval with User-Centered and Multi-Modal Strategies (MIRUM) at the ACM Multimedia conference, which grew into a dedicated submission area on Music, Speech and Audio in the main conference. Her honors include being selected as a Google European Doctoral Fellow. Recently, she also was chosen to participate in the Rising Stars Symposium at the ACM Multimedia (MM) 2016 conference.
Also trained as a concert pianist, Liem still performs, most notably with the Magma Duo, which won first prize at the international competition "A Feast of Duos."