Recognition of Handwritten Music Scores and Pianola Rolls

Overview

OCR / HTR Digitization of collections Tools for data mining/extraction Musical Heritage

The automatic recognition of printed music scores can be considered quite mature, as many commercial OMR (Optical Music Recognition) products exist. However, the existing solutions are still not able to deal with other kind of music documents, such as old handwritten music scores or pianola rolls.

Researchers in computer vision and musicology from the Computer Vision Center and the Universitat Autònoma de Barcelona are collaborating towards the preservation, cataloguing and access to the contents of historical music documents in archives, libraries and museums. For example, the Archive of the Liceu’s Opera House, the Gran Teatre del Liceu, is a very important artistical and cultural heritage that is starting to be digitized and cataloged (http://www.bib.uab.cat/human/arxiusocietatliceu/publiques/indexeng.php). Among the technical, artistical and administrative documentation, the archive contains more than 600 music works corresponding to the different representations of Operas, Concerts and Ballets. It is estimated that, in total, there are more than 350.000 handwritten music pages.

HANDWRITTEN MUSIC SCORES.

A first research line consists in the recognition of old handwritten music scores. The objective of Optical Music Recognition is to understand and interpret the music information contained in music score images and convert them into an editable file format (ex. MIDI, MusicXML, MEI). Since the recognition of historical manuscripts is difficult due to paper degradation and the differences in the handwriting styles, we are exploring deep learning architectures together with synthetic data generation and transfer learning for this task.

A second research line consists in the alignment of handwritten music scores. Since there are different versions of the same music work in archives of Opera Theaters, the goal is to compare them in order to study these variations from the musicological point of view. For this purpose, we have explored the adaptation of sequence alignment techniques to visually compare two versions of the same music work and highlight the sections that contain such differences. In addition, this method could be also used to compare the printed and handwritten version of the same music work, and consequently, validate the transcription of these music manuscripts.

Finally, we have also focused on the identification of the writer of a music score by analysing and comparing its handwriting style with a known set of writers. Thus, this technique can help scholars to determine the authorship of an anonymous music work. Indeed, this research line aims to speed up the classification and cataloguing of such amount of music works.

PLAYER PIANO ROLLS.

Player piano rolls have also a great interest from the historical point of view. They are an impressive witness for both music interpretation and reception during the first decades of the 20th century. Indeed, some of these rolls were created from a real piano recording from a pianist, so they can be considered as historical recordings because they contain the additional perforations to playback the tempo and dynamics recorded by the pianist.

Since the experts estimate that, only in Spain, there are more than 10.000 pianola rolls, our research has also focused on the preservation and recognition of player piano rolls. Our developed system is able to preserve a digital copy of the pianola roll and generate its correspondent MIDI file automatically (https://www.youtube.com/watch?v=vmTryKCM_e8&feature=youtu.be).

Project partners

With financial support by