Cambridge project reconstructs centuries-old music with machine learning

A Cambridge project has reconstructed and enabled a new performance of a medieval chant that had been partly obscured for centuries. The work combined multispectral imaging, machine learning and musicological expertise to recover notation from a damaged manuscript.
- A Cambridge project reconstructed the medieval chant <em>O Felix Mortale</em>.
- The work combined multispectral imaging, machine learning and musicological expertise.
- The damaged manuscript contains three 13th century chants with two voices.
- The CHARM toolbox is freely available and can process whole manuscripts.
- Future work will test the methods on broader document types and case studies.
Recovering music from a damaged manuscript
The project focused on historical sheet music whose notation had become illegible through physical or chemical degradation. It was based at the Cambridge Image Analysis group and involved the Cambridge University Library, its Cultural Heritage Imaging Laboratory, the Fitzwilliam Museum, the Gonville & Caius Library and the Department of Music.
One central case study was an anonymous fragment held by Gonville & Caius College Cambridge. The vertically torn fragment contains three 13th century chants written for two voices, an example of early polyphony. One part remains legible, while damage to the other left sections of the notation unresolved.
The fragment had been partially transcribed in 1970, but blank sections remained because of the severe degradation. The project concentrated on the second chant, O Felix Mortale, which was mainly written on the damaged part. Work with Dr. Johanna-Pauline Thöne of Maynooth University Ireland and PhD candidate Jack Stebbing of the University of Cambridge enabled a full transcription. The chant was then performed in the chapel of Gonville & Caius College, probably for the first time in centuries.
Combining imaging with expert-guided machine learning
The researchers used multispectral imaging, a technique that captures information across multiple wavelength bands, including wavelengths beyond visible light such as infrared and ultraviolet. These different bands can reveal features that are not visible to the naked eye or through conventional visible-light imaging.
Two imaging systems were used. A portable multispectral camera borrowed from the Fitzwilliam Museum provides 8 frequency bands. The in-house system at the Cultural Heritage Imaging Laboratory uses 49 band pass filters and captures images at higher resolution, but it cannot be moved, making it unsuitable for manuscripts that are too valuable or fragile to transport.
The team first applied standard processing methods and then developed representations designed with feedback from musicologists. The methods sought to suppress bleed-through from the reverse side of folios and material noise while highlighting the signal from the notation. Collaborating experts created annotation masks by labelling small regions according to important signal information. Those annotations helped train the system to produce visualizations focused on the features most relevant to reading the damaged music.
A toolbox for degraded documents
The researchers say the enhanced images enabled musicologists to reconstruct and transcribe notation in most of the project’s case studies. The results also showed the importance of the imaging hardware. Data from the Cultural Heritage Imaging Laboratory’s in-house device provided more detailed information in the case where both systems were used, leading to higher reconstruction quality. The portable system nevertheless produced helpful results when manuscripts could not be moved.
Following the initial research, the team developed the CHARM toolbox with a software engineer through follow-on funding from the impact Accelerator scheme. The toolbox is now freely available and shows each processing step. Although it was developed through work on degraded music notation, it is designed to enhance legibility in any imaged document and can process a whole manuscript at once.
What the project means for cultural heritage
The project demonstrates how damaged archival material can be studied without relying only on visible-light images or manual interpretation. Its approach connects non-invasive image acquisition with automated processing and the judgement of specialists who understand the documents being recovered.
Future work will apply the methods in CHARM to a broader range of case studies and document types. The stated aim is to evaluate their applicability, robustness and transferability beyond the materials examined in this project. For libraries, museums and researchers, the freely available toolbox offers a way to inspect processing decisions while exploring whether previously unreadable content can be made legible.
Cambridge Accelerate Programme for Scientific Discovery: How can we…bring together machine learning and expert knowledge to reconstruct music unheard for centuries?

