Music Tech Monday: Soloists can receive musical accompaniment with an app

During the Acoustical Society of America’s 167th meeting from May 5 to 9 in Providence, Ri, an innovative computer program was unveiled that enables soloists to practice and perform with a full digital backup band. Geared toward musicians who would not normally be in the position to play along with accompaniment, this system called Informatics Philharmonic will help musicians progress with their technique.

Developed by Christopher Raphael, chair of Computer Science in the School of Informatics and Computing at Indiana University, he stated that “classical musicians spend untold hours learning to play the solo literature featuring their instrument, but very few ever perform this music with the accompanying ensemble. The reason is it takes many players to make an orchestra, but only one to be a soloist.” As a former professional oboe player he used his musical expertise and computer science tech-savviness to develop a Bayesian Belief Network that acts as “a simple model for musical timing that understands the nominal note values from the score and what they imply about duration, and the way tempo changes fluidly in a performance.”

The Informatics Philharmonic system has 3 modules — one for computing a musical score that uses a Markov model algorithm; another to produce output audio through phase-vocoding an already existing audio track; the last for linking all of the audio together and guesstimating the timing with a Kalman-Filter module. This technology can listen, predict, and then play the track. The module for listening uses variable detection latency between 30 and 90 ms. The prediction phase uses a Gaussian graphic model that is comparable to a Kalman Filter. The play mode creates output audio through phase-vocoding from an accompaniment recording. 

The computer program intuitively adapts to the soloist’s style and playing technique. It can identify the aspects of different musical genres; for example, jazz uses many musical call and response sequences and can digitally realize jazz is being played and prepare itself for improvisation.

A similar downloadable program, called Cadenza, is also available for soloists to play with a virtual orchestra. Cadenza sprouted from the bright minds of the Harvard Innovation Lab in Boston. So far, this program is only available for Macs. The company is currently working on producing an app for smartphones and tablets. 

Story via Communications of the ACM