An important part of the project consists on comparing a performance with a score: The matching of differences between them will lead us to errors detection.
Such systems have been implemented for other purpose, mainly for real-time and interactive music. Known as score-following systems, their goal is to synchronise live instrumental music with computer activated samples, what is in fact very close to our goal: to activate samples, we need to know exactly when and how the expected note was played. These data could be compared to the expected ones in order to detect potential errors.
Score following for jMax - IRCAM, centre Pompidou (Paris, France)
jMax is a visual programming environment for interactive real-time music and multimedia, developped at IRCAM.
For our purpose, the most important feature of jMax among all the different packages which have been so far implemented is the Score Following Package.
Based on Hidden Markov Models, this package is being implemented for 2 years in the real-time system team at IRCAM. The score is modeled as a markov chain, with 2 states for each note or chord: One state for the right note (normal state, or n-state), one state for anything but the right note (ghost state, or g-state), and transitions between states, with their own probabilities, as described in [OD01], and shown on figure 1.1 (see [Rab89] for more informations about Hidden Markov Models).
This states succession is very interesting for our purpose: if we know which state has been reached, we have the possibility to know where an error occured... Furthermore, it is possible to extend this model: type of used datas, other parameters, etc.
Two objects have been implemented for score following: suivimidi and suiviaudio, adapted w.r.t. to MIDI and audio input. For both, played flow is considered as the observation and matching is done with a classical decoding using Vitterby algorithm (see [ALB99]). Improvements have been made by using temporal parameters in HMM ([Mou01]), and by creating a new model for polyphonic music ([Mat02]).
It would be a good solution to use jMax and its Score Following Pqckage for the project. The best way to do it would be to implement a Virtual Music Teacher package in jMax. The main object of this package would have as input the output of the Score Follower.
