Virtual Music Teacher

The system will have to compare a score and a performance of the same piece. We need to define exactely what we mean with "score" : which data will be recorded in this score, which file type will it be, and how we can build a score from a given piece.

I - Needed data

We have defined which kind of errors will be detected. We must now define all data we need to reach this goal.

I.1 - Note errors

Essential datum is the pitch of the note. We will use the MIDI definition of the pitch : an integer value between 0 and 127, centered on C3 = pitch 63.
But we also need the position of the note to determine which kind of note error we have (wrong note, skip note or extra note. See "evaluation criteria" for more details). Position will be defined as a three values structure : [measure number, beat number, beat fraction]

I.2 - Rhythm and tempo errors

We need a tempo parameter, which could be different from a part of the piece to another (precise tempo change, accelerando...). We will take tempo values between 0 and 256, which will correspond to the number of beats pro minute.
Note position and note duration are also needed for start time errors, duration errors and lack of regularity errors. Duration will be the "relative" duration of the note, i.e. the note value (1, ½, ¼, etc...) linked to the current tempo.

I.3 - Dynamics and phrasing errors

We need for each note its velocity to find accentuation errors. We will use the MIDI definition of the velocity : an integer value between 0 and 127.

For general dynamics errors, a global dynamic parameter would be better (ex : mf from meas.21 to meas.27, f from meas.27 to meas.28, etc.). Values will be taken between :
[ppp, pp, p, mp, mf, f, ff, fff]

For the staccato/legato errors, we need a special parameter which would allow to indicate in the score how the part shall be played. Value of this parameter will be chosen between three values : [staccato, normal/undefined, legato]

I.4 - Thresholds and user defined parameters

We said in the "evaluation criteria definition" that thresholds could be used to determine which tolerance level would be used for each piece. Since these threshold values are depending on the piece, the can be linked to the score. We have for the moment 2 parameters : duration tolerance parameter and accentuation parameter.
As we have said for the tempo parameter, those threshold values can change from a part of the piece to another (we may want to indicate in the score a "recitative" part, where the duration tolerance would be very high, for example for 2 or 3 measure, and then come back to a low tolerance).
The duration tolerance will be defined as a number between 0 and 1, corresponding to the tolerance percentage. The accentuation parameter will be a binary parameter : 1 if accentuated, 0 if not.

II - jMax score object

The jMax Score Following object uses a jMax score object as input. This object is a derivation of a MIDI file : mainly used data are, for each note : pitch, duration, time and velocity.

If we want to use the Score Following Package without having to make changes in it, a part of our score shall be a jMax score. It is possible to build a jMax score from a MIDI file. But we need other data : the jMax score will not be enough to store all the information we want. We will have to join another file to it in order to have the complete score we need.

III - joint file for other data

In this file, we will have to store : note value, note position, note accentuation, tempo, time signature, dynamic, legato parameter and duration tolerance.

III.1 - Note fields

Value, position, and accentuation are linked to each note. One part of our file will be the list of all notes with those parameters, like :
Index value position accent
0001   1/4   1:1:00   0
0002   1/8   1:2:00   0
0003   1/8   1:2:48   1
0004   1/4   1:3:00   0
...

III.2 - Parameters - controlers

As in a MIDI file, we can set parameter values with "controlers" : one controler type for each parameter to be set. For example :
Type        value     position
Time sig.   3:4     1:1:00
Tempo     120     1:1:00
Dur. Tol   0.2     1:1:00
Dynamic   mf      1:1:00
Legato      1       1:1:00
Dynamic   f        3:2:00
Dynamic   mf     4:1:00
...

III.3 - File structure

We can fill our file as a sequence : notes and controlers are placed in the order they shall appear in the score. We can define numeric values for each type :
0 : note
1 : tempo
2 : time signature
3 : legato parameter
4 : duration tolerance
5 : dynamic

First read number will indicate the data type : if we have 0, we know we will have to read successively index, value, position and accent.
The problem we will have here is that the score won't be very simple to fill for unexperienced users… In the future, it would be a good idea to create a tool to fill it. But let's forget it for the moment. Our linked file will look like :

2   3:4   1:1:00
1   120   1:1:00
4   0.2   1:1:00
5   mf   1:1:00
3   1   1:1:00
0   0001   1/4   1:1:00 0
0   0002   1/8   1:2:00 0
0   0003   1/8   1:2:48 1
0   0004   1/4   1:3:00 0
5   f   2:1:00
0   0005   1/4   2:1:00 1
0   0006   1/2   2:2:00 0
3   0   2:3:00
0   0007   1/8   2:3:00 0
0   0008   1/8   2:3:48 0
3   0   2:3:00
5   mf   2:1:00
0   0009   1/2   3:1:00 0

IV - Data overview

IV.1 - Used data

IV.2 - Data values - more details

Note value : defined as a fraction of a 4/4 measure.

Note position : 3 values structure : [measure number, beat number, beat fraction]. The beat number is linked to time signature. Beat fraction is an integer between 0 and 95 (48 is the center between 2 beats. 96 values is the best solution to have a precise value for 4-multiple and 3-multiple note values.

Time signature : 2 values, corresponding to the number of beats in the measure, and the note value that fills a beat (musical definition of time signature)

Dynamic : integer value between 0 and 7, with the following correspondance : 0 for ppp, 1 for pp, .... 6 for ff, 7 for fff

Legato : integer value : 0 for staccato, 1 for normal/undefined, 2 for legato

Duration tolerance : numerical value between 0 and 1 : 0.5 will mean we will detect no error if note is played between (t - note duration * 0.5) and (t + note duration * 0.5)