Transcript The optimisation of multiresolution FFT Parameters

The evaluation and optimisation of
multiresolution FFT Parameters
For use in automatic music transcription algorithms
Automatic music transcription (AMT)
AMT Algorithms
Time & Frequency Resolution
Short Window
Time Resolution Increases
Frequency Resolution Decreases
Long Window
Time Resolution Decreases
Frequency Resolution Increases
Multiresolution FFT (MRFFT)
High Time
Resolution
High Frequency
Resolution
FFT A
FFT B
FcA
FFT C
FcB
FFT D
FcC
FcD
256 Data Length
256 Data Length
Time Freq Plane - Dressler
512 Data
Length
512 Data
Length
Frequency
1024
Data Length
1024
Data Length
2048 Data Length
Time
Window Length - Bin Alignment
 Note-bin alignment – The position of a fundamental
frequency relative to a FFT bin frequency.
Note bin alignment
A 2048 FFT Decomposition of a 376.83Hz Sine Wave
250
FFT Bin Magnitude
200
150
100
50
0
215.33 236.87 258.40 279.93 301.46 323.00 344.53 366.06 387.60 409.13 430.66 452.20 473.73 495.26
FFT Bin (Hz)
Note bin alignment
A 2048 FFT Decomposition of a 366.06Hz Sine Wave
600
FFT Bin Magnitude
500
400
300
200
100
0
215.33 236.87 258.40 279.93 301.46 323.00 344.53 366.06 387.60 409.13 430.66 452.20 473.73 495.26
FFT Bin (Hz)
MRFFT Optimisation
 Cut off frequencies
 Subband FFT Length
 Optimised based on 3 characteristics determined by
window length
 Time Resolution
 Frequency Resolution
 Note Bin Alignment
Scoring
 Calculate score for time, freq, and note-bin alignment in
each subband
 Weight score according to notes in subband
 Range correct score to be between 0 and 1
 Sum all scores across all bands to generate MRFFT Score
Note Bin Scoring
Weighted Sub-band FFT Bin Score =
Sub-band FFT Bin Score * (notes in sub-band/total notes across all bands)
Error Scores
1
0
0.5
1
0.5
Bin A
Value
0
0.5
Half way
point
1
0.5
Bin B
Value
If 2 note frequencies fall within same bin, FFT length is discounted as unsuitable
Scoring Process
 The algorithm moves the cut off frequencies A, B and C through
all combinations of positions. For each position, all FFT lengths
between 256 and 8192 samples in increments of 128 are
evaluated on each sub-band. All combinations of FFT lengths
on all combinations of subbands are evaluated and scored.
Subband B
Subband A
Subband C
80 Hz
FcA
FcB
FcC
Subband D
5KHz
FcD
Solutions
1.
2.
3.
4.
5.
6.
4 band MRFFT
3 band MRFFT
Dressler 4 band MRFFT
Dressler fixed FFT Length variable bands
4 band MRFFT
1 band FFT
256-8192 range
256-8192 range
256-2048 range
256-2048 range
256-2048 range
8192
Results – Subband Divisions
Determined by MRFFT Scoring System
Karin
Dressler
Frequency
MRFFT 4
(Hz)
Band
Parameters
98.00
103.83
110.00
116.54
123.47
130.81
138.59
146.83
155.57
164.82
174.62
185.00
196.00
207.65
220.00
233.08
246.94
261.63
277.19
293.67
311.13
329.63
349.23
370.00
392.00
415.31
440.01
466.17 2048 FFT
493.89 FcA
523.26
554.37
587.34
622.26
659.26
698.46
740.00
784.00
830.62
880.01
932.34
987.78
1046.51
1108.74
1174.67 1024 FFT
1244.52 FcB
1318.53
1396.93
1479.99
1568.00
1661.24
1760.02
1864.68
1975.56
2093.03
2217.49
2349.35
2489.04 512 FFT
2637.05 FcC
2793.86
2959.99
3136.00
3322.48
3520.04
3729.35
3951.11
4186.06
4434.97
4698.69 256 FFT
4978.09 FcD
Solution 1
Solution 2
Solution 4
Solution 5
Band A
Band B
6016 FFT
FcA
6016 FFT
FcA
Band C
3328 FFT
FcB
Not Analysed Not Analysed
1664 FFT
FcA
2048 FFT
FcA
1408 FFT
FcB
FFT 2688
FcC
1792 FFT
FcC
1024 FFT
FcB
896 FFT
FcC
1408 FFT
FcD
1408 FFT
FcC
512 FFT
FcC
FcD 256 FFT
768 FFT
FcD
Band D
Results – MRFFT Score
Transcription Test – Low F Bands
Original
Solution 6
FcA
FcB
High F Resolution of solution 6 is
reflected in
Low frequency transcription accuracy
Solution 1
Transcription Test – High F Bands
Solution 1
Solution 3
Solution 6
F-Measure Results
Recall refers to the fraction
of the relevant notes that
were retrieved i.e. how
many of the correct notes
the system extracted.
1.000
0.900
0.800
0.700
Score
Precision refers to the
fraction of relevant notes
retrieved, relative to the
total number retrieved. I.e.
how many of the
extracted notes that were
correct.
F-Measure is the weighted
mean of precision and
recall.
0.600
0.500
Recall
0.400
Precision
Fmeasure
0.300
0.200
0.100
0.000
1
2
3
4
Solution
5
6
Peak Picker
 A threshold is dynamically set for each analysis window of
the STFT as a percentage of the maximum magnitude
within the window, with a minimum threshold heuristically
decided. If a bin magnitude exceeds the threshold a
note is transcribed at that point.
Peak Picker Robustness
25%
Threshold
Solution 1 Vs Solution 6 Picker
25%
Threshold
25%
Threshold
MRFFT Implementation
6016 FFT is performed on the entire frequency spectrum. The spectral information is then filtered to
include only the frequencies required by that band.
6016 FFT
Not Used in MRFFT
FcA
F
note frequency (orange magnitude) not in the frequency band considered, generates cross
channel interference (red magnitudes) that contributes to the magnitudes in the sub-band of
interest.
Cross talk indicators
Solution
1
2
3
4
5
6
average
power
average
magnitude in power in non
note bins
note bins
538.79 23.4338906
594.48 32.7189717
157.7 29.0526256
216.75
45.368934
208.73 35.0150222
1527 27.4607458
average
power in non
note bins as
% of power
in note bins
4.35
5.50
18.42
20.93
16.78
1.80
Adjacent bins
Note
Bin
Frequency
Frequency
98.00
96.90
103.83
102.28
110.00
107.67
113.05
116.54
118.43
123.47
123.82
130.81
129.20
134.58
138.59
139.97
155.57
145.35
150.73
155.57
156.12
161.50
164.82
166.88
174.62
172.27
177.65
185.00
183.03
188.42
196.00
193.80
199.18
204.57
207.65
209.95
215.33
220.00
220.72
226.10
233.08
231.48
236.87
242.25
246.94
247.63
253.02
258.40
261.63
263.78
269.17
274.55
 Adjacent bins in optimised MRFFT
represent fundamental frequencies.
Therefore any cross channel
interference will contribute to energy
contained in FFT bins representing note
frequencies.
Note FrequencyBin Frequency
98.00
95.30
103.83
102.63
110.00
109.96
116.54
117.29
123.47
124.62
130.82
131.95
138.59
139.28
146.84
146.61
155.57
153.94
161.27
164.82
168.60
174.62
172.27
185.00
185.52
196.00
198.77
207.66
212.02
220.01
225.27
233.09
238.52
246.95
251.77
261.63
265.02
 This may contribute to false positives.
F Measure conclusions
 The results of the F-Measure are largely disappointing,
and can be attributed to the inadequacies of the
implemented peak picker to handle fluctuations in
magnitude of local maxima. Characteristics of the MRFFT,
like adjacent note representing bins, and interference
generated by sub-band division methods contribute to
this problem.
 Large variations of spectral magnitudes also contribute
Conclusions
 The theoretical scoring of MRFFT parameters resulted in
favourable results for the optimised FFT.
 The ‘real world’ sinusoidal extraction test demonstrated
initially disappointing F-Measure results for the MRFFT
solutions compared to the single band 8192 FFT. However,
upon closer analysis of the transcribed files, positive aspects
of the MRFFT analysis were found as performance improved
in the higher frequencies.
 Further investigation of the results revealed inadequacies of
the peak picker implemented and also indicated issues with
the construction of the MRFFT that require further
investigation.