Transcript Easily extensible unix software for spectral analysis

Easily extensible unix software
for spectral analysis, display
modification, and synthesis of
musical sounds
James W. Beauchamp
School of Music
Dept. of Electrical and Computer Engineering
University of Illinois at Urbana-Champaign
[email protected]
http://www.staff.uiuc.edu/~j-beauch
Talk Topics
• SNDAN features
–
–
–
–
–
time-domain utilities
phase vocoder harmonic spectrum analysis
spectral graphics, modification, resynthesis
frequency tracking analysis, graphics, synthesis
pitch detection, conversion to harmonic format
• SNDAN applications
• Future developments
• Conclusions
SNDAN OVERVIEW:
signal
ANALYSIS FRONT END
analysis data
GRAPHICS
MODIFICATION
RESYNTHESIS
SNDAN OVERVIEW:
SIGNAL VIEWING AND EDITING
SNDAN OVERVIEW:
SPECTRUM ANALYSIS
SNDAN OVERVIEW:
SPECTRUM VIEWING, MODIFICATION, AND
RESYNTHESIS
Phase Vocoder Analysis
fs (sample
frequency)
s(t)
sound
signal
frequency)
BANDLIMITED
INTERPOLATION
RESAMPLER
COMPUTE
AMPLITUDES
AND PHASES
Ak (tn ),  k (tn )
fa (analysis
s’(t) HAMMING
WINDOW
w(t)s’(t)
(double period)
OVERLAP BY
1/2 PERIOD
THROW AWAY
K/2, K ODD
COMPUTE
FREQUENCY
DEVIATIONS
DISCRETE
FOURIER
TRANSFORM
(FFT)
harmonic data
Ak (t n ),f k (tn ),  k (0)
Harmonic Data Graphics
harmonic data
Ak (t n ), f k (tn )
1D: Amplitude vs. Frequency
(snapshot bar, line; comp. overlay)
1D & 2D: Frequency vs. Time
(individual, spectrogram)
3D: Amplitude vs. Frequency
EPS graphics
and Time
Inharmonicity vs. Time
display
Spectral Centroid vs. Time
Spectral Centroid vs. RMS Ampl.
Spectral Irreg. vs. Time
Inverse Spectral Density vs. Time
Musical Pitch vs. Time
Example 2D graph
Example 3D graph
Harmonic Data Modification
Smooth Ak vs. time (t)
Make Ak(t) proportional to Arms(t)
Smooth Ak vs. frequency (k)
Scale Ak by kp to achieve new
harmonic data
average centroid
Ak (t n ), f k (tn ),  k (0) Scale Ak to achieve designated
harmonic data
spectrum or aux. spectrum
Smooth fk vs. time (t)
aux. harmonic data Make all f (t) harmonic to f (t)
k
ave
Ak' (t n ), f k' (tn ),  k' (0) Flatten fk to average or harmonic
Quantize fund. freq. to ET pitch
Warp attack time
Reduce duration without
affecting attack and decay.
Harmonic Data Resynthesis
AMPLITUDE & FREQUENCY
LINEAR INTERPOLATION
harmonic data
Ak (t n ), f k (tn ),  k (0)
synthetic
(scale amplitude, freq, duration) signal
AMPLITUDE & PHASE
QUADRATIC INTERPOLATION
ˆs (t)
Signal Modification Example
Original
Flute
Time-smoothed
Amplitudes
Time-smoothed
Amplitudes
& Flattened
Frequencies
Time-smoothed
Amplitudes
& Frequencies
Time-smoothed,
Time-smoothed,
RMSed, &
Spectrum Envelope
Smoothed &
Spectrum Envelope
Flattened Frequencies
Smoothed &
Flattened Frequencies
Frequency Tracking (MQ) Data Analysis
fs
fmin
Athresh
s(t)
sound
signal
FFT WITH
TYPICAL 6 MS
HOP
KAISER WINDOW
WITH 100% ZERO
FILL
AT EACH FRAME
COMPUTE EACH PEAK’S
AMPLITUDE, FREQUENCY
AND PHASE
CONNECT PEAKS
TO NEXT FRAME
PEAKS (TRACKS)
APPLY THRESHOLD
IDENTIFY AND SAVE
SPECTRAL PEAKS
partial (MQ) data
Ak (tn ), f k (tn ),  k (tn ), k [n], 1 k  kmax [n]
Partial Data Processing
Graphics
partial data
Ak (tn ), f k (tn ), k [n]
2D: FREQUENCY VS. TIME
3D: AMPLITUDE VS.
FREQUENCY VS. TIME
EPS display
time
scale
Synthesis
partial data
Ak (tn ), f k (tn ),  k (tn ),  k [n]
INTERPOLATION:
AMPLITUDE - LINEAR
PHASE - CUBIC
synthetic
signal
ˆs (t)
Partial Data Processing
Pitch Detection
fmin
fmax
TWO-WAY MISMATCH
HARMONIC MATCHING
METHOD
partial data
Ak (tn ), f k (tn )
fundamental
frequency
data
F0(t)
Harmonic Separation
nhar (no. of
partial data
Ak (tn ), f k (tn )
fund freq data
F0(t)
harmonics)
HARMONIC
SIEVE
frequency
tolerance
harmonic data
Ak (t n ), f k (tn )
Saxophone Solo
2D Peak Track Plot:
Original Sound
Partial Data
Stretched x2
Synthesized from
Partial Data
Harmonic Data
Stretched x2
Pitch Plot:
Synthesized from
Harmonic-reduced
Data
Harmonic Data
Stretched x2
Smoothed Freq
Applications
• Synthesis instrument development
– nonlinear and frequency modulation
– wavetable trumpet and piano
• Timbre investigations
–
–
–
–
simplified sounds for discrimination studies
normalized sounds for MDS studies
perturbed sounds for discrimination studies
synthesis quality evaluation
• Music composition using Music 4C
Future Developments
•
•
•
•
•
•
•
More features for partial data format
Integrate programs into single program
More advanced analysis front end
Multi-Channel
Create GUI interface
Real time
Port to more platforms
SNDAN Conclusions
•
•
•
•
•
•
Provides analysis, graphics, modification, and synthesis
Specialized for musical sounds
Two spectrum data formats: harmonic and partial
Contains pitch detector
Unix source code modular and easily extensible
Source code available at:
– http://www.staff.uiuc.edu/~j-beauch/software/sndan/
• DOS binary version available at:
– http://ftp.cs.bath.ac.uk/pub/dream/SNDAN32/
• Real-time GUI spinoff analyzer for Mac available at:
– http://www.staff.uiuc.edu/~j-beauch/software/armadillo/