Prerequisites: [Trig, Physics, C/C++ Programming, Data Structures,
Operating Systems]
Textbook: Audio
Anecdotes – Tools and techniques for
Digital Audio
Ken
Greenebaum, Ronen Barzel (ISBN: 1-56881-104-7)
Description: Sound synthesis offers an exciting alternative to the use of
pre-recorded sounds in simulation and video game software. Synthesis techniques
allow sound to directly correlate to the simulated interactions which generate the sound. For instance
collision sounds may be dynamically synthesized using parameters from the
simulation or physics engine such as the material types involved, the speed of impact,
and the angle of impact. As a result every collision is unique and conveys much
more information than simply re-playing a pre-recorded engineered sound. The entire sonic chain may be modeled
including the sound source, the transmission environment and the listener. The
results are directional sounds which convey a great deal of information to the
user and correspondingly appear very rich.
This course introduces
the fundamentals needed to understand, create and manipulate digital sound: the
physics of sound, the human perception of sound, and digital sampling theory.
The class surveys significant algorithms and techniques used in the field
emphasizing tradeoffs in accuracy and performance. Assignments provide a
hands-on opportunity to experiment with implementing and using many of these algorithms
and implementation techniques. The instructor, Ken Greenebaum, provides insights
from his 15 years creating digital media systems for companies including
Silicon Graphics and Microsoft. Demos, videos, and guest speakers will be used
to enhance the lecture.
(A follow-on course may
be offered providing a deeper exploration of advanced algorithms)
Class requirements:
(1) Mid-term -25%
(2) Final -25%
(3) Project(s) -50%
Project:
Construct dynamic audio synthesis engine. To keep this project manageable it
will be broken into 10 pieces assigned synchronously to the topics introduced
in lecture.
(1) Audio
“Hello World” using either DirectSound or PABLIO libraries
(2) Parametric
Sine wave synthesizer
(3) Mixer with
de-zippering
(4) Rate and
quantization conversion to engine
(5) Spectral
filtering
(6) FM synthesis
(7) Additive
synthesis
(8) Subtractive
synthesis
(9) Physical
modeling
(10) 3D
Spatialization
Grading policy:
(1) Correctness
–
70%
(2) Design and
representation –
20%
(3) Programming
style –
10%
Programming requirements:
(1)
C/C++ submitted with handwritten Makefiles (MUST
BUILD)
(2)
DirectSound or PABLIO audio libraries
(3)
VisualStudio/Windows or Linux required
Office
Hours:
Course
webpage: http://cs245.sworks.com/
Contact
info:
Email: keng@sworks.com
Phone: 206.310.5253
Contents:
Week 1
·
Introduction
·
Physics of sound (Compression rarefaction of a medium,
collision, cricket (comb), speaker cone)
·
Attenuation ((spherical dissipation), point, dipole,
planar radiators)
·
Propagation of sound
waves (Speed, Reinforcement/cancellation,
Reflection, Diffraction, Doppler Effect)
·
Measurement (The Decibel, Sound Pressure Level)
·
EZ- synthesis (Square-waves, saw-tooth)
·
Assignment: Audio
“Hello World” using DirectSound or PABLIO audio libraries
Week 2
·
Human perception of
sound (
·
Sampling theory (Nyquist limit, Quantization, Sampling rate,
Reconstruction filter)
·
Sine-wave synthesis (sin(x), Lookup table, Oscillators)
·
Assignment: Parametric
Sine wave synthesis
Week 3
·
Wave table synthesis (Mixing, Enveloping, De-zippering techniques)
·
Computer techniques (Realtime programming, Streaming using
threads, Buffering: circular ‘ring-buffer’ queue)
·
Assignment: mixer with
de-zippering
Week 4
·
Dynamic rate
conversion (Why rate convert?, Challenges?,
Optimal solutions, Practical solutions)
·
Compression concepts (Encodings: (non-linear, mu-law, delta), Lossless:
entropy coding)
·
Assignment: add rate
conversion to mixer to playback simultaneous sounds of different quantization
and sample rate
Week 5
·
Compression concepts
cont. (Lossy compression (Auditory masking, MP3, WMA))
·
Spectral filtering (FIR, IIR filters, FFT)
·
Assignment: add
parametric filtering to sound engine
Week 6
·
Sound and interaction
·
Causality
·
Synthesis vs. playback
of pre-recorded sounds (1980’s standup
arcade games (tentative guest speaker: David Thiel Gotlieb programmer,
Microsoft Researcher and the creator of Q*Bert’s voice))
·
Dynamic parametric
synthesis
·
Synthetic Foley sound
·
Timbre trees
·
DirectAnimation
·
Assignment: No
assignment
Week 7
·
Midterm
·
Synthesis techniques
overview
·
No Assignment
Week 8
·
FM Synthesis (Explosions, Musical sounds, Exploring the
FM audio space)
·
Computer
techniques:Pipelining (Accurate timing, Synchronization,
Doppler via accumulation buffer)
·
Assignment: add FM
synthesizes to sound engine (mixer), find 3 interesting sounds in the sound
space
Week 9
·
Additive Synthesis
·
What makes wood,
metal, hollow, solid, etc. sound that way?
·
Resonance/Damping
·
Fundamental
frequencies/Harmonics
·
Assignment: Add
parametric Additive synthesis to sound engine
Week 10
·
Subtractive Synthesis
·
What makes wind,
explosions sound that way?
·
Noise generators (white, pink, 1/f noise)
·
Filtering
·
Assignment: Add
parametric Subtractive synthesis to mixer, create 2 interesting sounds
Week 11
·
Physical Modeling
·
What makes actual
objects have their characteristic sound?
·
The spring mass model
·
Modeling bars/bells
·
Modeling strings
·
Modeling complex
systems: Finite element Analysis
·
Simplification: modal
synthesis
·
Assignment: (2-week)
pick physical system and excitation model to add to sound engine
Week 12
·
Exciting physical
models (Plucking, Striking, Scraping
(Friction))
·
Assignment: (part-2)
Week 13
·
Spatialization (Surround sound, Panning vs. fading, Head
Related Transform)
·
Spatialization issues (Speaker array, headphones, crosstalk
cancellation, moving sources)
·
Assignment: add
parametric spatializer to sound engine
Week 14
·
Granular synthesis (sounds of nature, rivers, the wind)
·
Ambient synthesis (city streets, baseball game)
·
Review
·
Assignment: No
assignment (semester assignment due + Final)