Audio Engine Programming

Lecture 4 Introduction to FMOD programmers API

3D Sound

 Sounds have position and velocity  There is a listener component  Relationship between the two  Attenuation (with distance)  Occlusion (low-pass filter)   Doppler (relative velocities) Lots of “psycho-acoustic” options

Fmod audio development

 There are three primary components:   The Sound Designer The Sandbox   The API We’ll use all three  Start with Sound Designer

Data Driven vs. Low Level API

 Low-Level API   Everything is done using programming  The programmer has complete control The benefits are lower memory overhead

Continued…

 Data-Driven API   Takes assets from FMOD Designer  Sound Designer has control over the audio Integrates them into the code using the

EventSystem

object

Key Features

 The data-driven approach of the EventSystem:  Support’s event logic  Allows the Sound Designer to fine tune or ‘tweak’ the final mix without the need for Programmer intervention  Creates a logical separate of concerns for the Programmer and Sound Designer

Overview of event model

FEV event data FSB audio data FMOD EX

FMOD EventSystem FMOD Low Level

Assets from the sound designer

 The build process creates the following assets:  .FEV file - contains the event meta data     .FSB file(s) - contains the audio data The ‘Programmer report ‘- a human readable list of event names, sound definitions [Optional] A ‘header’ file - A source code header file [Optional] Cache files - Files used to speed up subsequent builds (These files are not required by the Programmer)

Linking the library

 Include the EventSystem header file  #include "fmod_event.hpp“  Declare a pointer to an EventSystem  FMOD::EventSystem *eventsystem;  Create the object (allocate memory)  FMOD::EventSystem_Create(&eventsystem);   Initialize the object eventsystem->init(64, FMOD_INIT_NORMAL, 0, FMOD_EVENT_INIT_NORMAL);

Loading files

 Set the media path (location of files)  eventsystem->setMediaPath((char *)MEDIA_PATH);  Load the .FEV file  eventsystem->load("examples.fev", 0, 0);   Load an event group eventsystem->getGroup("examples/examples", FMOD_EVENT_DEFAULT, &eventgroup);   Load an individual event eventgroup >getEvent(“car", FMOD_EVENT_DEFAULT, &car);

Loading Audio

 The required .FSB file(s) are listed in .FEV file  Both the .FEV and .FSB files should be located in the directory specified with

setMediaPath()

 The location of audio within the .FSB files is stored in the .FEV

 FMOD will load the audio data from the .FSB files as required for an event, automatically

Loading Options

 FMOD_EVENT_DEFAULT    Specifies default loading behaviour Event data for the whole group is NOT cached Loading will block any other task until complete.  FMOD_EVENT_NONBLOCKING   FMOD will use a thread to load the data Use Event::getState to find out when loading is complete.

Continued...

 FMOD_EVENT_INFOONLY   Does not allocate instances or load data Creates a ‘handle’ that allows the programmer to get information from the event

EVENT Playback

 Each events have a number of methods and attributes  To start playback of an event:  example->start();  To stop an event:  Example->stop();

Updating

 Updating the event system should be called once per 'game' tick (or once per frame)  eventsystem->update();  This allows the FMOD Engine to refresh data such as:    the position of 3D sounds changes to volume, pitch, effects as well as internal mechanisms

Demonstration

 Simple Event  Demonstrates the necessary code to start playback of an event

PROVIDING Parameter DATA

 Get the parameter called ‘test’  example >getParameter(“test", &test);  test will be a EventParameter object  To get the range of the parameter  test->getRange(&rangemin, &rangemax);  To set the value of the parameter  test->setValue(value);

Demonstration

 Parameter  Demonstrates the necessary code to provide parameter data to an event

PROVIDING a KEY-OFF

 If an event parameter is currently sustaining on a ‘sustain point’, triggering a keyoff will release it and allow the parameter to continue  test->keyoff()

Demonstration

 Keyoff  Demonstrating the necessary code to leave a sustain point

Selecting sounds at run-time

 The sound to be played can be selected at runtime  This is required for situations such as ‘colour commentary’ in sport games or other situations that require dynamic dialogue  The are two methods:  Programmer Sounds  Programmer Selected

Demonstration

 Programmer Sound  When a sound definition containing a ‘ProgrammerSound’ is to be played, a callback is made  The code must then provide a sound object at runtime just before it's played  Demonstrates the necessary code to select a sound to play at runtime, using the ‘ProgrammerSound’ method

Demonstration

 Programmer Selected    When a sound definition with the

playmode

‘ProgrammerSelected’ is to be played, a callback is made The code must then provide the index number at runtime just before it's played The index number identifies which file within the sound definition is to be played  Demonstrates the necessary code to select a sound to play at runtime, using the ‘ProgrammerSelected’ method

Closing the Event System

 Unload event data  eventsystem->unload();  Release audio data in memory  fsb->release();  Release memory and close eventsystem memory  eventsystem->release();

Setting up a Project

 Take the advice of the video tutorials    Have a separate folder Copy sounds into a “sounds” directory  Keeps the safe  Can have sub-directories Create an “output” directory

Designer Interface

Can delete this if you want, but need to have at least one group

Events

 Used to define sound properties for an event  Can be comprised of     One sound Several sounds randomly chosen Monophonic vs Polyphonic Can randomize pitch and attenuation  Events can be   Simple Multi-track

A Simple Event

Granular Sounds

 These sounds aren’t looping  Allows for a sounds to occur between time ranges  Allows for polyphony

Event Options

 Can vary attenuation (dropoff with distance)  Can vary pitch  Creates a sound considerably smaller than a “soundtrack”  Plays forever and is always random!

Sound Definitions

 Required for multi-track events

Multi-track Events

 Comprised of sound defs in layers

Multi-track Events

 Careful when adding sounds    Choose sound def Ambient noise is looping Other sounds are granular, so choose “Oneshot”  Have parameters  Not based on time!

Multi-track Events

 Can cross-fade and set fade out time

Effects

Parameter Properties

 Can define range and velocity (to simulate time)

Engine Designer

 fmod can specifically work with engine sounds   Need for Speed 2 Based on the “load” of the engine  Right-click on a sound instance->properties  Auto-pitch  Window->fmod Engine Designer

The Build Process

 Know which platform you’re targeting  Changes are applied only to that platform  Project->Clean, Project->Build

Interactive Music

 Comprised of several

short

music segments of     Intro music Darkening or discovery Fighting/intense fighting Release  Before you begin, you must know  Tempo of music (beats per minute – bpm)  Beats per measure (time signature)

Cues

Themes and Auditioning

Transitioning via Parameters

Other Things

 Supports deployment of multiple languages  Can deploy different builds based on   Platform Language

A Look at the API

 Basic API  Designer API   Built on basic API Can read .fev files  Written for C/C++   #include // C #include // C++  Read the documentation fmodex.chm

API Parts

  A System is the fmod engine A Sound is the raw resource   2D uses FMOD_2D 3D uses FMOD_3D result = system->createSound(“music.wav", FMOD_2D, 0, &sound);

API Parts

 A Channel is an

instance

of a Sound     Each time you play a sound, you get a new channel Channels can start out paused You can set the   Volume (0.0f – 1.0f) Pan (-1.0f – 1.0f)  Frequency (in Hz as a float) Always use FMOD_CHANNEL_FREE to pick for you

#include #include #include using namespace using namespace std; FMOD; void main () { FMOD_RESULT result; System* system; // Create the fmod system. We only need one of these result = System_Create(&system); // Specify a max number of simultaneous channels result = system->init(100, FMOD_INIT_NORMAL, 0); Sound* sound; // Decompress the entire mp3 into 16-bit PCM in memory result = system->createSound(" winning.mp3

", FMOD_DEFAULT, 0, &sound); if (result != FMOD_OK) { cout << " Couldn't open it! " << } FMOD_ErrorString(result) << endl; Channel* channel; // Used for setting volume, pan, pausing...

// You have the option of passing that channel as the last parameter result = system->playSound(FMOD_CHANNEL_FREE, sound, false , &channel); channel->setFrequency (44100.0f); if (result != FMOD_OK) { cout << " Couldn't play it! " << } while ( true ) { cout << "1"; system->update(); FMOD_ErrorString(result) << endl; // Now required in fmod } }



Virtual Voices

Supports “virtual voices” when hardware is limited   Set from System::init() System::getCPUUsage()  Try to keep < 1000   May need to set the priority of a channel if it’s important channel->isVirtual()

3D Sound in fmod

  Sound attenuation   Logarithmic Set the mindistance attenuation of a sound channel to start   Bee = 0.1f

Jet = 50.0f

 Leave max distance alone (default is 10,000) Set 3D settings with System::set3DSettings()    Doppler Distance factor (in cm, m, feet) Rolloff scale (attenuation models)

3D Sound

 fmod uses a left-handed coordinate system result=system->init(100, (FMOD_MODE)(FMOD_INIT_3D_RIGHTHANDED|FMOD_3D), 0);  With velocity, you must pass it as metres per second velx = (posx-lastposx) * 1000 / timedelta; vel = 0.1 * 1000 / 16.67 = 6 meters per second vel = 0.2 * 1000 / 33.33 = 6 meters per second

} void main () { FMOD_RESULT result; FMOD_VECTOR soundPos, soundVel; FMOD_VECTOR listenerPos, listenerVel, listenerForward, listenerUp; System* system; result = System_Create(&system); result = system->init(100, (FMOD_MODE)(FMOD_3D), 0); int numHC = 0; result = system->getHardwareChannels(&numHC); cout << "Hardware channels: " << numHC << endl; Sound* sound; result = system->createSound (“train.mp3", FMOD_3D, 0, &sound); } if (result != FMOD_OK) { cout << "Couldn't open it! " << FMOD_ErrorString(result) << endl; Channel* channel; result = system->playSound(FMOD_CHANNEL_FREE, sound, false, &channel); channel->setFrequency(44100.0f); if (result != FMOD_OK) { cout << "Couldn't play it! " << FMOD_ErrorString(result) << endl; } float dsp, stream, geometry, update, total; soundPos.y = soundVel.x = soundVel.y = 0.0f; soundPos.z = -100.0f; soundPos.x = 5.0f; soundVel.z = 6.0f; channel->set3DMinMaxDistance(10, 10000); listenerPos.x = listenerPos.y = listenerPos.z = 0.0f; listenerVel.x = listenerVel.y = listenerVel.z = 0.0f; listenerForward.x = listenerForward.y = listenerUp.x = listenerUp.z = 0.0f; listenerForward.z = listenerUp.y = 1.0f; system->set3DListenerAttributes(0, &listenerPos, &listenerVel, &listenerForward, &listenerUp); } while (true) { system->update(); system->getCPUUsage(&dsp, &stream, &geometry, &update, &total); cout << total << endl; soundPos.z+=0.01f; channel->set3DAttributes(&soundPos, &soundVel);

Interfacing with Sound Designer

 You should have created  .fev file (Designer file – event data)   .fsb file (raw audio data) .txt file (describes the events and parameters)  May want to start by creating a helper function void } checkErrors(FMOD_RESULT result) { if (result != FMOD_OK) { cout << "fmod error: " << FMOD_ErrorString(result) << endl; exit(1); }

Imagine That This is our Project

#include #include #include #include using namespace std; using namespace FMOD; void main () { EventSystem* eventSystem = NULL; EventGroup* eventGroup = NULL; Event* myEvent = NULL; EventParameter* eventPar; FMOD_RESULT result = EventSystem_Create(&eventSystem); checkErrors(result); result = eventSystem->init(64, FMOD_INIT_NORMAL, 0, FMOD_EVENT_INIT_NORMAL); eventSystem->setMediaPath("..//"); result = eventSystem->load("fmodTest.fev", 0, 0); result = eventSystem->getGroup("fmodTest/beeps", false, &eventGroup); result = eventGroup->getEvent("PossessedComputer", FMOD_EVENT_DEFAULT, &myEvent); result = myEvent->getParameter("proximityToComputer", &eventPar); myEvent->start(); } float dir = 0.0001f; float currentParVal = -1.0f; eventPar->setValue(0.1f); while (true){ cout << currentParVal << endl; eventPar->getValue(¤tParVal); currentParVal+=dir; eventPar->setValue(currentParVal); if ((currentParVal >= 1.0)||(currentParVal <= 0.0)) { dir = -dir; } }

Interactive Music

void main () { EventSystem* eventSystem = NULL; MusicSystem* musicSystem = NULL; MusicPrompt* introPrompt, *fightPrompt, *fight2Prompt, *releasePrompt; FMOD_MUSIC_ITERATOR cueIter; FMOD_MUSIC_ITERATOR paramIter; FMOD_MUSIC_ID intensityParID = -1.0f; FMOD_RESULT result = EventSystem_Create(&eventSystem); result = eventSystem->init(64, FMOD_INIT_NORMAL, 0, FMOD_EVENT_INIT_NORMAL); result = eventSystem->load("acid_test.fev", 0, 0); result = eventSystem->getMusicSystem(&musicSystem); musicSystem->setVolume(1.0f); result = musicSystem->getCues(&cueIter); cout << "Iter ID: " << cueIter.value->id << endl; result = musicSystem->prepareCue(cueIter.value->id, &fightPrompt); result = musicSystem->getNextCue(&cueIter); cout << "Iter ID: " << cueIter.value->id << endl; result = musicSystem->prepareCue(cueIter.value->id, &fight2Prompt); result = musicSystem->getNextCue(&cueIter); result = musicSystem->prepareCue(cueIter.value->id, &introPrompt); result = musicSystem->getNextCue(&cueIter); result = musicSystem->prepareCue(cueIter.value->id, &releasePrompt); musicSystem->getParameters(¶mIter); intensityParID = paramIter.value->id; cout << "paramIter.value->name is " << paramIter.value->name << endl; musicSystem->setParameterValue(intensityParID, 7.5f); result = introPrompt->begin(); while (true){ eventSystem->update(); } }

Other Things You Should Look Into…

 The 3D Reverb API  Asynchronously loading data (by default)  Memory management for non-PC platforms  FMOD::Memory_Initialize() // fix size of fmod  Using compressed samples  Built-in DSP

Configuration Notes

 You’ll be working with   Header files  C/C++ -> General->Additional Include Directories Lib files  Linker->General->Additional Library Directories  Linker->General->Additional Dependencies   DLLs (which you should put into Windows/System32) There’s also a FMOD.NET site…

Configuration Notes

 You have the option of 64-bit and 32-bit libraries  You might have to create a new platform