Transcript Virtual Marimba Creating a novel musical instrument that
Virtual Marimba
Creating a novel musical instrument that produces visual representations of sounds.
An Interactive Media Thesis by Chris Hackett & David Delaney CS6022 Principles of Interactive Media - May 4th
David Delaney profiles
• • • • • Business Studies Graduate Multi-Instrumental Musician Collaborative Musician (Palestinian/South African,etc) Sound Effects/Soundtracking Focus: Interactive Media in Tradtional Musical Instruments.
Sound Synthesis & Manipulation CS6022 Principles of Interactive Media - May 4th
Chris Hackett profiles
• • • • • Computer Systems Graduate Specialism: Web/Graphic Design Works with: Dreamweaver, Photoshop, Flash, Sketch-Up Interests: Graphics, Animation,Music, Audio-visual Art, Film, Photography Focus: Graphics, Visual Representations of Sound.
CS6022 Principles of Interactive Media - May 4th
inspiration
• • • • • • Interest in Music Similar Thesis Proposals Existing artefacts e.g. Marimba Lumina Ailbhe’s ‘Rhythm Manifesto’ Composer/Designers e.g. Todd Winkler Audio - Visual Art Culture CS6022 Principles of Interactive Media - May 4th
design process
• • • • • • User-Centred Design Research Requirements Gathering Design Prototyping Evaluation CS6022 Principles of Interactive Media - May 4th
visual music
• • • • • • • ‘A means of expressing music in visual form requiring the active involvement of an artist, designer, or director to interpret the music and find the means to express it visually.’ (Friedlander, ‘98) ‘A visualization of music which is the translation of a specific musical composition(or sound) into a visual language, with the original syntax being emulated in the new visual rendition.’ (Ox, J and Keefer, C, ‘06) Origins: Pythagoras, Aristotle, Sir Isaac Newton Correspondences of the colour spectrum and sound waves, music and colour, sound and light.
18th Century - Castel’s Clavecin Oculaire Late 19th/Early 20th - Experimentation, abstract film 20th Century - Art e.g. Scott Drave’s Electric Sheep Present - Artists, composers, designer, virtual djs CS6022 Principles of Interactive Media - May 4th
• • • • • • • VJ: Audio-visual Art and VJ Culture (Faulkner, M) On Curating Recent Digital Abstract Visual Music (Ox, J & Keefer C) Amplifying Reflective Thinking in Musical Performance (Johnson, A et al) Visual Music-Searching for an Aesthetic (DeWitt, T) Exploration of the correspondence between visual and acoustic parameter spaces (Gerhard et al) Issues for Designing a flexible expressive audiovisual system for real-time performance and composition (Franco et al) Rhythm Manifesto – Ailbhe Keogan CS6022 Principles of Interactive Media - May 4th
benchmarks
• • • • • malletKat Marimba Lumina ‘ Music Plays Images X Images Play Music ’ Ryuichi Sakamoto ‘Falling Up’ - Todd Winkler ‘Rhythm Manifesto’ - Ailbhe Keogan MPI X IPM CS6022 Principles of Interactive Media - May 4th
websites
• • • • • • • • • NIME www.nime.org
The MEGA Project www.megaproject.org
InfoMus Lab infomus.dist.unige.it
SARC www.sarc.qub.ac.uk
The s2S2 www.s2s2.org
ConGas www.cost287.org
www.centerforvisualmusic.org
www.howstuffworks.com
www.acm.org
CS6022 Principles of Interactive Media - May 4th
empirical work undertaken
• • • • • Benchmark Research e.g. malletKat Informal discussions with classmates Interview with Ailbhe Keogan, 27.02
Interview with David Merrill, MIT, 16.03
Early sketches, prototyping, scenarios CS6022 Principles of Interactive Media - May 4th
further empirical work
• • • • • • Interviews Cormac Byrne, Marimba player Evelyn Glennie, Killarney 12.07
Micheal O Suilleabhain, Musician Music Academics, e.g. Jurgen Simpson Further sketches, prototyping, scenarios
artefact
artefact
CS6022 Principles of Interactive Media - May 4th
layout
CS6022 Principles of Interactive Media - May 4th
layout
projection screens
CS6022 Principles of Interactive Media - May 4th
visuals
• • • • • • • • • Mapping Techniques Colour - 7 colours to 7 units of the scale Vibrato – sine wave Interrupted Cadence – expectation Relative Minor – sinister, dark, gothic Timbre of Sound Mikael’s Manifesto PD & GEM 3Ds Max, Photoshop, Flash CS6022 Principles of Interactive Media - May 4th
architecture
Camera Device Sensors
Sound (Speakers) Images (Projector)
Electronic Marimba App Max/MSP, Pure Data, VVVV, Director Device Drivers OS (Windows) BIOS Sound Parameters can be manipulated PC Hardware (USB, Firewire or Wireless) CS6022 Principles of Interactive Media - May 4th
hardware required
• • • • • • • Arduino board – prototyping Piezo sensors Pico device Projector – possibly 3 Webcamera Computer (PC/Mac) Under discussion – whether to build device from scratch or use existing commercial product CS6022 Principles of Interactive Media - May 4th
software required
• • • • • • • • Pure Data and GEM - probably MAX/MSP - not so likely VVVV - not so likely Director & Flash (Lo-Fi Prototyping) Photoshop (Graphics) 3D studio Max (Modelling, Graphics) Google Sketch-Up (sketches) Logic Pro (Synthetic Sounds for lo-fi’s) CS6022 Principles of Interactive Media - May 4th
extra scope for instrument
• • • • • • Multi Instrumental Display Dynamic Visuals Pitch bend, vibrato Velocity Sensitivity Pressure Sensitivity (Aftertouch) Recording and Playback of loops.
CS6022 Principles of Interactive Media - May 4th
schedule
June July August May Brainstorming Presentation & Report Developing Concept Users & Requirements Research Theory Research Technology Designing Prototyping Building Evaluation Documentation Thesis Write up Assessment & Dawn
CS6022 Principles of Interactive Media - May 4th
September
documenting the process http://0030511.blogspot.com
CS6022 Principles of Interactive Media - May 4th
concerns
• • • • • • • • • Finalising concept Building device Vs Existing product Real-time Vs Pre-Rendered Latency Problems Pressure/Velocity Sensitivity Number of Projectors Needed Software Skills Cost Time CS6022 Principles of Interactive Media - May 4th
issues for discussion
• • • • • • Suitable & novel interface?
Existing products Ideas for graphics Thesis Title Anyone know PD, GEM?
Have a good weekend!
CS6022 Principles of Interactive Media - May 4th
CS6022 Principles of Interactive Media - May 4th