"The Interactive Audio Journal"
www.iasig.org
*************************************************************************************
Editor: Alexander Brandon
*************************************************************************************
In This Issue:
Section I: From the Chairman: Some opening comments from IA-SIG Chairman, Mark Miller
Section II: Official IA-SIG Announcements: All official announcements regarding IA-SIG members, activities, and special events. In this issue, plans for GDC.
Section III: Working Group Reports: Status Reports from the 3D Working Group, the Interactive Composition Working Group, the Intellectual Property Working Group, the DSP Working Group, and the Multi Format working Group.
Section IV: Features: "The Composition of Ultima Ascention": an interview with George Oldziey about his work on the soundtrack of one of the year's most broad sweeping computer role playing games, and details about one of the first computer role playing games to use live orchestral music interactively.
Section V: Industry Corner: This section contains news, interviews and information from our member companies.
<Note: Items printed and re-printed in this section do not necessarily represent the views of the IA-SIG or its management although we do try and insure that there is useful content contained in everything that we publish.>
Section VI: Developer's Corner: An interview with Masaya Matsuura, the creator of the music as well as the producer for "Parappa the Rapper" and "Um Jammer Lammy", two of the most musically interactive games ever released.
If you are interested in contributing to "The Interactive Audio Journal" please contact Mark Miller (mark@groupprocess.com)
*************************************************************************************
Please Join the IA-SIG!
The Interactive Audio Special Interest Group (IA-SIG) exists to allow developers of audio software, hardware, and content to freely exchange ideas about "interactive audio". The goal of the group is to improve the performance of interactive applications by influencing hardware and software design, as well as leveraging the combined skills of the audio community to make better tools.
The IA-SIG has been influential in the development of audio standards, features, and APIs for Microsoft Windows and other platforms, and has helped numerous hardware companies define their directions for the future. Anyone with a commercial interest in multimedia audio is encouraged to become a member of the IA-SIG and participate in IA-SIG discussions.
*************************************************************************************
IA-SIG Steering Committee
Chairman: Mark Miller (mark@groupprocess.com)
Steering Committee:
Rob Hubbard (EA), Monty Schmidt (Sonic Foundry), Danny Petkevich (Staccato Systems), Brian Schmidt (Microsoft), Alexander Brandon (Straylight Productions), Tom White (MIDI Manufacturers Association).
IASIG Advisory Board
Thomas Dolby Robertson (Beatnik), David Mash (Berkelee School of Music), Craig Anderton (EQ Magazine), Gordon Currie (Portal Productions), Dale Gulick (AMD), Rudy Helm (At the Helm Productions)
***********************************************************************************
A message from IA-SIG chairman, Mark Steven Miller (Mark@GroupProcess.com)
This past quarter has been one of transition for the IA-SIG.
As in the past, we have lived through another 'winter' slump'
where the reflector goes quiet and one wonder's where everyone
has gone. I don;t know about the rest of you, but I have been
hard at work on the Game Developer's Conference Audio Track and
the accompanying IA-SIG events.
The full schedule for GDC Audio can be found elsewhere in the
newsletter (or on the site), but I though that I would present
a few highlights from this year's proceedings. In addition to
meetings of the ICWG (Friday 1:00PM - 2:00PM), the IPWG (Saturday
12:30PM - 1:30PM) and this years General Meeting (Friday 6:00PM
to 8:00PM), we will also see a presentation of papers by the 3DWG
member companies on proposed I3DL3 directions (Sunday 9:00AM to
11:30AM). This double length session and round table will provide
a great opportunity for IA-SIG members to provide the 3DWG with
feedback at the beginning of the L3 process.
As to the main conference audio track, here are some not to be
missed highlights...
Friday
10:30 am to 11:30 am
Building Better Soundtracks: Licensing Hit Songs in Your Game
Victor Rodriguez-Sony
Music licensing is not new to the film or television media, but
it is fairly new to computer gaming, and the rights a game producer
must secure are very different than the sort of rights a film
producer would be interested in, for example. As computer games
approach the visual sophistication of motion pictures, there is
a growing, competitive need to make the games sound better as
well. Likewise, the music companies see an opportunity to cross-promote
new artists by featuring them in computer games (often before
the songs have any radio play). The next few years are guaranteed
to be an exciting time in this area. Legal precedents will be
set, and soon an "industry standard" will emerge from
the chaos of creative and licensing issues being currently debated.
Saturday:
4:00 pm to 5:00 pm
The LucasArts iMUSE System: How Game Music Should Work
Michael Land and Peter McConnell
Who says streamed digital music can't be as interactive as MIDI? For almost a decade, the LucasArts iMUSE System has been at the cutting edge of interactive music in games, providing seamless soundtracks for LucasArts titles from the days of the Adlib card to the live bands of today. During these years, the system has evolved a unique and powerful paradigm, one that simplifies the job of the game programmer while providing the composer with extensive creative options. This makes it easier for both sides to see the big picture and mold it into a coherent and responsive interactive score. And unlike most interactive music systems, we put control into the hands of the composer without resorting to scripting -- it's all done with simple data entry fields. Until now the iMUSE systeM has been known only by its results. In this session we expose its inner workings to the game community for the first time.
Sunday:
2:30 pm to 3:30 pm Keynote Session
'Designing Sound for Media and Designing Media for Sound' Keynote
Randy Thom
Designing sound for movies, designing movies for sound. From pre-production through to final mix, legendary sound designer Randy Thom walks us through the ways in which sound design can and should influence and improve other aspects of the integrated media production process. >From its utilitarian uses like suggesting a mood, evoking a feeling, setting a pace, or indicating a geographical locale to the more adventurous such as filling in the ambiguous darkness around the edges of the frame or providing an opportunity to hear the fullness or emptiness of a vast landscape, Randy provides his views on the value that sound can bring to the final integrated product. In addition, he discusses the ways in which visual media can be designed in order to make maximum usage of audio's vital contribution. Drawing upon his experience working on films such as Apocalypse Now, Mars Attacks, Forrest Gump, Contact, and Indiana Jones, Temple of Doom, Randy's keynote ranges from philosophical exploration to hands on advice to provide "not to be missed" insights. All are welcome and encouraged to attend.
Mark Steven Miller
IA-SIG Chairman
President
GroupProcess Consulting
***********************************************************************************
Section II: Official Announcements
The Game Developers Conference (GDC) is at hand and with it
comes the flurry of activity around the SIG.
Interactive Composition Working Group Meeting
1:00 pm to 2:00pm
Friday, March 10th, 2000
The Interactive Composition Working Group of the IA-SIG will hold
an open organizational meeting to discus the groups current progress
and plans for the future.
The meeting will be led by ICWG Vice Chairmen, Jim Hedges and
David Javelosa.
IA-SIG General Meeting
7:00 pm 9:00
Friday, March 10th, 2000
Mark Miller, the acting IA-SIG Chairman will present and overview of the SIG's progress since the last GDC and his vision of the future of the organization, followed by reports from the chairpersons of several working groups.
Intellectual Property Working Group Meeting
12:30 pm to 1:30
Saturday, March 11th, 2000
The Intellectual Property Working Group of the IA-SIG will hold
an open organizational meeting to discus the groups current progress
and plans for the future.
The meeting will be led by IPWG Working Group members, Jennifer
Hruska and Geoff Kirk.
3D Working Group Paper Presentations
9:00 am to 10:00 am
Sunday, March 12, 2000
Brian Schmidt and Conrad Maxwell
The 3D Working Group of the IA-SIG hosts this double length
session on the future of interactive 3D audio. The group has recently
completed and released the I3DL2 guideline which extend interactive
3D audio to allow for real time acoustical simulations (reverberation).
In moving forward from this accomplishment, the group opens their
planning process for the pending Level 3 guideline to include
input and feedback from the GDC community. The session opens with
panel style presentations of proposals for or perspectives on
a "level three" Interactive 3D Audio guideline. Presentations
are ten minutes in length followed by five minutes of audience
Q&A. Up to six presentations are scheduled. Following the
presentations, the session becomes a large format, roundtable
discussion co-moderated by Conrad Maxwell and Brian Schmidt. Audience
participation is highly encouraged.
**************************************************************************************
Section III: Working Group Reports
The Working Group is the main functional aspect of the IA-SIG.
Working Groups generally form around issues of current concern
for the industry. Once formed, they meet either in person or via
the Internet and develop standards and recommended practices document.
These documents represent industry consensus and are published
and made available to all interested parties. This is where the
Working Groups (WGs) report their quarterly progress.
click here for a description of the Working
Group Process
3D AUDIO WORKING GROUP
(3DWG)
Chairman: Conrad Maxwell, Conexant <conrad.maxwell@conexant.com>
Overview:
----------
The group focuses on creating 3D Audio rendering guidelines
to define more realistic audio environments. This effort has lead
to extensions to the Microsoft DirectSound 3.0 API to enable hardware
acceleration, and to the publication in 1998 of the IA-SIG
Interactive 3D Audio Rendering and Evaluation Guidelines (Level
1), describing "minimal acceptable" 3D audio features
for all platforms. The group this year documented the recommended
enhancements to current 3D audio technology, such as Reverb parameters,
object reflections and occlusions, and more, published as the
IA-SIG Interactive 3D Audio
Rendering Guidelines (Level 2).
The I3DL2 document details the 3DWG's recommended approach
to providing state-of-the art Interactive 3D audio using a pre-set
reverberation to simulate a common environment for sound source
and listener. Occlusion and obstruction effects are also specified.
An example API implementation (two 'C' header files) is included
in the specification. enumerating a recommended "DirectSound
Property Set" implementation of the low level I3DL2 source
and listener controls. The Property Set also includes a number
of useful presets for rooms and material types for occlusions
and obstructions.
Current Status:
--------------
Currently the 3DWG will be holding a meeting at GDC to discuss
the future of 3D audio, starting with their ID3L3 guideline discussion.
Level two has been reached with success and it is now time to
move to the next stage.
INTERACTIVE COMPOSITION
WORKING GROUP (ICWG)
Chairman: Brad Fuller <bfuller@pacbell.net>
ICWG FAQ:
Q: What is Adaptive Audio?
A: Adaptive Audio is audio that is delivered via a system that
allows for direct or indirect control of the data and/or the data
stream.
Q: What is the ICWG's definition of an "Architecture"?
A: Architecture: a collection of components and interactions among
those components. A description of elements from which systems
are built, the interactions among those elements, the patterns
that guide their composition, and the constraints on these patterns.
Q: What is the desired output or result of the ICWG?
A: We hope to build a common lexicon of adaptive and interactive
music terms by the end of the year. This is an important step
as it will give us a foundation to move discussions and decisions
along. More importantly, we hope to have the first draft of the
Adaptive Audio Architecture completed by CGDC 2000.
Current Status
___________
IA-SIG Interactive Composition Lexicon
This issue has a proposal from David Javelosa involving terms
and methodologies we use in our interactive music engines. It
takes the form of a lexicon that is intended to help make sense
of the vastly nebulous and undefined terms and techniques composers
and sound designers oftentimes take for granted. Take a few minutes
and pore over it, decide for yourself how you and your company
think interactive music should perform!
Simply print out this issue of the newsletter, fill out the form
as best you can, and fax it to David Javelosa at (310) 581-9705.
"I'd like to hear from all the in-house systems: Crystal
Dynamics, EA, LucasArts, as well as the academic groups, commercial
apps, historic apps, etc..
This could be a very cool collection of concepts, AND a very valuable
doc for the future of interactive music and composition."
-David Javelosa
| IASIG Interactive Composition Lexicon | |||
| MS DirectMusic Producer | Lucas Arts | "Your Company and Tool/Application" | |
| Interactive : any media experience in which progress through the media is determined in some fashion by the experiencer. | |||
| Audio : sound in a broad enough sense to include (in any combination): | |||
| * Musical compositions, fragments, and cues | |||
| * Environmental sounds and ambiance | |||
| * Sound effects | |||
| * Voice over | |||
| I.A.E: Interactive Audio Environment: A logical grouping of audio data in which progress through the data is determined in some fashion by the experiencer. (more detail below) | Player - an audio renderer | ||
| Service: a type of interactive functionality to be provided in real-time by an I.A.E. playback mechanism. | Cue Manager - a software module that executes cue commands received from the game. | ||
| Stream: any logical grouping of sound data within and I.A.E. that occurs over time and can be acted upon discretely by a Service | Playfile - a standard music or sound file containing a linear stream of time-based data, such as an AIFF or MIDI file. | ||
| Intra-stream: a level of I.A.E. functionality that describes manipulations of Streams that do not alter the data within the Stream. (i.e.branching, muting, over-all volume changes, tempo, etc...) | |||
| Inner-stream: a level of I.A.E. functionality that describes manipulations of Streams that do alter the data within the Stream. (i.e. harmonic transposition, rhythmic manipulation, etc....) | |||
| Segment: any logical set of audio data that plays over time. Examples would be a 'one shot' sound effect, stereo Redbook audio file, a single MIDI channel MIDI file, a multi channel MIDI file, a multi-channel digital audio file... etc.... Segments' may ov | segment, motif, band, style | Cue - the basic unit of currency for communications between the game and the audio system. Cues are triggered by the game and contain the composer's instructions for what should happen when a particular game state or event occurs. | |
| Track: a logical subdivision of data within a Segment. | |||
| Structure: The underlying logical framework around which an I.A.E. is constructed (if one exists). In some cases this might be a chord chart that defines the harmonic progression of an I.A.E.. In others, it might be a rhythmic grid that defines the | Style | ||
| Pool: A logical set of Segments within a I.A.E.. Pools are most often defined by a certain functionality or role such as Segments of music data for the percussion instruments that can be strung together to create the rhythm bed for an I.A.E.. | Style | Cue Chart - a collection of cues contained in a single file. | |
| Group: A logical set of Tracks within a Segment. Groups are most often defined by a certain functionality or role such as Groups of Tracks that can be played by the guitar instrument. | Style | ||
| Path: A logical or defined progression through a Pool or Group . A Path or Paths can be preset or generated on the fly. | |||
| Polling Interval: The interval between navigation/decision making points in the logic scheme. | |||
| GameState: An arbitrarily, pre-defined, gameplay condition that can be detected by the game program or the music driver. (Examples: player has died, player achieves a goal, player enters the blue room, player is in peril, player is in combat, etc....) | |||
| IL (interactivity logic): The scheme by which 'the application' decides 'what to do' in a given situation | |||
| Example: If a system were to use MIDI channels to separate data within a Segment then: | |||
| * A Segment comprised of a single MIDI channel MIDI file would have one Track, | |||
| * A multi MIDI channel MIDI file would have as many Tracks as MIDI channels. | |||
| Example: If a system were based upon the playback of multi channel digital audio, then: | |||
| * A Segment comprised of a single channel of digital audio would have one Track, | |||
| * A multi channel digital audio (DA) file would have as many Tracks as DA channels. | |||
| Interactive Audio Environment: The entire group of Segments that one wishes to consider at any given time. | |||
| Please send responses to: | |||
| David Javelosa | |||
| IASIG Interactive Composition Work Group | |||
| djavelosa@yamaha.com | |||
| davidja@brandx.net |
Intellectual Property Working
Group (IPWG)
Chairman: Mark Miller, Mark@GroupProcess.com
Overview
-----------
PURPOSE: The purpose of the IPWG is to facilitate and insure the continued development, availability, growth, and profitability associated with the marketing, distribution, and licensing of sounds and sound sets to the interactive audio community.
GOAL: The immediate goals of the IPWG are to facilitate improvements in interactive audio and to make use and licensing term recommendations regarding reasonable and appropriate distribution of sounds and sound sets into interactive media and playback environments. The recommendations will be based on broad industry input and be consistent with the IPWG Purpose above and copyright-owner rights as intended under existing and evolving copyright laws.
For additional information or to join this group, please contact Mark at Mark@GroupProcess.com
Current Status
--------------
The IPWG has completed its first main milestone. The milestone involved identifying the various constituencies who have a stake in DLS IP issues and charting their respective interests. The next step involves identifying where these interests align and where they conflict. This analysis will provide the basis from which to begin drafting a set of proposed 'deal point' upon which to base future DLS licensing agreements.
Submitted by Mark Miller, Interim Chairman IPWG
Submitted by Mark Miller <Mark@GroupProcess.com>
MULTI FORMAT WORKING GROUP
(MFWG)
Chairman: Michael Land, LucasArts <mland@lucasarts.com>
Overview:
---------
The MFWG was formed to address the problem of taking audio authored
in a variety of channel formats and playing it back on all the
possible types of speaker configurations out there. We started
out with with a chart developed at BBQ '98, in which several input
formats of audio were matrixed to several output formats, with
appropriate channel mapping instructions at each point in the
grid. We began by examining and evaluating this chart, as well
as some general comments that accompanied it. In the course of
discussion, we explored some of the principles and cxommon elements
inherent in the problem being addressed by the chart. As a result,
the chart itself was distilled into a set of much more readable
and understandable rules and procedures. Some other key points
were also identified, such as that low frequency content would
automatically be routed to the subwoofer by the speaker system,
and that total signal power needed to be conserved during format
conversion. The result is that the guidelines are now much simpler
and more useful, without having lost any of the original problem
solving value of the chart.
Current Status
--------------
Since last newsletter we've been going through various approval processes, resulting in a tweaked word or phrase here and there in our final report. An announcement for the closing of the MFWG will be made soon once everything is complete.
Submitted by Michael Land.
MFWG Chairman
Updated by Mark Miller
DSP WORKING GROUP (DSPWG)
Chairman: Keith Weiner, DiamondWare <keith@dw.com>
Overview:
----------
The groups mission is to develop and recommend standard methodologies
for applying real time DSP to sound streams.
Current Status
--------------
Since Thanksgiving there has been no activity for the DSPWG, continued
effort will be made to accomplish a preliminary set of goals.
********************************************************************************
This section contains features and columns on varied topics of IA-SIG interest.
The Composition of Ultima: Ascention
Ultima: Ascention, the ninth and final chapter in a computer game series of sequels spanning over ten years and thousands of hours of gameplay has been one of the most highly anticipated and eagerly received titles of 1999 on any system. The game's reception was dampened only by a few bugs, and regardless of that it still stands on its own as one of the greatest RPGs ever made, with an entire world rendered in beautiful 3d, an incredibly immersive story, and a soundtrack played by a real orchestra. Not like the beeps of the early Ultima titles (1-5) or the Adlib / Soundblaster / MIDI soundtracks of the later ones (6-8). This was a big leap forward in every way for the series, and George Oldziey (GO) was on hand to comment on his work on the musical side. Here is his tale...
AB: Describe your background... what experience do you have in music, music for games, the companies you have worked for, etc.? What, for instance, first got you interested in music composition?
GO: I knew I wanted to be a musician since I was about 8 years old but my path ended up being quite a bit different than I originally anticipated. I planned to be a studio and orchestral trumpet player. I started playing trumpet professionally when I was 15, went to Manhattan School of Music and earned a BM in trumpet performance. I got to play in some great groups while I was in New York, but I ended up having nerve damage to my embouchure (facial muscles) and had to quit performing. I then spent quite a bit of time trying to heal the injury and work my way back. While doing that I took up jazz piano as a hobby and eventually gave up trumpet for the piano. I had composed some pieces during my trumpet playing days but I became more dedicated to the craft of composition after taking up the piano. I eventually earned a Masters degree in Composition (Southwest Texas State University) while continuing to work on my piano chops. I now have a nice balance of composing, university teaching (jazz ensemble at Southwestern University), and playing piano gigs, though I still dream of the days of playing trumpet in orchestras.
In terms of composing for games, it wasn't a specific goal when the opportunity was first presented to me. In early 1994 I had heard about an opening for a composer at Origin Systems in Austin, TX. I submitted a demo of some MIDI arrangements I had done. A few weeks later I got a call with a job offer. Fortunately for me the project I was to work on was a rather high profile and quality title, Wing Commander III. That was my first credit, and it got me hooked on doing music for games.
AB: How did you become involved in the Ultima: Ascention (UA) project?
GO: I had worked for Origin Systems full time from 1994 until 1998, during which time I composed the scores for three Wing Commander games (III, IV and Prophecy), so I have a history with them. When I left to work on my own we agreed that I would do some contract work for them. The first project to come up in that scenario was Ultima Ascension. Even though I had achieved some success with the Wing Commander music I had to re-audition for the job in a sense. I think I had become somewhat typecast by having only done the "Wing" games to that point. I had to prove that I could also provide a sound and style that would complement the world of Ultima.
AB: Describe the orchestra that was used to record the soundtrack..was it a private or publicly hired orchestra.. instruments, accoustic environment, recording equipment, editing environment, anything you would like to list.
GO: The music for Ultima was scored for 30 players. We did
all of the recording here in Austin using local symphony players.
Since we had a rather modest recording budget I ended up hiring
15 players and doing 2 passes for each cue. The score called for
4 pairs of winds including flutes (doubling on piccolo),
oboes (dbl. on English Horn), clarinets (dbl. on bass clarinet),
bassoons, 2 trumpets, 2 horns, 2 trombones, tuba, 8 violins, 4
violas, 2 cellos and 2 basses. They played along with a click
track as well as with pre-recorded tracks of sampled strings and
percussion that I had recorded to DA-88's at my studio. The live
recording was done at Dream Creations Studios in Austin and recorded
straight to DA-88's. All the tracks were then taken to Origin
Systems and dumped into ProTools for the final mix-down into surround.
AB: Speaking in general terms, how would you compare the budget (time as well as money) required for UA's soundtrack using an orchestra to more (currently) conventional means of game music soundtrack composition and recording? Naturally one would expect it to be significantly larger, but to reword the question, is that "surprisingly" significant, or "expectedly" significant?
GO: In terms of money, I would say it was not significantly larger. It ended up being about 20% percent of the entire music budget. The orchestral part of the process was not really that expensive, at least compared to most movie sound track recording. One thing that helped is that Texas is a right to work state, which means that you can not be denied work for not being in a union. As a result one is not bound by all of the standard scales and extra payments associated with recording. We paid the musicians a buy-out fee for their time, which both sides considered fair. Being a player myself, I wanted to make sure the players were paid a fair wage, which they were.
In terms of time, it definitely took extra work on my part. Besides presenting the producers with synth/sampler versions of all the music for pre-approval, I had to orchestrate the music for live performance. I ended up paying a copyist to take my Finale generated scores and format and print out the parts for the sessions. Of course, I also made sure there was a little extra in the budget to pay the orchestrator :) I also conducted all of the live performances, which was definitely a blast.
AB: Now, the million dollar question: after initial responses after the game's release, would you say first time buyers as well as longtime fans noticed a significant increase in soundtrack quality and effectiveness over other similar titles and / or previous games in the Ultima series?
GO: I haven't been keeping up with much of the initial response since the game's release, so I can't really answer that fully. All the feedback I've heard has been positive though, especially at Origin. They recently asked me do a similar score for Ultima Online 2. Hopefully this will become a regular feature in my work. One can only hope.
AB: Describe the interactive techniques you used in the soundtrack for UA.
GO: Creating the foundation for the interactivity of the music was the most fun and rewarding part for me. Richard Garriott had a vision in which he wanted individual themes to represent each of the six main virtues (love, courage and truth as well as their antitheses, hatred, cowardice and falsehood). Assigning musical themes to characters and such is nothing new. But Richard wanted to have situations where there were a certain number of the virtues present in different situations as well as present in varying degrees. That required composing themes that not only could stand on their own but could be layered together, with up to 6 present at any one time! I couldn't just layer them flat out. I had to create variations of them in which the thematic material still had it's own integrity yet could be combined in different ways. Finally got to stretch my counterpoint chops That was very challenging at times, but a lot of fun.
AB: What format was the music played back as (MP3, compressed WAV, custom format?)
GO: All of the music is digitally streamed off of the CD in WAV format. The game play music plays at 16 bit 22.050 kHz and the music in the movies (which are MPEG files) plays at 16 bit 44.1 kHz.
AB: Were other systems (such as DirectMusic or Miles) considered before deciding on using a full orchestra? If so, what pros and cons were discussed?
GO: I think there was some discussion early in the development cycle to use a MIDI format of some sort. But I came on board after the decision to stream the music was made. I had already composed about half of the music before the orchestra was added to the process. The only con, so to speak, that was discussed regarding the orchestra was the fact that once the live performances were recorded there could be no changes to the music. This is an idea that is probably foreign to many game producers :) I've worked for a few that liked to tweak the music right up until the last minute, and beyond. So, in a way, it added a lot of stability to the audio schedule. Once we scheduled the musicians we had concrete dates to work with that could not be changed later on.
AB: Were other interactive techniques considered as well such as multiple channel fading and song layering (for instance, one part of a song is played continuously and other parts are layered to match events and objects the player interacts with)?
What kind of schedule did the UA programming team allocate to music interactivity? Would you have wanted more functionality, or was there a comfortable compromise reached?
I'll answer all this at the same time. First of all, I wasn't really in on the discussions involving how interactive the music would eventually be. I was given a scenario that had already been decided upon. But I do know that the game engine did not provide for the level of interactivity that had been desired by the audio director (does it ever?). UA is such a visually intensive game that the programmers were a little reticent about allotting much bandwidth for the audio side of things. It pretty much ended up being a situation where a music cue is triggered by an event, and then interrupted by another event that would trigger a different cue. There is no multi-layering to speak of. But, given the complexity of trying to weave as many as 6 themes together in any one cue, I think it would have been a little crazy to also worry about layering entire pieces together. Unless of course it was a situation in which each layer was a separate theme that could be added or subtracted depending on vicinity or intensity. Now, that would be pretty cool. Maybe next time.
AB: What would you say is the most effective method of adding interactivity to a soundtrack using current and even possibly future technology and game engine capabilities?
This might seem like a copout, but I don't really think about that very much. I'm the kind of composer that likes the parameters given to him, not the other way around. Stravinsky was once quoted as saying that when he started working on a piece he was terrified of the infinite number of choices before him. It wasn't until he, or someone else who had commissioned him, had decided on the form, length and other parameters of the work that he was able to deal with it. I work much the same way. However, once the discussion of music interactivity is taking place during a production cycle, I like to participate and give my 2 cents.
AB: Describe your experience working on UA with an orchestra. (anything from "it was incredible" to "it was also a nightmare at times because of xyz" is welcomed)
It was incredible. Possibly the most fulfilling thing I've done in my musical career so far. They say that one of the reasons orchestra conductors live so long, besides the arm exercise, is the fulfillment and gratification of having control over so many other artists whose egos are as big as yours. It was a real rush conducting that group and hearing my music being played back as I had envisioned it. It was particularly rewarding for me because there had been discussion about using real orchestras for each of the 3 Wing Commander games I worked on, but it always got cut out of the budget. To finally have the opportunity with such a great game as UA was really a dream come true.
AB: As mentioned, I can only scratch the surface of your experiences and opinions you might have to offer concerning the UA project.. if there's anything you'd like to add or comment about please please do so.
I think that pretty much sums it all up. It was fun, fulfilling, challenging, rewarding, and it helped pay the mortgage. What else can anyone ask for?!
**************************************************************************************
Section
V. INDUSTRY CORNER
This section contains news, interviews and information
from our member companies.
<Note: Items printed and re-printed in this section do not
necessarily represent the views of the IA-SIG or its management
although we do try and insure that there is useful content contained
in everything that we publish.>
Nothing new this quarter for this issue. Check out our next issue
for reports from GDC!
********************************************************************************
An Interview with Masaya Matsuura
In this issue we have an interview with famed Japanese composer Masaya Matsuura, head of Nana-On sha, a development company that has produced the sterling Playstation titles "Parappa the Rapper" and "Um-Jammer Lammy". Mark Miller and Alexander Brandon find out how Masaya and his team work their magic.
How did you get started in Interactive Entertainment, and what inspired you?
MM) As you know, I had been a musician. My main interest in becoming a musician came when I listened to Isao Tomita's analog Synthesizer record and saw his studio on TV. I thought I could do this by myself. By the time I was working as a musician, all of the living environment just started to be computerized. I was into making music. I remember I had a hard time getting used to using a computer. As my everyday life skills using a computer grew, my computer music skills grew as well. As my computer skills grew and my creative environment became more computerized, I started to wonder whether this was the best way to create a product.
The fun part of making music is experimenting and making choices. On a CD, you can only represent the final product, not the process or the experimentation.
For Example, (This is not an example for music) if you want
to put Japanese on PC, you always have to spell out by the alphabet
and transfer to Japanese. If that software is not proper it, they
transfer to different and silly Japanese (Chinese) characters.
Many Japanese people have this experiences and laugh. This is
"FUN" side of interactivity and computers. If you print
this out, and show it somebody, however, nobody laughs and this
is just a mistake. There is always this gap for interactive expression.You
have to know what you wanted to know, then you see what kind answer
you get. When I say "The expression process of experiment"
I mean 'to share the experiment with the consumer'.
Please describe some of your early projects. Which ones did
you most enjoy?
MM) I had a band called PSY.S (Pronounced SIZE) since 1985 for 10 years. That's a Pop unit with Female vocals. I think we released 10 albums during that time.
After that, til "Parappa the Rapper" , there were about 10 CD-ROM's that I think I produced myself or produced with somebody else. It is difficult to say what I enjoyed the most. I think ... when I made the soundtrack for the movie called "Sweet Home" (Horror movie by Jyuzo Itami who passed away) This soundtrack was a perfect project for music produced by computers. When I did it, I used FairLight 3. It was very exiting to build the orchestral arrangement. I did everything by hand. I remember I only recorded 2 bars in each for 6 months.
Which publishers and companies have you worked with?
MM) Sony Computer Entertainment
Sony Music Entertainment
BMG
BMI
NEC
Fujitsu
Tokyo-Shoseki
What is your current definition of interactive audio?
The most important element of interactive audio is "LIVE". Basically, it is very important to perform the music 'live' or in real time. When you make music by computer, it is necessary to stay away from live performance. But interactive audio is able to create the same beauty of result as live performance.Professional musicians use a wide variety of professional quality tools to create music. Consumer level tools break down from the professional tools. Interactive audio should not be made with these consumer level tools. MIDI and HD Recording are the same thing as MS Office to me. Simply using MS Word to write sentences does not make the text interactive.
"Parappa the Rapper" and "Um Jammer Lammy" could be called revolutionary game ideas in which music and sound is vital to the game's enjoyment. What was your inspiration for these titles?
There is no one single inspiration, rather there are many inspirations involved in making these games. As I told you before, it is from my music experience, but that is not all. Maybe I can give you one example: As you know, Eizenstein's montage skill is well known. I use that skills for translating between music and image, music and character, music and scenario naturally.
I don't make music to match picture exactly. I design the music to express the discrepancy between meaning and sense. What the user receives from my creativity is not simply the music or the images, it is something else, a mystery point. Also this design is only possible with interactive media and it is not possible to do without holding the position of producer.
When I think of the relationship between each expression, the
compilation of the expressions should be different from individuals.
It is totally different from the process and the details of when
I make music for music's sake It looks like limited possibility,
but if you think the other way around, it is a new expression
space. This is one of my methodologies. The individual pieces
may be perceived of as 'some poor work' by some. But viewing the
pieces in the context of the montage and the associated images
creates totally fresh value. But remember I don't make "some
poor work" intentionally. If I want to produce "Some
Poor Work" intentionally, I have to spend a lot of energy
doing it.
What was it like to work on them?
MM) I can't say it is a sophisticated scene. I don't know about the 'scenes' at game companies. I heard there are some companies that hire 100 people to make huge scale titles. It looks like "COMPANY". On the other hand, if it is a small scale, they can communicate really well.
My company "Nana-On sha" has about 20 staff. We are
very flexible to work with. For example, while doing Um Jammer
Lammy, some staff already started another game called "Vib-ribbon".
I think everybody works for our company is very flexible and works
freely. "Vib-ribbon" is the next steps from "Parappa"
and "Um Jammer". It is like a chain reaction. I think
this environment is very important for me.
Before "Parappa the Rapper" and "Um Jammer Lammy",
what techniques did you use to get interactive audio in your projects?
MM) Technically speaking, there was a time when we experimented with switching many audio channels in real time by using CD-ROM XA Format on CD-I. I think that was beneficial. And we also used MacroMedia Director and in collaboration with a C language programmer.
Did you make all of the creative decisions for "Parappa" and "Um Jammer" or was there a team process for making decisions?
We have one team which works on story, script and situation formed by 4 people including myself.
This team examines how to express concepts in English, how
to deal with Japanese, contrast, name,..etc. I make the final
decision for expression of image and music myself. I think that
I am the only on who can conceive of the completed project from
the very start when we are making a music game. Add to this that
we received very serious opinions from Sony from the marketing
point of view. This is very hard...
What equipment did you use to write the music? Did you work
in your own studio or an outside facility? Did you hire musicians,
use MIDI or both? Do you play or sing on the soundtracks yourself?
MM) I am wondering if I am talking too much, I might break the fantasy... as I told you before, I used to use electronic instruments and sequencers. But I don't use them anymore to compose. If I want to make a recording using samplers and electronic instruments, I will use sequencers. I think they are very nice tools for young people who don't know much about music as they can make an ensemble on the desktop. I think this, however, is not a tool to make wonderful music.
Also it is a dangerous tool if you use it for a long time. It might drive you crazy. I strongly believe the developers of these tools should study the relationship between using these tools and changing human emotions. For me, the best use of computers for making music is HDR Recording and Analog Mixing Board for Computer automation.
I try to play instruments by myself even if I am not able to play them perfectly. Even though I've been playing for a long time, I still can't play well. I am better, though, every time I make something.
Chris Parks who is a NY based bass player (he played our release party at the Roxy in NYC Live), he is a great musician. Every time I think I am getting better, I feel the power of his performance. That's the reason why I want to play instruments. This is the same feeling of my wanting to learn English or you (Mark Miller, the editor) thinking you want to learn Japanese.
Also I do compose music in sessions with live musicians. This talent is just like being able to do an international conference without an interpreter. This is the creativity that musicians have. I do these types of sessions in my studio in Tokyo. If you remember those musicians who played at the Roxy, worked on Um Jammer Lammy. But they are as important to the team as Chris Parks is to me.
You seem to have an excellent relationship with Sony. The game soundtracks have even been published by Sony Records. Did Sony approach your company or did you approach Sony? How has the relationship developed over time?
MM) For the soundtrack, we think that we approached Sony Music,
and Sony music thinks that they approached us. I am not sure if
the relationship with Sony Music is excellent or not. I think
it's not a small thing to have developed a relationship and trust
within last 10 years. I learned a lot about the entertainment
music and music business. My current and future style has been
developed based upon all that I have learned over the past ten
years of working with Sony Music.
What are your plans for future projects? Perhaps a sequel,
or something entirely different?
MM) I am sorry, but I can not tell you now.
Pity... looking forward to it though. Who is your favorite video game soundtrack composer?
MM) No particular favorite.
What is your favorite game soundtrack of all time?
MM) No particular favorite.
**************************************************************************************
COMING IN THE NEXT ISSUE (MAY '00)
"The GDC Report"... how did one of the biggest conference
hubs for interactive audio go this year? Find out!
"Plans for the future"... an update on the future of
the IA-SIG and its efforts.
"Interactive Streaming"... MP3 is expensive, ADPCM too
big... just what will be the next big standard in music and sound
streaming?
IASIG NEWSLETTER VOLUME 1 ISSUE 4 FEBRUARY 2000