IXMF Working Group

Introductory Document, January 2002

The purpose of this workgroup is to design an interactive audio system that puts creative control in the hands of the author, ensures implementation of the author’s intent, works on virtually any platform and is available to everyone. Interestingly, the file format to facilitate this legendary goal already exists! It’s called XMF and it’s brand new on the scene, fully worked out.

This group will design an interactive (or “adaptive”) audio system that utilizes the open XMF standard. The goal is to define the specification of features and functionality at a relatively high level but with enough detail to submit to the MMA (MIDI Manufacturers Association) and for a developer to actually build it. With XMF being an open, published standard, anyone can create XMF authoring tools or players for in-house or commercial use.

XMF is essentially a sophisticated file format that can contain various types of data and metadata, organized like a container of files, folders and special information. It’s that special information part that points the way toward this format’s suitability as a vehicle for audio and MIDI assets along with instructions as to their implementation.

XMF (the eXtensible Music Format) was developed by a MIDI Manufacturer's Association (MMA) Working Group (XMF-WG) in 2000-2001, and adopted by the MMA (and its counterpart for Japan, AMEI) in 2001.  The XMF-WG includes representatives of several leading musical instrument and computer multimedia companies, and Beatnik has contributed a reference XMF parser implementation.  The MMA's first XMF File Types are basically an open version of Beatnik's RMF file format containing Standard MIDI Files and any custom DLS instruments needed to play them.  Chris Grigg of Beatnik (and, in his spare time, Control-G) and Rob Rampley of Line 6 are the XMF-WG chair and co-chair, and at 2001's Project BBQ they participated in a 'Rogue Group' that outlined a vision for how XMF might be used for an advanced interactive audio system.

One of the main advantages of an XMF file is that it may contain various types of data, including MIDI and/or audio assets and DLS2, making a single file capable of delivering what it normally takes dozens or even hundreds of files to accomplish. The format is also capable of data compression for efficiency and encryption for security.

The file format is now finished and ready for use by anyone who wishes to take advantage of its eXtensibility. Now it’s time for this workgroup to leverage the existing format as the unit of currency in a cross-platform interactive audio system that we brainstorm and design.

Recommended Workgroup Process:

1. Jan 15, 2002 - Feb.15, 2002
   Workgroup gains a detailed understanding of XMF and its capabilities

   Members read Chris Grigg’s document “Group Report: Towards Interactive XMF” on The Interactive Music Conference Project BAR-B-Q website (http://www.projectbarbq.com/bbq01/bbq01r5.htm), the XMF overview on the MMA website (http://www.midi.org/xmf) and ensuing discussion.

2. Feb. 15, 2002 – June 1, 2002
   Workgroup defines features and functionality of an interactive audio system

   Through discussion of creative objectives in various media, workgroup derives “what it wants the system to do”. Many factors will play a role in this discussion including individual working styles, musicality, platform idiosyncrasies, authoring tool graphic user interface, author/developer communications, formats of implementation instructions, and many more.

3. June 15, 2002 – July 15, 2002
   Sub-Group goes off-line and distills group discussion down to design proposal.

   IA-SIG creates sub-group based on member willingness, orientation towards the subject and experience in formalizing system designs.

4. July 15, 2002 - August 1, 2002
   Design Proposal gets submitted to full group for consensus.

5. August 1 – September 1, 2002
   Sub-Group completes and submits design documentation to the MMA.

DESIGN CONCEPTS

The following is taken from Chris Grigg’s “Group Report: Towards Interactive XMF” on the PROJECT BAR-B-Q website (http://www.projectbarbq.com/bbq01/bbq01r5.htm).

As I outlined in my talk Friday morning [at BAR-B-Q 2001], previous BBQ groups have done work articulating parts of this vision. It is perhaps best appreciated by contrast with the usual working method for interactive audio, which unfortunately in most cases can be stated simply as "I talk to the programmer, we decide what sounds are needed, I make a file for each sound, I give him the files, he puts them in the game but often doesn't use them the right way." This, as the kids like to say, sucks.

We'd rather work according to these principles:

• Defend against misuse of delivered audio
• Use MIDI & audio together
• Simplify tool development
• Standard formats lead to better tools
• Avoid least-common-denominator functionality limits

The following tactics have been identified to help put the audio artist in control:

• Use data-driven runtime audio systems, not code-driven
• Abstract the interface between coding and audio teams
• Have the game request audio services by symbol, not filename
  (Which relates to Brian Schmidt's immortal comment "Anyone who still thinks 
  there's a 1:1 relationship between a sound and a WAV file just doesn't get it.")
• Put audio guy in control of the data (i.e. by providing an editor)

Going further, some niceties we might like to see in an ideal world might include:

• Simplify & formalize content management & hand-off procedures
• Attach audio artist notes to individual sounds
• Generate reports & databases easily from sound, & vice-versa
• Facilitate collaboration among multiple audio artists

Of course this is all high-level goal stuff; there are additional objectives like controlling sound parameters while a sound is playing, making it easy to do commonly needed things like randomization and spatial placement, synchronizing new pieces of music to already-playing pieces of music, providing callbacks so events can be triggered when a sound gets to a certain point, etc.

ENTER XMF

XMF, it turns out, helps achieve all that. It's designed for bundling media files together, and allows content to be organized as files & folders like a computer file system. It also lets you attach metadata to files and folders in a simple and consistent way, so you can attach name tags or composer comments, so simple programs can generate reports based on the contents. Or slightly more elaborate programs could create XMF files based on entries made in a database. And it's extensible, so we can easily add new media formats, data formats, and metadata fields as needed.

Because XMF already handles so much of the organization and housekeeping, it helps us focus our design concepts into a concrete design, and allows us to focus just on the parts of the problem that are left. But it’s important to understand that the current XMF standards don’t include anything like the high-level interactivity functionality we want to see. XMF technology per se is just about providing the vessel that the data rides in. All of that new functionality needs to be invented and represented in data structures before it can be carried inside XMF.

So one useful way to look at the work of our [BBQ] Rogue Group is an attempt to answer the question Larry the O posed: "What do XMF and BBQ-style interactive audio mean for each other?" Of course, that begs another question: "What exactly does 'BBQ-style interactive audio' mean?"

The purpose of this workgroup is to define what we mean by 'BBQ-style interactive audio' and brainstorm the design of a system that accomplishes it. An attempt at a short definition would be: a system by which composers and sound designers create an interactive soundtrack and audition it by simulating target application control input while working in the authoring environment. Once the audio behavior works to the designer’s satisfaction, the assets are delivered and the author’s intent flawlessly realized in the end product.

It’s time to create this system…