kelvinluck.com

a stroke of luck

Second steps with Flash 10 audio programming


A while back I did some experimenting with the new Flash 10 audio features. Since then I’ve received a couple of emails from people who have noticed that the flash player can freeze up when the mp3 file is initially extracted with the Sound.extract command – especially with longer mp3 files.

The solution is to simply extract only as much of the sound as you need to work with on each sampleData callback. However, this can get confusing when you combine it with the speed changing code from my first example. So I’ve put together another example which uses this method:

The code is available for download here or you can see it below:

package com.kelvinluck.audio
{
   import flash.events.Event;
   import flash.events.SampleDataEvent;
   import flash.media.Sound;
   import flash.media.SoundChannel;
   import flash.net.URLRequest;
   import flash.utils.ByteArray;    

   /**
    * @author Kelvin Luck
    */

   public class MP3Player
   {
     
      public static const BYTES_PER_CALLBACK:int = 4096; // Should be >= 2048 && < = 8192

      private var _playbackSpeed:Number = 1;

      public function set playbackSpeed(value:Number):void
      {
         if (value < 0) {
            throw new Error('Playback speed must be positive!');
         }
         _playbackSpeed = value;
      }

      private var _mp3:Sound;
      private var _dynamicSound:Sound;
      private var _channel:SoundChannel;

      private var _phase:Number;
      private var _numSamples:int;

      public function MP3Player()
      {
      }

      public function loadAndPlay(request:URLRequest):void
      {
         _mp3 = new Sound();
         _mp3.addEventListener(Event.COMPLETE, mp3Complete);
         _mp3.load(request);
      }

      public function playLoadedSound(s:Sound):void
      {
         _mp3 = s;
         play();
      }
     
      public function stop():void
      {
         if (_dynamicSound) {
            _dynamicSound.removeEventListener(SampleDataEvent.SAMPLE_DATA, onSampleData);
            _channel.removeEventListener(Event.SOUND_COMPLETE, onSoundFinished);
            _dynamicSound = null;
            _channel = null;
         }
      }

      private function mp3Complete(event:Event):void
      {
         play();
      }

      private function play():void
      {
         stop();
         _dynamicSound = new Sound();
         _dynamicSound.addEventListener(SampleDataEvent.SAMPLE_DATA, onSampleData);
         
         _numSamples = int(_mp3.length * 44.1);
         
         _phase = 0;
         _channel = _dynamicSound.play();
         _channel.addEventListener(Event.SOUND_COMPLETE, onSoundFinished);
      }
     
      private function onSoundFinished(event:Event):void
      {
         _channel.removeEventListener(Event.SOUND_COMPLETE, onSoundFinished);
         _channel = _dynamicSound.play();
         _channel.addEventListener(Event.SOUND_COMPLETE, onSoundFinished);
      }

      private function onSampleData( event:SampleDataEvent ):void
      {
         var l:Number;
         var r:Number;
         var p:int;
         
         
         var loadedSamples:ByteArray = new ByteArray();
         var startPosition:int = int(_phase);
         _mp3.extract(loadedSamples, BYTES_PER_CALLBACK * _playbackSpeed, startPosition);
         loadedSamples.position = 0;
         
         while (loadedSamples.bytesAvailable > 0) {
           
            p = int(_phase - startPosition) * 8;
           
            if (p < loadedSamples.length - 8 && event.data.length <= BYTES_PER_CALLBACK * 8) {
               
               loadedSamples.position = p;
               
               l = loadedSamples.readFloat();
               r = loadedSamples.readFloat();
           
               event.data.writeFloat(l);
               event.data.writeFloat(r);
               
            } else {
               loadedSamples.position = loadedSamples.length;
            }
           
            _phase += _playbackSpeed;
           
            // loop
            if (_phase >= _numSamples) {
               _phase -= _numSamples;
               break;
            }
         }
      }
   }
}

You can compare it to the code in the original post to see the changes I made.

One thing to note is that there is still a delay when you load an MP3 in my example. This is because I am using the same FileReference.browse > Sound object hack as last time and this needs to loop over the entire loaded mp3 file while turning it into a Sound object. This wouldn’t be an issue in most use-cases where you have loaded the sound through Sound.load.

I also removed the option of playing the sound backwards in this example as that would have added further complexity to the code and hurt my head even more!



First steps with flash 10 audio programming


As I was reading my RSS feeds yesterday I came across a blog post by Andre Michelle where he released some sourcecode for using the new Sound APIs in Flash Player 10. I had a little spare time so I decided to finally set up FDT to allow me to author flash 10 swfs (which was easier than I expected) so I could do some playing.

The idea was to re-create my wave sequencer experiment using the APIs but to get started I did something simpler. I wrote a little class which allows you to load an MP3 file and play it back with the ability to change the playback speed dynamically. Here it is:

You can see the sourcecode for the relevant file below. The interesting stuff from an audio point of view is happening in the onSampleData callback. This is triggered by the Flash player whenever it needs a new buffer of audio samples to play. The code in that function is commented and hopefully pretty self explanatory. It is derived from code in my old wave sequencer experiment which was itself derived from some code in the popforge library.

package com.kelvinluck.audio
{
   import flash.events.Event;
   import flash.events.SampleDataEvent;
   import flash.media.Sound;
   import flash.net.URLRequest;
   import flash.utils.ByteArray;

   /**
    * @author Kelvin Luck
    */

   public class MP3Player
   {

      private var _playbackSpeed:Number = 1;

      public function set playbackSpeed(value:Number):void
      {
         _playbackSpeed = value;
      }

      private var _mp3:Sound;
      private var _loadedMP3Samples:ByteArray;
      private var _dynamicSound:Sound;

      private var _phase:Number;
      private var _numSamples:int;

      public function MP3Player()
      {
      }

      public function loadAndPlay(request:URLRequest):void
      {
         _mp3 = new Sound();
         _mp3.addEventListener(Event.COMPLETE, mp3Complete);
         _mp3.load(request);
      }

      public function playLoadedSound(s:Sound):void
      {
         var bytes:ByteArray = new ByteArray();
         s.extract(bytes, int(s.length * 44.1));
         play(bytes);
      }
     
      public function stop():void
      {
         if (_dynamicSound) {
            _dynamicSound.removeEventListener(SampleDataEvent.SAMPLE_DATA, onSampleData);
            _dynamicSound = null;
         }
      }

      private function mp3Complete(event:Event):void
      {
         playLoadedSound(_mp3);
      }

      private function play(bytes:ByteArray):void
      {
         stop();
         _dynamicSound = new Sound();
         _dynamicSound.addEventListener(SampleDataEvent.SAMPLE_DATA, onSampleData);
         
         _loadedMP3Samples = bytes;
         _numSamples = bytes.length / 8;
         
         _phase = 0;
         _dynamicSound.play();
      }

      private function onSampleData( event:SampleDataEvent ):void
      {
         
         var l:Number;
         var r:Number;
         
         var outputLength:int = 0;
         while (outputLength < 2048) {
            // until we have filled up enough output buffer
           
            // move to the correct location in our loaded samples ByteArray
            _loadedMP3Samples.position = int(_phase) * 8; // 4 bytes per float and two channels so the actual position in the ByteArray is a factor of 8 bigger than the phase
           
            // read out the left and right channels at this position
            l = _loadedMP3Samples.readFloat();
            r = _loadedMP3Samples.readFloat();
           
            // write the samples to our output buffer
            event.data.writeFloat(l);
            event.data.writeFloat(r);
           
            outputLength++;
           
            // advance the phase by the speed...
            _phase += _playbackSpeed;
           
            // and deal with looping (including looping back past the beginning when playing in reverse)
            if (_phase < 0) {
               _phase += _numSamples;
            } else if (_phase >= _numSamples) {
               _phase -= _numSamples;
            }
         }
      }
   }
}

As you can see, there are three public methods in the above class. loadAndPlay will load an mp3 file into a sound object and start playing it at the desired playbackSpeed. stop will stop the currently playing mp3. And playLoadedSound will start playing an already loaded sound object at the desired playbackSpeed. This is useful if you have already preloaded your sound objects but it is also useful for another important reason as you can see in the demo.

Thanks to some great work from an old friend of mine, it is possible to dynamically create a Sound object based on an MP3 loaded through the new FileReference.load() functionality in Flash 10. This is why in the demo you can browse for an mp3 file on your local machine which can then be dynamically controlled by Flash immediately without sending it to a server first.

You can download the complete FDT project of my demo here if you want to look through all of the code. I’m excited by the possibilities that are opening up in flash now that Adobe made some noise – I’ve got a long way to go before I can do anything nearly as incredible as the Hobnox audio tool but I’ve got some ideas and I’m looking forward to playing around with them :)

Update: Check out my follow on post where I examine how to extract the audio on demand rather than up front.