News

Unleash JS to sync audio and video to animation

Popcornjs01

Learn how to trigger events easily at any timeframe and play back music or video by using Mozilla’s Popcorn.js?

Popcornjs01

GET THE TUTORIAL CODE NOW


Once upon a time syncing up music or video with animations, overlays or other content could be done quite easily with Flash, but since the demise of that plugin it’s become harder to achieve this kind of effect.

HTML5 has given us a great way to natively play audio or video in the browser but it’s still a hassle to try and get something to happen at a key point in your media. Fear not though because Mozilla has come to the rescue with their media library Popcorn.js. This has a very simple premise to enable the developer to code content that can be triggered at various times during the playback of their media.

If you’ve ever seen interactive music videos by Arcade Fire and Rome for example, then this is the sort of library that enables the triggering of new scenes or animations at key points in their music videos. In the tutorial we are going to have a 3D scene in the background of the page, created in WebGL with three.js. Over the top of that will be some regular DOM content, which will be hidden. Popcorn will be used to load audio, play it and then trigger camera moves in the 3D content, while also fading in with CSS transitions, with the regular div-based content over the top.


1. Up and running

Open the start project folder in Brackets or a similar code editor and open index.html. Scroll down to the body tag and add the following div tags for our on-screen messages that will be shown at key points with the music. Copy and paste this code three more times and change the id to two, three and four.

<div id="one" class="outer hide">
<div class="middle">
<h1>&ldquo;Globally, loss-related floods have more than tripled since 1980&rdquo;</br><small>Munich Re Insurance</small></h1>
</div>
</div>

2. Link to popcorn

Change the text inside the <h1> tag for each of those copied sections and you can refer to the text in the finished file or add your own. A little further down the body section you will see a bunch of script tags for the 3D animation, add this at the bottom to link to the Popcorn library.

3. Add the music file

Scroll back up the page to just under the div tags from Step 1 and add this HTML5 audio tag to link to the MP3 audio. Later on, this audio tag will be hidden on the screen using CSS and the Popcorn.js library will link to this so that animation can be triggered based on the current time of the song playing.

<audio id="myAudio">
<source src="autumn-leaf.mp3" type="audio/mpeg">
</audio>

4. Start styling

In the head tag on the page, add an opening and closing style tag, then add the following CSS into that. Here the body tag gets the padding and margin removed and the right typeface is set for all content. Any content that overflows the page will be set to hidden, as is the audio from the previous step.

body {
padding:0;
margin:0;
overflow:hidden;
font-family: 'Oswald', sans-serif;
color:#ffba00;
}
audio { display:none; }

5. CSS animation

Now the code adds some very simple CSS3 animation. The hide class has been applied to all the content in the body section of the page. A transition for the opacity is applied, which sets the length to half a second. When content needs to fade in, the show class can be applied to the div and it will fade in, taking half a second to do so.

.hide {
opacity: 0;
transition: opacity .5s ease-in-out;
-moz-transition: opacity .5s ease-in-out;
-webkit-transition: opacity .5s ease-in-out;
}
.show {
opacity: 1;
}

6. Position the messages

As there will be animated 3D content using WebGL in the background, any new content to be displayed needs to be shown over the top of this. As such the z-index, which is like the height, is set to be higher than the rest of the page. This section is set to fill the browser width and height, positioned absolutely in the top left.

.outer {
z-index: 10;
width: 100%;
height: 100%;
position: absolute;
top: 0; left: 0;
}

7. Vertical alignment

Inside the fullscreen div of the outer created in the previous step is another div. The text inside here should appear in the centre of the screen horizontally and vertically. The middle class here will ensure that happens by aligning it on the vertical axis with the page centre.

.middle{
min-height: 100%;
min-height: 100vh;
width: 100%;
display: -webkit-flex;
display: flex;
align-items: center;
-webkit-align-items: center;
}

8. Set the headings

All of the text on the page is set with headings so here, the CSS for that is set. Because the background has a number of colours, there is a text shadow on the text to help it stand out against the background. The size of the text is also increased and centred horizontally.

h1{
font-size: 3.8em;
display: inline-block;
width: 40%;
margin: 0 auto;
text-align: center;
text-shadow: 3px 3px #000;
}

9. Wait for the audio

Before the audio is set to play, it is important to ensure that it has fully loaded and can be played. Add the code here to just under the existing script tags on the page. This will load all of the scripts and content in the body tags before calling the init function. The init function will show the 3D background, so test that in the browser.

<script>
document.addEventListener("DOMContentLoaded", function () {
init();
}, false);
</script>

10. Calling the music

If you’ve loaded the content from a local server or webhost, the 3D scene will show, but no music is playing just yet. Just under the init(); line in the previous step add the next line, which will call a function to play the music. This hasn’t been created yet so don’t run it in a browser.

initMusic();

11. Link and play

Before the closing script tag in Step 9, add the following function. This links to the audio with the id of myAudio, which was added way back in Step 3. Once the link is established, the audio is set to play. Save the document and try this in the web browser, the music will start to play and the 3D scene is visible in the background.

function initMusic(){
popcorn = Popcorn( "#myAudio" );
popcorn.play();
}

12. Call the first animation

The scene needs a little movement so that the first message can be displayed. Add the line below inside initMusic. This is calling a function on line 111 of the scene.js file if you want to look at it. Save this and refresh your browser and the camera will move forward in the scene.

camMove1();

13. Cache the divs

To speed up DOM manipulation of adding and removing classes, variables are created to hold the four div tags that contain text. The ‘show’ class will be added and removed and this will make the appropriate message fade in or fade out at the appropriate time. Add this code inside the initMusic function.

var one = document.getElementById( 'one' );
var two = document.getElementById( 'two' );
var three = document.getElementById( 'three' );
var four = document.getElementById( 'four' );

14. Sync to music

All of the remaining code in this tutorial is added before the closing bracket of the initMusic function at each subsequent step. To make something happen in time to music, the following code is used. This calls its own function after one second of the music playing. Test in the browser to see the message show after one second.

popcorn.cue( 1, function() {
one.classList.add('show');
});

15. Move forward

Adding the next code causes the camera to start to move forward to the next section in the 3D scene by calling camMove2 inside the scene.js file. The text on the screen is made to fade out by removing the ‘show’ CSS class, and so the opacity is removed to 0 again.

popcorn.cue( 13.3, function() {
camMove2();
one.classList.remove('show');
});

16. Add the message

If you looked at the last move in the browser you will have seen the camera move forward towards the buildings and the rain. Now the next code will show the text in the div with the id of ‘two’. This takes place after almost 16 seconds. Save and test to see the effect.

popcorn.cue( 15.8, function() {
two.classList.add('show');
});

17. Lightning flashes

Just after the last message displays on the screen you will hear two bass stabs as part of the music. It’s possible to use this with a lightning effect in the 3D scene to make it more dramatic. Here we turn a flash of light on for just less than half a second in time with the first bass stab.

popcorn.cue( 17.2, function() {
lightOn();
});
popcorn.cue( 17.6, function() {
lightOff();
});

18. Slight change to the colour

Let’s do the same again on the second bass stab. The code here gets the timing just right for that. There are more bass stabs a little later and you can add more by listening to the audio and noting the times yourself.

popcorn.cue( 18, function() {
lightOn();
});
popcorn.cue( 18.4, function() {
lightOff();
});

19. Continuing the journey

Now we wait for the song to be almost 28 seconds in to call the camera to move again with camMove3. The second message is faded out on the screen as the camera moves forward. Save and refresh now.

popcorn.cue( 27.7, function() {
camMove3();
two.classList.remove('show');
});

20. The next message

Just after the camera arrives at the bridge, the next message needs to display on the screen, so in this code the function is called at just after 30 seconds so that the message displays. You’ll notice in the browser window that an animation of a car drives past at this particular point and this is no coincidence as the message is relevant to that.

popcorn.cue( 30.2, function() {
three.classList.add('show');
});

21. Final camera movement

Once the car has driven past the screen it’s time to move the camera forward again to the final destination. At just after 40 seconds the final camera move is called, and the third message fades out on the screen by removing the ‘show’ CSS class. The camera swings up to the mountain for the final text to display.

popcorn.cue( 40.5, function() {
camMove4();
three.classList.remove('show');
});

22. Finish off

As the camera eases into the final position in the background, the final text is displayed on the screen. At 43 seconds the fourth text block gets the CSS class of ‘show’ added so that it fades in, and shortly after this the music finishes playing. Now just save and test the document in your browser to see the full animation and music sync up together.

popcorn.cue( 43, function() {
four.classList.add('show');
});
×