How to Animate Lottie in Response to Audio

The Lottie format is exceptionally well suited for animation tasks both on web and mobile, especially where it involves a sequential progression. We rely on this capability in our interactivity library, which uses scroll events to progress an animation in response.

Today we’re going to quickly look at another set of events that can also be used to make a Lottie animation interactive in a fun way - Audio! An audio signal is in essence a continuous stream of signal values over time. So what we’re going to try to do is take those signal values, and use it to animate a Lottie.

Before we go further, let’s have a look at the results. The embed below is for the web version of this tutorial.

The code in this tutorial itself however, applies to Android, but we've also created versions for iOS and the Web. You can find the code for our demo apps on our GitHub.

If you'd like to test drive it first, skip towards the end and find our web demo embedded in this post. Looks good? Let's get started!

Obtaining an audio signal

Android offers a number of ways to read audio. For this tutorial, we’ll be reading audio directly from the microphone.

We initialise an instance of AudioRecord, and start recording audio. Note: You will need to ensure runtime permissions for RECORD_AUDIO first.

private var recorder: AudioRecord? = null
private val minBuffer = AudioRecord.getMinBufferSize(
        16000,
        AudioFormat.CHANNEL_IN_MONO,
        AudioFormat.ENCODING_PCM_16BIT
)

private fun listen() {
    // lazy init audio recorder
    if(recorder == null) {
        recorder = AudioRecord(
            MediaRecorder.AudioSource.MIC,
            16000,
            AudioFormat.CHANNEL_IN_MONO,
            AudioFormat.ENCODING_PCM_16BIT,
            minBuffer
        )
    }

    // start recording and sampling
    recorder.startRecording()
}

That should allow us to start listening for audio on our microphone. Now we need to get some measurements from it to use with our Lottie.

Measuring the input signal

Now that we have our signal source, we will need to periodically sample the signal to obtain a measurement to which our animation will respond.

We will be considering the RMS (root mean square) value of the amplitude in decibels (dB) for this. Wikipedia has a good explanation here.

private fun sampleAudio(): Double {

    // read audio buffer from the recorder
    val buffer = ShortArray(minBuffer)
    recorder.read(buffer, 0, minBuffer)

    val rms = buffer.map { abs(it.toDouble()).pow(2) }
                    .average()
                    .let { sqrt(it) }

    val rmsdB = 20.0 * log10(rms)
    return rmsdB
    
}

Selecting an appropriate sampling method is probably the most difficult part. Depending on your needs, you may need to tweak this a little. For our purposes though, the RMS value works well.

Cleaning up the input signal for a smoother animation

When sampling the audio, we expect there to be quite a bit of fluctuations. This does not translate very well for an animation so we will need to post-process our signal a little bit. In the demo, we apply these two methods:

1. Apply exponential smoothing over subsequent samples. Once again, Wikipedia has a good explanation. The end result is less flickering, meaning a smoother animation.
2. Apply a small gain to the input signal. We will use this to control the sensitivity of the animation to sounds.

private var ema = 0.0
private val alpha = 0.8
private val preGain = 0.3

private fun sampleAudio(): Double {

    // read audio buffer from the recorder
    val buffer = ShortArray(minBuffer)
    recorder.read(buffer, 0, minBuffer)

    val rms = buffer.map { abs(it.toDouble()).pow(2) }
                    .average()
                    .let { sqrt(it) }

    ema = ema * alpha + (1 - alpha) * rms
    val preGained = preGain * ema
    return 20.0 * log10(preGained)
    
}

We should now have a pretty good set of values coming out of our signal source. We still need to call these methods at specified intervals.

Continuously sampling the audio and animating the Lottie

Now that we are able to obtain a value to use to animate the Lottie with, we can start sampling the audio signal at regular intervals and apply that value to animate the Lottie.

To animate the Lottie, we will need to convert the dB level into a fraction so we can set the animation progress.

fun getProgress(input_dB: Double): Double {
    var perc = input_dB.coerceAtLeast(min_dB) - min_dB
    perc /= (max_dB - min_dB) 
    return perc
}

Here, the values for min_dB and max_dB represent a range limit for the input signal. As a reference, average human speak is about 60 dB and anything above 80 dB is considered “loud” and “unpleasant”.

Continuous sampling can be achieved in a number of ways. Here we use a simple Handler to call our sampleAudio() method at specified intervals. You can also use a Timer to similar effect.

val handler = new Handler(mainLooper);
val SAMPLING_RATE = 100L;

private fun sampleAndAnimate() {
    val inputdB = sampleAudio()
    val progress = getProgress(inputDb)
    lottieView.progress = progress
}

// The handler will call itself at sampling intervals
handler.postDelayed({
        sampleAndAnimate()
        handler.postDelayed(this, SAMPLING_RATE);
}, SAMPLING_RATE);

Here's our web demo that incorporates everything up to this point.

Not bad at all! You will probably notice that while the Lottie progresses to the desired points in response to audio, and as a result the animation isn’t very smooth. This is because we’re jumping from one point to another with no intermediate steps. Let's see if we can fix that!

Making the Lottie animate between progress values

To make the final animation smooth, we need to ensure that the animation progresses between each value gradually. For this, we can use a standard android ValueAnimator like so;

val animator = with(ValueAnimator.ofFloat(0.0f)) {
    interpolator = AccelerateInterpolator()
    duration = SAMPLING_RATE

    addUpdateListener {
        val progress = it.animatedValue as Float
        binding.lottieView.progress = progress
    }

    this
}

This allows the animator to be responsible for progressing our animation. So we need to change our previous method accordingly. We set it up so that whenever a new progress value is obtained, the animation stops in its tracks, and then proceeds to the new value from there.

private fun sampleAndAnimate() {
    val inputdB = sampleAudio()
    val progress = getProgress(inputDb)
    
    with(animator){
        cancel()
        val currentStop = animatedValue as Float
        val maxStop = progress.coerceAtMost(1.0).toFloat()
        setFloatValues(currentStop, maxStop)
        start()
    }
}

And that’s it! We now have a nice, visually pleasing Lottie animation interacting with us! Here's a demo of the web version for you to try out!

Once again, you can find the code for our demo apps on our GitHub. A tutorial video on how to make a Lottie animation sync with audio is also included as below.

Did you find this useful? Let us know!

As you saw, making a Lottie interesting and interactive can be a simple matter animating its progress! We think the technique broadly described above can be used in a number of applications ranging from Audio recorders to your podcasting app!

As always, we are interested to know how you use Lottie animations in your apps! Join our Discord! We love to hear from you.