Summoning a Plague of Frogs in Three.JS in Support of Waterparks

It’s been about 2 years since 300 Entertainment, MDDN and I last worked on a Waterparks campaign. This time around, I was inspired by Awsten’s explanation of the frog on the new album cover being a symbol of both vileness and impurity in some cultures and prosperity and luck in others. For Awsten, this new record, Intellectual Property, has been a way for him to address growing past religious guilt, which I certainly get.

When I think about frogs, the first visual that comes to mind is the plague of frogs which (spoiler if you haven’t seen it) caps off Paul Thomas Anderson’s Magnolia and in particular Stanley Spector’s memorable line, “This happens. This is something that happens.” Thinking back to our previous successful exploits in using Twitter activity to drive an audio/visual activation, I proposed that we visualize a plague of frogs on the web, driven by tweets of the hashtag #IntellectualProperty. In addition to a swarm of amphibians appearing, a new sort of audio mechanic would also be utilized: signal strength. Similar to an old radio, the signal strength grows (as the plague multiplies,) allowing fans to tune into a transmission of new audio from the album. However, when the plague diminishes, so does the signal strength. Developed in web based 3D, the experience feels like a dynamic participatory music video. In the end, it is a very thematic way for fans to work together to unlock content while also spreading the album hashtag on Twitter.

The plague is now over but please read on to learn how it came together.

Overview

For this dev blog, I'd like to discuss those jumping frogs but first, here's a quick overview of how everything else is functioning.

We are listening for new tweets on the #IntellectualProperty hashtag using the Filtered Stream API. For more info on that, check out the last Waterparks dev blog but be wary about the new Twitter terms, conditions, and costs as our app was shut down (after launch) and I'm still investigating the cause.

I developed a few simple custom Vue.js components for this build, including the SignalMeter in the bottom right corner and the Marquee of incoming tweet usernames. The meter is just a simple red div sliding across a measurement axis I designed in Figma. The marquee relies on Tom Jenkinson's excellent dynamic-marquee library to usher new elements in and understand when new space is available for another. Well done Tom! (A fellow SoundCloud alumnist.)

Naturally, the bulk of this experience is created using Three.js. In the center of the scene, is a Salt rock lamp modeled (and offered for free) by Timothy Ahene. Thanks Timothy, enjoy all those free coffees!

We use a pair of Audio objects to create the tune in radio effect: one static noise audio and another of the clear audio transmission. Each tweet strenghtens the audio signal to 100% and then Greensock decays it back to 0%. This signal strength is then used to dynamically adjust the volume of both audio files. So, if the signal is at 100% you hear the audio transmission and when it is at 0% you only hear static. Then you can just setup two computed properties to handle the volume adjustments.

// Noise volume
const noiseVolume = computed(() => {
  // Return volume
  return 1.0 - signal.value

})

// Transmission volume
const transmissionVolume = computed(() => {
  // Return volume
  return signal.value

})

The last element worth mentioning before we get into the frog business is the use of post-processing in Three.js to achieve the static overlay and glowing effect of the lamp. Similar to the audio technique used above, we use a FilmPass and HueSaturationShader to desaturate and add static to the scene when the signal strength is at 0%. I really like this visual because it makes the static scene look like an inactive nature camera. Then we use the UnrealBloomPass and some selective blooming to just make the lamp glow. This glow is also connected to the signal strength and inherently the incoming tweets. Now, let's talk frogs.

Frog

I can't recall dealing with pre-rigged animated meshes in Three.js before and I certainly have never instantiated hundreds of them. This was a bit of a learning experience but in the end, we were able to get a decently performative solution going.

Loading Model

I purchased a 3D model of a frog that was rigged with many different animations but took some time to simplify the model a bit and only extract the single jumping animation I required. I also highly compressed the textures to keep the overall model size low. In the end, I had a small .glb file which I could load in using GLTFLoader. This is also a good point to adjust any aspects that will be shared over all frogs, such as materials and shadow settings.

Cloning

Since this model includes a skeleton which is used for the animations, cloning it requires using the SkeletonUtils.clone method. Once, the model is cloned, we can begin working with this particular frog instance. First, we'll want to position it. In the case of our app, frogs appear in the round and hop pass the salt rock lamp before disappearing again. So, we can position each frog initially at a random point on a circle outside the perimeter of the rock.

// Random start position
let startAngle = Math.random() * 360
let startRadians = THREE.MathUtils.degToRad(startAngle)
let startX = Math.cos(startRadians) * 3
let startZ = Math.sin(startRadians) * 3

// Position frog
frog.position.x = startX
frog.position.z = startZ

Now, let's work on setting up our jump animation.

Animation

Three.js uses an AnimationMixer to playback different clips of animations associated with a model. Let's start by initializing one of those.

// Initialize mixer
let mixer = new THREE.AnimationMixer(frog)

We then need to create an animation clip of just the jump animation. Since our model originally had all of these clips running one after another, we can use the AnimationUtils.subclip method to clip the frames we require.

// Get jumping subclip
let jumpClip = THREE.AnimationUtils.subclip(frogModel.animations[0], 'jump', 0, 100)

Then, we can create an AnimationAction on the mixer for the jump clip. I've noticed that clamping the animation to the final frame let's the jumps flow better, so we'll do that too here.

// Create action
let jumpAction = mixer.clipAction(jumpClip)

// Clamp action
jumpAction.clampWhenFinished = true

You can call the play() method on this jumpAction now but in order to see anything, you'll need to update the mixer in your render loop. We're going to need a mixer for every single frog so all of these mixers are stored in an mixers[] array and then updated in the render.

// Calculate mixer delta
let mixerUpdateDelta = clock.getDelta()

// Loop through mixers
mixers.forEach(mixer => {
  // Update mixer
  mixer.update(mixerUpdateDelta)

})

At this point, you'll see a frog jumping in place. Let's use Greensock to animate it jumping forward across the scene without smacking into the salt rock lamp.

Jump!

After a bit of back of the napkin math, I determined I wanted our frogs to jump 4 times at a distance of 1.5 before being removed from the scene (for performance reasons.) I found a nice jump angle (155°) originating from the starting angle, that had the frogs jumping right pass the rock, rather than into it. Let's first establish those variables.

// Initialize jumps
let jumps = 0
let maxJumps = 4
let jumpDistance = 1.5

// Initialize jump angle
let jumpAngle = startAngle + _.sample([155, -155])

I could then establish a jump function. First, we'll determine where the frog if jumping to based on the frog's current position, jump angle, and jump distance. Then, we'll have the frog face that direction using the lookAt method.

// Jump
function jump() {
  // Initialize jump position
  let jumpRadians = THREE.MathUtils.degToRad(jumpAngle)
  let jumpX = Math.cos(jumpRadians) * jumpDistance + frog.position.x
  let jumpZ = Math.sin(jumpRadians) * jumpDistance + frog.position.z

  // Look at jump position
  frog.lookAt(new THREE.Vector3(jumpX, 0, jumpZ))

  // ...

}

To wrap up our jump function, we'll use Greensock to tween the frog's position to the established jumpX and jumpY. Luckily, Three.js provides the duration of the jump animation which we can pass into the gsap function as jumpClip.duration. Now, when the gsap animation starts, we'll reset the jump action and play it, synced alongside with the repositioning. This gives us the jump animation we were looking for! Each time this animation completes, we'll incremenent the jump variable we estalished earlier until the maximum amount of jumps is achieved. At this point, the frog will be removed from the scene and mixer removed from the mixers array.

// Animate jump
function jump() {
  // ...

  // Animate jump
  gsap.to(frog.position, {
    duration: jumpClip.duration,
    x: jumpX,
    z: jumpZ,
    onStart() {
      // Play jump action
      jumpAction.reset().setLoop(THREE.LoopOnce).play()

    },
    onComplete() {
      // Increment jumps
      jumps += 1

      // If less than max jumps
      if (jumps < maxJumps) {
        // Jump again
        jump()

      } else {
        // Remove frog from scene
        scene.remove(frog)

        // Remove mixer from array
        mixers.shift()

      }
    }
  })
}

Now, all we need to do is jump!

jump()

Acknowledgements

Thanks to Freddie Morris, Miles Sherman, and the teams at 300 Entertainment, MDDN, and Fueled by Ramen for giving me an opportunity to work on Waterparks again. Congrats to Awsten and his Waterparks bandmates on their new record, Intellectual Property, out everywhere now.