A tutorial on how to build a fluid media gallery in React powered by WebGL.
Intro
This delightfully playful gallery is perfect for spicing up a user flow between multiple full pages of content. It’s responsive and very simple to use.
The implementation of react-fluid-gallery is broken up into two layers: React and WebGL. The majority of the gallery’s code is agnostic to React, which allows us to easily extend the technique to other UI frameworks like Vue, Angular, or plain HTML+JS.
Play around with the demo & then let’s dive into how it all works!
TL;DR Show me the code!
React Wrapper
The
ReactFluidGallery component kicks things off by mounting an HTML5 canvas. This canvas will serve as the render target for the underlying WebGL simulation, FluidGallery, which we’ll talk about in a bit.It’s worth noting that is a common pattern in graphics programming. React stores a retained scenegraph that handles render updates when changes occur, and we’re introducing an imperative escape hatch into this retained tree of components via an HTML5
canvas. (more info on this distinction)In addition to mounting the canvas render target,
ReactFluidGallery includes logic for passing on resize, scroll, and touch events to the underlying FluidGallery simulation.WebGL Simulation
The underlying
FluidGallery class implements the core gallery scrolling and WebGL display logic.All of the WebGL bits are greatly simplified thanks to Three.js, which provides some convenient abstractions and renders its output to the previously mounted
canvas.Let’s break down what’s going on here in detail.
FluidGallery.contructor(): First, we initialize all of our slides as WebGL Textures._initTexturetakes special care to differentiate between video and image textures. We then follow fairly standard Three.js initialization steps, including creating a WebGLRenderer, a PerspectiveCamera, a ShaderMaterial that wraps our custom vertex and fragment shaders, and a 3D Scene that contains one object, a single Plane that will be resized to fit the entire render canvas. The Plane has our ShaderMaterial attached, so our custom fragment shader will run once per pixel on the output canvas.
FluidGallery.onScroll(event): This is called from our React wrapper every time the user scrolls via the mouse wheel or touch events.
FluidGallery.update(): This gets called once per animation frame. We’re usingrequestAnimationFramevia the raf package to ensure that our simulation updates and renders smoothly. Internally, we’re aggregating the current scroll velocity viathis._speedand adding it tothis._positioneach frame. We also apply some friction to the scroll speed so it eventually slows down.this._positionis a floating point number that goes from zero up to the number of slides in the gallery, where an integer value means a single slide is being shown and a floating point value between two integers represents a partial transition between those two slides. Finally, we update all of the shader parameters to reflect their current values depending on which slide(s) are currently visible.
FluidGallery.render(): This also gets called once per animation frame. It just renders the Three.js scene to the targetcanvas.
The Three.js scene uses a shader material to render pixels on the canvas. WebGL v1 shader pipelines include a vertex shader and a fragment shader. For our purposes, we’re using a very simple vertex shader with a more complicated fragment shader displayed above.
Note that these shaders are not JavaScript, but rather GLSL source files that are exported from JavaScript via ES6 template strings for simplicity of bundling. Most bundlers don’t know how to handle
glsl files, but by exporting them as strings from JS files, they’ll work with any JavaScript bundler.The real magic happens in this fragment shader, which takes in two textures and some parameters detailing the current state of the simulation. For example, if
progress is 0, texture1 will be the first slide, texture2 will be the second slide, and the output of the fragment shader will be a pixel for pixel reconstruction of texture1. If progress is 2.5, for instance, texture1 will be the third slide, texture2 will be the fourth slide, and the output will be a roughly 50% mix between texture1 and texture2.If you’d like to learn more about the power of GLSL shaders, check out this great list of resources!
Credit
The original version of this awesome gallery technique was published on the personal website of Tao Tajima.
The React package was bundled using create-react-library.
Follow me on twitter for more awesome stuff like this @transitive_bs