I'm now playing around with this new basic blog template from Next.js examples --> available on Next.js GitHub.

Luciano Lupo Notes.

Glitch images using WebGL

Cover Image for Glitch images using WebGL

Wanted to play around for a bit with WebGL and glitching images, was working with Three.js and react and I decieded to make a little project out of it. One day was trying to make images glitch and traditonal way with css was not good enought, so I decided to try ThreeJS with in a React Project. I implemented it in a project I already had with three.js and react, so I could use the same camera and renderer.

This is the end result:

Glitch effect

This are the important libraries involved:

react
three
@react-three/drei
@react-three/fiber

the Plane that will be the "image"

import React, { forwardRef, useRef } from "react";
import { useFrame } from "@react-three/fiber";
import "../../MyCustomMaterial";
import storedState from "../../store";

const Plane = forwardRef(
  ({ color = "white", shift = 1, opacity, args, map, ...props }, ref) => {
    const material = useRef();
    let last = storedState.top.current;

    useFrame(({ clock }) => {
      const { top } = storedState;

      const elapsedTime = clock.getElapsedTime();
      const randomSignFactor = Math.floor(elapsedTime * 100) % 2 === 0 ? 1 : -1;
      const random = Math.random();
      const random2 = Math.random();

      const randomDispFactor =
        (1 + Math.sin(material.current.shift / 10) + random / 10000) *
        randomSignFactor;
      material.current.dispFactor =
        Math.floor(random2 * 100) > 91
          ? randomDispFactor
          : material.current.dispFactor;
      material.current.dispFactor =
        Math.floor(random2 * 100) > 93 ? 0 : material.current.dispFactor;

      last = top.current;
    });
    return (
      <mesh ref={ref} {...props}>
        <planeBufferGeometry args={args} />
        <myCustomMaterial
          ref={material}
          color={color}
          transparent
          opacity={opacity}
          tex={map}
          disp={map}
        />
      </mesh>
    );
  }
);

export default Plane;

This Plane component is part of a larger React application that uses Three.js for 3D graphics. The component represents a plane in the 3D space, and it's defined as a functional component that accepts props such as position, rotation, and scale. for this particular case I'm using it plane and with no rotation, so its behaves kinda like a 2D image. The stored state is not so relevant for this case, the main interesting things are the random generation using clock and the use of useFrame hook to update the displacement map. and of course the myCustomMaterial component which is a custom shader material that uses a displacement map to create the glitch effect

The Shader:

import * as THREE from "three";

import { extend } from "@react-three/fiber";

export class MyGlitchMaterial extends THREE.ShaderMaterial {
  constructor() {
    super({
      uniforms: {
        effectFactor: { value: 1.0 },
        dispFactor: { value: 1.0 },
        tex: { value: undefined },
        disp: { value: undefined },
        hasTexture: { value: 0 },
        scale: { value: 0 },
        shift: { value: 0 },
        opacity: { value: 1 },
        color: { value: new THREE.Color("white") },
      },
      vertexShader: `varying vec2 vUv;
      void main() {
        vUv = uv;
        gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );
      }`,
      fragmentShader: `



      varying vec2 vUv;
      uniform sampler2D tex;
      uniform sampler2D disp;
      uniform float _rot;
      uniform float dispFactor;
      uniform float effectFactor;
      uniform float hasTexture;
      uniform float shift;
      uniform float scale;
      uniform vec3 color;
      uniform float opacity;


      void main() {
        
        vec2 uv = vUv;
        vec4 disp = texture(disp, uv);
        vec2 distortedPosition = vec2(uv.x, uv.y + dispFactor * (disp.r*effectFactor));
        vec2 distortedPosition2 = vec2(uv.x, uv.y - (1.0 - dispFactor) * (disp.r*effectFactor));
        vec4 _texture = texture(tex, distortedPosition);
        vec4 _texture2 = texture(tex, distortedPosition2);
        vec4 finalTexture = mix(_texture, _texture2, dispFactor);
        
        // Seed value
        float v = fract(sin(dispFactor * 1.) * 1000.);
        
        vec4 newTex = texture(tex, uv);
        
        
        
        // Prepare for chromatic Abbreveation
        vec2 ruv = uv;
        vec2 guv = uv;
        vec2 buv = uv;
        
        // Randomize 
        float dispBias = 0.01;
        float amplitude = 20.;


        if (v>.1){
          float y = floor(uv.y * amplitude * sin(dispFactor)) ;
          float x = floor(uv.x * amplitude * sin(dispFactor)) ;
        
          if (sin(50. * y * v) > -0.01) {

            // image row mix moves right left
            ruv.x = fract(uv.x + sin(10. * y * dispFactor) * dispBias);
            guv.x = fract(uv.x + sin(15. * y * dispFactor) * dispBias);
            buv.x = fract(uv.x + sin(100. * y * dispFactor) * dispBias );
            
            //image row mix moves up or down 
            ruv.y = fract(uv.y + sin(10. * y * dispFactor) * dispBias );
            guv.y = fract(uv.y + sin(15. * y * dispFactor) * dispBias );
            buv.y = fract(uv.y + sin(100. * y * dispFactor) * dispBias );
          
          }
          
  
  
          if (sin(50. * x * v) > -0.01) {
            
            //  image column mix moves right left
            ruv.x = fract(uv.x + sin(10. * x * dispFactor) * dispBias  );
            guv.x = fract(uv.x + sin(15. * x * dispFactor) * dispBias  );
            buv.x = fract(uv.x + sin(100. * x * dispFactor) * dispBias  );
            
            // image column mix moves up or down
            ruv.y = fract(uv.y + sin(10. * x * dispFactor) * dispBias  );
            guv.y = fract(uv.y + sin(15. * x * dispFactor) * dispBias  );
            buv.y = fract(uv.y + sin(100. * x * dispFactor) * dispBias  );
           
          }
          


        }

        newTex.r = texture(tex, ruv).r ;
        newTex.g = texture(tex, guv).g ;
        newTex.b = texture(tex, buv).b ;
        newTex *= 1. * opacity;

        // this is to use the same material for images and texts
        if (hasTexture == 1.0) {
          gl_FragColor = newTex;
        } else {
          gl_FragColor = vec4(color, opacity);
        }
      }
      `,
    });
  }

  get effectFactor() {
    return this.uniforms.effectFactor.value;
  }
  set effectFactor(v) {
    return (this.uniforms.effectFactor.value = v);
  }
  get dispFactor() {
    return this.uniforms.dispFactor.value;
  }
  set dispFactor(v) {
    return (this.uniforms.dispFactor.value = v);
  }

  get disp() {
    return this.uniforms.disp.value;
  }
  set disp(v) {
    return (this.uniforms.disp.value = v);
  }

  set scale(value) {
    this.uniforms.scale.value = value;
  }

  get scale() {
    return this.uniforms.scale.value;
  }

  set shift(value) {
    this.uniforms.shift.value = value;
  }

  get shift() {
    return this.uniforms.shift.value;
  }

  set tex(value) {
    this.uniforms.hasTexture.value = !!value;
    this.uniforms.tex.value = value;
  }

  get tex() {
    return this.uniforms.tex.value;
  }

  get color() {
    return this.uniforms.color.value;
  }

  get opacity() {
    return this.uniforms.opacity.value;
  }

  set opacity(value) {
    if (this.uniforms) this.uniforms.opacity.value = value;
  }
}

extend({ MyGlitchMaterial });

This Shader ( which is an extension of ShaderMaterial ) has 3 parts:

  • Displacement
  • Chromatic Aberration

Displacement

        vec4 disp = texture(disp, uv);
        vec2 distortedPosition = vec2(uv.x, uv.y + dispFactor * (disp.r*effectFactor));
        vec2 distortedPosition2 = vec2(uv.x, uv.y - (1.0 - dispFactor) * (disp.r*effectFactor));

Here's what's happening, the shader samples a displacement texture (disp) and creates two distorted versions of the UV coordinates:

distortedPosition: Shifts pixels upward distortedPosition2: Shifts pixels downward

The amount of shift is controlled by:

dispFactor: How much displacement to apply (0-1) effectFactor: Overall intensity multiplier disp.r: Red channel of the displacement texture

Mixing

vec4 _texture = texture(tex, distortedPosition);
vec4 _texture2 = texture(tex, distortedPosition2);
vec4 finalTexture = mix(_texture, _texture2, dispFactor);

The shader samples the main texture twice using both distorted positions, these samples are then mixed together based on dispFactor which creates a smooth transition between the two displaced versions

Additional randomness

        // Seed value
        float v = fract(sin(dispFactor * 1.) * 1000.);

        vec4 newTex = texture(tex, uv);



        // Prepare for chromatic Abbreveation
        vec2 ruv = uv;
        vec2 guv = uv;
        vec2 buv = uv;

        // Randomize
        float dispBias = 0.01;
        float amplitude = 20.;

Here I generate a pseudo-random v value, when v is above 0.1, it creates additional displacement and use amplitude to control how many "slices" the effect creates.

// Horizontal displacement
ruv.x = fract(uv.x + sin(10 * y * dispFactor) * dispBias);
// Vertical displacement
ruv.y = fract(uv.y + sin(10 * y * dispFactor) * dispBias);

Here the displacements get applied in a "Grid" type of way, the sin function is used to create the spaces in between the glitch and not gitched parts.

Chromatic Aberration

        // Prepare for chromatic Abbreveation
        vec2 ruv = uv;
        vec2 guv = uv;
        vec2 buv = uv;

Here I define the red green blue channels and then:

// image row mix moves right left
ruv.x = fract(uv.x + sin(10 * y * dispFactor) * dispBias);
guv.x = fract(uv.x + sin(15 * y * dispFactor) * dispBias);
buv.x = fract(uv.x + sin(100 * y * dispFactor) * dispBias);

//image row mix moves up or down
ruv.y = fract(uv.y + sin(10 * y * dispFactor) * dispBias);
guv.y = fract(uv.y + sin(15 * y * dispFactor) * dispBias);
buv.y = fract(uv.y + sin(100 * y * dispFactor) * dispBias);

each channel gets a different distortion.

newTex.r = texture(tex, ruv).r;
newTex.g = texture(tex, guv).g;
newTex.b = texture(tex, buv).b;
newTex *= 1 * opacity;

Finally the original image tex gets added with the new 3 channel distorted image ruv guv buv, beacuse this 3 separated channels will have pixels only in the places that are visible thanks to the v condition and sin() function.

        if (v>.1){

        //... code ...

          if (sin(50. * y * v) > -0.01) {
            //... code ...
          }

          if (sin(50. * x * v) > -0.01) {
            //... code ...
          }

The clock and the randoms from the React components generate randomness thru time, and update the dispFactor variable, the rest of the randomness happens in the glsl shader, based on the dispFactor variable.