Material Design on the GPU

One of the things I like about Material Design is that it builds on principles we see in the real world. Depth is used to convey information, light is used to indicate seams, and drop shadows follow convincing behaviours.

Material design is inspired by tactile materials, such as paper and ink. […] Surfaces can have elevation (z-height) and cast shadows on other surfaces to convey relationships.

^{- Polymer}

What if we take these ideas to the extreme, and treat this as a graphics programming exercise?

The GPU could be used to simulate shading and reflections, new algorithms could be conceived that operate more effectively in a 2D domain, and surfaces could react to user interactions in a more tactile and realistic manner.

There is lots of potential in this idea. In this article, I will focus specifically on rendering text, or “paper and ink.”

Text in the Real World #

^{references from flickr}

When we look at text in the real world, it bears little resemblance to the typography we see on our computer screens and mobile devices. In reality, text erodes, fades, glows, drips, glistens, absorbs, and so forth.

With modern GPUs many of these effects can be simulated convincingly. Below is a real-time demo applying text to three different surfaces.

^{click here for the full experience}

Scroll/pinch to zoom; drag to rotate. Click the arrow icons to change materials.

If your browser doesn’t support WebGL, this GIF shows a few stills of the effect.

Implementation #

Layout & Rendering #

The first step to a text renderer is getting layout, word-wrapping, kerning, and bitmap rendering to work reliably. For this we will leverage some BMFont tools and a few open source modules. These are not suitable for complex text layouts, but good enough for simple Latin-language phrases.

• Hiero to generate the bitmap fonts
• load-bmfont to load various BMFont formats
• word-wrapper for a basic word wrap algorithm
• layout-bmfont-text to layout the glyphs
• three-bmfont-text to pack glyphs into a ThreeJS BufferGeometry

Signed Distance Field #

With the above modules, we have bitmap fonts, but these will scale poorly in 3D.

To smooth glyph edges, we will use a technique popularized by Valve^[1] known as Signed Distance Field (SDF) text rendering. You can read more about it here.

Hiero is used to generate the SDF glyph atlases.

For accurate anti-aliasing across all zoom levels, we use standard derivatives and glsl-aastep. The glslify shader looks a bit like this:

#extension GL_OES_standard_derivatives : enable
#pragma glslify: aastep = require('glsl-aastep')

void main() {
  float sdf = texture2D(sdfMap, vUv).a;

  gl_FragColor.rgb = vec3(0.0);
  gl_FragColor.a = aastep(0.5, sdf);
}

Result:

^{special thanks to some helpful people for showing me the derivatives trick}

Feathered Edges #

Next, we mimic some of the feathering and roughness that comes from real ink on paper. A more involved technique might use a fluid solver, but for our purposes we will just use simplex noise: glsl-noise.

We distort the distance field with multiple layers of noise, summing them up to fake the feathering. The values below are exaggerated for demonstration.

#pragma glslify: noise = require('glsl-noise/simplex/2d')
#pragma glslify: aastep = require('glsl-aastep')

float ink(float sdf, vec2 uv) {
  float alpha = 0.0;
  alpha += aastep(0.5, sdf + noise(uv * 1000.0) * 0.1) * 0.3;
  alpha += aastep(0.5, sdf + noise(uv * 50.0) * 0.2) * 0.3;
  alpha += aastep(0.5, sdf + noise(uv * 500.0) * 0.2) * 0.3;
  return alpha;
}

Result:

Absorption #

We can again use noise to lerp between different strengths and distorted distances. This can add some variety to the text and help make it appear faded and absorbed by the paper surface.

Result:

Lighting #

Next, we apply per-pixel lighting to the ink color.

Each bitmap glyph is submitted with a second set of UVs that align with the background plane’s texture coordinates. Then, we can sample the normal, diffuse and specular maps of the paper and apply basic lighting to our ink.

For this we are using a few glslify modules:

• glsl-face-normal
• glsl-perturb-normal
• glsl-diffuse-oren-nayar
• glsl-specular-blinn-phong

The lighting dramatically improves the realism of the scene and provides a subtle shine while moving the camera. Different BDRFs could also be explored, e.g. for wet paint or ink.

Result:

Further Thoughts #

The technique here works well for materials like brick, paper, and burlap. It may be possible to pre-compute the distortions into lookup tables to optimize the shaders.

Animating the text can be done easily by modulating the threshold passed to aastep().

float alpha = aastep(animate, sdf);

For a “hand drawn” animation, the SDF atlas could store motion vectors in the red and green channels. Other effects, like dripping paint, emboss, displacement and environment mapping would also be interesting to explore in this domain.

Source #

The demo was written in ES6 and uses a host of open source tools, including ThreeJS and glslify.

https://github.com/mattdesl/physical-text