Material Design on the GPU

One of the things I like about Material Design is that it builds on principles we see in the real world. Depth is used to convey information, light is used to indicate seams, and drop shadows follow convincing behaviours.

Material design is inspired by tactile materials, such as paper and ink. […] Surfaces can have elevation (z-height) and cast shadows on other surfaces to convey relationships.

- Polymer

What if we take these ideas to the extreme, and treat this as a graphics programming exercise?

The GPU could be used to simulate shading and reflections, new algorithms could be conceived that operate more effectively in a 2D domain, and surfaces could react to user interactions in a more tactile and realistic manner.

There is lots of potential in this idea. In this article, I will focus specifically on rendering text, or “paper and ink.”

 Text in the Real World

reference

references from flickr

When we look at text in the real world, it bears little resemblance to the typography we see on our computer screens and mobile devices. In reality, text erodes, fades, glows, drips, glistens, absorbs, and so forth.

With modern GPUs many of these effects can be simulated convincingly. Below is a real-time demo applying text to three different surfaces.

click here for the full experience

Scroll/pinch to zoom; drag to rotate. Click the arrow icons to change materials.

If your browser doesn’t support WebGL, this GIF shows a few stills of the effect.

 Implementation

 Layout & Rendering

layout

The first step to a text renderer is getting layout, word-wrapping, kerning, and bitmap rendering to work reliably. For this we will leverage some BMFont tools and a few open source modules. These are not suitable for complex text layouts, but good enough for simple Latin-language phrases.

 Signed Distance Field

With the above modules, we have bitmap fonts, but these will scale poorly in 3D.

To smooth glyph edges, we will use a technique popularized by Valve[1] known as Signed Distance Field (SDF) text rendering. You can read more about it here.

Hiero is used to generate the SDF glyph atlases.

For accurate anti-aliasing across all zoom levels, we use standard derivatives and glsl-aastep. The glslify shader looks a bit like this:

#extension GL_OES_standard_derivatives : enable
#pragma glslify: aastep = require('glsl-aastep')

void main() {
  float sdf = texture2D(sdfMap, vUv).a;

  gl_FragColor.rgb = vec3(0.0);
  gl_FragColor.a = aastep(0.5, sdf);
}

Result:

sdf

special thanks to some helpful people for showing me the derivatives trick

 Feathered Edges

Next, we mimic some of the feathering and roughness that comes from real ink on paper. A more involved technique might use a fluid solver, but for our purposes we will just use simplex noise: glsl-noise.

We distort the distance field with multiple layers of noise, summing them up to fake the feathering. The values below are exaggerated for demonstration.

#pragma glslify: noise = require('glsl-noise/simplex/2d')
#pragma glslify: aastep = require('glsl-aastep')

float ink(float sdf, vec2 uv) {
  float alpha = 0.0;
  alpha += aastep(0.5, sdf + noise(uv * 1000.0) * 0.1) * 0.3;
  alpha += aastep(0.5, sdf + noise(uv * 50.0) * 0.2) * 0.3;
  alpha += aastep(0.5, sdf + noise(uv * 500.0) * 0.2) * 0.3;
  return alpha;
}

Result:

noise

 Absorption

We can again use noise to lerp between different strengths and distorted distances. This can add some variety to the text and help make it appear faded and absorbed by the paper surface.

Result:

absorb

 Lighting

Next, we apply per-pixel lighting to the ink color.

Each bitmap glyph is submitted with a second set of UVs that align with the background plane’s texture coordinates. Then, we can sample the normal, diffuse and specular maps of the paper and apply basic lighting to our ink.

For this we are using a few glslify modules:

The lighting dramatically improves the realism of the scene and provides a subtle shine while moving the camera. Different BDRFs could also be explored, e.g. for wet paint or ink.

Result:

lighting

 Further Thoughts

The technique here works well for materials like brick, paper, and burlap. It may be possible to pre-compute the distortions into lookup tables to optimize the shaders.

echo

Animating the text can be done easily by modulating the threshold passed to aastep().

float alpha = aastep(animate, sdf);

For a “hand drawn” animation, the SDF atlas could store motion vectors in the red and green channels. Other effects, like dripping paint, emboss, displacement and environment mapping would also be interesting to explore in this domain.

 Source

The demo was written in ES6 and uses a host of open source tools, including ThreeJS and glslify.

https://github.com/mattdesl/physical-text

 
603
Kudos
 
603
Kudos

Now read this

Drawing Lines is Hard

Twitter: @mattdesl Drawing lines might not sound like rocket science, but it’s damn difficult to do well in OpenGL, particularly WebGL. Here I explore a few different techniques for 2D and 3D line rendering, and accompany each with a... Continue →