Vertex Displacement Mapping + Normalmap

Normal mapping and vertex-displacement mapping from the same texture.
This ones uses the Kineme texture tools to create the displacement texture. It’s a classic use of the technique.

Not sure I’ll ever use this one ‘in anger’ but it’s an interesting test. I’m glad the lighting works so well, too.

The base colour comes from the normal map, incidentally.

The final screenshot shows the mesh from the back. Not quite sure why it looks so different from that side…

Next I have to work out if I can use the same technique with radial displacement. Think that might be a bit more tricky.

Bump Mapping

Thought I’d have a go using the Normal Map code from the last post to do some bump-mapping.
Turns out it’s quite easy to use the output of the Normal Map CIFilter as normals in a Fragment Shader. All that needs to be done is to transform it into the correct range by multiplying the RGB value of each pixel by 2.0, then subtracting 1.0.

Some examples using a simple Phong lighting model, and normals extracted from first a static image, then the live video from the iSight camera, run through the Normal Map CIFilter:

Here’s the GLSL code:

Vertex Shader:

varying vec3 normal, lightDir, eyeVec;

void main()
{
	// Normal from mesh
	normal = gl_NormalMatrix * gl_Normal;

	// Current Vertex coordinates transformed into World space
	vec3 Vertex = vec3(gl_ModelViewMatrix * gl_Vertex);

	// Light Direction
	lightDir = vec3(gl_LightSource[0].position.xyz - Vertex);
	// Eye Vector
	eyeVec = -Vertex;

	gl_Position = ftransform();

	gl_FrontColor = gl_Color;
	gl_TexCoord[0] = gl_TextureMatrix[0] * gl_MultiTexCoord0;
}

Fragment Shader:

varying vec3 normal, lightDir, eyeVec;
uniform sampler2D texture;
uniform sampler2D normalMap;

void main (void)
{
	// Base color
	vec4 final_color =
	(gl_FrontLightModelProduct.sceneColor * gl_FrontMaterial.ambient) +
	(gl_LightSource[0].ambient * gl_FrontMaterial.ambient);

	vec3 N = texture2D(normalMap, gl_TexCoord[0].st).rgb * 2.0 - 1.0;
	//vec3 N = normalize(normal);	// Normal
	vec3 L = normalize(lightDir);	// Light Direction (normalized)
	float lambertTerm = dot(N,L);	// Base lighting (no ambient or specular contribution)

	if(lambertTerm > 0.0)
	{
		// Add diffuse contribution
		final_color += gl_LightSource[0].diffuse * gl_FrontMaterial.diffuse * lambertTerm;
		// Compute specular contribution
		vec3 E = normalize(eyeVec);
		vec3 R = reflect(-L, N);
		float specular = pow(max(dot(R, E), 0.0), gl_FrontMaterial.shininess);
		// Add specular contribution
		final_color += gl_LightSource[0].specular * gl_FrontMaterial.specular * specular;
	}

	// Output
	gl_FragColor = final_color;// * texture2D(texture, gl_TexCoord[0].xy);
}

Note
You’ll need to stick the GLSL shader inside a Lighting Patch in order to be able to set the properties for the light.
You’ll get an error in the Frag shader about the varying normal not being used.

Normal Map

Inspired by Angelo Pesce’s comment here, I decided this morning I’d have a go at generating a Normal Map from a live video input, using a CIFilter patch. The code is surprisingly simple.

///////////////////////////
////	FUNCTIONS      ////
///////////////////////////

/*
Function to calculate the luminosity of a pixel
*/

float luminosity(vec4 color)
{
	vec4 lumcoeff = vec4(0.299,0.587,0.114,0.0);
	return dot(color, lumcoeff);
}

/*
Creates Normal Map for use in Displacement Map / Height Map GLSL shaders
*/

vec4 tb_normalMap(sampler inPix)
{
	// Sampler dimensions
	vec2 dims = samplerSize(inPix);
	// Working pixel coordinates
	vec2 xy = samplerCoord(inPix);
	// Normalised coordinates of working pixel (used to calculate Normal Z component)
	vec2 xyNorm = xy / dims;
	// Normal grid size (in pixels)
	float offset = 1.0;

	// Working pixel luminosity
	vec4 pix = sample(inPix, samplerTransform(inPix, xy));
	float lumCurrent = luminosity(pix);

	// Derivitive X
	float lumdx = luminosity(sample(inPix, samplerTransform(inPix, vec2(xy.x+offset, xy.y))));
	float dx = lumCurrent - lumdx;
	// Derivitive Y
	float lumdy = luminosity(sample(inPix, samplerTransform(inPix, vec2(xy.x, xy.y+offset))));
	float dy = lumCurrent - lumdy;
	// Normal Z. I'm not sure if this is strictly-speaking correct, but it seems to work!
	float dz = 1.0;

	// Output Normal Map
	return vec4(dx,dy,dz,1.0);
}

///////////////////////////
////    MAIN LOOP      ////
///////////////////////////

kernel vec4 normalMap(sampler Image)
{
	// Output Normal Map
	return tb_normalMap(Image);
}

The idea is that this map can be used to aid lighting of meshes created with Vertex Displacement Mapping / Heightmap techniques.

May or may not be useful…

Here’s what the effect looks like (I actually think it’s quite a nice effect in it’s own right).

EDIT:
Corrected mistake in function name in main loop.

More Radial VDM Screenshots (again)

In the third and fourth screenshots, I’ve swapped the Blending Mode of the mesh. I like the ethereal look of these. The downside is, it’s much more difficult to discern the 3-dimensional form of the mesh. I may try adding some kind of fog-like effect, so that the mesh colour or alpha changes with the z-coordinate of each fragment. This may help  things look more 3D (and is something I’ve been meaning to try, anyway).

More Radial VDM Screenshots

I’ve changed the blending mode of the GLSL Mesh in the last two (to ‘Over’ and ‘Add’ respectively).

Radial Vertex Displacement Mapping

Thanks to some great work (and a fortuitous accident) from the good people at Kineme.net, it’s now possible to use moving images as Displacement Maps in a GLSL Vertex Shader in QC. So I thought I’d give it a go.

The input is the iSight camera, with luminosity modulating the Radius parameter of a function to bend a GLSL grid into a sphere, so what we have is essentially a radial distortion of a sphere, based on the luminosity of the input texture.

This is very cool, and something I’ve been wanting to do for ages. It opens the doors to all sorts of mesh-related fun, as you can potentially use custom CIFilter or GLSL Fragment shader code to create different displacement images to feed into the patch.

The only problem is, it runs quite slowly on my machine. It’s undoubtedly much faster to do this kind of thing with Vertex Buffer objects, but that means custom OpenGL code in a QC plugin, which is way beyond my capabilities currently. Sadly.

Spherical Harmonics Rendering Issues

It seems some people have been having problems getting the Spherical Harmonics qcClip to work properly. Everything works fine on my 1st-generation MacBook Pro, but it seems some other systems/graphics cards have issues. In fact, I just tried it on my soon-to-be-replaced G5, and it didn’t render correctly on this system either.

Apologies to anyone else effected. I’m not sure what the issue is as yet, but I’ll try and track it down.

Any clues anyone might have as to why it might not be working greatly appreciated.

tb Spherical Harmonics 1.0 Released

Get it here.

Lookup Table for Below

This needs to be plugged in to the LUT input of the GLSL shader.

Spherical Harmonics GLSL (No Lighting, Lookup Table)

Vertex Shader:

/*
Spherical Harmonics code Paul Bourke, adapted from
http://local.wasp.uwa.edu.au/%7Epbourke/surfaces_curves/sphericalh/

GLSL assistance and general encouragement Memo

GLSL implementation alx @ toneburst, 2008
*/

/////////////////////
////  CONSTANTS  ////
/////////////////////

#define TWOPI 6.28318531
#define PI    3.14159265

/////////////////////
//// TWEAKABLES  ////
/////////////////////

// Pre-Transform controls
uniform vec4 TT_0;		// TT_0(X) = Pre-Scale X (range 0.0 > 1.0)
uniform vec4 TT_1;		// TT_1(Y) = Pre-Scale Y (range 0.0 > 1.0)
uniform vec4 TT_2;
uniform vec4 TT_3;		// TT_3(X),(Y) = Pre-Translate X,Y (range 0.0 > 1.0)

// Spherical Harmonics controls (range 0.0 to 10.0)
uniform float M0,M1,M2,M3,M4,M5,M6,M7;

// Light position
uniform vec3 LightPosition;

/////////////////////
////  VARYINGS   ////
/////////////////////

// Passes result of shading calculation to Fragment Shader
varying float colpos;

/////////////////////
////  FUNCTIONS  ////
/////////////////////

// The actual Spherical Harmonics formula (operates on Spherical coordinates)
vec3 sphericalHarmonics(float theta, float phi, float m0,float m1,float m2,float m3,float m4,float m5,float m6,float m7)
{
	vec3 point;
	float r = 0.0;
	r += pow(sin(m0*phi),m1);
	r += pow(cos(m2*phi),m3);
	r += pow(sin(m4*theta),m5);
	r += pow(cos(m6*theta),m7);
	
	point.x = r * sin(phi) * cos(theta);
	point.y = r * cos(phi);
	point.z = r * sin(phi) * sin(theta);

	return point;
}

/////////////////////
////  MAIN LOOP  ////
/////////////////////

void main()
{
	// Create pre-transform matrix from uniform vec4s
	mat4 TT = mat4(TT_0,TT_1,TT_2,TT_3);
	
	// Get vertex coordinates (cartesian)
	vec4 vertex = gl_Vertex;
	
	// Initial vertex position pre-transformed
	vertex = TT * vertex;
	
	// Spherical coordinates to send to Spherical Harmonics function
	float theta = (vertex.x + 0.5) * TWOPI;	// set range to 0 > TWOPI
	float phi = (vertex.y + 0.5) * PI;		// set range 0 > PI
	
	// Spherical Harmonics function
	vertex.xyz = sphericalHarmonics(theta, phi, M0, M1, M2, M3, M4, M5, M6, M7);
		
	// Shading calculation
	colpos = length(vertex.xyz + LightPosition);

	// Transform vertex by modelview and projection matrices
	gl_Position = gl_ModelViewProjectionMatrix * vertex;
	
	// Forward current color and texture coordinates after applying texture matrix
	gl_TexCoord[0] = gl_TextureMatrix[0] * gl_MultiTexCoord0;
}

And Fragment Shader:

/////////////////////
//// TWEAKABLES  ////
/////////////////////

// Base color
uniform vec4 Color;
// Lighting range. Range 0.1 > 1.0 (use exponential control)
uniform float LightRange;

/////////////////////
////  VARYINGS   ////
/////////////////////

// Shading calculation from Vertex Shader
varying float colpos;

/////////////////////
////  TEXTURES   ////
/////////////////////

// Shading lookup table input
uniform sampler2D LUT;
// Lookup y-position. Range 0.0 > 1.0
uniform float LUT_Y;

// Surface texture input
uniform sampler2D TileImg;
// Surface texture scale. Range 0.001 > 0.5
uniform vec2 Tile;

/////////////////////
////  MAIN LOOP  ////
/////////////////////

void main()
{
	// Fake lighting shading with Lookup Table
	float lookupX = clamp((1.0-LightRange) * colpos,0.1,0.9);
	// Lookup shade across x-axis of LUT at y-position set by LUT_Y input
	vec4 shade = texture2D(LUT, vec2(lookupX,LUT_Y));

	// Surface tiling texture
	vec2 xy = gl_TexCoord[0].xy;
	vec2 phase = fract(xy / Tile);
	vec4 texTile = texture2D(TileImg,phase);

	// Output color compute
	gl_FragColor = Color * shade * texTile;
}