Antialias Frustration

Pictures uploaded to illustrate a forum post.
(Yes, this is me wearing a colourful scarf and talking to my cat, incidentally).

Blending horizontally

Blending vertically

Blending in both directions causes edges to become visible

Blendy-Pixelly GLSL

A Happy New Year to anyone who actually reads this

Some code, and some screenshots:

Fragment Shader:
// Control inputs
uniform float TileX;
uniform float TileY;
uniform float DirectionMix; // X-Y direction of blend. Range 0.0 (horizontal only) to 1.0 (vertical only)

//Declare a 2D texture as a uniform variable
uniform sampler2D Texture;

void main()
{
vec2 texCoord = gl_TexCoord[0].xy;
vec2 p1 = vec2(floor(texCoord.x / TileX) * TileX, floor(texCoord.y / TileY) * TileY);
vec2 p2 = clamp(vec2(p1.x + TileX, p1.y + TileY), 0.0, 1.0);
vec2 mixXY = vec2(fract(texCoord.x / TileX), fract(texCoord.y / TileY));
vec4 mixPixX = mix(texture2D(Texture, p1), texture2D(Texture, p2), mixXY.x);
vec4 mixPixY = mix(texture2D(Texture, p1), texture2D(Texture, p2), mixXY.y);

// Output
gl_FragColor = mix(mixPixX, mixPixY, DirectionMix);
}

And the result, with various settings:

This experiment started off as an attempt to test an antialias technique suggested by -NiCo- on the OpenGL GLSL shader forum, in reply to this post.

-NiCo-’s suggestion was to use the ‘fract’ function to get a normalised X-Y position for each pixel within its cell/tile (still not sure what the correct terminology is). This can then be used to mix between colour-values at the left and right, and top and bottom of each cell.

As an initial step, I thought I’d try antialiasing a standard Cartesian pixellation effect.
This is the effect of using the un-scaled position of each pixel in its cell to mix smoothly across the entire width/height of the cell. I think it looks a bit like some of the artifacts you get with really heavy JPEG compression (particularly in the 2nd example above).

The next step is to scale and clamp mixXY so the blend happens over a much shorter distance, which should result in the edges of each tile appearing blurred. I’ll also need to add another point, so I can blend both sides of the cell. I will then try and implement the same kind of thing with the polar version of the pixellation effect I was working on earlier.

I’ll try and work this simple version into a proper VDMX qcFX, when I can work out how to scale the controls properly.