Shader help (convolution matrix for a simple blur)

[Azzkikr]

I'm relatively new to shaders. Been reading a bit through https://thebookofshaders.com/, and using Moonshine as a reference. However, I struggle quite a bit just to get a boxblur shader working. I know how to do it in 'normal' code, but I can't seem to figure out how to get an arbitrary pixel position relative to the currrent pixel in the shader pipeline. So suppose given the default shader function:

Code: Select all

vec4 effect(vec4 color, Image tex, vec2 tc, vec2 sc) {
    vec4 current = Texel(tex, tc);
    return current * color;
}

I would like to get the pixels surrounding the 'current' pixel, e.g. the pixel topleft, top, topright, left, right, bottom left, bottom, and bottom right, and then averaging it and set that color as the current pixel. I tried something like this:

Code: Select all

// I know it's verbose and maybe there's better way to do this... but bear with me.
vec4 effect(vec4 color, Image tex, vec2 tc, vec2 sc) {
    vec2 scale = love_ScreenSize.xy;

    vec4 topleft  = Texel(tex, (tc + vec2(-1.0, -1.0) / scale));
    vec4 top      = Texel(tex, (tc + vec2(0,    -1.0) / scale));
    vec4 topright = Texel(tex, (tc + vec2(1.0,  -1.0) / scale));

    vec4 left     = Texel(tex, (tc + vec2(-1.0, 0) / scale));
    vec4 curr     = Texel(tex, tc);
    vec4 right    = Texel(tex, (tc + vec2(1.0, 0) / scale));

    vec4 botleft  = Texel(tex, (tc + vec2(-1.0, 1.0) / scale));
    vec4 bottom   = Texel(tex, (tc + vec2(0, 1.0) / scale));
    vec4 botright = Texel(tex, (tc + vec2(1.0, 1.0) / scale));

    vec4 avg = (topleft + top + topright + left + curr + right + botleft + bottom + botright) / 9.0;

    return avg * color;
}

but it's not really working (nothing is blurred). Can somebody point me into the right direction?

[pgimeno]

Works for me. Is your texture fullscreen? If not, you need to pass to the shader the size of the image you're going to draw, and use that instead of love_ScreenSize.xy.

[Azzkikr]

Works for me. Is your texture fullscreen? If not, you need to pass to the shader the size of the image you're going to draw, and use that instead of love_ScreenSize.xy.

Thank you for your response.

I think I figured something out. It looks like an image (sprite) is in fact blurred (although I expected the effect to be more prevalent but that's a different matter), but I'm also drawing primitives like rectangles, but I don't see any blur on those.

My drawing logic is very basic (a bit dumbed down):

Code: Select all

love.graphics.setShader(boxblur)
love.graphics.draw(image) -- draws a sprite, which is blurred
particles:draw() -- this will eventually just draw some rectangles, which are not blurred

I tried using the boxblur from Moonshine: the image is blurred like crazy, but the rectangles are not. What am I missing here?

[Azzkikr]

I think I found the culprit: when I draw everything to a canvas first (including sprites and 'primitives'), then it works as I expected. I'm not sure why though.

[pgimeno]

I think I found the culprit: when I draw everything to a canvas first (including sprites and 'primitives'), then it works as I expected. I'm not sure why though.

Rectangles are rendered as quads (pairs of triangles). The texture in their interior would be blurred if it wasn't plain, but they are rendered with a texture which consists just of one plain colour, and the effect of blurring can't possibly be seen there. The blurring can't affect the borders, because they are the hard edge of the rectangle and beyond it, nothing is drawn. It can only affect the image drawn inside.

When you draw the rectangle to a canvas that encloses said rectangle, then the whole interior of the canvas is blurred, including the border of the rectangle which is now part of the image. The border of the canvas will still not be blurred, so if it's smaller than the screen, you will still see the hard edges of the canvas when you draw it. Drawing it in full screen should work as you expect.

As for the intensity of the effect, note that (1) you're doing one single pass of an average filter with radius 1 (using "taxi metric", which is the metric where a constant radius delimits a square, not a circle), and (2) the effect can be smaller if the dimensions aren't right.

A radius 1 average is a very crude approximation to a Gaussian filter, and it has a small effect. If you want the effect to be stronger, you need to apply it more times, and/or use a bigger kernel. This "kernel" thing is a fancy word for the weights of the pixels to sum; in your case each pixel has a weight of 1/9 therefore your kernel is a 3x3 matrix full of 1/9's with the target pixel being in the middle of the matrix.

This page: http://homepages.inf.ed.ac.uk/rbf/HIPR2/gsmooth.htm gives this radius 2 kernel as an approximation of a gaussian filter:

Code: Select all

        1  4  7  4 1
        4 16 26 16 4
1/273 * 7 26 41 26 7
        4 16 26 16 4
        1  4  7  4 1

with the target pixel at the centre of the matrix, of course. This means that instead of x-1, x, x+1 you need to go from x-2 to x+2, and same with y, doing something like this (this is pseudocode; in practice you would do it similarly to how you're doing it, but with the bigger radius):

Code: Select all

  s = 0
  s += image[x-2, y-2] * 1
  s += image[x-1, y-2] * 4
  s += image[x, y-2] * 7
  s += image[x+1, y-2] * 4
  s += image[x+2, y-2] * 1
  s += image[x-2, y-1] * 4
  s += image[x-1, y-1] * 16
  ...
  s += image[x+2, y+2] * 1
  s /= 273

If the effect is still not enough for your purposes, I suggest doing two passes: render the canvas to another canvas with the filter active. This method converges quickly to a gaussian no matter the kernel, but even more quickly if the kernel is "good" to start with. http://nghiaho.com/?p=1159 shows an example with a box kernel (simple average using taxi metric, like the one you're using) that is virtually indistinguishable from a Gaussian after 4 iterations.

As for your comment "I know it's verbose and maybe there's better way to do this... but bear with me", don't be sorry: unrolling the loops is good for performance. Jumps (e.g. backward jumps like the ones needed for loops) badly hurt performance in shaders. Also, applying the kernel is easier with the loops unrolled.

[Azzkikr]

Wow, thanks for that extensive response. If I can give you a donation for your time and effort, let me know

The blurring can't affect the borders, because they are the hard edge of the rectangle and beyond it, nothing is drawn. It can only affect the image drawn inside.

I figured something like this was happening, so this makes sense.

As for the intensity of the effect, note that (1) you're doing one single pass of an average filter with radius 1 (using "taxi metric", which is the metric where a constant radius delimits a square, not a circle), and (2) the effect can be smaller if the dimensions aren't right.

I already read up quite a bit on how it works (I was triggered by Gimp, to see if I could implement a simple convolution matrix blur in a shader), but the extra information is certainly helpful. I will check out about increasing the kernel or multiple passes. It's just a toy for now anyway to see how shaders work for simple effects

As for your comment "I know it's verbose and maybe there's better way to do this... but bear with me", don't be sorry: unrolling the loops is good for performance. Jumps (e.g. backward jumps like the ones needed for loops) badly hurt performance in shaders. Also, applying the kernel is easier with the loops unrolled.

Alright, good to know. I'll take your comments into account and fiddle a bit more with these shaders.

Thanks so much again!