Page 1 of 1

[Closed] Change stencil when depth test fails

Posted: Mon Jan 02, 2023 9:13 am
by siberian
Hello.
Is it possible to increase or decrease the value in the stencil only if the pixel is not affected by the stencil function in love.graphics.stencil?

Re: Change stencil when depth test fails

Posted: Mon Jan 02, 2023 2:06 pm
by MrFariator
As far as I know, the stencil function can only alter the values of pixels it touches, as per the StencilAction page on the wiki (except maybe Invert? Never used it). As such, depending on if you want to limit your rendering to things the stencil touches, or to limit to things the stencil is not touching, you'll have to consider different CompareModes in love.graphics.setStencilTest. All depends on your stencil setup, of course.

Re: Change stencil when depth test fails

Posted: Mon Jan 02, 2023 2:09 pm
by slime
love doesn't support that right now. What sort of thing would you use it for?

Re: [Closed] Change stencil when depth test fails

Posted: Mon Jan 02, 2023 3:02 pm
by siberian
slime wrote: Mon Jan 02, 2023 2:09 pm love doesn't support that right now. What sort of thing would you use it for?
For stencil shadows with "Carmack’s Reverse".
It's okay if it doesn't work, I'll do something else.
2023-01-02_21h58_12.png
2023-01-02_21h58_12.png (202.75 KiB) Viewed 1764 times

Re: [Closed] Change stencil when depth test fails

Posted: Mon Jan 02, 2023 3:17 pm
by slime
I think pretty much all 3D games these days use shadow mapping (which love does expose functionality for) instead of shadow volumes.

That's not to say stencil actions when the depth test fails definitely won't be exposed in the future, just that their practical use is pretty limited - shadow volumes via depth-fail stenciling was the main thing which used that feature but its downsides compared to shadow mapping mean people generally choose the latter instead.

Re: [Closed] Change stencil when depth test fails

Posted: Mon Jan 02, 2023 3:23 pm
by siberian
slime wrote: Mon Jan 02, 2023 3:17 pm I think pretty much all 3D games these days use shadow mapping (which love does expose functionality for) instead of shadow volumes.
I haven't made friends with the shadow maps yet :3 . So far I have only had problems with them.
But I think I'll learn someday.
Thank you.