How to efficiently play an animation/video of 500 frames?

[ecoste]

I have a video animation that consists of 500 frames in total, and I would like to display it in the game like an animation/video. Each frame is around ~1MB and is a 1080x1920 PNG.

If I make a .ogv of the 500 frames, it is 7MB in size and love.video plays it fine. The catch is that the video has a white background which I need to be transparent in game, which OGV (or any other video format to my knowledge edit: here's a list https://pixelbakery.co/recipes/list-of- ... a-channels) doesn't support. Thus, love.video doesn't work.

If I load up each of the 500 frames using love.graphics.newImage and play it in a sequence, the animation also plays perfectly fine and has transparency enabled. It does exactly what I want it to do. However, loading up the 500 frames takes around a minute(and I have a fast SSD), and also takes up 2GB of RAM which is pretty rough. Although I don't really care about the RAM, but the minute load time is unacceptable to me.

Does anyone have any ideas how to store/load up the frames so it's faster?

[zorg]

Hi and welcome to the forums.

Two ideas:

- use a separate thread to pre-load only a smaller fraction of the frames, and as they are "used up", keep requesting loading new ones from storage; there's already at least one library that implements this feature (kinda, you probably need to load the images as ImageData on the other thread, whether you use the lib or not, and create images from those on the main thread, which will be faster than creating from just a path)

- If you can isolate a specific color that's not used, and different enough from all others that you do use, you might be able to use a shader to replace that specific color with transparency (since i havent played around with this yet, i'm not 100% sure you can affect video with shaders, but i assume that it's possible.)

[grump]

That's tricky. Only 2 GiB? Seems a bit low - 32 bit textures should require >4 GiB of VRAM (1920x1080x4x500). There's compressed texture formats that would require less memory at runtime, but that wouldn't help with the loading times.

Do you really mean the white background should be transparent, or rather alpha channel and alpha blending? Can you post the video or at least a single, representative PNG frame?

If you really have to do it with PNGs, my first idea would be a ring buffer of n Images and a background thread that loads the image data while the animation plays. It would only take a couple seconds to fill the buffer.

zorg is a ninja

If you can isolate a specific color that's not used, and different enough from all others that you do use, you might be able to use a shader to replace that specific color with transparency

A problem with this approach is color bleeding/fringing that will occur due to video compression. This can be tweaked with compression settings but it will never be pixel-perfect.

[ecoste]

Thanks for your replies and ideas

A couple of points:

• That's tricky. Only 2 GiB? Seems a bit low - 32 bit textures should require >4 GiB of VRAM (1920x1080x4x500).

Yeah, it's actually 4GB. I just eye-balled 2GB on the task manager incorrectly. So this means that that RAM also becomes a problem.

• Do you really mean the white background should be transparent, or rather alpha channel and alpha blending? Can you post the video or at least a single, representative PNG frame?

I just want the white background to be fully transparent (alpha=0 on those pixels).

• If you really have to do it with PNGs, my first idea would be a ring buffer of n Images and a background thread that loads the image data while the animation plays.

• and as they are "used up", keep requesting loading new ones from storage

The whole thing takes a minute to load but only plays for 10 seconds, I don't think a ring buffer that's loading things from storage is going to help much here. One image takes 0.12s to load up on my SSD, that means we can load 80 images images in 9.6 seconds. So in the best situation we're only going to keep 80 less frames in memory which isn't super significant.

• There's compressed texture formats that would require less memory at runtime

I might try these to at least reduce the RAM usage, the loading can be done in a thread before the level loads at least. So then having 2 of these animations in memory will be hopefully less than 8GB.

• you might be able to use a shader to replace that specific color with transparency

I thought about this as well, but it just seems super hacky and would rather explore other options.

edit: I'll try loading up the images in multiple different threads, maybe the loading was a bit CPU bound by processing the image on load instead of storage bound. Although if it's CPU bound it wouldn't really matter since lua is single threaded.
edit edit: coroutines make absolutely no difference

[zorg]

If it's applicable, one more idea would be to do something akin to "skeletal" animation; basically, have just a few cut-outs of separate parts of the image (e.g. with a body, the limbs and head and torso), and have some numeric data that define transformations on each part, hierarchically at that. It would consume less memory, and would allow transparency inherently.

[Gunroar:Cannon()]

picture:mika

The cheese is that? Hentai? . Though I guess it's drawn pretty well if you're the one that did it
(Though the chest area is highly unrealistic and kinda spoils the nice picture)

[grump]

The cheese is that?

It's fucking ridiculous and somewhat disgusting. My willingness to help has dropped to negative levels.

[ReFreezed]

I don't know how well this would work, if at all, but how about using two ogv files - one with the RGB info, and one with just the alpha (stored in one of the RGB channels) and then using a shader to combine everything when drawing.

[pgimeno]

I'm more inclined to the solution of using a key colour with a threshold. It might bleed but that happens sometimes.

Untested:

Code: Select all

vec3 key = vec3(0.,1.,0.); // green as key colour
float threshold = 0.01; // adjust appropriately
vec4 effect(vec4 colour, Image tex, vec2 texture_coords, vec2 screen_coords)
{
    vec4 texturecolour = VideoTexel(texture_coords);
    if (length(texturecolour.rgb - key) < threshold) texturecolour.a = 0.0;
    return texturecolour * colour;
}

Black could be an option as a key colour, judging by the sample image, and that's great because bleeding effects might be reduced.

With a few more calculations, the "frontier area" with the key colour could be interpolated instead of using hard alpha.

[Xii]

If the posted image is representative of the animation, I observe that the color green isn't used much at all. Therefore you could repurpose the green color channel as the alpha channel using a shader.