Keeping track of memory used by images
Posted: Sat Apr 12, 2014 4:12 pm
(Note: This post used to named "How to handle a lot of images?", but I felt that was a bad name for this...)
Hello everyone
SHORT VERSION:
How are images held in memory? Like, does imageData:getSize() and love.filesystem.getSize() give the size of the loaded image, or the size of the just the file? I'm not an expert on computers, but isn't the images 'expanded' when it's loaded, and takes up a different amount of RAM when loaded than it does on disk? I want to count the amount of space my loaded images take up, so what is the best way to do this?
LONG VERSION: (Wow, you must like to read my redundant stories... well, if you want to...)
So I was thinking one day: "Gee, it sure is annoying to have all my images stored in separate tables all over my game". I felt if two files are to use the same image, the image should be loaded somewhere general so both can reference it, rather than one file 'own' the image, and the other file have to reference the first.
I had an 'Image' table that looked something like this:
And that worked very well until I wanted to load sprite sheets and animations. They need quads, and are all in the same image, which was a problem for my current method of doing things. Now, i'm not gonna try and reinvent that code, but you could imagine how it worked.
Rather than loading images like this:
I could do it like this:
And that worked very well also. Not only did it look nice, but it automatically cut that quad out of the main image, and pasted it into a new ImageData, so that each section could be treated like a separate image, as if they all had separate files.
Now, what happens if I want to load TONS of images? Like a few big sprite sheets, or tons of separate files?
I'm not the best at memory and techy computer things, but I'm pretty sure I would run into a problem if I tried to load too many...
I thought maybe I could rethink the way I store images. Take a look at this next one I made. This is pretty close to what I had, but this one doesn't handle quads like the previous, but my older one did.
This one will store the information you give it first, and load the image when you actually need it. It keeps track of order of the images loaded, and when we begin to run out of space, it will keep unloading the oldest until there is enough space for the new one.
Just try to interpret this one... I put comments.
So that one works well too. Keep in mind when I say I 'unload' an image, I just remove the reference so that the garbage collector can take it eventually. So in reality, the current memory may be larger than what it actually says, but it's close enough to keep tabs on how much memory I'm using at any time. I used the same idea for my 'Audio' and 'Font' tables.
Now, what you've all been waiting for, MY REAL PROBLEM!
While this might work I'm my head, I'm not sure if it works for real. For instance, I heard somewhere that when you load an image, it actually is much larger than the few kilobytes windows says it is. It is expanded in some way. So this means that imageData:getSize() is giving me one of those sizes; the file size, or the loaded size.
Another thing, I'm still using love-0.8.0
I see in love-0.9.0 they added love.filesystem.getSize(). Since you don't need to even load the file you want to get the size of, is it giving the unloaded size? If it doesn't, that means I can use it to get the size of all my images as I add them to my 'Image.roster' table, rather than waiting until I actually load them, and get the size then. (Thats why I had to initially set the image size to false, because I couldn't figure it out without loading it, and I wanted to avoid loaded EVERY image)
So if anyone could explain how those two functions work, as far as where it's getting the number it gives me, then I guess that would be all I need. If I have any other questions in the near future, I'll post back here so that my code is already explained. Because I probably didn't need to explain all that to get to my point. Then again, you did want to read the long version...
------------------------------------------------------------------------
And just on the side, I would like to add I found how to keep track of the size of unloaded images. What I could do, rather than just set the image to false when I want to unload it, I can put the image in a weak table, so while it's waiting to be collected, I could still use it if i needed it, or even save it from being collected. If the image lasts long enough in the weak table to be collected, then I probably won't need it for a while anyways, and so it should be collected, rather then if I just ran out of space, but still need to draw that image within the next few frames, ya know?
Hello everyone
SHORT VERSION:
How are images held in memory? Like, does imageData:getSize() and love.filesystem.getSize() give the size of the loaded image, or the size of the just the file? I'm not an expert on computers, but isn't the images 'expanded' when it's loaded, and takes up a different amount of RAM when loaded than it does on disk? I want to count the amount of space my loaded images take up, so what is the best way to do this?
LONG VERSION: (Wow, you must like to read my redundant stories... well, if you want to...)
So I was thinking one day: "Gee, it sure is annoying to have all my images stored in separate tables all over my game". I felt if two files are to use the same image, the image should be loaded somewhere general so both can reference it, rather than one file 'own' the image, and the other file have to reference the first.
I had an 'Image' table that looked something like this:
Code: Select all
Image = {
roster = {},
newImage = function(self, filepath, imageName)
self.roster[imageName] = love.graphics.newImage(filepath)
end,
get = function(self, imageName)
return self.roster[imageName]
end
}
Rather than loading images like this:
Code: Select all
Image:newImage("blah/blah/bloop_1.png", "BloopBody")
Image:newImage("blah/blah/bloop_2.png", "BloopTop")
Image:newImage("blah/blah/bloop_3.png", "BloopLeft")
Image:newImage("blah/blah/bloop_4.png", "BloopRight")
Code: Select all
Image:newImage("blah/blah/bloopsheet.png", {
{"BloopBody", 0, 0, 32, 32},
{"BloopTop", 32, 0, 32, 32},
{"BloopLeft", 0, 32, 32, 32},
{"BloopRight", 32, 32, 32, 32}
})
Now, what happens if I want to load TONS of images? Like a few big sprite sheets, or tons of separate files?
I'm not the best at memory and techy computer things, but I'm pretty sure I would run into a problem if I tried to load too many...
I thought maybe I could rethink the way I store images. Take a look at this next one I made. This is pretty close to what I had, but this one doesn't handle quads like the previous, but my older one did.
This one will store the information you give it first, and load the image when you actually need it. It keeps track of order of the images loaded, and when we begin to run out of space, it will keep unloading the oldest until there is enough space for the new one.
Just try to interpret this one... I put comments.
Code: Select all
Image = {
roster = {}, -- Stores data for recreation
list = {}, -- Stores the order images were loaded in
currentMemory = 0, -- Current memory useage
maxMemory = (1024^2) * 64, -- 64 Megabytes (in bytes)
-- Creates a table rather than load the image
-- right away, because we may not need this
-- image until much later, and it would take
-- up space and take up loading time
newImage = function(self, filepath, imageName)
self.roster[imageName] = {
image = false, -- Where image will go once loaded
filepath = filepath, -- Where the file for this image is
size = false -- The size of this image (in bytes)
}
end,
get = function(self, imageName)
if not(self.roster[imageName].image) then -- If the image isn't there...
self:reload(imageName) -- then go reload it!
end
return self.roster[imageName].image -- Return the image (should be loaded)
end,
reload = function(self, imageName)
local img = love.image.newImageData(self.roster[imageName].filepath) -- Load the ImageData
local imgSize = img:getSize() -- The size of the image (in bytes)
local oldImg -- Image to unload
while self.currentMemory + imgSize > self.maxMemory do -- If we loaded too much already...
oldImg = table.remove(self.list) -- Get the oldest image in the list...
self.roster[oldImg].image = false -- and remove the reference to the image...
self.currentMemory = self.currentMemory - self.roster[oldImg].size -- to make more space!
end
self.roster[imageName].image = img -- Store the new image
self.roster[imageName].size = imgSize -- Record the size for later
self.currentMemory = self.currentMemory + imgSize -- Update the current memory usage
table.insert(self.list, 1, imageName) -- Add this image as the newest (first) on the list
end
}
Now, what you've all been waiting for, MY REAL PROBLEM!
While this might work I'm my head, I'm not sure if it works for real. For instance, I heard somewhere that when you load an image, it actually is much larger than the few kilobytes windows says it is. It is expanded in some way. So this means that imageData:getSize() is giving me one of those sizes; the file size, or the loaded size.
Another thing, I'm still using love-0.8.0
I see in love-0.9.0 they added love.filesystem.getSize(). Since you don't need to even load the file you want to get the size of, is it giving the unloaded size? If it doesn't, that means I can use it to get the size of all my images as I add them to my 'Image.roster' table, rather than waiting until I actually load them, and get the size then. (Thats why I had to initially set the image size to false, because I couldn't figure it out without loading it, and I wanted to avoid loaded EVERY image)
So if anyone could explain how those two functions work, as far as where it's getting the number it gives me, then I guess that would be all I need. If I have any other questions in the near future, I'll post back here so that my code is already explained. Because I probably didn't need to explain all that to get to my point. Then again, you did want to read the long version...
------------------------------------------------------------------------
And just on the side, I would like to add I found how to keep track of the size of unloaded images. What I could do, rather than just set the image to false when I want to unload it, I can put the image in a weak table, so while it's waiting to be collected, I could still use it if i needed it, or even save it from being collected. If the image lasts long enough in the weak table to be collected, then I probably won't need it for a while anyways, and so it should be collected, rather then if I just ran out of space, but still need to draw that image within the next few frames, ya know?
