Tiles and memory usage :(

[zugamifk]

Hey ho all, I've been working on an RPG for a little while now and it's coming along smoothly, despite the fact that I'm not anywhere close to having something playable yet.

Anyway, my ultimate plan (that is, my plan for the far, far future) is to have a huge open world RPG with (hopefully) seamless transitions between maps.

Now, I don't plan on having that for now, but I would very much like the ability to have very large maps, on the order of 1000x1000 tiles if possible. The problem is that the memory usage of the game seems to go up exponentially as the number of tiles increases. With a 100x100 tile map, I get around 30MB of memory used, but with 500x500 it goes up to around 1GB and at 1000x1000 the game freezes.

I'm wondering, is there some way to get around this? I have no idea why this happens, and the code itself is just a version of the simple tile-based RPG example on the wiki. It's not trying to draw them all at once either, I only have a 32x24 area of the map drawn at any time.

This certainly may be an aspect of coding I'm not aware of, but I'd love some help if I could get it. Many thanks

[bartbes]

Well, it depends on the structure of your map, I hope you only load the images once for example, the best would be an array of arrays filled with numbers representing tiles (probably), so I'd do that.

[Jasoco]

Is there any reason why you need any one map to be 1000x1000? Can't you do what other games do and break it up into sections?

Even old RPG's before 3D only went up to 256x256 or so and those maps were relatively large world maps. Heck, I doubt any Sim City or Sims game used tile grids that large. I'd just break it up into world sections. Like the valley, the mountains, the forest, the beach, and so on. As well as one or two or more for each town and one for each house inside the town.

In my top-down adventure/RPG engine I've tested creating large maps, but you need to realize that the larger the map the longer it takes to initialize the entire 2-dimensional array. I create one 2D array (i.e. mapTile[x][y] = whatever) for each layer of the map. The background and four decoration layers, then a layer for the collisions. It takes no time at all with a maximum of 255x255. (Which is pretty huge as it is.) but if I try to use 1000x1000 it takes upwards of 5-10 seconds to create that array during which time nothing can happen so it ends up blank screen while the game loads. (Obviously I could use a little elbow grease and make the array creation run in steps in the love.update() function.. IF I really thought it was important to have one single continuous map to play in. As I said, even the biggest of old 2D RPG's only had maps of an average of 256x256 tiles and they were still pretty big overworlds.

I use the same 2D arrays for every map and just load the data from map files every time I change screens. Rather than loading all of them in at once into their own arrays or a single big map. I simply erase the map first, then take the data from my plain-text map file to be loaded next, parse it and place each tiles data as the values to the array.

Of course I use fade down and fade up to hide the instantaneous map replacement.

[Nikolai]

The problem is that the memory usage of the game seems to go up exponentially as the number of tiles increases. With a 100x100 tile map, I get around 30MB of memory used, but with 500x500 it goes up to around 1GB and at 1000x1000 the game freezes.

You see that as a problem? Consider that, from the statistics you quote, with 10,000 tiles, the memory cost is about 3K per tile; at 250,000 tiles, the memory cost is about 4K per tile. Even if it could somehow be managed to occupy only 2K per tile, 1,000,000 tiles would require roughly 2 GB; this doesn't include room for the plot, the animations, the operating system, or anything else. You might need some kind of hard drive swapping just to make that work at all, and believe me, you'd notice the enormous hit in performance.

Is a map this large absolutely necessary?

[Robin]

Heck, I doubt any Sim City or Sims game used tile grids that large.

Not even The Sims 3, and that features a seamless neighbourhood. The world is still divided into “lots”, but they use a certain array of tricks to make it seem seamless.

[zugamifk]

Is there any reason why you need any one map to be 1000x1000? Can't you do what other games do and break it up into sections?

Unfortunately, no. For the time being, my game is essentially a wilderness exploration/survival game with a focus on allowing the player to be creative. That's why I want a large map, so the player can go where they please and feel like they're in a large open world. I'm trying to play around with how I can convey that sense, but it's look like simply having one huge map to play in isn't going to work.

A couple example of games that use gigantic amounts of tiles that came to mind for me were Transport Tycoon, which can have maps as big as 2048x2048 and Stronghold which uses an unbelievably huge number of tiles, although I'm not exactly sure how. I was hoping they had figured something out that worked for them, and that I was simply missing something.

I am deathly afraid of loading screens, I find they break immersion horribly and I want to avoid them at all costs. It's certainly a big obstacle, but I feel that if I work at it hard enough, I can figure something out.

You see that as a problem? Consider that, from the statistics you quote, with 10,000 tiles, the memory cost is about 3K per tile; at 250,000 tiles, the memory cost is about 4K per tile. Even if it could somehow be managed to occupy only 2K per tile, 1,000,000 tiles would require roughly 2 GB; this doesn't include room for the plot, the animations, the operating system, or anything else. You might need some kind of hard drive swapping just to make that work at all, and believe me, you'd notice the enormous hit in performance.

You're absolutely right, I hadn't even thought of that. Of course it goes up exponentially. I feel silly now. Especially since my tiles actually hold quite a bit of information already, I should have figured that out. Ah well

I may just make a large number of small maps divided into quadrants and then load and unload adjacent maps based on what quadrant
of the map you're in. This was the other idea I had in mind. One thing is for sure, it seems the large maps are not a good idea.

[Robin]

Maybe you could use something similar to IFS, so you can define a whole part of the map with just a few number coordinates.

[Jasoco]

Well, the loading screen would only be shown at load. Before the player is even immersed.

If you must have a huge grid, create the arrays in steps in the update function so you can draw stuff like a progress bar labeled "Reticulating Splines" (I love Sim City 2) while it creates the array.

I did some tests a while ago to see how long it took for my computer's Löve to create a multidimensional array. A few seconds is nothing if you do it at the start. Now if you were doing this loading after the player is immersed then yeah. Avoid it if you can. But at start it's nothing. Not a thing.

This is untested Pseudo-code. Don't expect it to work right off.

Code: Select all

function love.load()
  loadingScreen = 0
  nextGameModeStepAfterLoadingScreen = 1
  gameMode = loadingScreen
  tileGridLoaded = 0
  tileGrid = {}
end

function.update(dt)
  if gameMode == loadingScreen then
    if tileGridLoaded < maxTileRows then
      tileGridLoaded = tileGridLoaded + 1
      tileGrid[tileGridLoaded] = {}
      for y=0,maxTileColumns do
        tileGrid[tileGridLoaded][y] = tileData
      end
    else
      gameMode = nextGameModeStepAfterLoadingScreen
    end
  elseif gameMode == nextGameModeStepAfterLoadingScreen
    --STUFF FOR THE NEXT "GAME MODE"
  end
end

function love.draw()
  if gameMode == loadingScreen then
    --DRAW THE PROGRESS BAR HERE OR WHATEVER LOADING SCREEN YOU WANT
  elseif gameMode == nextGameModeStepAfterLoadingScreen then
    --DRAW WHATEVER HAPPENS NEXT LIKE THE MENU OR WHATEVER
  end
end

Bottom line, if you want a huge map, you WILL have to load at some point or another. My method above would probably make it more pleasant, instead of a long pause while it generates.

[Thursdaybloom]

I'm hijacking because this has now become relevant to my interests (and I don't want to start a new thread on an existing topic). I not only would like to experiment with massive maps, but I need to understand more about loading and the loading process - so here come the noob questions:

• Would I break one huge map into a grid and have an array to represent each section? example: 1000 x 1000 total tiles broken into separate 100x100 lots then have "section1 = { grid info }" etc?
  
  If not, what is meant by 'creating arrays in steps'?
  
  Does Jasoco's example code then just replace what is shown on screen while this loading is happening in the background so it's not just black and/or a frozen frame?

I'm not fussed about a "loading screen" with a progress bar, I'm just interested in large maps and seamless transitions between sections. In fact, I have no practical reason to know this, I'm just curious about how to load data correctly, what's happening while things are loading and techniques for any future need for a huge map

[Jasoco]

Apparently my code would actually slow the process down more. I did a test where I created a 2D array of 1000x1000 and it took longer to create it while allowing for updating and drawing than it did if I just leave it in one place. It may freeze the app for a few seconds but it is faster than what i did. Maybe some more tweaking would work, but I deleted my project and would have to start over.

By steps I meant creating each row in update each update instead of all at once in load. Simple. But it didn't work as well as I wanted it to. So..