GameMaker Tiledata handling

A

Alexander123

Guest
Hello,

I've been trying to create my own 'map editor' in order to speed up game development process as much as possible.
The problem I've encountered is handling data because of the actual data size. For example if I were to have a 2.000x2.000 world, I'd have a total of 4.000.000 grid cells (tiles).

Now then, if we assume that each cell can hold the following:
  1. ground image,
  2. an add-on on top of that image (something like a flower, or any type of decoration),
  3. a wall or anything like that, a box, a tree, etc
  4. an add-on on top of a wall (a torch, spiderweb, etc)
And if we assume that each of those takes up to 8 bits (basically that sprite takes a stored ID from 0 - 255) it would take a total of 2.000*2.000*32 bits of data, which is equivalent to 15.625 megabytes to store a map of that size. Now then, the size itself isn't a problem if I were to have only 1 floor (you can look at floors as z-axis). But what if I wanted to have several floors? The size would be equivalent to 15.625*floor megabytes which gets ridiculous over time.

My question would be, what do you think would be the best way to store this type of data, or tiledata in general when working with maps which have enormous size?
 
Last edited by a moderator:
M

MarisFrance

Guest
I prefer using Lua for structures. I really love Lua tables which are very flexible.
I'd use 2D array for keeping tiles (1.)
And for other objects (2. 3. 4.) I prefer to create instances of objects - trees, stones, houses.

Of course you shouldn't create all instances in the world at once. I tried... it freezes the game. It's a bad idea.
But you can spawn objects only near player (in certain radius) and delete objects if they are too far (but they will be stored in Lua tables).

Sorry for my English. I hope you understand me. :)
 

TheouAegis

Member
There are lots of ways. I do agree that you should have more than one set of data for separate things.

You could consider tile aseemblies - don't save each tile, save each set of tiles.

Another method is prepending tile data with tile counts. So if you had 15 copies of tile 4 in a row, instead of writing 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4, you'd write 15 4.

Also you likely have less than 256 tiles, so you can already store 4 or more tiles in a single variable already.

As for walls, you could store them by bit.
 
A

Alexander123

Guest
There are lots of ways. I do agree that you should have more than one set of data for separate things.

You could consider tile aseemblies - don't save each tile, save each set of tiles.

Another method is prepending tile data with tile counts. So if you had 15 copies of tile 4 in a row, instead of writing 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4, you'd write 15 4.

Also you likely have less than 256 tiles, so you can already store 4 or more tiles in a single variable already.

As for walls, you could store them by bit.
Well, I'm already saving the data directly into the buffer while using buffer_u8, so there's no need to create additional variables and then shift the respective bit positions in order to store 4 values into a single variable. Furthermore, I'm creating a buffer of size (room_width div TILE_SIZE) * (room_height div TILE_SIZE) * 4 which means that I'm already leaving the right amount of memory needed for every single tile (for every single depth) which is probably an overkill knowing that most of those tiles won't need that much memory. I'm saving it directly in groups like:

0:0 [Ground][Add-on][Wall][Add-on]
1:0 [Ground][Add-on][Wall][Add-on]
.
.
n:n[Ground][Add-on][Wall][Add-on]

Which is quite handy because I can avoid storing the data inside arrays/lists and I draw directly from the buffer to the desired surface.

I did some research and I've noticed that there are two new functions in GMS 2 buffer_compress() and buffer_decompress(). I'm wondering if there's anyone here who used those functions and if he/she could tell me what would be the good practice of using them?
Would it be okay to compress only a part of a buffer? If so how would I access the memory which wasn't compressed? Has it been removed from the buffer altogether and have all the bits shifted by a certain amount (the same size which was compressed)? Or can I just compress the empty part of the buffer and keep working with the rest of it with the previously mentioned offset? Could it be that the main purpose of compression / decompression is just to save space on the local disk, so a good practice would be just compressing the whole buffer, then saving the compressed one on the disk for future use?

I prefer using Lua for structures. I really love Lua tables which are very flexible.
I'd use 2D array for keeping tiles (1.)
And for other objects (2. 3. 4.) I prefer to create instances of objects - trees, stones, houses.

Of course you shouldn't create all instances in the world at once. I tried... it freezes the game. It's a bad idea.
But you can spawn objects only near player (in certain radius) and delete objects if they are too far (but they will be stored in Lua tables).

Sorry for my English. I hope you understand me. :)
Well, I'm already using buffers to save/load all the data. Arrays/Lists would work if the map size wasn't in millions.. I've already explained the process of saving/loading and bypassing arrays altogether!
 
D

dannyjenn

Guest
Are you concerned about the RAM or about the file size, or both?


I am also wondering, do you really need that many combinations? 256*256*256*256 = 4294967296 possibilities for any given tile, which seems like a lot. I see why 4 separate layers could be a good idea, but do you really need 256 of each? I can't image why you'd need 256 different tiles for ground, for example. Especially considering that the ground is likely going to have stuff on top of it anyway. Seems a bit unnecessary. Maybe simplify the graphics a little.

Also, how often do you think that a single tile is going to actually need to using all four layers? If you could limit yourself to just two layers then you'd cut the overall size in half.


I don't know about GMS2's buffer_compress() function (haven't used it yet), but the suggestion that TheouAegis gave (writing 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 (which takes 15 bytes) as 15 4 (which takes only 2 bytes)) is a pretty standard sort of compression algorithm. I wouldn't be surprised if buffer_compress() used some variant of that (though by the bit, not by the byte).

Another idea for compression is to use four 1-bit maps (one for each layer) to keep track of which tiles are at which positions in each layer. That way if you have something that's just ground without any walls or add-ons, you can store that particular tile with just a single byte (plus a bit) rather than as 4 bytes.

e.g. Suppose you have a pretty small room: a bedroom, just 8 tiles by 8 tiles. By your method, this would take 8*8*4 (=256) bytes. But instead, you could do it like this:
You first make a 1-bit map for the ground layer, which might look like this:
Code:
11111111
11111111
11111111
11111111
11111111
11111111
11111111
11111111

(64 ones and no zeros)
And then maybe you have some carpeting on the first add-on layer:
Code:
00000000
00111100
00111100
00111100
00111100
00000000
00000000
00011000

(18 ones and 46 zeros)
And walls around the edges, with a table and chairs in the center:
Code:
11111111
10000001
10111101
10111101
10000001
10000001
10000001
11100111

(34 ones and 30 zeros)
And maybe a picture frame or a window or something on the back wall on the second add-on layer:
Code:
00010000
00000000
00000000
00000000
00000000
00000000
00000000
00000000

(1 one and 63 zeros)
Those are your 1-bit maps, each of which takes up w*h bits (=8 bytes in this example). So you are increasing your overall size by 4 bits per cell, though in the long run this is likely to decrease the size since you won't need to store as much tile data. (1s in the bit-map indicate which cells are occupied by some tile, while the 0s indicate unoccupied cells. You only need tile data for occupied cells, not unoccupied cells.) e.g. Notice how there is a total of only 117 ones? That means you only need 117 bytes of actual tile data (which you'd then place in the room by getting the correct positions from the bit-map data), for a total of 117+8*4 (=149) bytes (which is smaller than the 256 bytes).
But in the worse case scenario (i.e. every cell is occupied by a tile, and every tile uses all four layers), this method would increase your overall size by 4 bits per cell (i.e. total size would be w*h*4.5, not w*h*4). In this example, the 8x8 room would take 288 bytes instead of 256. And a 2000x2000 room would take 18000000 bytes which is considerably higher than 16000000 bytes.
 
Last edited by a moderator:
Top