GMS 2 Need File Size to Determine Buffer Size?

Urto

Member
The player in my game has to explore a rather large world that can't reasonably fit into memory. Instead, they simply load parts of the world from files as they approach and unload as they walk away. This was working fine at first, but as I added more complexity in how designers can build these pieces of the world, there started to be a bad hiccup in FPS when these loads occurred. They're just too big right now. I've done some other optimizations, but the big one I seem to need is asynchronous file loading. And gratefully, GMS2 has the buffer_load_async() function.

Here's my issue: one of the arguments of buffer_load_async requires me to specify a buffer size. Presumably, I just want the buffer to be exactly as big as the file I'm going to load. But near as I can make out, GMS2 has no functions to get the size of a file on disk.

Since I can make the buffer beforehand, I've tried setting it's type to buffer_grow, but that doesn't seem to matter; no matter what I do, the size I specify in the 4th argument of buffer_load_async is the maximum amount of characters I can load into the buffer as a string. Making it a grow-type buffer offers me no extra room.

I can just make all of the buffers WAY too big, but that seems like a waste of memory and could potentially lead to some bad data when I read the buffer. The only solution I can even come up with is: on game load, read every file the normal synchronous way, but discard the findings and store the length of the final string in a map (with the key being the filename). Then, when I have to load the file later, I can get the size of the buffer by looking up the file name in that map...

... but that solution just seems to suck. Surely there's some better way to use buffer_load_async and to actually know how big I need the buffer to be based on the file I'm loading, right? Or at worst to let it be a grow-type buffer?
 

Binsk

Member
file_bin_size()?

Also, how in the hell are you making worlds so big that they wouldn't fit in memory? That would require gigabytes per world, which I highly doubt they equate to.
 

Urto

Member
file_bin_size()?

Also, how in the hell are you making worlds so big that they wouldn't fit in memory? That would require gigabytes per world, which I highly doubt they equate to.
I definitely didn't know I could do this even though it's not a bin file. I'm worried there's any quirkiness there I'm unaware of, but I tried it out and it seems to get the correct buffer size, so I can go forward with it for now.

I'm still all-ears for other ways to do this. It seems really weird that there's only functionality to get the size of bin files. But if nothing else, I think I can work with this, so thank you for the help!
 

Binsk

Member
Glad to help. Realize that EVERY file in existence is a binary file. Even your text files. The only reason they are "text" files is because the binary data stored is intended to be read a specific way (aka, strings).

Same goes for buffer files and any other kind of file. That's why the bin size works since it is just returning the number of byes in the file (which conveniently coincides with the buffer since buffer data is just raw binary essentially).
 

Urto

Member
So I know basic comp-sci stuff like that. I just was figuring that since GMS2 explicitly has different functions for bin files, it was expecting a literal .bin that has actually been serialized into binary.

I am also a little bit worried that I still have to read the size of the file during the synchronous section of the load since disk operations are exactly what was too slow to begin with (more specifically the hiccup usually occurred with certain types of SSDs). Still, this is a much cheaper operation than reading the entire file, so I'll deal.
 
Top