Any way to store really big numbers?

So let's say I have the number `9444732965739290427392`. That's 2^65. Is there any way to store this in GML? I'm coming to the sad realisation that I can only store up to 64 flags for bitmask operations.
 

chamaeleon

Member
If it's just for a large set of bitflags, perhaps create a struct that has an array of integers, and use i div 64 to get which integer and i mod 64 to pick which flag in the selected integer with functions added to this struct.?
 
If you're just using it as a bitmask, you probably don't need it all in one variable.
If it's just for a large set of bitflags, perhaps create a struct that has an array of integers, and use i div 64 to get which integer and i mod 64 to pick which flag in the selected integer with functions added to this struct.?
Sorry fellas, you may have misunderstood me (or I misunderstand you). I have 64+ flags, let alone a bitmask with a size > 2^64.

To give you an example:
GML:
enum ExampleFlags
{
    Flag0= 2,
    Flag1= 4,
    Flag2= 8,
    Flag3= 16,
    Flag4= 32,
    Flag5= 64,
    Flag6= 128,
    Flag7= 256,
    Flag8= 512,
    Flag9= 1024,
    Flag10= 2048,
    Flag11= 4096,
    Flag12= 8192,
    Flag13= 16384,
    Flag14= 32768,
    Flag15= 65536,
    Flag16= 131072,
    Flag17= 262144,
    Flag18= 524288,
    Flag19= 1048576,
    Flag20= 2097152,
    Flag21= 4194304,
    Flag22= 8388608,
    Flag23= 16777216,
    Flag24= 33554432,
    Flag25= 67108864,
    Flag26= 134217728,
    Flag27= 268435456,
    Flag28= 536870912,
    Flag29= 1073741824,
    Flag30= 2147483648,
    Flag31= 4294967296,
    Flag32= 8589934592,
    Flag33= 17179869184,
    Flag34= 34359738368,
    Flag35= 68719476736,
    Flag36= 137438953472,
    Flag37= 274877906944,
    Flag38= 549755813888,
    Flag39= 1099511627776,
    Flag40= 2199023255552,
    Flag41= 4398046511104,
    Flag42= 8796093022208,
    Flag43= 17592186044416,
    Flag44= 35184372088832,
    Flag45= 70368744177664,
    Flag46= 140737488355328,
    Flag47= 281474976710656,
    Flag48= 562949953421312,
    Flag49= 1125899906842624,
    Flag50= 2251799813685248,
    Flag51= 4503599627370496,
    Flag52= 9007199254740992,
    Flag53= 18014398509481984,
    Flag54= 36028797018963968,
    Flag55= 72057594037927936,
    Flag56= 144115188075855872,
    Flag57= 288230376151711744,
    Flag58= 576460752303423488,
    Flag59= 1152921504606846976,
    Flag60= 2305843009213693952,
    Flag61= 4611686018427387904,
    Flag62= 9223372036854775808,
    Flag63= 18446744073709551616,
    Flag64= 36893488147419103232,
    Flag65= 73786976294838206464,
    Flag66= 147573952589676412928,
    Flag67= 295147905179352825856,
    Flag68= 590295810358705651712,
}

So the question is how am I able to store Flag64+ ? Storing a struct with an array of ints would still require that the 65th int is of a size greater than 2^64. Unless I am misunderstanding you, which may very well be possible.

EDIT: @chamaeleon On second thought, I think I may see what you're alluding to. In this case, what does i represent? Is it the original bitmask? And then each flag is also encoded within another bitmask of their own?
 
Last edited:

TsukaYuriko

☄️
Forum Staff
Moderator
You could have an array with the size of the total amount of flags. Each cell contains a boolean, representing a flag. Flag definitions are in standard ascending-by-one order, representing the index of the cell that stands for the flag.
Replace array with list or buffer if that's more your type of thing.
 

chamaeleon

Member
Instead of using the big 1<<n numbers, just use n.
GML:
function bitmask() constructor {
    bits = array_create(10, int64(0));
    
    static set = function(i) {
        bits[i div 64] = bits[i div 64] | (1 << (i mod 64));
    }
    
    static clear = function(i) {
        bits[i div 64] = bits[i div 64] & ~(1 << (i mod 64));
    }
    
    static test = function(i) {
        return (bits[i div 64] & (1 << (i mod 64))) != 0;
    }
}
GML:
var a = new bitmask();
for (var i = 60; i <= 70; i++) {
    a.set(i);
}
for (var i = 55; i <= 75; i++) {
    show_debug_message("Bit " + string(i) + " => " + string(a.test(i)));
}
Code:
Bit 55 => 0
Bit 56 => 0
Bit 57 => 0
Bit 58 => 0
Bit 59 => 0
Bit 60 => 1
Bit 61 => 1
Bit 62 => 1
Bit 63 => 1
Bit 64 => 1
Bit 65 => 1
Bit 66 => 1
Bit 67 => 1
Bit 68 => 1
Bit 69 => 1
Bit 70 => 1
Bit 71 => 0
Bit 72 => 0
Bit 73 => 0
Bit 74 => 0
Bit 75 => 0
 
Instead of using the big 1<<n numbers, just use n.
GML:
function bitmask() constructor {
    bits = array_create(10, int64(0));
  
    static set = function(i) {
        bits[i div 64] = bits[i div 64] | (1 << (i mod 64));
    }
  
    static clear = function(i) {
        bits[i div 64] = bits[i div 64] & ~(1 << (i mod 64));
    }
  
    static test = function(i) {
        return (bits[i div 64] & (1 << (i mod 64))) != 0;
    }
}
GML:
var a = new bitmask();
for (var i = 60; i <= 70; i++) {
    a.set(i);
}
for (var i = 55; i <= 75; i++) {
    show_debug_message("Bit " + string(i) + " => " + string(a.test(i)));
}
Code:
Bit 55 => 0
Bit 56 => 0
Bit 57 => 0
Bit 58 => 0
Bit 59 => 0
Bit 60 => 1
Bit 61 => 1
Bit 62 => 1
Bit 63 => 1
Bit 64 => 1
Bit 65 => 1
Bit 66 => 1
Bit 67 => 1
Bit 68 => 1
Bit 69 => 1
Bit 70 => 1
Bit 71 => 0
Bit 72 => 0
Bit 73 => 0
Bit 74 => 0
Bit 75 => 0
Ah okay, I see now. Thanks for the clarification. The use-case for these flags are as types, so I was hoping to be able to have GML bitwise encoding as part of the implementation. But I appreciate the solution. I'll have to fiddle with it a bit to see if I can get the implementation feeling more user-friendly.

GML:
function ExampleSetter(_name, _painter, _createNew)
{
    var _error = Test.CheckInput([
        [_name,           T.STRING],
        [_painter,        T.ACTOR_PAINTER | T.TERRAIN_PAINTER | T.TEXTURE_PAINTER],
        [_createNew,      T.BOOLEAN | T.UNDEFINED]
    ])
    if d(_error) throw _error
}
 
Last edited:

chamaeleon

Member
You could add functions to perform bitmask operations between bitmasks instead of individual bits only. My code is starting to get into the territory of semantics now. Should test functions return true if all bits are set or if some bits are set of the provided bits to test? Current implementation returns true if at least one bit is set in both sets, changing to ensure all bits are set is left as an exercise.
GML:
function bitmask() constructor {
    sz = 10;
    bits = array_create(sz, int64(0));
    
    for (var i = 0; i < argument_count; i++) {
        set(argument[i]);
    }
    
    static set = function(i) {
        bits[i div 64] = bits[i div 64] | (1 << (i mod 64));
    }
    
    static set_bits = function(bm) {
        for (var i = 0; i < sz; i++) {
            bits[i] = bits[i] | bm.bits[i];
        }
    }
    
    static clear = function(i) {
        bits[i div 64] = bits[i div 64] & ~(1 << (i mod 64));
    }
    
    static clear_bits = function(bm) {
        for (var i = 0; i < sz; i++) {
            bits[i] = bits[i] & ~bm.bits[i];
        }
    }
    
    static test = function(i) {
        return (bits[i div 64] & (1 << (i mod 64))) != 0;
    }
    
    static test_bits = function(bm) {
        for (var i = 0; i < sz; i++) {
            if ((bits[i] & bm.bits[i]) != 0)
                return true;
        }
        return false;
    }
}
GML:
var a = new bitmask();
var clearing_bits = new bitmask(64, 65, 66); // or use named values
var test_bits1 = new bitmask(63, 64); // or use named values
var test_bits2 = new bitmask(65, 66); // or use named values

for (var i = 60; i <= 70; i++) {
    a.set(i); // setting individual bits
}

a.clear_bits(clearing_bits); // clearing all bits set in clearing_bits

show_debug_message("Bitmask 1 test => " + string(a.test_bits(test_bits1))); // testing bits in test_bits1
show_debug_message("Bitmask 2 test => " + string(a.test_bits(test_bits2))); // testing bits in test_bits2
Code:
Bitmask 1 test => 1
Bitmask 2 test => 0
 
Thanks
You could add functions to perform bitmask operations between bitmasks instead of individual bits only. My code is starting to get into the territory of semantics now. Should test functions return true if all bits are set or if some bits are set of the provided bits to test? Current implementation returns true if at least one bit is set in both sets, changing to ensure all bits are set is left as an exercise.
GML:
function bitmask() constructor {
    sz = 10;
    bits = array_create(sz, int64(0));
   
    for (var i = 0; i < argument_count; i++) {
        set(argument[i]);
    }
   
    static set = function(i) {
        bits[i div 64] = bits[i div 64] | (1 << (i mod 64));
    }
   
    static set_bits = function(bm) {
        for (var i = 0; i < sz; i++) {
            bits[i] = bits[i] | bm.bits[i];
        }
    }
   
    static clear = function(i) {
        bits[i div 64] = bits[i div 64] & ~(1 << (i mod 64));
    }
   
    static clear_bits = function(bm) {
        for (var i = 0; i < sz; i++) {
            bits[i] = bits[i] & ~bm.bits[i];
        }
    }
   
    static test = function(i) {
        return (bits[i div 64] & (1 << (i mod 64))) != 0;
    }
   
    static test_bits = function(bm) {
        for (var i = 0; i < sz; i++) {
            if ((bits[i] & bm.bits[i]) != 0)
                return true;
        }
        return false;
    }
}
GML:
var a = new bitmask();
var clearing_bits = new bitmask(64, 65, 66); // or use named values
var test_bits1 = new bitmask(63, 64); // or use named values
var test_bits2 = new bitmask(65, 66); // or use named values

for (var i = 60; i <= 70; i++) {
    a.set(i); // setting individual bits
}

a.clear_bits(clearing_bits); // clearing all bits set in clearing_bits

show_debug_message("Bitmask 1 test => " + string(a.test_bits(test_bits1))); // testing bits in test_bits1
show_debug_message("Bitmask 2 test => " + string(a.test_bits(test_bits2))); // testing bits in test_bits2
Code:
Bitmask 1 test => 1
Bitmask 2 test => 0
Thanks for your thoughts - you are correct in that it is definitely a discussion of semantics. The underlying concept is solid, just a matter of tweaking how it's implemented. Having said that, I'm not that interested in a system that requires arrays to supplement bitmasks. After all, bitmasks are a way to optimise lists into a computationally faster process - it kind of defeats the purpose, in my view at least, to have to also use arrays. In the end, I decided to treat types as n, as you suggested, but each just represents a position in the array. Nothing fancier than that. The type checking process will just also use arrays to store multiple typesToCheck.

E.g.
Code:
function ExampleSetter(_name, _painter, _createNew)
{
    var _error = Test.CheckInput([
        [_name,           T.STRING],
        [_painter,        [T.ACTOR_PAINTER, T.TERRAIN_PAINTER, T.TEXTURE_PAINTER]],
        [_createNew,      [T.BOOLEAN, T.UNDEFINED]]
    ])
    if d(_error) throw _error
}
It's a bit heavier on the CPU, but that's the price of using less memory.
 

zbox

Member
GMC Elder
Might be worth throwing in incase anyone finds this on google - I've got a free asset that does this "cookie clicker number" sized number sort of stuff. Addition and subtraction, not bitmasking, so probably not helpful in this case.

There are a few bugs here and there, but the bones are setup. It works via doing basic addition on strings so it's slower (technically it's slower. You won't notice unless you're using it to do serious math which I don't recommend).
 

TheouAegis

Member
. Having said that, I'm not that interested in a system that requires arrays to supplement bitmasks.
I don't think he was saying to use an array instead of bitmasks, but to use an array of bitmasks. In the code, i is understood to bs the bit you want to check. If you want to check bit 65, you take 65 div 64 to find out which bitmask 65 (i) is in, then i&63 will tell you which bit in that mask corresponds to 65. Your masks are still 64 bits, but you would have an array of them. This is how it would have been done on 8 bit systems as well.
 

chamaeleon

Member
I'm not that interested in a system that requires arrays to supplement bitmasks
In the end, I decided to treat types as n, as you suggested, but each just represents a position in the array.
So in the end you use arrays, while ensuring it takes up 64 times more memory (using 64 bits to encode the value of one bit) plus any additional overhead related to more array entries.

With some small additions (leaving them out) you can do this
GML:
#macro FOO 1
#macro BAR 2
#macro BAZ 3

var a = new bitmask();

a.set([FOO, BAZ]);

show_debug_message("FOO         => " + string(a.test(FOO)));
show_debug_message("BAR         => " + string(a.test(BAR)));
show_debug_message("BAZ         => " + string(a.test(BAZ)));
show_debug_message("FOO|BAR     => " + string(a.test([FOO,BAR])));
show_debug_message("FOO|BAZ     => " + string(a.test([FOO,BAZ])));
show_debug_message("BAR|BAZ     => " + string(a.test([BAR,BAZ])));
show_debug_message("FOO|BAR|BAZ => " + string(a.test([FOO,BAR,BAZ])));
Code:
FOO         => 1
BAR         => 0
BAZ         => 1
FOO|BAR     => 1
FOO|BAZ     => 1
BAR|BAZ     => 1
FOO|BAR|BAZ => 1
If all bits need to be set the implementation can of course produce
Code:
FOO         => 1
BAR         => 0
BAZ         => 1
FOO|BAR     => 0
FOO|BAZ     => 1
BAR|BAZ     => 0
FOO|BAR|BAZ => 0
 
I don't think he was saying to use an array instead of bitmasks, but to use an array of bitmasks. In the code, i is understood to bs the bit you want to check. If you want to check bit 65, you take 65 div 64 to find out which bitmask 65 (i) is in, then i&63 will tell you which bit in that mask corresponds to 65. Your masks are still 64 bits, but you would have an array of them. This is how it would have been done on 8 bit systems as well.
Yep I got that thanks. I'm not denying it's not a clever system, just that it doesn't suit my purposes to have an array of bitmasks when I could suffice with just the array.

So in the end you use arrays, while ensuring it takes up 64 times more memory (using 64 bits to encode the value of one bit) plus any additional overhead related to more array entries.

With some small additions (leaving them out) you can do this
GML:
#macro FOO 1
#macro BAR 2
#macro BAZ 3

var a = new bitmask();

a.set([FOO, BAZ]);

show_debug_message("FOO         => " + string(a.test(FOO)));
show_debug_message("BAR         => " + string(a.test(BAR)));
show_debug_message("BAZ         => " + string(a.test(BAZ)));
show_debug_message("FOO|BAR     => " + string(a.test([FOO,BAR])));
show_debug_message("FOO|BAZ     => " + string(a.test([FOO,BAZ])));
show_debug_message("BAR|BAZ     => " + string(a.test([BAR,BAZ])));
show_debug_message("FOO|BAR|BAZ => " + string(a.test([FOO,BAR,BAZ])));
Code:
FOO         => 1
BAR         => 0
BAZ         => 1
FOO|BAR     => 1
FOO|BAZ     => 1
BAR|BAZ     => 1
FOO|BAR|BAZ => 1
If all bits need to be set the implementation can of course produce
Code:
FOO         => 1
BAR         => 0
BAZ         => 1
FOO|BAR     => 0
FOO|BAZ     => 1
BAR|BAZ     => 0
FOO|BAR|BAZ => 0
Indeed. And taking up all 64 bits for just one number is certainly wasteful. I mean, isn't that true for any number less than 18446744073709551616 ? Heck, I may as well bitmask the player's HP, the enemies' movement speeds, the button input flags, and the state flags, too. I appreciate your solution, but it doesn't feel nice to use. You might call that a dumb thing to say. That's alright, that's the beauty of semantic arguments. But truthfully, if I have to create a new instance of a bitmask struct, and run a method on the struct every time I want to allocate flags, and then also to clear those flags - just for the sake of saving memory, then at the scale I'm dealing with, I'm going to hit CPU problems - and it's not going to be any easier to use:

GML:
function ExampleSetter(_name, _painter, _createNew)
{
    // Note the extra steps involved:
    var _bm = new Test.TypeBitmask()
    var _error = Test.CheckInput([
        [_name,           _bm.Set(T.STRING)],
        [_painter,        _bm.Set([T.ACTOR_PAINTER, T.TERRAIN_PAINTER, T.TEXTURE_PAINTER])],
        [_createNew,      _bm.Set([T.BOOLEAN, T.UNDEFINED])]
    ])
    if d(_error) throw _error
    _bm.CleanUp() // Calls Clear() then delete self, or whatever.
}
If you can advise on ways to modify GM's allocation of bits to integers (in either direction) that's a different story, but in the absence of that, I'd prefer to stick with the solution that is easiest to use.

Thanks all for contributing :)
 

Roldy

Member
- just for the sake of saving memory, then at the scale I'm dealing with, I'm going to hit CPU problems
If performance is an issue then I would suggest profiling. Good chance that @chamaeleon proposal is both faster and more memory efficient.

While in general optimization is a choice between size vs speed, it is actually a choice between runtime computing (smaller size) vs precomputing (faster speed). i.e. You spend memory in the form of precomputation in exchange for speed. The other potential speed advantage would be memory layout; in that case, again @chamaeleon proposal is probably better.

If you are spending memory without getting the benefits of precomputation then you won't get the speed advantage, and very likely will incur a performance hit due to memory access. Typically the CPU spends a great deal of time waiting on memory transfer and cache misses. If you go with your larger memory footprint solution it is likely to also run slower.

But profiling, and clever implementation will let you know for sure.

I would also suggest that you consider (i.e. profile) using strings or buffers over arrays. Strings and buffers ARE arrays however they may have better access overhead than GML arrays; who knows. Again, profiling would help answer that question.

However, if you don't care about performance and are more leaning towards a system that is easier to implement then I agree using array indices as your flags is about the most straight forward way.
 
Last edited:

TheouAegis

Member
You only need to calculate what array to access if you don't know offhand, such as when looping through every bit. E.g., if you want to check bit 65 explicitly, you don't need to calculate which bitmask to read, because you already know it would be bitmask[1]&1.

Heck, I may as well bitmask the player's HP, the enemies' movement speeds, the button input flags, and the state flags, too.
I bitmask button inputs all the time. That's actually a logical use of bitmasks. Movement speeds wouldn"t work well if your speeds are non-integer. But anyway, if you are looking at bitmasking only in terms of memory, then it's relatively pointless outside of saving and loading these days, but it is much better in terms of multiple values actually needing to be lumped together, such as input checks, pause types, etc.
 
Top