Use which program to compress data for buffer_decompress

[jzh]

I need to supply compressed data ( compressed wav files) for buffer_decompress(), but tried "pigz -z", which will compress data to zlib format, but buffer_decompress() complains "unsupported compress method".
Any one know what the problem is? What else program I could try to compress data?
BTW, I am using HTML5 platform, GMS2.

Thanks

 

[rytan451]

As a stopgap measure, could use buffer_compress to compress the data.

 

[jzh]

As a stopgap measure, could use buffer_compress to compress the data.

Thanks for your reply, but I need a simple program to process multiple files, i.e. a linux command like pigz, so I could make a script to generate the files.

 

[chamaeleon]

Thanks for your reply, but I need a simple program to process multiple files, i.e. a linux command like pigz, so I could make a script to generate the files.

I'd recommend saving the content of a GMS compressed buffer to a file, and examine the initial couple of bytes and compare. Ideally, a buffer that contains the same content as the content you are attempting to compress using a commandline tool (doesn't have to be an audio file buffer, can be anything, a bit of lorem ipsum text or whatever). My first guess is that you are not accounting for gzip compressed content having a different header than zlib.

Zlib Wikipedia

The gzip header, used in the ubiquitous gzip file format, is larger than the zlib header, as it stores a file name and other file system information.

If you can determine the starting offset of the zlib part, you may be able to decompress. What you actually need to do to determine the number of bytes to skip in the beginning is left as an exercise for the interested reader.

 

[YellowAfterlife]

buffer_compress is pretty much stock zlib - with header, but without checksum.

You can check my zlib extension source code for compatible implementation.

 

[jzh]

buffer_compress is pretty much stock zlib - with header, but without checksum.

You can check my zlib extension source code for compatible implementation.

Thanks for this tool, but I don't have Visual studio to build it. It would be nice to include a make file to build it under Linux.
Is the problem only being the checksum? So my understanding is libz's checksum is 4 bytes at the end of file, but if I manually removed the 4-byte it still complains unsupported compression method.

 

[jzh]

I'd recommend saving the content of a GMS compressed buffer to a file, and examine the initial couple of bytes and compare. Ideally, a buffer that contains the same content as the content you are attempting to compress using a commandline tool (doesn't have to be an audio file buffer, can be anything, a bit of lorem ipsum text or whatever). My first guess is that you are not accounting for gzip compressed content having a different header than zlib.

Zlib Wikipedia


If you can determine the starting offset of the zlib part, you may be able to decompress. What you actually need to do to determine the number of bytes to skip in the beginning is left as an exercise for the interested reader.

Thank you very much for your advice!
The problem is I use HTML5 export, which saves the file to its cache, and can't find it by the name.
I will try buffer compress a short string and dump the buffer content without saving to a file, see whether that will reveal anything.
Thanks again!

 

[jzh]

Okay, now did some test to dump the compressed buffer content:

var filename = obj_assets_load.file_name; // a file contains only string "abc" and a new line.
var buf_id_test = buffer_load(filename); // checked the size and content of this buffer, it is 6 byte: 97, 98, 99, 10, 0, 0 (the last two zeros are padded, I think)
var buf_id = buffer_compress(buf_id_test, 0, 4); // so only compress the first 4 bytes, because the original file only contains 4 bytes.

The resulting content of compressed buffer is 23 bytes:
0x78, 0x9c, 0x05, 0xc0, 0x31, 0x11, 0x00, 0x00, 0x00, 0x01, 0xc0, 0x5d, 0x4b, 0xf4, 0xef, 0xf0, 0xd7, 0x3d, 0x00, 0x03, 0x73, 0x01, 0x31

However if I use command "pigz -z -4" to compress the original file containing "abc" and a new line, the result is only 12 bytes:
0x78, 0x9c, 0x4b, 0x4c, 0x4a, 0xe6, 0x02, 0x00, 0x7e, 0x03, 0x01, 0x31

They have the same header, but different content!

Anyone could shed any light on the matter?

Many thanks!

 

[YellowAfterlife]

Thanks for this tool, but I don't have Visual studio to build it. It would be nice to include a make file to build it under Linux.
Is the problem only being the checksum? So my understanding is libz's checksum is 4 bytes at the end of file, but if I manually removed the 4-byte it still complains unsupported compression method.

You are not expected to build it, only take a look at the code to see how to invoke zlib yourself to compress data in a way that matches GM's buffer_compress output.

Thank you very much for your advice!
The problem is I use HTML5 export, which saves the file to its cache, and can't find it by the name.
I will try buffer compress a short string and dump the buffer content without saving to a file, see whether that will reveal anything.
Thanks again!

Since zlib format is consistent across exports, you could run buffer_compress on a desktop target and take the resulting file to your HTML5 project.

They have the same header, but different content!

It is possible that default compression level differs, or implementation does.