Android / Amazon Fire [SOLVED!] copying a file using buffer problem

W

Wraithious

Guest
Hello, in my project I want to just copy a file in the included files without modifying or doing anything to it and save it in a folder.
So after realizing GMS' file handling functions do not work with .mp3 files I converted my file to an .ogg and I got it to load the file into a buffer and transfer it to a public folder through my extension, but the resulting file is not playable, the file is all there right down to the last byte, but always reports the track length is 00:00:00, same deal with .wav files, really weird, so I'm guessing maybe it's a problem with how I create the buffer? I'm not too good with buffers and alignments but my reasoning was if I made a u16 buffer with a 1 alignment it would just copy the file byte for byte and not change anything, but I'm guessing the meta data somehow gets lost (doubtful because every single byte is accounted for) or that maybe I have to use an alignment of 2 or 4 to account for mono or stereo but I really have no idea? Can anyone help with this? here is my code on the GMS side:
Code:
fname = "wildDubz.ogg";
if(file_exists(fname))
{file = file_bin_open(fname,0);
size = file_bin_size(file);
gameTransfer = buffer_create(1024, buffer_grow, 1);
for(var i=0;i<size;i+=1;){buffer_write(gameTransfer,buffer_s16,file_bin_seek(file,i));}
var buffaddress = buffer_get_address(gameTransfer);
SaveFromAs(buffaddress,global.myFolder,"wildDubz.ogg",size);
file_bin_close(file);}
if !file_exists(fname) show_debug_message("file_open_bin has failed to open the file");
and if it helps, here is my code in my extension that extracts the buffer data to an android public folder:
Code:
//buffer address, absolute path to output folder, new file name, file size in bytes
        public void SaveFromAs(String inputPath, String opp, String opn, double setToRing) {
            int numOfBytes = (int)setToRing;
            Log.i("yoyo", "295: Buffer size: " + String.valueOf(numOfBytes));
            try
            {
                mf = new MemoryFile(inputPath, numOfBytes);
                InputStream in = null;
                OutputStream out = null;
                in = mf.getInputStream();
                Log.i("yoyo", "302: Found the file, input stream created: " + String.valueOf(in));
                out = new FileOutputStream(opp + "/" + opn);Log.i("yoyo", "303: Output stream created");
                copyFile(in, out);
                in.close();
                in = null;
                out.flush();
                out.close();
                out = null;
                OsUpdt(opp + "/" + opn);
                ReturnGMS("buff","buff_finished","","",-4,-4);
            }
            catch(IOException e)
            {
                Log.i("yoyo", "315: Couldn't find file");
                Log.e("tag", "Failed to copy asset file: " + inputPath, e);
            }
        }

    private void copyFile(InputStream in, OutputStream out) throws IOException {
      byte[] buffer = new byte[1024];
      int read;
      while((read = in.read(buffer)) != -1)
          {
          out.write(buffer, 0, read);
          }
    }
 

FrostyCat

Member
You read from the file as bytes (u8) but wrote in the buffer as s16. This will make a mess of the copy.

Also, the function for reading a byte is file_bin_read_byte(), not file_bin_seek().
 
W

Wraithious

Guest
ahh I didn't realize that, I'm still in the learning stage in using buffers, thanks I'll change it to u8 and see what happens.
 
W

Wraithious

Guest
You read from the file as bytes (u8) but wrote in the buffer as s16. This will make a mess of the copy.
hmm, that still didn't work, I changed this line
Code:
for(var i=0;i<size;i+=1;){buffer_write(gameTransfer,buffer_s16,file_bin_seek(file,i));}
to this:
Code:
for(var i=0;i<size;i+=1;){buffer_write(gameTransfer,buffer_u8,file_bin_seek(file,i));}
This is a visual of the file properties I get from the copied file:
wd1kb4.png
and it should look like this:
wd1kb5.png
 
W

Wraithious

Guest
Again, file_bin_seek() isn't the function for reading a byte, file_bin_read_byte() is.
ok sorry I missed that, but now I'm realizing I need to use file_bin_position along with that, but the documentation is extremely vague on that, it only shows how you get the byte at position 0,
Code:
pos = file_bin_position(file);

This would store the current position in the variable "pos".
how do I loop through the positions? I'm not understanding why that function wouldnt be file_bin_position(file,position), or instead why wouldn't file_bin_read_byte(file) actually be file_bin_read_byte(file,byte), that makes absolutly no sense to me, how do you define what position to get the byte? are there any examples I could look at?

...
 
Last edited by a moderator:

FrostyCat

Member
how do I loop through the positions? I'm not understanding why that function wouldnt be file_bin_position(file,position), or instead why wouldn't file_bin_read_byte(file) actually be file_bin_read_byte(file,byte), that makes absolutly no sense to me, how do you define what position to get the byte? are there any examples I could look at?

...
You simply call file_bin_read_byte() iteratively, then each call would give you the current byte and move onto the next byte, just like what buffer_read() does with buffers. If you need to read a byte out of order, use file_bin_seek() and then file_bin_read_byte().

Also, the Manual's description clearly reads "current position". This is not vague, it is you missing an important cue.
 
W

Wraithious

Guest
You simply call file_bin_read_byte() iteratively, then each call would give you the current byte and move onto the next byte, just like what buffer_read() does with buffers. If you need to read a byte out of order, use file_bin_seek() and then file_bin_read_byte().

Also, the Manual's description clearly reads "current position". This is not vague, it is you missing an important cue.
ahh ok I see what you mean thanks for clarifying that. So I tried a few different combinations of what you said and an example @hippyman had posted on sound files but still I'm getting the same results with the file output, here's all the things I tried:
try#1:
Code:
for(var i=0;i<size;i+=1;){buffer_write(gameTransfer,buffer_u8,file_bin_read_byte(file));}
try#2:
Code:
for(var i=0;i<size;i+=1;){var byte=file_bin_read_byte(file);buffer_write(gameTransfer,buffer_u8,byte);}
try#3 overkill method:
Code:
for(var i=0;i<size;i+=1;){file_bin_seek(file, i);var byte=file_bin_read_byte(file);buffer_write(gameTransfer,buffer_u8,byte);}
try#4, I've seen that littleEndian and bigEndian (wav and raw?) audio files sometimes get read backwards so I reversed the loop:
Code:
for(var i=size;i>0;i-=1;){file_bin_seek(file, i);var byte=file_bin_read_byte(file);buffer_write(gameTransfer,buffer_u8,byte);}
So I'm at a total loss now as to why nothing makes any difference when the file is saved :(

EDIT: I also tried it with a text file, allmost the same thing happens, the file opens but theres no text there, it's also labled as ASCII, This is what the text file attempt looks like when opened in notepad++, I'm assuming that's what my audio files also look like if I could open them in notepad++ So what am I missing here? there has to be something I'm supposed to be doing that I don't know about...
wd1kb6.png
 
Last edited by a moderator:

PNelly

Member
ahh ok I see what you mean thanks for clarifying that. So I tried a few different combinations of what you said and an example @hippyman had posted on sound files but still I'm getting the same results with the file output, here's all the things I tried:
try#1:
Code:
for(var i=0;i<size;i+=1;){buffer_write(gameTransfer,buffer_u8,file_bin_read_byte(file));}
try#2:
Code:
for(var i=0;i<size;i+=1;){var byte=file_bin_read_byte(file);buffer_write(gameTransfer,buffer_u8,byte);}
try#3 overkill method:
Code:
for(var i=0;i<size;i+=1;){file_bin_seek(file, i);var byte=file_bin_read_byte(file);buffer_write(gameTransfer,buffer_u8,byte);}
try#4, I've seen that littleEndian and bigEndian (wav and raw?) audio files sometimes get read backwards so I reversed the loop:
Code:
for(var i=size;i>0;i-=1;){file_bin_seek(file, i);var byte=file_bin_read_byte(file);buffer_write(gameTransfer,buffer_u8,byte);}
So I'm at a total loss now as to why nothing makes any difference when the file is saved :(

EDIT: I also tried it with a text file, allmost the same thing happens, the file opens but theres no text there, it's also labled as ASCII, This is what the text file attempt looks like when opened in notepad++, I'm assuming that's what my audio files also look like if I could open them in notepad++ So what am I missing here? there has to be something I'm supposed to be doing that I don't know about...
View attachment 14403
Slow down and and try to think a little more carefully about the changes you're making to your code. Your first, second, and third iterations are all effectively doing the same thing, and if isn't working it indicates the issue probably isn't there. If you're simply copying a file from one place to another then the endianness of the file shouldn't make any difference. Even so, endianness impacts how you have to read individual, multi-byte numbers as you move through the file, rather than just simply reading it backwards (it's actually a huge pain the butt if you need to go against the grain).

So I've never written Java for Android, but I see a couple things that don't make sense to me. You're using buffer_get_address() to get a pointer to your buffer and passing it as a string to your Java extension, then you're creating a new MemoryFile and providing the optional "name" argument as the string representation of that pointer. Java has no conception of pointers, and isn't designed to allow direct memory access to begin with, so trying to use a buffer address isn't the right approach if you want to accomplish this with Java. I'm pretty confident what's happening is that you're passing the size of the buffer through to your extension correctly, which makes your write loop proceed through the correct number of iterations, but you're feeding bunk data from MemoryFile to your write operations, which would be why you see a bunch of null bytes piling up to the same size of the original file.

I think you might be making this a lot harder than it needs to be. If all you want to do is copy a file then I'd recommend just passing the file's path out to the extension and using Java to open and read it in the first location, then write it out to your other location. You should really only need input and output streams, I don't see any need for MemoryFile, and it doesn't really seem suited to the task if you look at the docs: (https://developer.android.com/reference/android/os/MemoryFile.html#MemoryFile(java.lang.String, int))
 
G

grixm

Guest
I didn't read the whole thread carefully so sorry if I misunderstand, but it looks like part of your code loads the file as a binary file and then manually transfers the data into a buffer. This is unnecessary, you can load the file into a buffer directly using buffer_load()
 
W

Wraithious

Guest
I'm pretty confident what's happening is that you're passing the size of the buffer through to your extension correctly, which makes your write loop proceed through the correct number of iterations, but you're feeding bunk data from MemoryFile to your write operations, which would be why you see a bunch of null bytes piling up to the same size of the original file.

I think you might be making this a lot harder than it needs to be. If all you want to do is copy a file then I'd recommend just passing the file's path out to the extension and using Java to open and read it in the first location, then write it out to your other location. You should really only need input and output streams, I don't see any need for MemoryFile, and it doesn't really seem suited to the task if you look at the docs: (https://developer.android.com/reference/android/os/MemoryFile.html#MemoryFile(java.lang.String, int))
Yep my only goal here is to copy the file byte for byte to a new location without changing it. And yes I believe you are 100% correct about the file size being correct and the data being bunk, and I had read the memory file docs quite a few times and looked at examples before trying any of this, 90% of the info I found on memoryFile showed how to put a file into memoryFile and very little info on how to get it back out, but I did finaly find an example and followed that. what I had thought from my reading tho was that all I needed to do was pass the buffer address to memory file and the size, which I have done, and memoryFile would do the rest if coupled to a filestream in/out, the only thing I was doing by passing the file name was for the output stream to name the file, which I give the exact same name to. I don't know of a way to use fileOutputStream without providing a name, unless you mean I only need to use the output path without the name?

the biggest problem, and I would have loved to do it the way you said by just passing the path address and using input/output streams directly but that seems to be broken in GMS as per this thread I started a week ago, I marked it solved only because @rwkay gave me a few tips on a work around, the result of which, + my additions / modifications to it, you see here in this thread. So it's like a catch 22, one thing works one way, the other things work the other way but they don't work together.

I didn't read the whole thread carefully so sorry if I misunderstand, but it looks like part of your code loads the file as a binary file and then manually transfers the data into a buffer. This is unnecessary, you can load the file into a buffer directly using buffer_load()
Yes I definatly tried that too, at the time I tried it tho I was trying to use an .mp3 file only in testing, which I found out later GMS cannot load a .mp3 file into a buffer at all (at least not from an included file), so maybe I'll give that another shot, I really don't have any more options that I'm aware of

EDIT: nope, that still didn't work, I set it up like this in gms:
Code:
fname = "clint.ogg";
if(file_exists(fname))
{
gameTransfer = buffer_load(fname);
size = buffer_get_size(gameTransfer);
var buffaddress = buffer_get_address(gameTransfer);
SaveFromAs(buffaddress,global.myFolder,"clint.ogg",size);
}
if !file_exists(fname) show_debug_message("file_open_bin has failed to open the file");
same results, I think @PNelly is on to something tho, I'm going to try and rethink how the java method should be, I don't know tho at this point, I've tried other ways allready before the memoryFile way such as using assetManager, but can't pass a path to it from gms, as I said its a catch22
Also just tried it with your way and wrkay's suggestion of using file_copy to first move the file from included files to the save area,
Code:
file_copy("clint.ogg","clint.ogg");
fname = "clint.ogg";
if(file_exists(fname))
{
gameTransfer = buffer_load(fname);
size = buffer_get_size(gameTransfer);
var buffaddress = buffer_get_address(gameTransfer);
SaveFromAs(buffaddress,global.myFolder,"clint.ogg",size);
}
if !file_exists(fname) show_debug_message("file_open_bin has failed to open the file");
but my extension breaks in that case with the dreadded
Code:
11-20 14:24:44.916 29401 29483 I yoyo    : 245: got file:
11-20 14:24:44.918 29401 29483 I yoyo    : Can't find method on extension class:null
error because there is no valid buffer address or size being passed there.
 
Last edited by a moderator:
W

Wraithious

Guest
@PNelly and @FrostyCat and anyone else who may be able to help, So over the last 2 days I've made some good progress with this, it is 99.333 (repeating of course lol) % successful, meaning text files can be copied perfectly, the file is there, correctly named, and openable and readable by humans, whereas .wav and .ogg crash my extension with thousands of "Can't find method on extension class:null" errors.
so what I did was, in my extension, I first defined a byte list array named bytesRead:
Code:
List<Byte> bytesRead = new ArrayList<Byte>();
Then changed my method to read and convert the series of strings representing each byte passed from gms into Bytes and load them into the arrayList, then convert Bytes[] to bytes[] using a byte array converter, then pass the completed byteArray into a byteArrayInputStream, and finally write them out to the file:
Code:
   public void SaveFromAs(String inputPath, String opp, String opn, double setToRing) {
            int numOfBytes = (int)setToRing;
            if(byteCount<numOfBytes)
            {
            bytesRead.add(Byte.valueOf(inputPath));
            byteCount++;
            }
            if(byteCount>=numOfBytes){
                byte[] byteArray = new byte[bytesRead.size()];
            for (int index = 0; index < bytesRead.size(); index++) {
                byteArray[index] = bytesRead.get(index);
                }
                try{
                    OutputStream out = null;
                    InputStream in = new ByteArrayInputStream(byteArray);
                    out = new FileOutputStream(opp + "/" + opn);Log.i("yoyo", "303: Output stream created");
                    copyFile(in, out);
                    in.close();
                    in = null;
                    out.flush();
                    out.close();
                    out = null;
                }
            catch(IOException e)
            {
                Log.i("yoyo", "315: Couldn't find file");
                Log.e("tag", "Failed to copy asset file: " + inputPath, e);
            }
            OsUpdt(opp + "/" + opn);
            ReturnGMS("buff","buff_finished","","",-4,-4);
            Log.i("yoyo", "295: Buffer size: " + String.valueOf(numOfBytes));
            byteCount=0;
            }
                                        }

    private void copyFile(InputStream in, OutputStream out) throws IOException {
      byte[] buffer = new byte[1024];
      int read;
      while((read = in.read(buffer)) != -1)
          {
          out.write(buffer, 0, read);
          }
    }
And this is my rebuilt codeblock from gms to get and pass the file as individual bytes converted to strings (you cannot pass anything but strings and doubles to an extension):
Code:
fname = "test50.txt";
if(file_exists(fname))
{gameTransfer = buffer_load(fname);
var buffaddress = buffer_get_address(gameTransfer);
size = buffer_get_size(gameTransfer);
var type = buffer_get_type(gameTransfer);
for(var i=0;i<size;i+=1;)
{var byte = string(buffer_read(gameTransfer,type));
SaveFromAs(byte,global.myFolder,"test50.txt",size);
}
if !file_exists(fname) show_debug_message("Can't find file");
}
So what do I do now? why won't this work on .wav and .ogg files? am I still missing something or doing something wrong?
 

FrostyCat

Member
It's probably choking on the null characters.

My take would be to pass the buffer in Base64 form, then you can use strings straight without attempting to handle pointers with Java.
 
W

Wraithious

Guest
It's probably choking on the null characters.

My take would be to pass the buffer in Base64 form, then you can use strings straight without attempting to handle pointers with Java.
Ok thanks I'll check that out, I saw some posts and other info on base64 but I'll have to study that, I've never used it before.

EDIT:
I checked over at stackOverflow and now understand what you're telling me, I think you are correct and this is the way I'm going to have to learn to do it to make it work. I have to do some thanksgivingly stuff but I'll post back here if I get it to work, thanks again
 
Last edited by a moderator:

PNelly

Member
Ok thanks I'll check that out, I saw some posts and other info on base64 but I'll have to study that, I've never used it before.

EDIT:
I checked over at stackOverflow and now understand what you're telling me, I think you are correct and this is the way I'm going to have to learn to do it to make it work. I have to do some thanksgivingly stuff but I'll post back here if I get it to work, thanks again
Indeed, base64 will be your friend. Doing it that way you can pass the whole file as a single string, and you eliminate the null bytes as Frosty points out. Your code still has way more going on than it needs to. I think bringing together the things discussed so far (and peeking at SO a little bit), you can probably reduce your approach to something like this:

Code:
/// GML
fname = "wildDubz.ogg";
if(file_exists(fname)){
  file = file_bin_open(fname,0);
  size = file_bin_size(file);
  gameTransfer = buffer_create(1024, buffer_grow, 1);
  for(var i=0;i<size;i++;){
    buffer_write(gameTransfer, buffer_u8, file_bin_read_byte(file));
  }
  file_bin_close(file);}
  var dataString = buffer_base64_encode(gameTransfer, 0, buffer_get_size(gameTransfer));
  SaveFromAs(dataString, fname);
} else {
  show_debug_message("file open failed");
}
Code:
/// Java
public void saveFromAs(String dataString, String fname){

  byte[] data = Base64.decode(dataString, 0);

  try{
    // not sure what directory you're needing to point towards but it sounds
    // like this might be the place you want
    File outFile = new File(Environment.getExternalStorageDirectory() + fname);
    FileOutputStream fos = new FileOutputStream(outFile, true);
    fos.write(data);
    fos.close();
  } catch (Exception e){    
    Log.i("yoyo", "315: Couldn't find file");
    Log.e("tag", "Failed to copy asset file: " + inputPath, e);
  }

  // I've never been an extension guy so I dunno what's required around here
  ReturnGMS("buff","buff_finished","","",-4,-4);
}
Happy Turkey.
 
W

Wraithious

Guest
Indeed, base64 will be your friend. Doing it that way you can pass the whole file as a single string, and you eliminate the null bytes as Frosty points out. Your code still has way more going on than it needs to. I think bringing together the things discussed so far (and peeking at SO a little bit), you can probably reduce your approach to something like this:
Thanks very much for the advice and example! Just for informational purposes there's 2 ways I'm getting the folder address to save stuff to in java to the user's device so they can access the content from any app, by allowing the player or the developer to name their folder that the files will be put into:
Code:
        public String GetFldr(String arg0) {
        namedFolder = arg0;
                state = Environment.getExternalStorageState();
        if (Environment.MEDIA_MOUNTED.equals(state)&& !arg0.equals("")) { //Check if we can access public storage
            Root = Environment.getExternalStorageDirectory();
            folder = new File(Root.getAbsolutePath() + "/" +arg0 + "/"); //make the folder with name passed from GMS
            if (!folder.exists()) {
                folder.mkdirs();}      
                msg = folder.toString();
                editor.putString("folderaddress", msg).apply();
            }
            String myString = msg; //save the absolute address to the folder in variable msg
            Log.i("yoyo","89: "+ myString);
            return myString; //pass the absolute folder address back to GMS
    }
///////////// get the absolute path to a default folder, in this case the ringtones folder
//....
File f = new File(inputPath);
String ext = (f.getAbsolutePath().substring(f.getAbsolutePath().lastIndexOf(".")));
                ringFolder = (Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_RINGTONES));
                if (!ringFolder.exists()) {
                    ringFolder.mkdirs();
                    Intent rint = new Intent(RingtoneManager.ACTION_RINGTONE_PICKER);
                    rint.putExtra(RingtoneManager.EXTRA_RINGTONE_TYPE,RingtoneManager.TYPE_RINGTONE);
                    RunnerActivity.CurrentActivity.startActivityForResult(rint, 0);
                }
                msg = opn + ext;
                opp = ringFolder.toString(); // set variable opp (output path) to the ringtone folder's absolute address
//....
tomorrow after the family leaves I will work on this in-depth and solve this, Happy turkey to you too!
 
W

Wraithious

Guest
Indeed, base64 will be your friend. Doing it that way you can pass the whole file as a single string, and you eliminate the null bytes as Frosty points out. Your code still has way more going on than it needs to. I think bringing together the things discussed so far (and peeking at SO a little bit), you can probably reduce your approach to something like this:
Wowwww thanks man that worked nice!! I tweaked the java and gms code slightly and it works perfect, in gms I just added my other 2 arguments to call the method, and in the java code i did this:
Code:
public void SaveFromAs(String inputPath, String opp, String opn, double setToRing) {
            byte[] data = Base64.decode(inputPath, 0);

        try{
            File outFile = new File(opp + "/" + opn);
            FileOutputStream fos = new FileOutputStream(outFile, false);
            fos.write(data);
            fos.close();
        } catch (Exception e){   
            Log.i("yoyo", "315: Couldn't find file");
            Log.e("tag", "Failed to copy asset file: " + inputPath, e);
        }           
            OsUpdt(opp + "/" + opn);
            ReturnGMS("buff","buff_finished","","",-4,-4);
            Log.i("yoyo", "295: Buffer size: " + String.valueOf(numOfBytes));
        }
I had to change the boolean argument in the fileOutputStream to false, true makes it write the data to the end of the file (that would be usefull for appending to a file)
and I just grabbed my file destination path from gms that originally gets the path when you start the game and make your folder.
Again thanks for your help!
 
Top