SOLVED Sending a surface or buffer to a C++ DLL and getting processed image from it to a surface or buffer back

mbeytekin

Member
I'm trying to send a surface to dll and process it in DLL, and after this get this processed image back. I tried send surface as buffer (argument as double) but it's not working.
My mean goal is using opencv library in C++.
Anyone can help please ?
 

Bart

WiseBart
What is it that you're doing precisely at the moment?

Definitely not an expert at the matter but I'll try to outline what I expect to be the way of doing things.
First of all, you'll want to copy the surface's contents to a buffer using buffer_get_surface (modifying the surface directly isn't possible since it's stored in VRAM).
I assume it's what you're doing already since you mention buffers.
Then, you'll need to pass that buffer's address in memory to the DLL. GM allows you to retrieve the address using buffer_get_address. And that's what you pass to the DLL using external_call so it knows where to find the data in RAM.
I'm not sure if it's possible to send the address using a real (i.e. double on the C++ side). You probably need to use ty_string as argument type in external_define instead. On the C++ side you then interpret that value as a pointer.
After you finish modifying the buffer's data and the function returns on the GM side you can get the resulting image back to the surface using buffer_set_surface.
 
Last edited:

mbeytekin

Member
Thank you for your answer..
I've just found a C++ code for processing an image that uses SLIC (superpixel creation) method. This code uses opencv libraries. But in that code it loads an image and saves after processing. But I want to control this process within my GM app.So I must send raw image to it and get it back from memory.
I tried to implement 'https://github.com/YoYoGames/GameMakerStudio_ExtensionExample' for copying buffer to DLL , but I can't do anything. I want to create extension with functions.I didn't try external call function. As you said when I try to copy buffer to DLL it crashes because of memory issues or double type I guess.
 

Bart

WiseBart
That's an interesting example! I didn't know such thing existed.
It seems like there's an example function on buffers in there as well, the following declaration:
C++:
EXPORTED_FN  char*  MyExtension_BufferPointer( void* _pBuffer, double _size)
{
    // ...
}
So you need to pass a pointer to the buffer (simply the value returned by buffer_get_address, I'd assume) and its length.
(So you don't send the actual image or buffer, you just let the DLL know where to find it in RAM and how many bytes it takes up; both GM and the DLL write to and read from that same contiguous block of memory, which is the buffer)

But at the moment it's not entirely clear to me where you're stuck.
Have you been able to build the DLL? Did you create the DLL extension already?
And how does your code look on the GameMaker side? How are you making the calls?
Are you calling extension functions?

Quite a few questions but it's important to figure out where things go wrong.
 

mbeytekin

Member
Yes I created DLL and normally it works. But as you said I'm trying send all buffer data to DLL.This is not necessary and unuseful. Only starting address and size will work I guess. I'll try this. Main problem is C++ side. My C++ knowledge is very limited.
 

mbeytekin

Member
Now I have another problem
DLL has some opencv functions..
Gamemaker shows some errors when run
I put all DLL files from Opencv directories to my included files but nothing changed
LoadLibraryA failed with error code 193
LoadLibraryA failed with error code 193
LoadLibraryA failed with error code 193
LoadLibraryA failed with error code 193
LoadLibraryA failed with error code 193
LoadLibraryA failed with error code 193


Opencv libraries are 64bit and I must compile it as x64 in C++. Maybe this is the problem
 
Last edited:

rIKmAN

Member
Now I have another problem
DLL has some opencv functions..
Gamemaker shows some errors when run
I put all DLL files from Opencv directories to my included files but nothing changed
LoadLibraryA failed with error code 193
LoadLibraryA failed with error code 193
LoadLibraryA failed with error code 193
LoadLibraryA failed with error code 193
LoadLibraryA failed with error code 193
LoadLibraryA failed with error code 193


Opencv libraries are 64bit and I must compile it as x64 in C++. Maybe this is the problem
That is the problem - the Windows Runner is still only 32bit.
 

Sybok

Member
Firstly, the buffer argument should be sent as a string (string is a pointer in GMS).
And secondly the 193 error means that your DLL isn't seeing a dependency somewhere. Is the OpenCV DLL in the same folder as your extension?

As rIKmAN said the DLL's need to be 32 bit, so that's why your extension isn't seeing it correctly.
 

mbeytekin

Member
Ok. I solved my 32bit 64bit problem with using old opencv libraries. Now DLL works but I can't create a working buffer pointer for image in C++

My GM Code like this;

GML:
/// @description Insert description here

// You can write your code in this editor

_ww=sprite_get_width(sprite0)
_hh=sprite_get_height(sprite0)

surf= surface_create(_ww,_hh)
surface_set_target(surf)
draw_clear_alpha(c_black,0)
draw_sprite(sprite0,0,0,0)
surface_reset_target();
buff=buffer_create(_ww*_hh*4,buffer_fixed,1)
buffer_get_surface(buff,surf,buffer_surface_copy,0,0)

DLLFunc = external_define("DLLGAMEMAKER.dll", "createImage", dll_cdecl, ty_real, 4, ty_real, ty_real, ty_real, ty_real);
var _a = external_call(DLLFunc, buffer_get_address(buff),_ww*_hh*4,_ww,_hh);
and my C++ code is;
C++:
fn_export double createImage(char* buffer, char size,int width,int height) {
   
    int buf;
    (unsigned char *) buf;

    memcpy(&buf,buffer, size);

     cv::Mat img = cv::Mat(720,1280, CV_8UC4, buf);
   
    cv::imshow("Original", img);

    return 1;
}
But it shows a blank blue page in DLL. so my buffer is no correct in DLL.

By the way I never send width (_ww) and height(_hh) values to DLL. When I enter manually cv::Mat img = cv::Mat(720,1280, CV_8UC4, buf); it works with blank page but if I change to cv::Mat img = cv::Mat(height,width CV_8UC4, buf); it gives an error in GML because of cv::Mat img = cv::Mat(height,width, CV_8UC4, buf); causes a 0x0 image...

Have you any advice about this?
 
Last edited:

Bart

WiseBart
Hm... So you create a surface, draw something to it and copy the contents to a buffer with the correct size (width*height*(1 byte per rgba component)).
The code looks alright on the GM side.
But the types in the function signature of createImage don't correspond exactly to the types used by GameMaker.
Not sure how the DLL handles the conversion of size, width and height (all three being ty_real).
The way you define size at the moment makes it so that it can only hold a maximum value of 255 which is not enough for the 720*1280*4 that you need.
buf is also declared twice for some reason and you don't allocate any memory for it. Next you copy to its address (wherever that may be).
Does it change anything if you change the function definition to this:
C++:
fn_export double createImage(char* buffer, double size, double width, double height) {
    // ...
}
The example you linked to also mentions a void*. Not sure if that's a requirement and whether you can use a char* instead.
GML:
DLLFunc = external_define("DLLGAMEMAKER.dll", "createImage", dll_cdecl, ty_real, 4, ty_string, ty_real, ty_real, ty_real);
That's the things that catch my attention that might need a further look into.
Perhaps someone else here is able to give some more insight on the C++ side of things.

Hope this may help a bit anyways.

EDIT: seems like I'm wrong about the buf part. That's supposed to be the location of the pointer to the buffer.
But it looks like you're confusing the size of the pointer with the size of the buffer.
You know the size of the pointer, it's 32 bits or 4 bytes since GM's runner is 32bit, as mentioned above.
size is the size of the data that it points to.
 
Last edited:

mbeytekin

Member
Hmm yes you're right.. size must be 4. But now confused again. How can DLL know size of buffer? I must set the size of buffer pointer in C++ before memcpy? Really confused ... :)
 

Bart

WiseBart
You're right, I think my explanation may have caused more confusion.

I'm not too sure what you're trying to do with the memcpy:
C++:
int buf;
(unsigned char *) buf;

memcpy(&buf,buffer, size); // guess the size argument here should be either 4 or sizeof(buf)
Is it to create a reference to the buffer that you can pass as an argument to cv::Mat?

The DLL already knows the size of the buffer, you're already passing that in as _ww*_hh*4, which seems correct to me. But on the DLL side you take it in as a char. And you use that same value for the memcpy.
What I'd suggest is to try get the function's argument types right, both in the call to external_define and in the function signature in the DLL's code. char* corresponds to ty_string, double corresponds to ty_real.
Then see if you still end up with a 0x0 image.
I'm not even sure if you have to pass the buffer's size in this case. Since it can be derived from the image's dimensions combined with the datatype in that cv::Mat function.

Must admit I'm a bit limited by my C++ knowledge here so maybe someone else can help out further with the specifics :)
 

mbeytekin

Member
testd.png
Thank you very much for your suggestions.This is my first try with C++ DLL and I choosed a bit difficult work for first one. After million of tryings I see exported and returned variable types are very important. Especially working with memcpy. Because it crashes if any variable type is wrong. But I'm not sure about "The DLL already knows the size of the buffer, you're already passing that in as _ww*_hh*4" and GM's starting address. After trying to change variables finally I got an image like screenshot.
I tried convert buffer address Hex to Dec before exporting but nothing changed and this strange image doesn't change when I send a different address to DLL. So DLL's reading wrong address.
 

mbeytekin

Member
Finally it works! Bart you were right . This size thing is not like I think.

GML:
/// @description Insert description here
// You can write your code in this editor
_ww=sprite_get_width(sprite0)
_hh=sprite_get_height(sprite0)
surf= surface_create(_ww,_hh)
surface_set_target(surf)
draw_clear_alpha(c_black,0)
draw_sprite(sprite0,0,0,0)
surface_reset_target();

buff=buffer_create(_ww*_hh*4,buffer_fixed,1)
buffer_get_surface(buff,surf,buffer_surface_copy,0,0)

DLLFunc = external_define("DLLGAMEMAKER.dll", "createImage", dll_cdecl, ty_real, 3,  ty_string,ty_real, ty_real);
external_call(DLLFunc,buffer_get_address(buff),_ww,_hh);
C++ code;

C++:
fn_export double createImage(char* _pBuffer,  double width, double height) {
    unsigned char* pBuffer = (unsigned char*)_pBuffer;
    memcpy(&pBuffer,&_pBuffer, sizeof(pBuffer));
    cv::Mat img = cv::Mat(height,width, CV_8UC4, pBuffer);
    cv::imshow("Original", img);
    return 1;
}
Regards
 

Bart

WiseBart
It's definitely not the easiest DLL to start out with alright. All the data types, that opencv library, a bit of pointer trickery, ...
Nice to hear that you got it working!
 

mbeytekin

Member
Now the second part... :) Getting processed image to GM2 buffer back.
I was thinking like this: If I get data in C++ from GM2 buffer's address, after processing image memcpy again to original pointer address. But can't do this.

C++:
fn_export double createImage(char* address  double width, double height) {
    unsigned char* pBuffer = (unsigned char*)address;
    memcpy(&pBuffer,&address, sizeof(pBuffer));
    cv::Mat img = cv::Mat(height,width, CV_8UC4, pBuffer);
    cv::imshow("Original", img);
  
    memcpy(&address, &img.data[0], sizeof(address));
  
        return 1;
}
Mat is opencv image and image.data is array with image data.

I search google and GM forums but I can't find any example exactly.
There are some examples to get pointer address from C++ and buffer_seek from this address. This is the only way?
Or memcpy can copy from C++ to GM buffer?
 
Last edited:

chamaeleon

Member
Now the second part... :) Getting processed image to GM2 buffer back.
I was thinking like this: If I get data in C++ from GM2 buffer's address, after processing image memcpy again to original pointer address. But can't do this.

C++:
fn_export double createImage(char* _pBuffer,  double width, double height) {
    unsigned char* pBuffer = (unsigned char*)_pBuffer;
    memcpy(&pBuffer,&_pBuffer, sizeof(pBuffer));
    cv::Mat img = cv::Mat(height,width, CV_8UC4, pBuffer);
    cv::imshow("Original", img);
  
    memcpy(&pBuffer, &img.data[0], sizeof(pBuffer));
  
        return 1;
}
As you see pBuffer is my pointer for memcpy() and Mat is opencv image and image.data is array with image data.

I search google and GM forums but I can't find any example exactly.
There are some examples to get pointer address from C++ and buffer_seek from this address. This is the only way?
Or memcpy can copy from C++ to GM buffer?
You can certainly memcpy() from one to the other. The buffer you received from GML is a contiguous memory area of a fixed size, and you can treat it any way you want in the DLL (correctly or incorrectly as C/C++ allows you to...)

Your current memcpy() of image data is only copying 4 bytes though, since the size of a pointer is 4 bytes. You'll want to use some perhaps like width*height*sizeof(img.data[0]) under the assumption your buffer allocated in GML before calling the DLL function was made large enough to hold this amount of data (I don't know anything about this type of structure, nor how it correlates to your use in GML once the DLL call returns).

The first memcpy() is totally unnecessary as you already assigned the address in the first line declaring the pBuffer variable.
 

mbeytekin

Member
Yes you are right chamaeleon.. memcpy size was wrong..
I changed size and it works now...


memcpy(&address[0], &img.data[0], width*height*4.);


and this lines are unnecessary;

unsigned char* pBuffer = (unsigned char*)_pBuffer;
memcpy(&pBuffer,&_pBuffer, sizeof(pBuffer));

now my code is;

C++:
fn_export double createImage(char* address,  double width, double height) {
    
        Mat img = Mat(height, width, CV_8UC4, address);
    cv::imshow("Image from GM", img);
    // same image copy to buffer back;
    memcpy(&address[0], &img.data[0], width*height*4.);
    
    return 1;
}
Thank you very much again. Now my DLL works like a charm :)
Regards...
 
Top