Windows [UPDATED: 6/6/17] HTC Vive Support [Github Repo]

Discussion in 'Work in Progress' started by Binsk, Jul 26, 2016.

  1. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    EDIT: This project is long dead, poorly organized, and will need a complete rewrite to work with current systems. I may do so in the future but at this point in time it is only good as a casual reference.

    Hello everyone.

    Introduction:
    I have been trying to add support for the HTC Vive to GameMaker and finally have something to show. I have a fully working system here that is technically capable of making a full VR Game. The only thing holding it back currently is the need to copy the render surfaces into a buffer each step in order to pass the texture data to the DLL. This still happens fast enough that you can maintain 90fps with a decent CPU and, since most of VRs general limitations are because of the GPU, I expect decent games can still be made if designed with efficiency in mind.

    Download:

    You can get access to the source code and binaries from my github repo.
    My DLL and source is covered under the MIT license, however it requires some 3rd party libraries such as GLEW, SDL and of course OpenVR. Each library contains its own license so make sure to check those out if you decide to make something with this extension.

    Hardware:
    You need a beefy computer to be able to run VR as well as the appropriate VR hardware. The minimum GPU requirement is a Nvidia GTX 970 and you will need at least one base-station and a VR headset along with all attachments.

    If you have an underpowered GPU then you will experience severe jitter with your camera. If you do not have the VR setup then this system will not boot.

    I have not tried this with the Oculus because I do not (and will not) own one. OpenVR is supposedly compatible but I handled the controller input with the Vive in mind specifically so controls may map funny (and a few axes are not accounted for). If you have an Oculus and try this out let me know how it works!

    Final Notes:
    I started this project as a proof of concept and it remains as such. While it is fully functional there are a few things holding it back such as the buffer requirement. This means that to design anything of substance with this system you are going to have to have experience and know what you are doing in order to make it run smoothly.

    Also, you will need to know how to handle 3D via matrices. I provide several matrix functions in my GML code but you will need to know what matrices to multiply together and how to assign them and render to a surface correctly. I only provide the required matrices to render with but you have to know how to use them.

    As for features I have support for head tracking, vive controller tracking as well as related digital / analog button, touchpad, and trigger support. There is system and compositor error checking as well as playspace bound retrieval and chaperone state detection. That is where the features end as that is what I suspect is essential to make a game. There are other VR features that I have not implemented but if you need them let me know or feel free to contribute.

    I would love to hear feedback or, even better yet, try out something you are making with this project! Feel free to leave a post or a link here! Thanks for your interest. Should this garner enough interest then I may consider maintaining it and porting it efficiently over to GMS2.
     
    Last edited: Sep 9, 2019
  2. icuurd12b42

    icuurd12b42 TMC Founder GMC Elder

    Joined:
    Apr 22, 2016
    Posts:
    1,837
    Is there not a set of tool to send directx SBS to the Vive or Oculus
     
  3. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    Not that I am aware of, albeit I am still learning the system and very new to it.

    As far as I know, the Vive has two display modes: Direct and Extended.
    Direct is what I am using and requires that you push a texture (OpenGL or DirectX) to each eye.
    Extended makes the Vive act as a secondary monitor which I am assuming you can then use SBS images? I am unsure as I have never gotten the extended mode to work. Switching it with the provided commands always fails, and using SteamVR to switch it works (albeit it would be a hassle if you were to release a game that required this) but I cannot seem to actually get anything to display on the device in this mode. When I am in extended mode I always get a red display an the position matrices stop updating.

    EDIT:
    Looking at some Google results, it looks like I may need to avoid initializing the Compositor in order to use the extended mode. However, if I do this then I don't get any distortion processing and have to handle that and several other things manually.
     
    Last edited: Jul 27, 2016
  4. icuurd12b42

    icuurd12b42 TMC Founder GMC Elder

    Joined:
    Apr 22, 2016
    Posts:
    1,837
    There is a shader on the old forum that does the distortion required
     
  5. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    Giving this topic a little bump and an update.

    I haven't managed to switch the system to use Extended mode yet. I just haven't had time.
    Something I did find out, however, is a little setting in SteamVR called "Enable reprojection." It was on for me. I don't know if that is default or if I switched it and forgot. Either way, that is what was causing the lag on the headset's side. I can now get a solid 90FPS with that setting disabled.

    The buffer issue is still slow, and thus why I will be looking into Extended mode (unless I can find away around using buffers). All this to say, make sure that option is disabled and this actually runs at a decent framerate.

    EDIT: The buffer may not be as slow as I thought initially. Rendering a scene and then doing the same without passing it to the headset doesn't drop all that many frames. With about 500k polygons on the screen I can get about 87fps going to the headset. Without pushing I get about 120fps. That is completely acceptable. I will be working on getting the controllers to work soon.
     
    Last edited: Aug 5, 2016
  6. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    Update.

    I have uploaded a new version of the example and DLL. I have added controller tracking (no buttons yet, just movement) and support for retrieving the player's "play space" in VR. I also moved all the DLL calls to external_* calls because extensions give no errors whatsoever when things go wrong.

    If you try this, please give your specs and a rough frame-rate! Thanks.
     
  7. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    Update again.

    I decided to move to a GitHub repo. This makes it easier for people to modify and it makes it easier for you to get updates quicker.

    I currently added digital button detection so now you can detect when all (but the system) buttons are pressed and/or released. I don't have analog support yet, so no trackpad or precise trigger detection (just "on / off").
    I also added chaperone state detection. This makes it possible to read the states of the base stations. It's great for when SteamVR is starting slow as you can now detect if they are booted yet and whatnot.
    Lastly I implemented the fading functions. You can now fade the scene to a color and/or fade to the VR "Home" environment.

    That's it for now! Hopefully I can get the model stuff, haptic feedback, etc. working soon.
     
  8. trg601

    trg601 Member

    Joined:
    Jun 21, 2016
    Posts:
    144
    Unfortunately I don't have an HTC Vive, but I was just wondering if someone would make something for it with Gamemaker, so I think this is super awesome that you are making this!
     
  9. David Consolo

    David Consolo Member

    Joined:
    Jul 18, 2016
    Posts:
    3
    I am EXTREMELY impressed with this.

    I've been programming with Game Maker for 10 years now, but I had to make the switch over to Unity since I bought the Vive a few weeks ago and was hoping to develop for it.

    It was a hard decision, since I practically grew up with Game Maker and I never wanted to let it go. I've invested too much time into learning it.

    What you've done here has given me a glimmer hope, to the point of me crying when I put on the headset and ran your demo FLAWLESSLY, because I'll take any excuse to go back to developing with Game Maker. Thank you for working on this, and I hope to see more of it in the future.

    I will post my computer specs and fps data tommorow night.
     
    Binsk likes this.
  10. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    Thanks David for sharing! I'm glad I could make your day and look forward to seeing how it ran for you!

    There really isn't much left for me to do with this system, honestly. I just need to get haptic feedback and the analog inputs, really. The model loading is implemented (not pushed to the repo yet) but I think there is an error with OpenVR as grabbing the model name (required for everything else) is causing segfaults. Can't seem to figure out how to get around it so that feature may be dropped, but everything else should be a go once I get more time to work on it again!
     
  11. David Consolo

    David Consolo Member

    Joined:
    Jul 18, 2016
    Posts:
    3
    Sounds great!! Yeah, you are very close to a great proof of concept, and I can't wait for those features. I can live without the default model haha.

    So the fps is below 90, but it seems to run smooth in headset, perhaps I'll add an environment to make the lag more noticeable if there is any, and a simple archery mini game. I'm extremely busy with work, and any time I have free I dedicate to making my own game, but hopefully things will free up. Having a real world demo game could potentially reveal hidden problems.

    Here are some fps numbers.
    "Direct Mode" Enabled AND "Always-On Reporojection" Disabled: 44-45fps
    "Direct Mode" Enabled AND "Always-On Reporojection" Enabled: 68-70fps
    "Direct Mode" Disabled "Always-On Reporojection" Disabled: 44-45fps
    "Direct Mode" Disabled AND "Always-On Reporojection" Enabled: 49-52fps

    I get the best visual setting "Direct Mode" Enabled AND "Always-On Reporojection" Disabled.
    I get weird ghosting with "Direct Mode" Enabled AND "Always-On Reporojection" Enabled.

    My pc:
    i7-4790k
    Geforce GTX 1060 6GB
    32GB of RAM
    Windows 10 Home

    Let me know of any other settings to try.
     
  12. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    Oh my, you were only able to top out at 70FPS? That is surprising. I get a solid 90FPS with a i5-6600K and GTX 970.

    Hm... Yes, Direct mode and "Always-On Reprojection" disabled should be the fastest setup. I am unsure why you would be getting ghosting, but as most people should have re-projection disabled I'm not too worried at the moment.

    You are right with the real working demo. I'd love to try and design a mini-game of sorts to get more in-depth with the system. Say, if you are still willing to experiment, would you mind trying a quick profiling with the debugger while in "Direct Mode" with "Always-On Reprojection" disabled? I expect to see most of it going to the buffer copying, but I also want to see if pushing the texture to the HMD is causing a slowdown for you.
     
  13. David Consolo

    David Consolo Member

    Joined:
    Jul 18, 2016
    Posts:
    3
  14. Miles3298

    Miles3298 Member

    Joined:
    Oct 22, 2016
    Posts:
    3
    I hope this bump isn't too terrible a one, since Vive doesn't seem to be a super popular subject for GM. But I can say I have tried out your demo and have gotten similar results to David's. Smooth as smooth can possibly be in the headset with reprojection on but low FPS (caps at 45, as you documented yourself); juddery as heck with high framerate but reprojection off (hovers in the 87-90 range and looks smooth on monitor but feels absurdly low in-headset) . My specs are in line with yours - same processor, but slightly better GPU (1070). No project changes of any kind, I just built and ran it a few times.

    I gotta say, this is pretty awesome where it is. Thanks a ton for all the work you've done on it and shared with us. Have you by chance had any more time to fiddle around any more with this project?

    I am in the same boat as this fine fellow, and I think I've spent the same number of years with it. I hope your Unity learning is going smoother than mine! I wish you good luck with it.
     
  15. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    Hey Miles, nah the bump is welcome.

    It turns out I didn't fully understand reprojection earlier on in this topic. Reprojection will cut the framerate in half (45fps) whenever it thinks the game can't maintain a solid 90fps. It then uses the same frame for two updates (as the headset still refreshes at 90fps) while stretching / tweaking the second use to try and compensate for head movement. This results in decreased nausea when the framerate can't hit 90, as it gives a smooth experience and removes the jitter that you were seeing.

    As such, reprojection should be enabled, but forced reprojection disabled. This will give you a playable VR experience, but at the cost of a lower framerate and less accuracy.

    I still think that having to transfer the surface into a buffer every frame is what is making the solid framerate unattainable. As stated above, unless that can get sorted then we are at a bottleneck with the system. I haven't played with it much more since then, as I think the solution is finding out how to use GameMaker's texture pointers in a DLL. I will also assume that this will require DirectX (because GameMaker uses DirectX 9 on the Windows platform), and my current system is using OpenGL (which is the one I am familiar with). So, it would require me learning DirectX and re-writing all the graphical parts.

    If anyone knows how to use the pointers passed from *_get_texture functions in a DLL, it would go a long way to helping me get the next update out.

    All that said, thanks for the support. I'm glad you like it so far. I just haven't had the interest because of the surface issue (as it is a PAIN to debug / experiment with DLLs that interface with GameMaker).
     
  16. azb

    azb Member

    Joined:
    Dec 19, 2016
    Posts:
    2
    This is really cool!
    I've been trying to make some games with it but i'm stuck on something kindof basic...
    Suppose I wanted to get the rotation angles rx, ry, and rz of the hand controllers, how would I do this?
    I saw how you used matrix_set when setting the transformations for the controller, and I've been trying to get the rotation angles from
    that array that it uses, but whenever I try to do this the rotation of the controller ends up being incorrect, so i'm not sure really what i'm doing wrong.
    Is there an easy way to do this?
    Thanks!
     

    Attached Files:

    • img.png
      img.png
      File size:
      43.6 KB
      Views:
      11
  17. DennisGMC

    DennisGMC Member

    Joined:
    Jun 21, 2016
    Posts:
    42
    @Binsk
    Hey this is a great i use it in my 3d game!
    Here is my game that will use vr however i haven't posted screenshots with the vive yet since i just started working on it.
    https://forum.yoyogames.com/index.php?threads/coregrid-sandbox-building-2d-and-3d.16073/
    But will you write a script to get the analog detection for the trackpad and trigger, you have it in the op but it's been 6 months since you last pushed an update.
    So i'm just wondering are you still working on the project?
    Also how can i teleport the player to different coordinates in the map, also can i add 90 rotation to the z axis before drawing the scene?
    Is there some way to scale the vr up. When im standing in my game everything is absolutly HUGE! like 200 times bigger than it is supposed to be.
     
  18. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    Hello Dennis,

    Glad to hear someone is finding it useful. I am not currently working on this project. Read my previous post for a brief explanation. I haven't discarded the project, but it just isn't something I care to (or have the time to) work on at this point in time. If you feel like you could add to it, however, feel free to take a look at the source. That is why I am using git.

    To do any of the things you asked above (excluding the analog input, which won't happen soon unless you are willing to add it in yourself) you are going to have to mess around with the matrices. If I recall correctly, the system returns 3 matrices. The first is essentially the "head" position and orientation while the second and third are the relative locations of each eye to the head. These get multiplied together, but you could modify these directly beforehand in order to scale the world and translate the player's position in the world. Note, these matrices are relative to the play-space coordinates.

    I provide some matrix op scripts so you should be able to figure it out.
     
    DennisGMC likes this.
  19. DennisGMC

    DennisGMC Member

    Joined:
    Jun 21, 2016
    Posts:
    42
    @Binsk

    Alright i gave it a try and changed some things. But how can i compile this to a .dll? I edited the main.h file.
    How can i change this into a .dll and which other files need to be included and which .dll file should i replace it with?
    If it fixed anything ill post the code here.

    EDIT:

    Also the surface that is drawn is inverted on the x axis if you know what i mean. Is there any way to reverted the x axis?
     
    Last edited: Jan 24, 2017
  20. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    I use Code::Blocks (for my IDE) and MinGW (A windows port of GCC for my compiling) to get around Visual Studio and its compiler. As such, that is what the project file is. If you have those two set up then you can open the project file and it should have all the linking and such set up with relative paths so it should compile, in theory (as we know, this kind of thing is never straight-forward).

    If you don't use Code::Blocks or MinGW then you will need to import the c++ files and headers into whatever IDE you use and link the packages (OpenVR, GLEW, SDL) to your compiler. If you aren't using MinGW as your compiler then you should also change the include in main.cpp from #include "openvr_mingw.h" to #include "openvr.h" (assuming your paths are all correct), albeit it may still run just fine using the compatibility version.

    In regards to the inverted surface, I am not sure what you mean? Are you talking about in the headset? On the screen (the debugging view)?
     
    DennisGMC likes this.
  21. DennisGMC

    DennisGMC Member

    Joined:
    Jun 21, 2016
    Posts:
    42
    @Binsk
    Thanks for explaining i will try to figure it out using you explanation.

    About the inverted surface i actually ment flipped surface like this.
    The left image is how to would look on a computer screen without a vive.
    And the right image is how it would look like on a vive. Basically the whole game gets flipped horizontally on a vive.
     
    Last edited: Aug 16, 2018
  22. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    Does this happen for you with the demo project? The demo seemed to be working fine for me, so are you sure it is not an issue with your code?

    If you can't figure anything else out you can try modifying the vr_surface_set_* functions before applying your render. If you aren't using those functions and are setting the surfaces yourself then take a look at those functions to make sure you render to them correctly.

    If I recall, getting the orientation right on the Vive was odd and a bit wonky, but it should have been working with the example.
     
  23. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    Long time no update.

    I have rewritten this entire thing from scratch. It is significantly more organized, it uses newer versions of the libraries, a lot of wonky aspects of the system have been removed and there are more features!
    I think the biggest overhaul is how I handle detecting controller input. It is completely different and significantly easier to use. Not only that but I now have support added for every single button including grabbing the analog values for the trigger and touchpad.

    Another big difference is that I removed the convenience scripts that set up the matrices for you. They were buggy and required some workarounds that slowed things down (such as rendering extra times to a surface). Now you must manually multiply the HMD and EYE matrices manually and set it and the projection matrix manually before rendering. It is demonstrated in the example project and isn't that hard to do.

    Anyway, if you are interested please leave feedback!
     
  24. dadio

    dadio Potato Overlord Forum Staff Moderator

    Joined:
    Apr 15, 2016
    Posts:
    217
    Sounds very cool Binsk.
    Would love to try this out, but dont have a Vive.
    Keep up the great VR support work - Im very happy to see this!
     
  25. Miles3298

    Miles3298 Member

    Joined:
    Oct 22, 2016
    Posts:
    3
    Cool that there's been an update. Seems I'm having some trouble getting this to run, though. I stuck all the DLLs in the included files, and when I run the demo, I get this:

    Error defining an external function.
    at gml_Script_vr_system_init (line 76) - global.__gmvr_external[GMVR_DLL.system_hmdexists] = external_define(argument0, "gm_getHmdExists", dll_cdecl, ty_real, 0);
    ############################################################################################
    --------------------------------------------------------------------------------------------
    stack frame is
    gml_Script_vr_system_init (line 76)
    called from - gml_Object_demoObj_StartGameEvent_1 (line 2) - vr_system_init("GMVive Wrapper.dll");

    LoadLibraryA failed with error code 126 (And this error is printed again for 16 more lines.)

    Did I miss a step in setting this up? At the very least, "GMVive Wrapper.dll" exists as far as GM itself is concerned, otherwise it'd throw the custom "can't find this file" error you gave it. My apologies if I missed something obvious.

    I wonder if it's possible that the new "Steam Home" update might have screwed it up, because it broke Advanced OpenVR settings too. But that's just an assumption.
     
  26. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    Hm... Just to double check; did you have all these files added to the project as includes?
    • GMVive Wrapper.dll (which it seems you did)
    • SDL2.dll
    • openvr_api.dll
    • glew32.dll
    For safety sake, are they also the ones from this repo or did you download them elsewhere? I can't guarantee other versions will work. Also, they should all keep their names (minus the wrapper, as you can change that to whatever). I know if any one is missing then it will fail to define the external functions.

    I just tried the system with the new Steam Home update and it still worked fine. Are you using the compiler or the runner? I have yet to try it under the compiler because I recently re-installed my system and haven't set up VS yet.

    Those are the questions I can think of at the moment. Let me know and I'll see if I missed something while compiling the DLL in the meantime.

    EDIT: Just to clarify; adding the DLLs to the project should be all that is required. It should run fine after that.
     
    Last edited: Jun 11, 2017
  27. Miles3298

    Miles3298 Member

    Joined:
    Oct 22, 2016
    Posts:
    3
    Very odd, then. I did indeed add those as includes:
    [​IMG]

    And they indeed came straight from the repo, from the lib32 folders I think.

    NOTE: I added a show_debug_message at the beginning to indicate that GM itself knows that the first file exists. So what was Line 76 got bumped up to 77.

     
  28. icuurd12b42

    icuurd12b42 TMC Founder GMC Elder

    Joined:
    Apr 22, 2016
    Posts:
    1,837
    error 126 mean missing external module. it can be caused by any dll or driver missing from your system. It can be triggered by any of the dll listed here trying to load and getting stomped because what they need to run is missing
    I suggest you install every ms redist package from microsoft. and see if there are support files missing from your vive driver like if you were presented with a choice of a full install vs a fast install
     
  29. George Hennen

    George Hennen Member

    Joined:
    Dec 11, 2018
    Posts:
    22
    what do you mean.... like a 2d game from gamemaker in a vive? i am so lost and intrigued.
     
  30. Binsk

    Binsk Member

    Joined:
    Jun 22, 2016
    Posts:
    582
    Just as a precursor to this response; know that this code is extremely dated, made for GMS 1.4, and uses a very old version of OpenVR that has changed significantly. It is highly unlikely to work anymore.

    GameMaker is fully capable of making 3D games. It has the tools just none of the heavy-lifting has been done as in other 3D-focused engines. As many people like to make 3D projects with GM this project was aimed at these people. I suppose you could translate a 2D game into VR a number of ways but the conversion into 3D space would be on you.

    This project merely retrieved data from the hardware for you and pushed two GameMaker surfaces to the displays. What was pushed to the displays was on the developer.

    Rewriting this with my more recent knowledge is something I really want to do, especially with the numerous very nice looking 3D engines in the WIP forum now. However, it is low priority on a long list of projects I have to work on.
     
  31. BlueHarrier

    BlueHarrier Member

    Joined:
    Mar 31, 2017
    Posts:
    25
    I've read that this is not the first time this error appears, is there any way to fix it?

    Code:
    ___________________________________________
    ############################################################################################
    ERROR in
    action number 1
    of Other Event: Game Start
    for object demoObj:
    
    Error defining an external function.
     at gml_Script_vr_system_init (line 76) - global.__gmvr_external[GMVR_DLL.system_hmdexists] = external_define(argument0, "gm_getHmdExists", dll_cdecl, ty_real, 0);
    ############################################################################################
    --------------------------------------------------------------------------------------------
    stack frame is
    gml_Script_vr_system_init (line 76)
    called from - gml_Object_demoObj_StartGameEvent_1 (line 2) - vr_system_init("GMVive Wrapper.dll");
    
    I'm using Oculus Rift headset and Touches with 2 sensors, I've oppened Steam VR and the Oculus application, my system is fully VR-Ready (I7-7700K, 16GB and GTX 970).
     

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice