Dynamic Blocks - experimenting

B

Braffolk

Guest
What is this you ask? Well, it's a soon-to-be-game called 'Tuvo' that has been more or less in development since 2013. When it's released in 2045, it will be a procedurally generated dangerous roguelikelikelike combined with more peaceful dwellings in places that you've never heard of before - probably because they come from the Estonian and other Finno-Ugric mythologies with a slight mix of custom lore and worldbuilding.

In this thread I'll post both technical and gameplay related updates - most so far are of my adventures in trying to break what's technically possible with Game Maker, but hopefully soon enough you'll see more visual and gameplay related update posts.





Updates
May 21 - Gifs of the dynamic engine
June 2 - Technical troubles with networking, cameras and liquids in a dynamic world
July 25 - More on modular networking and camera update optimisations
July 27 - Procedural world tree & sneaky-peeky at networked terrain on the dynamic engine

Plans
writing...

Old demos:
Version 4
Version 3a
Version 2b
Version 6

Controls:
Middle Mouse Drag - Move Camera
K,L,left & right mouse button - place blocks
Scrollwheel - zoom

If you test, I'd be really grateful if you also post the average fps and any rendering bugs that you may find (plus your gpu model if you do find some).

Twitter for daily* updates: https://twitter.com/braffolk
* Whenever is the day that the game is being worked on.

Stealing suggestions.
 
Last edited by a moderator:
B

Braffolk

Guest
Well, here's a small update.

I managed to get block variations in. This means that one block has multiple images among which its image is picked. All of this is done in the terrain rendering shader, it uses really basic noise to randomly choose one of the variations for every block.


The variation noise:


I also added a second layer, on my PC it didn't have any major performance impact so it should be fine.


And of course, a new demo.


If you have any suggestions or find any bugs please let me know :)


@Zek & @lolslayer
It can never be too fast honestly. Once I start adding things like lighting, it needs to be as fast as possible.
 

Tirous

Member
Cool engine, not without its flaws however! ;D


BEHOLD!!!



also...


^ zooming only, just thought to say ;D

Best of luck on your engine mate, wish i could make one myself.
Maybe after my current project... maybe... :)
 
  • Like
Reactions: Zek
B

Braffolk

Guest
Cool engine, not without its flaws however! ;D

<snip>
^ zooming only, just thought to say ;D

Best of luck on your engine mate, wish i could make one myself.
Maybe after my current project... maybe... :)
Yeah, I've noticed that myself too. Not a clue yet how that is even possible but that'll be something that I'll be fixing later today.

On a random note, I managed to somehow break rendering for AMD GPUs in the newest version, even though I don't remember doing much with rendering... Still works fine on nvidia gpus.

If you've got an AMD GPU, it'd be great if you could test this https://www.dropbox.com/s/83gpvqxcqincq3j/Chunking2D_5_broken_on_AMD.exe?dl=0
 
  • Like
Reactions: Zek
B

Braffolk

Guest
Well the AMD bug is finally fixed, huge thanks to @Opticrow for being okay with all those .exes...

I've also started to play around with lighting and added some form of ambient occlusion.
Current state:

The colours are colder in darker areas and warmer in lighter ones, like this:


Ambient occlusion:

My ambient occlusion uses a bit of colour data from its neighbouring blocks for a bit of colour diversity.
 
B

Braffolk

Guest
So I wasn't really satisfied with the lighting I made last time and redid it.

I wanted the lighting to be as flexible as possible so that it would allow me to make the lighting feel more than just a simple gradient between dark and light, or whatever other gradients you may use.



Going to get a bit more technical here, so the basic idea was that I'd like to have a combination of image based lighting and 2D dynamic shadows. Plus the shadows should not be sharp, instead they should be smooth and reveal quite a bit of the terrain.

So the first step was to generate an obstacle surface from the terrain, I wrote a simple shader for this that is kind of the opposite to an outline. There are 2 extra steps to make this surface as nice as possible, first is to make it look a bit more random and the second is to smooth it out properly for smooth shadows.
Obstacle surface steps:


The second step is to fill another surface with light images. This is fairly simple and straight forward, I draw the lights on the surface with additive blending and it's finished for the final step.


The third and the final step is the most complex one. This is where all the shadow magic happens. All the data is passed into the shadow shader where ray marching and a few other neat tricks are used to create the shadows.
Shadows, with and without the blur that was applied in the first step

One of the really good things about this is that if I don't want smooth lighting it's extremely easy to make it non-smooth! I can simply use different light sprites like this.




There still is a small bug (probably visible on one of the gifs). It should be very easy to fix but for some reason all the fixes so far have introduced new bugs.

This should be fixed in no time though.



On a side note, this is my engine project for my main game that can be found here. http://gamejolt.com/games/game/16352
 
R

renex

Guest
This is so good, it makes me feel incompetent.

You could possibly sell some of these components on the marketplace as well.
 
F

FROGANUS

Guest
wow impressive work so far! lots of food for thought on ambient shading and lighting. dbox link is down for now but ill stay tuned.
 
T

ThunkGames

Guest
Oh yah, this.

I tested this a little while back but forgot to leave any feedback.

I'm glad someone took the time to do this right, and I eagerly wait to purchase this asset.
 
B

Braffolk

Guest
Coloured squares = chunks, red rectangle = camera



Well, now I can do dynamic large scale worlds (the regions on image are 2240x1600, 1760x1088 and 320x320). The scale will come out more once I give actual shape to the blocks, not just colours. Spaceships, worlds colliding and windmills are the future. Once I get down to actually optimising it, I'll be able to handle regions 10x the current scale on an average PC.
 

Morendral

Member
This is very impressive so far!

By spaceships though, do you mean "I dropped a bunch of squares here and it is a static piece that just happens to be in the shape of a spaceship" or do you mean something that actually moves around?
 
B

Braffolk

Guest
This is very impressive so far!

By spaceships though, do you mean "I dropped a bunch of squares here and it is a static piece that just happens to be in the shape of a spaceship" or do you mean something that actually moves around?
It won't be static, everything will be animated and moving. I've built the world so that I could take a bunch of block regions and create an animated entity out of them which can move and interact with its surroundings. The engine is flexible and allows me to create and modify dynamic world hierarchies on the run. (I'm not going to have actual spaceships but rather just flying ships.)

For example I could create a windmill by having two zones on top of each other and one rotating, though in game these regions would be much smaller.


Or a flying ship:
 
Last edited by a moderator:
B

Braffolk

Guest
I'll start actively working on the project after the exam period, which will last until the 15th of June. This is a more technical post about what's going on behind the scenes. I'll soon post more visual things too.

Networking
But now to what I've actually been doing. I'm restructuring the core engine to work with networking. Instead of going for an implementation later on, I'm attempting to avoid inconsistencies in the networking implementation by designing everything around a more universal networking object system that allows me to efficiently modularise different objects that either would contain too much data to send in one go, shouldn't actually have all of their data be sent because the client doesn't need it or in some cases would be sent in order of detail, e.g. region boundaries would be sent instead of the region block data to a joining player (regions can be seen in above gifs as those rectangles that can rotate and be relative to each other). Same with player data. One player doesn't have to know all details about another player before they actually meet.

As seen in the gifs above, many of the moving chunk regions are relative to each other. They are in a hierarchy that is processed and rendered starting from the parent (the world object) and then looping recursively through all of its children until all the regions in the game are processed and if needed, rendered. So this creates another potential problem - what if a networking object is received before its parent is known about.

All networking objects have a UUID that is unique to them and is used to sync states between objects in the server and its clients. So back to the issue at hand, I'm implementing a networking queue system that creates subqueues that wait for objects' parents to be received. In case we receive an object with a UUID that one of our queues is waiting for, the queue will be run after we've processed the parent. So what if in one queue there was an object waiting for its parent, but in another queue, there are objects waiting for the object in our first queue? Then the second queue will be executed after the first. Even if we receive an object and put it in a queue, some processing can still be done on it so that we wouldn't overload things later on. I'm playing around with two potential implementations, one is hierarchical and the other simply puts all queues in a UUID lookup map. I'm still doing performance tests to find out which one will be more efficient in potential game scenarios.

This queue system isn't just to solve the potential hierarchy issue, rather it also allows me to create large scenarios, entities and bosses that would otherwise create a large lag spike in both processing and networking if sent at once. A potential scenario would be that a group of players are moving towards a lair in which we have a boss that consists of 15 interlinked objects and some of those objects consist of blocks. This queue system allows me to first generate the boss on the server side in a few frames and then also send the pieces separately depending on what is required. So once all the clients have received the boss data, only a small packet has to be sent that creates the whole boss. The same applies to interlinked regions.

Putting it all back together
I've already developed a rendering pipeline for all the blocks and block objects as seen in the first demo. This is very simple to add to the core engine with little to no hassle. The three issues that may arise are related to lighting, liquids and cameras. I'm not yet sure whether I should handle lighting per-region or create one darkness surface from all visible regions that will be used for dynamic lighting, I'll have to play around with that.

Cameras

Cameras/views usually aren't an issue, but due to the blocky nature of the engine and the fact that all of these blocks, regions and chunks can actually be rotated, the issue gets quite a bit more complex.

So first a bit of technical information about the regions themselves. Each region has a link (parent) object. Link objects define their physical boundaries and handle everything that has to do with the exterior of the region, the region itself only deals with what is inside the region. I've written an extension which combines OBBs (Oriented Bounding Boxes) and Game Makers matrices so that I wouldn't have to waste performance on dealing with links separately in processing and rendering. This extension also allows me to easily project one OBBs edge coordinates into the local coordinates of another OBB, meaning that I can easily find the local area in chunk coordinates that is in view of our camera. Other than that it also allows me to do very optimised OBB in OBB collision checks that are also optimised with simple AABB checks.

Each region is also divided into zones. Zones deal with entities, liquids, saving, loading chunks etc.

Now to the issue. It's simple to find the chunks that should be loaded. Now what I don't want to do is to simply loop through cameras rectangular region in a region and check if the chunks in it are loaded. That's horrible performance wise when the camera deals with hundreds of chunks. What I do want to do is to first calculate the rectangular areas that previously were inside the camera, but now should be unloaded. Then I also need to calculate the areas that just appeared in the camera view.

So to start off, after loading/unloading chunks I have to store the previous state of our camera so that next time the cameras are processed, the difference can be used to find how much and where the camera moved. Quite a simple operation thanks to my OBB extension. But it gets more complex because the regions too can move and rotate. This means that I also have to cache the previous state of each regions OBB.

So each time a camera is processed, I first map the previous state of a camera on each visible regions previous state and do the same with the current state. Then once I have found the difference, I can finally load and unload what is necessary and nothing more.

I'm still working on optimising the whole process, but it's getting where I want it to be.

Liquids
When it comes to liquids, there are two approaches that could be taken. The first would handle liquids per-zone. So that in a zone liquids would always move on the grid and towards the bottom of that zone. In the case of region overlap, this could create serious inconsistencies in case they have a difference in rotation. The second approach that I'm considering involves separating liquids from block grid and making them always fall towards the bottom of the world. While this would visually be the better solution, it's technically very hard to implement as liquids would still have to collide with blocks and even handle water pressure without causing a serious performance impact. One implementation that I tested a while ago was to handle water as points rather than blocks, this would still work with the general blocky style as water overlap would be hidden behind the front layer of blocks.
 
Last edited by a moderator:

amusudan

Lousiest of Potatoes
This is absolutely amazing, great work and really looking forward to seeing more progress.
 
B

Braffolk

Guest
It's been too long since I posted the last update, so here's another one. I'm not going into everything I have done, but I'll just give a few interesting notes that I regard as important.

Networking

I've been working on finishing my networking core that will run behind everything. It's finally finished, though it may still get some polish in the future.The networking engine is modularised into different modules. Modules aren't technically objects, but they can be attached to objects as an extra layer of more universal code and functionality that will be synced across specified clients.

Modules can easily be attached to any object by calling netinst_attach( module_id ). Now to actually run these extra layers of code I call netinst_perform( event ). This simply loops through all events of modules that were added to the instance and runs their specifeid scripts, no extra checks or data handling is done here in order not to lose any performance. Modules don't have to be attached to instances though. Some modules are only used for sending/reading data and never get attached to any instances.

Because of the modular nature of the engine I've also written packets that fit this modular approach. Rather than having packet IDs and packet specific code, packets are instead divided into types: raw (doesn't have to know the client), general(needs to know the client), netinst create, destroy and update (executes the networking read code in the netinstance). A packet can have any amount of these types and under each there can be any amount of module updates, which can simply be called by doing packet_add( module_id ) in the appropriate instance.

Edit: I forgot! Clients and servers can both create and simulate netinstances, this makes sure that you don't have to wait for a silly old server to move your player. You can do it by yourself and the server just makes sure that what you did is a legal move. Cameras do this for example.

Here's a little example of it all coming together (the lag in the gif is due to hard-coded temporary chunk drawing and initiation, ignore it):


Camera optimisation
As seen in the gif above, cameras can be pretty dynamic. They can change size, angle and take pretty much any shape that is a rectangle. When it comes to interactions between cameras and the world I've always tried to make things as fast as possible, especially because in case we have multiple players connected, things can get quite slow in case the server has to constantly keep checking the updates on 8 or more cameras.

So the idea was to make sure that no processing speed is wasted on looping through large areas to make sure that there are chunks or zones that need creating or deleting. That's going to have some serious performance impacts in case your cameras cover tens of thousands of chunks.

First off, each linked region stores the last local state of the camera. The camera's rotated OBB shape will be mapped onto the region. It's converted into a rectangle in chunk coordinates which covers all visible chunks inside the camera. Then on each region or zone update in case the camera state has been modified, a simple algorithm finds all the rectangular areas that either a) need to be removed since they are no longer in the camera view b) need to be added because they have appeared in the camera view.

The algorithm had to meet a few rules in order to produce an efficient solution:
1) The areas to be removed must only be inside the camera's last position and nowhere else.
2) Removal areas must not overlap the current state of the camera in any way
3) The areas to be added must only be inside the view, but also have no overlap with the previous state of the camera

A gif of the algorithm:


Going forward
The core engine is coming together quite neatly. While there is little visual progress, the technical side of the engine is close to being finished. The tiling shader, lighting and many other necessary elements of the engine have been finished in earlier iterations of the engine, so it all comes down to simply putting it all back together.

The next step is cleanup and optimisation. After that I'm planning on working on worldgen. I feel that while there is some quite decent worldgen out there (especially when it comes to 3D), no one has really used the maximum potential of 2D. Most of the worldgen in 2D sandboxy games use a rather basic set of elements that are each on their own and rarely build on top of each other. The schema tends to divide into 2 types of generation. 1) Noise based generation: terrain, caves, plant positions, structure positions etc. 2) Structure generation: dungeons, buildings, rooms, mines etc.

I'm attempting a different approach, where rather than defining a set of generation features, I'm going for a more recursive approach - one that builds features on top of other features, but more on that later ;)
 
Last edited by a moderator:

GMWolf

aka fel666
Oh wow, that's is very, very cool indeed!
Being able to build things that can actually move, rotate, etc. That's very cool!
 

YanBG

Member
The old demo links are dead. I'm working on chunks and am interested how to reduce the loading time. Each time you step into a new chunk i make a lot of checks to generate a detailed scene(houses etc).
 
B

Braffolk

Guest
The old demo links are dead. I'm working on chunks and am interested how to reduce the loading time. Each time you step into a new chunk i make a lot of checks to generate a detailed scene(houses etc).
Hey, sorry about the broken links. Should be fixed now. I also included the previously not seen version 6 of the engine, but it may have more bugs than other versions since it has experimental lighting.

When it comes to loading chunks it's always better to do as few check as possible and just load the data. In case you have a lot of data to load, split it up. Render some more complex chunks in multiple steps, don't load too many chunks in a single step etc.

Always avoid a set of if/switch statements when loading large amounts of data or even looping through it, instead opt for storing scripts into data structures and run those using script_execute.

Update stuff
Since I'm posting here already, I'll throw in a couple images of what I've been messing around with.

Networking now syncs individual blocks across clients and proper rendering is in too.



Another thing I've been doing on the side is tree generation. In the middle of the world there will be a large old oak tree, the world tree, and on each of its branches sits an unique realm.

First came the branch generator. Despite their looks, they are definitely branches.


Then I threw a bunch of these branches together. While it didn't look too bad and kind of worked, it was still not up to the standards that I required. It had a random chance to generate a branch in case there was enough distance between the new branch and an old one or in case there hasn't been one in awhile, it'll generate one despite the random chance (the numbers on rectangles display the number of branches created).


So the question was how to generate branches in a way that they would grow naturally and also be placed in a way that doesn't scratch the eye too much. The answer was to use a force field based on nodes (so it's not really a field technically, but I'm rendering it as so). I can input any coordinate around the tree and it will neatly weigh in all the nodes based on their size and distance and return a single vector pointing to the direction where there are least obstructions and the length of the vector points to how much care should be given to it. If it's a zero, we can just generate a random branch and not care about the field (though that's almost never the case).

The force field can be seen rendered here, on the next images that include the force field it will just be rendered as points rather than actual vectors to keep things more clear, but it'll still be in there.


The first thing I did after that was to simply make the branch growth follow the force field - avoid other branches by finding the best path around them. Great people on /r/GameMaker Discord also suggested to make branches regrowable and it worked like a charm as demonstrated below. When a branch is deleted, the parent will simply have its generation retriggered.


So finally I made the whole generation follow the force field. In case there are no nodes around, we can feel pretty free to generate a new branch. The branche's angle is also calculated simply by using the same vector force field function. It returns the angle pointing away to the most empty area around a point, so it resembles how branches work in real life - they like to grow towards places where the sun shines.

Now the final problem that I hadn't given enough thought to was making the trees actually prefer growing upwards rather than in the general direction of what they were created in. At the first glance this seemed hard to combine while I already have the force field, but it actually made things way easier. I simply added a small upwards value to the returned vector and voila, the branches point where they are supposed to. It's also interesting to note that there's a slight gradient going on in the force field. The upwards force is stronger at the bottom of the gif and weaker at the top. The reason for that is because the tree grows from bottom-up. The first branches go for the sun and the upper branches are simply forced to follow their lead despite a weaker upwards force - because the bottom branches themselves provide this force for the branches above. Things might look slightly too upwards if this wasn't the case.


So right now the generator gives a pseudo spruce tree. Just needs a bit of decorating and fixing up. Once I feel that the generation is customisable enough, I'll make it look like a proper oak tree.
 
Last edited by a moderator:

YanBG

Member
Thank you! Indeed i was loading(procedural gen) 3x3 chunks with the player in the center and all their tiles but now i added a few more checks to load only the row with 3 chunks infront of the player(the direction of moving). Also i plan to place the tiles outside of for loop and in different step for each line(again according to direction).

First time the chunk needs to be generated but then i can save it to file(or distribute them with the game if i scrap the procedural gen). Do you reccomend txt or json? Right now i just set the unique seed(with x/y) before (re-)generating each chunk, so it always plays the same.

Always avoid a set of if/switch statements when loading large amounts of data or even looping through it, instead opt for storing scripts into data structures and run those using script_execute.
I'll look into that. Is it faster or easier to manage?
 
B

Braffolk

Guest
@ThePC007 Not sure tbh. Depends on whether they would fit the general style, be optimised enough and not require too much time to implement. It's definitely something to consider in the future though.

@YanBG I'd always suggest using buffers for saving/loading things. They're the fastest way to deal with any data that has to be either loaded/saved or networked in Game Maker. If you have Discord you can always add me: Braffolk#1160. I'm willing to help with things that don't take too much time.
 
Top