Fugue Devlog 9: More Natural Environments and Better Player Control

04.25.2021

As I mentioned in the last devlog, I implemented a two viewport system: one viewport for the half-resolution 3d rendering and then a second main (the root) viewport to render the UI and scale up the 3d output. One problem: shadows weren't rendering anymore. This was easy to fix; by default new viewports don't have a shadow atlas, so I just had to change that value from 0 to 1.

I also continued work on the outdoor/natural environment scenes, which brought up some more issues. The forest clearing scene dropped in performance (down to 11-15fps) once more trees were added, which I kind of expected. I reduced the tree model to about 330 vertices (from ~1.2k), which probably helped a little, but was less impactful than I'd hoped. I ended up playing around with the root viewport settings (which confusingly are mixed into Project Settings > General). By default the root viewport is set to render 3d, but now it's only rendering the 2d image it's passed by the half-resolution viewport. So I set the root viewport's render mode to 2d (under Project Settings > General > Intended Usage) and that boosted the scene to around 24-30fps. Still not the greatest, but fine for now.

I also wanted to set up a baked lightmap for ambient occlusion, but there's a bug where environmental lighting isn't taken into account (a fix should be out in 3.3.1).

I also put together a cave/grotto scene, which went quickly:

I realized the cave and forest clearing scenes are actually closer to indoor scenes: there's no skybox or distant environment to render. So I started playing around with a proper outdoor environment, where the sky is visible and there's distant geometry to handle:

A lot of new things to consider here. One, which is mostly specific to this particular scene, is grass and wind effects. I have a lot of grass here...I want to evoke that wind-sweeping-through-the-grass effect that comes up in a lot of Ghibli films. Of course this is a performance hit, with the scene rendering at 10-15fps.

A real-life example

If you look at the effect in Ghibli films, it's actually very simple. The grass is just a static image with trails moving on top:

From The Wind Rises

So I don't really need actual grass geometry for the effect to look good. I can probably put together some kind of shader that moves these trails over the surface of a static texture.

The other thing I'm not sure about is what to render as a skybox and what to render as in-game terrain. I'm using the HTerrain plugin to generate terrain and in the image above all the terrain is in-game. It's really lacking in detail/richness which is mostly because I'm only using one texture. I'm thinking I should probably only render the foreground as in-game geometry and everything else as skybox for performance's sake. Which means I need to figure out a workflow for modeling and exporting skyboxes from Blender. This is something I'll have to think on more, I'm really not sure how best to approach this.

A few other updates:

  • Set up a test character and started giving character animations a try, which I was dreading (I still am, but less so). The whole process was made way easier with Mixamo—this will sound like an ad, but it has an auto-rigger that makes a rigging a character very easy and their library of free motion-capture animations covers a lot of what I need right now. For the character itself I based a mesh off of Kiros from FF8 an am using a texture someone extracted from the game.
    • It's very helpful to have a character for scale. The scale of the indoor test environment, for example, was way too huge.
    • I spent quite a long time tweaking the third-person controls, eventually settling on a system that doesn't use the mouse. Basically, with the default keybindings of WASD: W moves the character forward in whatever direction they're facing, S moves the character backwards (i.e. they literally walk/job backwards), and A/D turns them to their left or right respectively. So far it feels smooth and intuitive.
  • Formalized the "scene" system a bit more. I implemented a node type called Scene which contains information like fixed camera configurations and where the player spawns. This is really simple at the moment, but I imagine it'll be more complicated as I need to switch between cameras in a scene, or track the player for certain segments, and so on.

Fugue Devlog 8: Testing Outdoor Environments and Performance

04.23.2021

I started experimenting with an outdoor environment to start trying out the various environmental effects and elements I looked into last week.

There were a few small victories: the Waterways add-on worked great for still pools by setting both Flow > Distance and Flow > Pressure to 0. I also found an off-the-shelf cloud shader that looks great. With these elements and the ones from last week the outdoor environment looked dreamy:

But the experience raised an important issue that I kind of hoped I would avoid: performance. The outdoor scene was very choppy. I originally had the ~6.6k vertices trees so I tried simplifying them down to about ~1.2k or so. It didn't seem to help much. After reading up on 3d game optimization, the vertex/triangle count was not as important as I thought.

It's possible that some slowdown is due to Godot, which isn't as optimized as more mature 3d game engines (for example, it lacks occlusion culling, which is when objects blocked/hidden by other objects are discarded). It sounds like major 3d improvements have been pushed to the 4.0 release, which may not happen for awhile.

My GPU (or rather my lack of a dedicated one) is no doubt a major bottleneck here, but I want this game to play well on integrated graphics. I'm going with the low-res textures and low-poly models not only for style reasons but also for performance. I thought that would go a long way in terms of performance, but I was completely wrong. They are important, but not nearly as important as light and shadows.

After some more reading and testing of my own:

  • Shadows have a huge effect. Turning off ambient occlusion for the world environment helps a lot and using simpler shadows helps a bit more. Glow doesn't seem to affect FPS that much.
  • Mesh amount and complexity still have an impact. Having a lot (~100) of the 1.2k vertices trees did slow things down a lot, if only because they make light/shadows more complicated.
  • The wind shader, on the other hand, didn't seem to affect performance much compared to a regular spatial shader. Though it does require dynamic shadows to work; otherwise I could probably bake those shadows.
  • The cloud shader also has a pretty substantial impact (10-15fps). Tweaking its settings to do fewer draw passes did help a little. I also tried porting another cloud shader, but that performed even worse and was a hassle to set up.

Throughout this testing I thought this was just an issue with the outdoor scene (with the high number of moving trees, especially), but the indoor test scene also had an FPS of only about 10-11. Turning off ambient occlusion, however, boosted that to around 30fps. I guess I completely underestimated how demanding ambient occlusion is.

Specifically, I'm using Godot's screen-space ambient occlusion (SSAO), which adds a lot of richness to the environment. But it's hard to justify such a massive drop in FPS. Fortunately, the screen shader effect kind of maintains some visual interestingness on its own.

I still wanted to preserve some AO if possible. Baking lightmaps is the usual approach and wasn't too complicated to get set up in Godot. The baked lighting does looks better than having no SSAO, and I get around 30fps consistently—basically same performance as without the baked lightmap. But the baked lightmaps don't look as good as the SSAO. According to the docs SSAO is a different approach that captures details in smaller geometries. But after digging a bit, it seems the main difference is that SSAO has a "Light Affect" parameter that controls the visibility of AO in direct light. Normally it's not visible in direct light, i.e. "Light Affect" is 0, but I set it to 1 to exaggerate corners with deeper shadows. This isn't an option for the AO produced by baking lightmaps.

So I tried another route. The screen shader effect I'm using already simulates half resolution for more pixel texture; it makes sense to just actually render at half resolution and draw fewer pixels in the first place. Below are a few comparisons:

1x resolution, with a screen shader that simulates 0.5x res. Runs at about 20-22 FPS

0.5x resolution, with a screen shader that simulates (an additional) 0.5x res. Runs at about 48-54 FPS

0.5x resolution, without a screen shader further downscaling. Runs at about 54-60 FPS

1x resolution, with a screen shader that simulates 0.5x res and SSAO enabled. Runs at about 15-18 FPS

0.5x resolution, with a screen shader that simulates (an additional) 0.5x res and SSAO enabled. Runs at about 50-60 FPS

0.5x resolution, without a screen shader further downscaling and SSAO enabled. Runs at about 40-44 FPS

The outdoor scene looks good under all settings. But I imagine the double downscaling looks worse as there are more objects on the screen. That's the case with the indoor scene, which looks too noisy with the double downscaling. But otherwise the FPS gains are very promising and give me enough space to switch SSAO back on for the indoor scene!

One issue here is that this downscales the entire viewport, so the text is also downscaled. Text should probably render at the highest resolution possible for readability. So instead I need to render the player camera to a separate downscaled viewport and then render the non-downscaled resolution text (and other UI elements) on top. This was surprisingly complicated to set up; at least, the way Godot's viewports and scene tree work isn't intuitive to me. I struggled to find a way to specify what node loaded scenes attach to so that I could automatically attach scenes to my downscaled viewport node. But that doesn't seem possible—instead I have my player camera as a child of this downscaled viewport node and let scenes attach to the scene tree normally. The "Main" scene that contains all of this is instantiated as a singleton (i.e. using autoload).

This is the route I'll go with for now. The look of the game is pretty much the same, and I can use SSAO for indoor scenes and maintain high framerates. Similarly, the outdoor scene can use the cloud shader and maintain high framerates. That will likely change though as one tree is obviously an unrealistically simple scene. I need to keep experimenting with simpler trees and laying out the scene differently so I don't actually have to have a small forest's worth of trees. My hope is that with this set up the marginal cost of additional meshes will be pretty small. Not small enough for an entire forest, but small enough that I won't have to worry much about performance again.

A related question is the player camera. Right now the player has free reign over it, and the dialogue system was designed with that in mind. However when I imagine the game being played I see a fixed camera setup. The benefit with that is the player is way more constrained in what they see, so I can be more minimal in scene dressing and maintain better performance that way. For example, I'd only need to seat a few trees at the edge of the camera's view rather than anywhere the player might possibly look.

Fixed camera example

This was just a quick test to see how it might work. To really see if this is the right approach I should start figuring out the character design more and flesh out the third-person system in general. So I'll probably tackle that next before prototyping more of the outdoor/natural environments.


Fugue Devlog 7: Environmental Effects and Objects

04.18.2021

A tree

With the dialogue system in decent shape, I'm moving onto exterior environments. There's a lot to figure out here: environmental effects like wind, fire, smoke, rain, and snow; terrain design; rocks; vegetation including grass, bushes, and trees; skyboxes; and water (which I'm categorizing into "still" water, like those in ponds and pools, and "active" water, like that of the sea). In general creating "natural" effects and objects is way harder than human-made interiors and objects!

Several of the effects (rain, snow, fire, smoke) can be implemented as particle systems. I played around with Godot's Particles node and am pretty happy with the results. Here are a couple demos showing falling leaves and petals:

Falling leaves using a particle system

Falling petals using a particle system

The only thing that doesn't seem possible is the leaves rotating along the x/z axes. It's not a huge deal since these are more of mood-setting elements and there'll be plenty else on the screen to focus on.

Many of the remaining effects are usually implemented with shaders: wind in grass and trees (this tree wind shader works pretty well), the grass itself, and water. I don't have much experience with shaders so I spent some time learning more about them and playing around with them. I haven't yet managed to finish anything substantial yet but will keep exploring there.

In terms of skyboxes, those seem not too difficult to implement in Blender: basically you build your skybox and then export it as a panoramic image into Godot.

Terrain, rocks, water, and vegetation are a lot harder. For terrain and water there are fortunately some Godot plugins available that make it much easier. For terrain, there's Zylann's HTerrain plugin for heightmap-based terrains. This works well for outdoor natural environments and it supports some things out of the box, like painting grass textures and basic wind for grass. Unfortunately the heightmap-based approach doesn't work for things like caves and overhangs, so it's not a panacea.

For caves and overhangs there's Zylann's VoxelTerrain plugin for voxel-based terrains. It's not as mature as HTerrain (lacks a few of its key features, like LOD and grass painting) and it requires compilation into Godot (i.e. not just an addon you can drop in), so I'm not sure if this is exactly what I want to use.

Alternatively, I could create the terrains in Blender using the sculpting and texture painting tools there and import them into Godot. Caves aren't too difficult in Blender (you can use a cube and some displace modifiers to create the right rocky walls as a starting point) so maybe this is the way to go.

For rivers there's Arnklit's Waterways. I wonder if I can stretch it to work for other bodies of water. In any case I may need something different for still bodies of water, like those in grottos:

A grotto

For trees, there were two plugins for Blender I looked at: tree-gen and modular-tree. I also looked at hand-making trees based on "How to Create a Low Poly Tree in 1 Minute" (a similar technique, mainly using Blender's particle system for leaves/small branches, came up in a few other videos). The generated tree-gen mesh has almost 1.3 million vertices which is way too many. In comparison the basic tree generated by modular-tree has about 21k vertices1 and the handmade low-poly tree has only 6.6k vertices. That might still be too many—I don't yet have an intuition of what number I should aim for, I just know that fewer vertices are better in my case. The low-poly one also better matches the look of the game so far (low-poly, photo-realistic textures, and dithering, which in this case is just an artifact of Blender).

tree-gen example

modular-tree example

Low poly example based on "How to Create a Low Poly Tree in 1 Minute"

I wrote a Blender addon that generates the base of the tree (trunk and branches, but without the leaves) in a manner similar to that low-poly tutorial. You still need to manually set up the vertices and particle system for the leaves, but I may expand the script to do some of that automatically. It was a struggle to get the script working properly (my linear algebra is really rusty and the Blender Python documentation, at least for the bmesh parts, is really lacking), and it's not nearly as powerful as tree-gen or modular-tree, but it does what I need and in the style suitable for the game.

Here's the tree with this basic wind shader:

Tree with wind

With trees and other vegetation, as well as rocks, you need not only to create the tree meshes, but also distribute them. This post details the use of a particle system to distribute vegetation throughout a terrain, but Zylann has yet another (!) plugin, Scatter3d, that lets you paint scene instances (e.g. meshes) in Godot. For wind, Godot has a tutorial for a wind shader.

For rocks, Blender has an add-on to generate rocks, which as far as I can tell is based around feeding noise into a displacement modifier (this is the basic approach for creating caves in Blender as well). I've also seen rocks generated with a particle shader, which probably makes more sense if you have a desert landscape that needs tons of small rocks scattered about. If I've learned anything while researching how to approach these natural elements, it's that a rock is not a rock, and water is not water, and so on...there are rocks that are big, for up-close use, or medium-sized ones for foreground elements, or tiny ones for a sprawling landscape; water in a calm pool is not the same as water running through a river which is not the same as water that's crashing against a beach.

A tricky thing with natural elements is the right amount of realism. There's of course a lot of effort spent on figuring out ways to most accurately simulate natural environments and effects, and realism usually comes with a big cost to performance. I'm not trying to make natural environments that approach reality but that are interesting and convincing enough. Whenever I come across a video or post detailing how to make a tree or a water shader or what not, I always have to figure out the context. Is these for rendering in a 3d short film? Is this for a triple-A video game where players are assumed to have the latest graphics cards? Is this a demo of how realistic you can get an effect but way too computationally demanding for practical use?

For example, I found this beautiful Breath of the Wild style grass created using a shader. It's a bit resource intensive and, as wonderful as it looks, it's just not the vibe Fugue's going for.

Next I need to pull these pieces together and try building a couple natural environments—probably a forest clearing and a grotto to start.


  1. It was a little tricky to set up modular-tree. One note is to checkout the blender_28 branch if you're using Blender 2.8 or above. The other is that there's a bug when adding a twig node. What worked for me is to first add a twig node and "execute" the node, creating a mesh called tree, then rename the mesh to twig (or whatever else). Then set up the tree nodes: a trunk node into a branch node, then a "tree parameters" node separately. Check create_leafs, then click the eyedropper for the leaf parameter and select your twig mesh in the scene hierarchy. Then click execute tree


Fugue Devlog 6: More Dialogue System!

04.15.2021

A few pain points came up after working with the dialogue system and editor more. The schema of having the text (what's being said) and varying choices and outcomes on the same level didn't make much sense, since the vast majority of dialogue is simple linear exchanges. Having the possibility of choices and outcomes attached to every utterance made the editor graph really unwieldy. This led to a few changes:

  • Because the type of an event can be inferred from who the speaker is, i.e. a "thought" is anything where the speaker is "self", and otherwise it's a "verse", the event types can be dropped.
  • I renamed "events" to "verses", because the term "event" is confusing in the context of dialogue.
  • Verses no longer have one piece of text attached to them but can encompass multiple "lines". Each "line" of a verse has at minimum some text and its speaker, but it can also have a timeouts and/or a delay, or emit a signal to trigger other parts of the game.
    • These are specified by a bit of syntax at the beginning of the line's text. E.g. !foo;t5;d8|This is the text means this line will: emit a signal called "foo", have a 5 second timeout, and a 8 second delay.

These changes meant the dialogue editor could be streamlined:

More streamlined script schema

Another small quality-of-life improvement is a button on each outcome to add a new verse already connected to that outcome. Before I'd have to add the new verse, then drag it next to the outcome so I could connect the cable.

The other update to the dialogue system is a better dialogue box layout system, to minimize overlapping dialogue boxes:

Demo of the dialogue box layout system

This was tricky to figure out. The primary constraint is that you can't reposition dialogue boxes so much that it's unclear who the associated speaker is. The safest axis of movement is along the x-axis, so the very simple approach is to just go from left to right and move boxes either left or right if there are overlaps. Of course there are many situations where this won't help, but I don't think the game will have much more than a few simultaneous speakers at once. The other simplification is that this layout adjustment is applied only to "clear" dialogue boxes; that is those within range that you can clearly "hear" the speakers. Anything out of range will be drawn below these if there's overlap.

I also implemented a custom rich text effect based on this tutorial. It seems like a powerful system:

Custom text effect


Fugue Devlog 5: Dialogue System Implementation

04.13.2021

I'm chipping away at implementing the dialogue system. It's daunting; sometimes it feels like trying to build a house all at once. Once you start to sort out what the foundation is, what part depends on what other part, etc, a build order starts to become clear and the whole implementation becomes more manageable. If you can sort that out on paper and think through most if not all of the possible issues, implementation is really straightforward.

Some of the key features like choice selection, tracking dialogue states (e.g. remembering how many times you've interacted with that speaker), and speaker tracking are demoed below:

Demo of choice selection and tracking dialogue state

I discussed an ambient dialogue system in a previous post and made some headway on implementing it. The off-screen dialogue box handling needs more work (really janky atm), but I have the dialogue box "blurring" (which fuzzes dialogue boxes as you get further from then, to mimic those voices becoming harder to hear) more or less working.

I had to compromise a bit though. I originally envisioned a gaussian blur effect, where at sufficient distances the dialogue boxes were basically smeared into nothingness. But blurring individual UI element is really complicated. It looked like the only approach was to setup separate viewports for each dialogue box, render those to textures, and then apply a blur shader to those textures. Way too complicated.

Here's the current implementation of dialogue "blurring". As the player gets further from the speakers, the dialogue boxes scale down and become transparent. It needs some tweaking, e.g. the transparency change looks awkward as a linear function, maybe should be using squared distance to feel more organic. But I think this works well as a general approach. This also means that distant dialogue takes up less screen space, so there'll be less clutter.

Demo of dialogue "blurring"

Here's the handling of off-screen/ambient dialogue. I ended overcomplicating it, spending too much time trying to implement this unnecessarily complicated version, then found that the simpler approach worked better anyways (just clamping off-screen dialogue boxes to screen space). There was some weirdness that needed fixing—basically, off-screen objects in perspective cameras are positioned counterintuitively, so the y-positioning of the dialogue boxes can look unexpected. The most noticeable case is when off-screen dialogue boxes are positioned at the top the screen for things that are behind the camera. It feels more "natural" to have those at the bottom of the screen; so there's a small snippet that ensures that's the case.

Demo of off-screen/ambient dialogue

The other feature, which is also a bit janky right now, is interrupting ongoing NPC dialogue. Discussions where the player isn't involved, i.e. among a group of only NPCs, advance automatically. If the player enters the vicinity of any of the NPCs involved in that discussion, and the player can speak with at least one of those NPCs, they will pause their discussion and address the player. If the player leaves the vicinity, they resume their discussion from wherever they left off.

There's a lot to tweak here, like timing around the discussion pausing and resuming, checking for race conditions (always a possibility when timers are involved), and figuring out how best to handle the lead-in/lead-out snippets (here: "Do you need something?" when the player interrupts and "What was I saying..." when the discussion resumes).

Demo of interrupting an NPC dialogue

Aside from cleaning up the implementations of these features, I'm mostly finished with this first pass at the dialogue system. Still many things to figure out about how dialogue is best triggered and how it should be associated with entities and so on. No doubt things will need fixing and I'll want to do things the system doesn't support as-is. And the tools will also change; I'm already finding pain points with the dialogue editor that need improving.

The next step is to start reviewing all of my notes for the game's world and story and start putting together a more concrete draft/design document. That'll help me figure out what other key mechanic systems are required.

<< >>