I realized I forgot to build out an important part of the dialogue system: "cut scene" support. These aren't true cut scenes in the sense that they aren't cinematic sequences; here I'm using the term to describe something more like a play: characters moving around, saying things, with audio and animation triggers. This meant expanding the dialogue system to support "stage directions", and that required re-writing a few parts.

The implementation also meant that I had to reckon with NPCs. So far the entities engaging in dialogue were just the player and static objects. Of course it's far more typical that NPCs will be the ones engaging in dialogue. I'd written an Interactable class for things that the player can interact with, but it didn't capture the necessary functionality for NPCs (such as moving around), so I had to re-write that a bit.

Dialogue advancement got more complicated—I wanted to support chains of stage directions (e.g. walking to a place, turning around, walking to another place, etc), so dialogue timeouts and/or manual dialogue advancement don't happen until all of the stage direction sequences are finished.

This all required enough changes that everything felt a bit brittle, so I also spent a bit of time refactoring the code a bit.

Here's a small example of the cutscene system (the moving character is an NPC and the behavior is entirely directed by a dialogue script). The stage directions here are: walk to the designated point, turn to look at the player, then run the "pick up" animation.

Stage directions can be attached to any line in a verse:

Sometimes NPCs should move around etc without saying anything, so I've added a special speaker name of <none> for these cases. The general syntax for a stage direction is npc node name:target node name (excluding directions that don't have an actor, e.g. playing an audio clip). If there are multiple directions for a single actor attached to one verse-line, they're executed in sequence. If there are directions for multiple actors, they're executed in parallel. I'm hoping this doesn't relinquish too much timing control, but who knows at this point.

One problem here is that these are just open-ended text inputs. There's nothing checking that these actor or target nodes actually exist.

I updated the dialogue editor script validation to now require an associated scene, against which all references of speakers/actors and target nodes are checked against:

That should now cover all the systems that'll be necessary no matter how the world, narrative, and other game mechanics shape up. Before moving forward with the rest of the development I need to sort those out to figure out what exactly I'll need.

Scene portals

One of the last two scene components were "portals", i.e. the connections to other scenes. I didn't give this a whole lot of thought; I just used Areas that have an associated scene that the player's sent to. It works fine, though there were a couple small snags at first.

The first was infinite scene looping: when the player leaves out of one portal and enters the new scene, they spawn in the portal of the other scene. That immediately sends them back to the previous scene, where they spawn in the portal, thus sending back to the other scene, and so on. I just needed a bit of code that starts with portals deactivated until the player leaves the entrance portal area.

The second was load times. There's some hanging after entering a portal while the new scene loads. I added a very simple preloading system, where a scene preloads any linked scenes. A cache of recent scenes is kept (right now the last ten, but could probably be higher than that) to reduce loading further. I imagine a more sophisticated preloading system might be ideal later, such as one that preloads not only the immediately linked scenes but those that are two jumps out.

Scene boundaries

The other last scene component was scene boundaries, i.e. preventing players from walking out of the camera's view or falling off the map. I don't know what the best practice around this is, but I just used static bodies and collision polygons; basically invisible walls. Godot's collision polygons were pretty frustrating to use at first, but eventually I got the hang of it.

Grass and wind shader

I took a shot at creating my own grass wind shader to emulate the Ghibli effect I talked about last time and didn't produce anything worthwhile. Shaders are still really confusing for me! I ended up following this grass shader tutorial for Godot which gave me a bit more insight into how to implement a shader for a particular effect, but still way above my knowledge level. In any case, it looks amazing and I gave it a try in the outdoor environment. The performance lags a bit (20-24fps) but it's not nearly as bad as I thought it'd be.

What I need to think on more is whether or not the effect needs to be reproduced in the background, and if so, how best to do it. I might be able to layout levels so that I don't even need to do that. For example here I shaped the terrain so that it looks like you're at the top of a hill, which is why you don't see any more grass in the closest part of the background. This is just a trick so I don't have to render more grass than absolutely necessary (I'm realizing a huge benefit of the fixed cameras is that you can do many of these kinds of tricks).

Throughout this whole process I'm learning a lot about how challenging emulating light is, especially in video games where you are far more resource-constrained (i.e. you need to hit a certain number of frames per second). I've kind of known that light/shadow is where a lot of the focus is in better video game graphics (e.g. raytracing) but didn't really appreciate it until now. I thought that by sacrificing mesh complexity and detail I'd have more room for light/shadow effects, but I had it backwards: the light/shadow effects were always the primary bottleneck. This is all to say that one problem with the grass/wind shader above is that shadows aren't cast onto the grass! But I think I have to let that slide. I did have a sketch of a very simple cloud shadow shader for grass; maybe I'll try to combine it with this one in the future.

Background environments and skyboxes

I'm still working on figuring out my general approach to these outdoor environments. I thought I should design the foreground to be more like an "indoor" environment with elements like the grass shader and then just have a pre-rendered skybox as the background. I'd create all the terrains in Blender, then just export the environment render and load it in Godot as a skybox.

I was not very successful at this. I tried creating a terrain of karsts, which itself was fun to do in Blender. The shader system there is powerful and you can generate a variety of environments with noise textures (probably much trickier to do if they have to have concave elements like overhangs though). The snag came when importing into Godot—it just looked terrible. I think skyboxes are literally just for skies.

This was really discouraging...I wasn't sure what to do. Eventually I decided to try just exporting the terrain as a mesh rather than as a background image or skybox and that worked surprisingly well.

There are still some issues with textures here (banding, doesn't seem to be the right resolution), and I simplified the geometry too much (way too pointy). But this captures the gist of what I was going for, and more importantly, the workflow is promising!

What's next

This terrain/environment process needs more tuning, but I'm happy with it for now. That was one of the last big unknowns for me in terms of the art asset workflow (the other one is character design, but I'm holding off on that for now). I think I'll take a step back from the art/game development and dig back into the world-building/narrative side.

As I mentioned in the last devlog, I implemented a two viewport system: one viewport for the half-resolution 3d rendering and then a second main (the root) viewport to render the UI and scale up the 3d output. One problem: shadows weren't rendering anymore. This was easy to fix; by default new viewports don't have a shadow atlas, so I just had to change that value from 0 to 1.

I also continued work on the outdoor/natural environment scenes, which brought up some more issues. The forest clearing scene dropped in performance (down to 11-15fps) once more trees were added, which I kind of expected. I reduced the tree model to about 330 vertices (from ~1.2k), which probably helped a little, but was less impactful than I'd hoped. I ended up playing around with the root viewport settings (which confusingly are mixed into Project Settings > General). By default the root viewport is set to render 3d, but now it's only rendering the 2d image it's passed by the half-resolution viewport. So I set the root viewport's render mode to 2d (under Project Settings > General > Intended Usage) and that boosted the scene to around 24-30fps. Still not the greatest, but fine for now.

I also wanted to set up a baked lightmap for ambient occlusion, but there's a bug where environmental lighting isn't taken into account (a fix should be out in 3.3.1).

I also put together a cave/grotto scene, which went quickly:

I realized the cave and forest clearing scenes are actually closer to indoor scenes: there's no skybox or distant environment to render. So I started playing around with a proper outdoor environment, where the sky is visible and there's distant geometry to handle:

A lot of new things to consider here. One, which is mostly specific to this particular scene, is grass and wind effects. I have a lot of grass here...I want to evoke that wind-sweeping-through-the-grass effect that comes up in a lot of Ghibli films. Of course this is a performance hit, with the scene rendering at 10-15fps.

If you look at the effect in Ghibli films, it's actually very simple. The grass is just a static image with trails moving on top:

So I don't really need actual grass geometry for the effect to look good. I can probably put together some kind of shader that moves these trails over the surface of a static texture.

The other thing I'm not sure about is what to render as a skybox and what to render as in-game terrain. I'm using the HTerrain plugin to generate terrain and in the image above all the terrain is in-game. It's really lacking in detail/richness which is mostly because I'm only using one texture. I'm thinking I should probably only render the foreground as in-game geometry and everything else as skybox for performance's sake. Which means I need to figure out a workflow for modeling and exporting skyboxes from Blender. This is something I'll have to think on more, I'm really not sure how best to approach this.

A few other updates:

• Set up a test character and started giving character animations a try, which I was dreading (I still am, but less so). The whole process was made way easier with Mixamo—this will sound like an ad, but it has an auto-rigger that makes a rigging a character very easy and their library of free motion-capture animations covers a lot of what I need right now. For the character itself I based a mesh off of Kiros from FF8 an am using a texture someone extracted from the game.
  • It's very helpful to have a character for scale. The scale of the indoor test environment, for example, was way too huge.
  • I spent quite a long time tweaking the third-person controls, eventually settling on a system that doesn't use the mouse. Basically, with the default keybindings of WASD: W moves the character forward in whatever direction they're facing, S moves the character backwards (i.e. they literally walk/job backwards), and A/D turns them to their left or right respectively. So far it feels smooth and intuitive.
• Formalized the "scene" system a bit more. I implemented a node type called Scene which contains information like fixed camera configurations and where the player spawns. This is really simple at the moment, but I imagine it'll be more complicated as I need to switch between cameras in a scene, or track the player for certain segments, and so on.

I started experimenting with an outdoor environment to start trying out the various environmental effects and elements I looked into last week.

There were a few small victories: the Waterways add-on worked great for still pools by setting both Flow > Distance and Flow > Pressure to 0. I also found an off-the-shelf cloud shader that looks great. With these elements and the ones from last week the outdoor environment looked dreamy:

But the experience raised an important issue that I kind of hoped I would avoid: performance. The outdoor scene was very choppy. I originally had the ~6.6k vertices trees so I tried simplifying them down to about ~1.2k or so. It didn't seem to help much. After reading up on 3d game optimization, the vertex/triangle count was not as important as I thought.

It's possible that some slowdown is due to Godot, which isn't as optimized as more mature 3d game engines (for example, it lacks occlusion culling, which is when objects blocked/hidden by other objects are discarded). It sounds like major 3d improvements have been pushed to the 4.0 release, which may not happen for awhile.

My GPU (or rather my lack of a dedicated one) is no doubt a major bottleneck here, but I want this game to play well on integrated graphics. I'm going with the low-res textures and low-poly models not only for style reasons but also for performance. I thought that would go a long way in terms of performance, but I was completely wrong. They are important, but not nearly as important as light and shadows.

After some more reading and testing of my own:

• Shadows have a huge effect. Turning off ambient occlusion for the world environment helps a lot and using simpler shadows helps a bit more. Glow doesn't seem to affect FPS that much.
• Mesh amount and complexity still have an impact. Having a lot (~100) of the 1.2k vertices trees did slow things down a lot, if only because they make light/shadows more complicated.
• The wind shader, on the other hand, didn't seem to affect performance much compared to a regular spatial shader. Though it does require dynamic shadows to work; otherwise I could probably bake those shadows.
• The cloud shader also has a pretty substantial impact (10-15fps). Tweaking its settings to do fewer draw passes did help a little. I also tried porting another cloud shader, but that performed even worse and was a hassle to set up.

Throughout this testing I thought this was just an issue with the outdoor scene (with the high number of moving trees, especially), but the indoor test scene also had an FPS of only about 10-11. Turning off ambient occlusion, however, boosted that to around 30fps. I guess I completely underestimated how demanding ambient occlusion is.

Specifically, I'm using Godot's screen-space ambient occlusion (SSAO), which adds a lot of richness to the environment. But it's hard to justify such a massive drop in FPS. Fortunately, the screen shader effect kind of maintains some visual interestingness on its own.

I still wanted to preserve some AO if possible. Baking lightmaps is the usual approach and wasn't too complicated to get set up in Godot. The baked lighting does looks better than having no SSAO, and I get around 30fps consistently—basically same performance as without the baked lightmap. But the baked lightmaps don't look as good as the SSAO. According to the docs SSAO is a different approach that captures details in smaller geometries. But after digging a bit, it seems the main difference is that SSAO has a "Light Affect" parameter that controls the visibility of AO in direct light. Normally it's not visible in direct light, i.e. "Light Affect" is 0, but I set it to 1 to exaggerate corners with deeper shadows. This isn't an option for the AO produced by baking lightmaps.

So I tried another route. The screen shader effect I'm using already simulates half resolution for more pixel texture; it makes sense to just actually render at half resolution and draw fewer pixels in the first place. Below are a few comparisons:

The outdoor scene looks good under all settings. But I imagine the double downscaling looks worse as there are more objects on the screen. That's the case with the indoor scene, which looks too noisy with the double downscaling. But otherwise the FPS gains are very promising and give me enough space to switch SSAO back on for the indoor scene!

One issue here is that this downscales the entire viewport, so the text is also downscaled. Text should probably render at the highest resolution possible for readability. So instead I need to render the player camera to a separate downscaled viewport and then render the non-downscaled resolution text (and other UI elements) on top. This was surprisingly complicated to set up; at least, the way Godot's viewports and scene tree work isn't intuitive to me. I struggled to find a way to specify what node loaded scenes attach to so that I could automatically attach scenes to my downscaled viewport node. But that doesn't seem possible—instead I have my player camera as a child of this downscaled viewport node and let scenes attach to the scene tree normally. The "Main" scene that contains all of this is instantiated as a singleton (i.e. using autoload).

This is the route I'll go with for now. The look of the game is pretty much the same, and I can use SSAO for indoor scenes and maintain high framerates. Similarly, the outdoor scene can use the cloud shader and maintain high framerates. That will likely change though as one tree is obviously an unrealistically simple scene. I need to keep experimenting with simpler trees and laying out the scene differently so I don't actually have to have a small forest's worth of trees. My hope is that with this set up the marginal cost of additional meshes will be pretty small. Not small enough for an entire forest, but small enough that I won't have to worry much about performance again.

A related question is the player camera. Right now the player has free reign over it, and the dialogue system was designed with that in mind. However when I imagine the game being played I see a fixed camera setup. The benefit with that is the player is way more constrained in what they see, so I can be more minimal in scene dressing and maintain better performance that way. For example, I'd only need to seat a few trees at the edge of the camera's view rather than anywhere the player might possibly look.

This was just a quick test to see how it might work. To really see if this is the right approach I should start figuring out the character design more and flesh out the third-person system in general. So I'll probably tackle that next before prototyping more of the outdoor/natural environments.

With the dialogue system in decent shape, I'm moving onto exterior environments. There's a lot to figure out here: environmental effects like wind, fire, smoke, rain, and snow; terrain design; rocks; vegetation including grass, bushes, and trees; skyboxes; and water (which I'm categorizing into "still" water, like those in ponds and pools, and "active" water, like that of the sea). In general creating "natural" effects and objects is way harder than human-made interiors and objects!

Several of the effects (rain, snow, fire, smoke) can be implemented as particle systems. I played around with Godot's Particles node and am pretty happy with the results. Here are a couple demos showing falling leaves and petals:

The only thing that doesn't seem possible is the leaves rotating along the x/z axes. It's not a huge deal since these are more of mood-setting elements and there'll be plenty else on the screen to focus on.

Many of the remaining effects are usually implemented with shaders: wind in grass and trees (this tree wind shader works pretty well), the grass itself, and water. I don't have much experience with shaders so I spent some time learning more about them and playing around with them. I haven't yet managed to finish anything substantial yet but will keep exploring there.

In terms of skyboxes, those seem not too difficult to implement in Blender: basically you build your skybox and then export it as a panoramic image into Godot.

Terrain, rocks, water, and vegetation are a lot harder. For terrain and water there are fortunately some Godot plugins available that make it much easier. For terrain, there's Zylann's HTerrain plugin for heightmap-based terrains. This works well for outdoor natural environments and it supports some things out of the box, like painting grass textures and basic wind for grass. Unfortunately the heightmap-based approach doesn't work for things like caves and overhangs, so it's not a panacea.

For caves and overhangs there's Zylann's VoxelTerrain plugin for voxel-based terrains. It's not as mature as HTerrain (lacks a few of its key features, like LOD and grass painting) and it requires compilation into Godot (i.e. not just an addon you can drop in), so I'm not sure if this is exactly what I want to use.

Alternatively, I could create the terrains in Blender using the sculpting and texture painting tools there and import them into Godot. Caves aren't too difficult in Blender (you can use a cube and some displace modifiers to create the right rocky walls as a starting point) so maybe this is the way to go.

For rivers there's Arnklit's Waterways. I wonder if I can stretch it to work for other bodies of water. In any case I may need something different for still bodies of water, like those in grottos:

For trees, there were two plugins for Blender I looked at: tree-gen and modular-tree. I also looked at hand-making trees based on "How to Create a Low Poly Tree in 1 Minute" (a similar technique, mainly using Blender's particle system for leaves/small branches, came up in a few other videos). The generated tree-gen mesh has almost 1.3 million vertices which is way too many. In comparison the basic tree generated by modular-tree has about 21k vertices¹ and the handmade low-poly tree has only 6.6k vertices. That might still be too many—I don't yet have an intuition of what number I should aim for, I just know that fewer vertices are better in my case. The low-poly one also better matches the look of the game so far (low-poly, photo-realistic textures, and dithering, which in this case is just an artifact of Blender).

I wrote a Blender addon that generates the base of the tree (trunk and branches, but without the leaves) in a manner similar to that low-poly tutorial. You still need to manually set up the vertices and particle system for the leaves, but I may expand the script to do some of that automatically. It was a struggle to get the script working properly (my linear algebra is really rusty and the Blender Python documentation, at least for the bmesh parts, is really lacking), and it's not nearly as powerful as tree-gen or modular-tree, but it does what I need and in the style suitable for the game.

Here's the tree with this basic wind shader:

With trees and other vegetation, as well as rocks, you need not only to create the tree meshes, but also distribute them. This post details the use of a particle system to distribute vegetation throughout a terrain, but Zylann has yet another (!) plugin, Scatter3d, that lets you paint scene instances (e.g. meshes) in Godot. For wind, Godot has a tutorial for a wind shader.

For rocks, Blender has an add-on to generate rocks, which as far as I can tell is based around feeding noise into a displacement modifier (this is the basic approach for creating caves in Blender as well). I've also seen rocks generated with a particle shader, which probably makes more sense if you have a desert landscape that needs tons of small rocks scattered about. If I've learned anything while researching how to approach these natural elements, it's that a rock is not a rock, and water is not water, and so on...there are rocks that are big, for up-close use, or medium-sized ones for foreground elements, or tiny ones for a sprawling landscape; water in a calm pool is not the same as water running through a river which is not the same as water that's crashing against a beach.

A tricky thing with natural elements is the right amount of realism. There's of course a lot of effort spent on figuring out ways to most accurately simulate natural environments and effects, and realism usually comes with a big cost to performance. I'm not trying to make natural environments that approach reality but that are interesting and convincing enough. Whenever I come across a video or post detailing how to make a tree or a water shader or what not, I always have to figure out the context. Is these for rendering in a 3d short film? Is this for a triple-A video game where players are assumed to have the latest graphics cards? Is this a demo of how realistic you can get an effect but way too computationally demanding for practical use?

For example, I found this beautiful Breath of the Wild style grass created using a shader. It's a bit resource intensive and, as wonderful as it looks, it's just not the vibe Fugue's going for.

Next I need to pull these pieces together and try building a couple natural environments—probably a forest clearing and a grotto to start.