# Fugue Devlog 8: Testing Outdoor Environments and Performance

04.23.2021

I started experimenting with an outdoor environment to start trying out the various environmental effects and elements I looked into last week.

There were a few small victories: the Waterways add-on worked great for still pools by setting both Flow > Distance and Flow > Pressure to 0. I also found an off-the-shelf cloud shader that looks great. With these elements and the ones from last week the outdoor environment looked dreamy:

But the experience raised an important issue that I kind of hoped I would avoid: performance. The outdoor scene was very choppy. I originally had the ~6.6k vertices trees so I tried simplifying them down to about ~1.2k or so. It didn't seem to help much. After reading up on 3d game optimization, the vertex/triangle count was not as important as I thought.

It's possible that some slowdown is due to Godot, which isn't as optimized as more mature 3d game engines (for example, it lacks occlusion culling, which is when objects blocked/hidden by other objects are discarded). It sounds like major 3d improvements have been pushed to the 4.0 release, which may not happen for awhile.

My GPU (or rather my lack of a dedicated one) is no doubt a major bottleneck here, but I want this game to play well on integrated graphics. I'm going with the low-res textures and low-poly models not only for style reasons but also for performance. I thought that would go a long way in terms of performance, but I was completely wrong. They are important, but not nearly as important as light and shadows.

After some more reading and testing of my own:

• Shadows have a huge effect. Turning off ambient occlusion for the world environment helps a lot and using simpler shadows helps a bit more. Glow doesn't seem to affect FPS that much.
• Mesh amount and complexity still have an impact. Having a lot (~100) of the 1.2k vertices trees did slow things down a lot, if only because they make light/shadows more complicated.
• The wind shader, on the other hand, didn't seem to affect performance much compared to a regular spatial shader. Though it does require dynamic shadows to work; otherwise I could probably bake those shadows.
• The cloud shader also has a pretty substantial impact (10-15fps). Tweaking its settings to do fewer draw passes did help a little. I also tried porting another cloud shader, but that performed even worse and was a hassle to set up.

Throughout this testing I thought this was just an issue with the outdoor scene (with the high number of moving trees, especially), but the indoor test scene also had an FPS of only about 10-11. Turning off ambient occlusion, however, boosted that to around 30fps. I guess I completely underestimated how demanding ambient occlusion is.

Specifically, I'm using Godot's screen-space ambient occlusion (SSAO), which adds a lot of richness to the environment. But it's hard to justify such a massive drop in FPS. Fortunately, the screen shader effect kind of maintains some visual interestingness on its own.

I still wanted to preserve some AO if possible. Baking lightmaps is the usual approach and wasn't too complicated to get set up in Godot. The baked lighting does looks better than having no SSAO, and I get around 30fps consistently—basically same performance as without the baked lightmap. But the baked lightmaps don't look as good as the SSAO. According to the docs SSAO is a different approach that captures details in smaller geometries. But after digging a bit, it seems the main difference is that SSAO has a "Light Affect" parameter that controls the visibility of AO in direct light. Normally it's not visible in direct light, i.e. "Light Affect" is 0, but I set it to 1 to exaggerate corners with deeper shadows. This isn't an option for the AO produced by baking lightmaps.

So I tried another route. The screen shader effect I'm using already simulates half resolution for more pixel texture; it makes sense to just actually render at half resolution and draw fewer pixels in the first place. Below are a few comparisons:

The outdoor scene looks good under all settings. But I imagine the double downscaling looks worse as there are more objects on the screen. That's the case with the indoor scene, which looks too noisy with the double downscaling. But otherwise the FPS gains are very promising and give me enough space to switch SSAO back on for the indoor scene!

One issue here is that this downscales the entire viewport, so the text is also downscaled. Text should probably render at the highest resolution possible for readability. So instead I need to render the player camera to a separate downscaled viewport and then render the non-downscaled resolution text (and other UI elements) on top. This was surprisingly complicated to set up; at least, the way Godot's viewports and scene tree work isn't intuitive to me. I struggled to find a way to specify what node loaded scenes attach to so that I could automatically attach scenes to my downscaled viewport node. But that doesn't seem possible—instead I have my player camera as a child of this downscaled viewport node and let scenes attach to the scene tree normally. The "Main" scene that contains all of this is instantiated as a singleton (i.e. using autoload).

This is the route I'll go with for now. The look of the game is pretty much the same, and I can use SSAO for indoor scenes and maintain high framerates. Similarly, the outdoor scene can use the cloud shader and maintain high framerates. That will likely change though as one tree is obviously an unrealistically simple scene. I need to keep experimenting with simpler trees and laying out the scene differently so I don't actually have to have a small forest's worth of trees. My hope is that with this set up the marginal cost of additional meshes will be pretty small. Not small enough for an entire forest, but small enough that I won't have to worry much about performance again.

A related question is the player camera. Right now the player has free reign over it, and the dialogue system was designed with that in mind. However when I imagine the game being played I see a fixed camera setup. The benefit with that is the player is way more constrained in what they see, so I can be more minimal in scene dressing and maintain better performance that way. For example, I'd only need to seat a few trees at the edge of the camera's view rather than anywhere the player might possibly look.

This was just a quick test to see how it might work. To really see if this is the right approach I should start figuring out the character design more and flesh out the third-person system in general. So I'll probably tackle that next before prototyping more of the outdoor/natural environments.

# Fugue Devlog 7: Environmental Effects and Objects

04.18.2021

With the dialogue system in decent shape, I'm moving onto exterior environments. There's a lot to figure out here: environmental effects like wind, fire, smoke, rain, and snow; terrain design; rocks; vegetation including grass, bushes, and trees; skyboxes; and water (which I'm categorizing into "still" water, like those in ponds and pools, and "active" water, like that of the sea). In general creating "natural" effects and objects is way harder than human-made interiors and objects!

Several of the effects (rain, snow, fire, smoke) can be implemented as particle systems. I played around with Godot's Particles node and am pretty happy with the results. Here are a couple demos showing falling leaves and petals:

The only thing that doesn't seem possible is the leaves rotating along the x/z axes. It's not a huge deal since these are more of mood-setting elements and there'll be plenty else on the screen to focus on.

Many of the remaining effects are usually implemented with shaders: wind in grass and trees (this tree wind shader works pretty well), the grass itself, and water. I don't have much experience with shaders so I spent some time learning more about them and playing around with them. I haven't yet managed to finish anything substantial yet but will keep exploring there.

In terms of skyboxes, those seem not too difficult to implement in Blender: basically you build your skybox and then export it as a panoramic image into Godot.

Terrain, rocks, water, and vegetation are a lot harder. For terrain and water there are fortunately some Godot plugins available that make it much easier. For terrain, there's Zylann's HTerrain plugin for heightmap-based terrains. This works well for outdoor natural environments and it supports some things out of the box, like painting grass textures and basic wind for grass. Unfortunately the heightmap-based approach doesn't work for things like caves and overhangs, so it's not a panacea.

For caves and overhangs there's Zylann's VoxelTerrain plugin for voxel-based terrains. It's not as mature as HTerrain (lacks a few of its key features, like LOD and grass painting) and it requires compilation into Godot (i.e. not just an addon you can drop in), so I'm not sure if this is exactly what I want to use.

Alternatively, I could create the terrains in Blender using the sculpting and texture painting tools there and import them into Godot. Caves aren't too difficult in Blender (you can use a cube and some displace modifiers to create the right rocky walls as a starting point) so maybe this is the way to go.

For rivers there's Arnklit's Waterways. I wonder if I can stretch it to work for other bodies of water. In any case I may need something different for still bodies of water, like those in grottos:

For trees, there were two plugins for Blender I looked at: tree-gen and modular-tree. I also looked at hand-making trees based on "How to Create a Low Poly Tree in 1 Minute" (a similar technique, mainly using Blender's particle system for leaves/small branches, came up in a few other videos). The generated tree-gen mesh has almost 1.3 million vertices which is way too many. In comparison the basic tree generated by modular-tree has about 21k vertices1 and the handmade low-poly tree has only 6.6k vertices. That might still be too many—I don't yet have an intuition of what number I should aim for, I just know that fewer vertices are better in my case. The low-poly one also better matches the look of the game so far (low-poly, photo-realistic textures, and dithering, which in this case is just an artifact of Blender).

I wrote a Blender addon that generates the base of the tree (trunk and branches, but without the leaves) in a manner similar to that low-poly tutorial. You still need to manually set up the vertices and particle system for the leaves, but I may expand the script to do some of that automatically. It was a struggle to get the script working properly (my linear algebra is really rusty and the Blender Python documentation, at least for the bmesh parts, is really lacking), and it's not nearly as powerful as tree-gen or modular-tree, but it does what I need and in the style suitable for the game.

Here's the tree with this basic wind shader:

With trees and other vegetation, as well as rocks, you need not only to create the tree meshes, but also distribute them. This post details the use of a particle system to distribute vegetation throughout a terrain, but Zylann has yet another (!) plugin, Scatter3d, that lets you paint scene instances (e.g. meshes) in Godot. For wind, Godot has a tutorial for a wind shader.

For rocks, Blender has an add-on to generate rocks, which as far as I can tell is based around feeding noise into a displacement modifier (this is the basic approach for creating caves in Blender as well). I've also seen rocks generated with a particle shader, which probably makes more sense if you have a desert landscape that needs tons of small rocks scattered about. If I've learned anything while researching how to approach these natural elements, it's that a rock is not a rock, and water is not water, and so on...there are rocks that are big, for up-close use, or medium-sized ones for foreground elements, or tiny ones for a sprawling landscape; water in a calm pool is not the same as water running through a river which is not the same as water that's crashing against a beach.

A tricky thing with natural elements is the right amount of realism. There's of course a lot of effort spent on figuring out ways to most accurately simulate natural environments and effects, and realism usually comes with a big cost to performance. I'm not trying to make natural environments that approach reality but that are interesting and convincing enough. Whenever I come across a video or post detailing how to make a tree or a water shader or what not, I always have to figure out the context. Is these for rendering in a 3d short film? Is this for a triple-A video game where players are assumed to have the latest graphics cards? Is this a demo of how realistic you can get an effect but way too computationally demanding for practical use?

For example, I found this beautiful Breath of the Wild style grass created using a shader. It's a bit resource intensive and, as wonderful as it looks, it's just not the vibe Fugue's going for.

Next I need to pull these pieces together and try building a couple natural environments—probably a forest clearing and a grotto to start.

1. It was a little tricky to set up modular-tree. One note is to checkout the blender_28 branch if you're using Blender 2.8 or above. The other is that there's a bug when adding a twig node. What worked for me is to first add a twig node and "execute" the node, creating a mesh called tree, then rename the mesh to twig (or whatever else). Then set up the tree nodes: a trunk node into a branch node, then a "tree parameters" node separately. Check create_leafs, then click the eyedropper for the leaf parameter and select your twig mesh in the scene hierarchy. Then click execute tree

# Fugue Devlog 6: More Dialogue System!

04.15.2021

A few pain points came up after working with the dialogue system and editor more. The schema of having the text (what's being said) and varying choices and outcomes on the same level didn't make much sense, since the vast majority of dialogue is simple linear exchanges. Having the possibility of choices and outcomes attached to every utterance made the editor graph really unwieldy. This led to a few changes:

• Because the type of an event can be inferred from who the speaker is, i.e. a "thought" is anything where the speaker is "self", and otherwise it's a "verse", the event types can be dropped.
• I renamed "events" to "verses", because the term "event" is confusing in the context of dialogue.
• Verses no longer have one piece of text attached to them but can encompass multiple "lines". Each "line" of a verse has at minimum some text and its speaker, but it can also have a timeouts and/or a delay, or emit a signal to trigger other parts of the game.
• These are specified by a bit of syntax at the beginning of the line's text. E.g. !foo;t5;d8|This is the text means this line will: emit a signal called "foo", have a 5 second timeout, and a 8 second delay.

These changes meant the dialogue editor could be streamlined:

Another small quality-of-life improvement is a button on each outcome to add a new verse already connected to that outcome. Before I'd have to add the new verse, then drag it next to the outcome so I could connect the cable.

The other update to the dialogue system is a better dialogue box layout system, to minimize overlapping dialogue boxes:

This was tricky to figure out. The primary constraint is that you can't reposition dialogue boxes so much that it's unclear who the associated speaker is. The safest axis of movement is along the x-axis, so the very simple approach is to just go from left to right and move boxes either left or right if there are overlaps. Of course there are many situations where this won't help, but I don't think the game will have much more than a few simultaneous speakers at once. The other simplification is that this layout adjustment is applied only to "clear" dialogue boxes; that is those within range that you can clearly "hear" the speakers. Anything out of range will be drawn below these if there's overlap.

I also implemented a custom rich text effect based on this tutorial. It seems like a powerful system:

# Fugue Devlog 5: Dialogue System Implementation

04.13.2021

I'm chipping away at implementing the dialogue system. It's daunting; sometimes it feels like trying to build a house all at once. Once you start to sort out what the foundation is, what part depends on what other part, etc, a build order starts to become clear and the whole implementation becomes more manageable. If you can sort that out on paper and think through most if not all of the possible issues, implementation is really straightforward.

Some of the key features like choice selection, tracking dialogue states (e.g. remembering how many times you've interacted with that speaker), and speaker tracking are demoed below:

I discussed an ambient dialogue system in a previous post and made some headway on implementing it. The off-screen dialogue box handling needs more work (really janky atm), but I have the dialogue box "blurring" (which fuzzes dialogue boxes as you get further from then, to mimic those voices becoming harder to hear) more or less working.

I had to compromise a bit though. I originally envisioned a gaussian blur effect, where at sufficient distances the dialogue boxes were basically smeared into nothingness. But blurring individual UI element is really complicated. It looked like the only approach was to setup separate viewports for each dialogue box, render those to textures, and then apply a blur shader to those textures. Way too complicated.

Here's the current implementation of dialogue "blurring". As the player gets further from the speakers, the dialogue boxes scale down and become transparent. It needs some tweaking, e.g. the transparency change looks awkward as a linear function, maybe should be using squared distance to feel more organic. But I think this works well as a general approach. This also means that distant dialogue takes up less screen space, so there'll be less clutter.

Here's the handling of off-screen/ambient dialogue. I ended overcomplicating it, spending too much time trying to implement this unnecessarily complicated version, then found that the simpler approach worked better anyways (just clamping off-screen dialogue boxes to screen space). There was some weirdness that needed fixing—basically, off-screen objects in perspective cameras are positioned counterintuitively, so the y-positioning of the dialogue boxes can look unexpected. The most noticeable case is when off-screen dialogue boxes are positioned at the top the screen for things that are behind the camera. It feels more "natural" to have those at the bottom of the screen; so there's a small snippet that ensures that's the case.

The other feature, which is also a bit janky right now, is interrupting ongoing NPC dialogue. Discussions where the player isn't involved, i.e. among a group of only NPCs, advance automatically. If the player enters the vicinity of any of the NPCs involved in that discussion, and the player can speak with at least one of those NPCs, they will pause their discussion and address the player. If the player leaves the vicinity, they resume their discussion from wherever they left off.

There's a lot to tweak here, like timing around the discussion pausing and resuming, checking for race conditions (always a possibility when timers are involved), and figuring out how best to handle the lead-in/lead-out snippets (here: "Do you need something?" when the player interrupts and "What was I saying..." when the discussion resumes).

Aside from cleaning up the implementations of these features, I'm mostly finished with this first pass at the dialogue system. Still many things to figure out about how dialogue is best triggered and how it should be associated with entities and so on. No doubt things will need fixing and I'll want to do things the system doesn't support as-is. And the tools will also change; I'm already finding pain points with the dialogue editor that need improving.

The next step is to start reviewing all of my notes for the game's world and story and start putting together a more concrete draft/design document. That'll help me figure out what other key mechanic systems are required.

# Fugue Devlog 4: Dialogue System Improvements

04.09.2021

Not a lot of coding today, but a lot more work on the dialogue system. Sketching the manager render sequence out, figuring out the speaker tracking, dialog box placement, etc.

I've tried to think through all possible cases but surely some have been overlooked. I just hope that none are serious enough to require a complete restructuring. There are so many cases to consider, but a few that are trickier include:

• Should multiple on-screen decisions be allowed? If so, how do players select between decisions? Right now I'm thinking that only one decision be allowed, but don't know of a robust way of enforcing that constraint in the dialogue editor validator.
• Should time stop/player movement be locked while in dialogue? I like the idea of being able to move around, break off from conversations, etc. Makes it feel more lively. But it adds in complications of pausing and resuming dialogue. And fixing the player in place while in a conversation feels like a reasonably "real" constraint to have.
• That being said, pausing and resuming is necessary for ambient conversations. For example, two people having a conversation, then pausing to address the player when they get close enough (e.g. "Can I help you?"), and resuming their conversation once the player leaves their vicinity. This is part of a broader question of whether or not the ambient dialogue system should be distinct from the "player" dialogue system. Ideally they are one and the same, and the player's involvement is a special case (e.g. time stops/player movement stops only if an active script includes the player as a speaker; dialogue always auto-advances/times out if the player isn't involved, etc).
• How should dialogue progression be handled? There are a lot of potentially conflicting cases here. In general there are three ways a statement can advance: the player hits the next dialogue input, the player selects a choice (if the statement is a decision), or the statement times out (if it has a timeout specified). This is straightforward if there's one statement on-screen. But if there are multiple, as with simultaneous dialogue, then what? If some statements have timeouts and some don't, does pressing the next dialogue input advance them all? Or do they wait for the timeouts to finish? There isn't really a clear answer here. I may just have to pick something and be ready to change it as the rest of the game develops.

This last point is kind of the case with the dialogue system more broadly. I won't really know how well any of this works until I implement it and start working through test cases.

I did get a very basic version of the speaker tracking implemented, was pretty straightforward to do in Godot. Basically I calculate the full bounding box ("AABB") of a spatial node and use that to determine its center x coordinate and maximum y coordinate. Then unproject that point to the camera/screen space. It will get more complicated with multiple speakers and potentially overlapping dialogue boxes, though I think I have a decent solution to that in the sketches above.

## Blender scripting

I also learned a bit of Blender scripting (which uses Python). Most people I know who do a lot of 3d modeling use Rhino/Grasshopper, and I've seen a lot of really amazing work done with scripting in that tool. It looks like Blender is just as capable, which is cool.

I made a big button for quickly exporting the current file to my Godot model folder. Normally I have to go to the export dialogue, navigate to the appropriate folder, then save. Navigating through folders is slow...it's a small thing but smoothes out the workflow a lot.

I'm also really impressed by the environment Blender provides for scripting. It has a console for trying things out and logs every interaction you have with the UI so that you can easily figure out what functions do what. There are also several templates provided that make it easy to quick prototype an idea.