I’m still working on Fugue. Funnily I had a draft update post exactly a year ago that I never finished, but this post is almost 2 years after the last one. A lot of preparatory progress has been made: characters and major plot points are starting to come together, and the whole production tooling/process is much further along.

Verses: Rust to C#

I’m trying to keep the core game logic outside of Unity/C# and move as much as possible into Rust, a language I’m more comfortable with. So I rewrote the Verses/Garland system, which is the scripting system (as in dialogue scripts) that manages most of the game state, into Rust and generated an FFI for it using uniffi (and its C# bindings generator). It’s just a lot easier for me to maintain and make changes. A few notes:

• I am a bit concerned about possible issues building Rust code for console platforms, but I’ll worry about that later.
• There’s a bit of plumbing required to wire up the library to Unity, but it’s not too bad.
• One minor annoyance is that Unity requires a full restart to load any changes to the library, though as the library stabilizes this should be less of a problem.
• Debugging is more complicated, and as far as I know there’s no way to hook into Unity’s logging system, so I log to a file instead.
• The generated code is for C# 12, while Unity supports only C# 9, so some unsightly post-processing is required to make the generated bindings Unity-compatible, but again it’s not too bad.
• There’s additional overhead calling over an FFI boundary, but I expect it to be negligible here.

So far this approach has been fine in terms of trade-offs. There’s also something nice about having a strict boundary between the “game environment” code and logic and a much more strongly enforced “single source of truth” in terms of the game state. But more complications could easily come up as I get more into writing the game’s scripts, so we’ll see.

Stagecraft: Blender to Unity

Similar to the Rust/C# divide I’m also trying to keep as much scene-building outside of Unity as possible. I find Blender a lot nicer to work with and it’s of course much more powerful for modeling, texturing, and so on. But it doesn’t have analogues to Unity’s components/MonoBehaviours, so there’s no clear way to represent the dynamic behavior necessary for a game. And many of Blender’s powerful features, like procedural textures, aren’t directly exportable to Unity.

To solve these issues I wrote a custom Blender add-on, Stagecraft. Where possible this add-on uses existing functionality, e.g. Blender’s baking system can be used for procedural textures. But what about game component behaviors? In my case I have a relatively small set of fixed behaviors that just require their data/configuration to be specified. For example, one behavior I have is ItemCarrier, which defines if the object has items the player can pick up. This only requires that the item IDs and quantities are specified, which are easy to attach to objects in Blender as custom properties. Unity built-ins like colliders can also be configured.

Stagecraft then exports the scene as its individual models (as GLTFs) and then a separate scene hierarchy that has all this additional data like ItemCarrier items. In Unity I have a custom importer which imports this scene hierarchy and creates all the appropriate game objects with the correct MonoBehaviours attached. It creates the correct lights and bakes the lightmap as well. The goal is that the scene is entirely specified in Blender and no edits should be made to the scene in Unity. We’ll see where the limits to this approach are, but so far it seems to work well.

There is one major improvement I’m figuring out, which is handling animation state machines and blend trees. I’d like to be able to bring in an asset with some animations (actions) and be able to define the equivalent of a Unity Animator from within Blender, with conditions and triggers that are handled by Verses/Garland.

Loom

I also have a new version of my world-building and content authoring tool, loom, which makes it easier to edit scripts, write quest outlines, and organize characters, factions, and locations.

Next steps

I’m still working on mechanics, but I need to have a playable prototype before I can really move further, hence the focus on getting the content production system further along. The next big tasks for this workflow are putting characters in scenes and having complex animation state machines, and then thoroughly testing the Verses/Garland script integrations in Unity. Hopefully soon I’ll have the whole skeleton of the game engine and production system finished and it’ll be easier to iterate on it and polish it. The hardest part is getting the foundation in place.

It’s been a while since the last update; I was away at a language immersion program and then very busy with our move to Paris but managed to find some time here and there to continue work on the Fugue. Though it’s probably more accurate to say the “pre-work” on the game, as I’m still deep in setting up all the tooling and infrastructure that will hopefully make the actual game development feel more like building with Legos.

The biggest development is that I re-wrote the GUI tools from tauri/typescript to egui/rust. Having a web-based UI is just too heavy and if I’m going to be using something frequently I’d want it to be quick and snappy. That in itself was challenging because the UI tooling is so much better for the web (makes sense, given that it’s the dominant type of UI these days) and the Rust ecosystem in general is still in its infancy. But ultimately it was worth it.

Two completely new tools are crane and depot. On the point of wanting to avoid clunky UIs, Unity is so sluggish on Linux (not sure if the same is true for Windows) and just adds so much mental overhead to assembling scenes within it. crane is instead a very simple scene editor with modal controls (à la vi/vim; I’m a zealot) which is capable of parsing and exporting Unity scenes. So the bulk of scene editing/creation can happen in there, with some parts (MonoBehaviour scripts, etc) that might still need to be done from within Unity, but are far less common tasks.

Depot & 3d overload

depot manages all the non-character 3d assets, tracking licensing and attribution where needed (e.g. CC-BY licensed assets), and common 3d processing tasks built-in. Primarily this is generating model variations with different textures and mesh simplification (way more on this below). It’ll eventually integrated with crane so crane can directly pull models from depot to add to a scene.

As I alluded to a long time ago, the 3d model aspect is probably the most difficult part of the technical aspect of the game’s development (writing, character development, etc are harder–and more fun–but leaving those aside for now) and I’ve decided the best approach is to set aside some budget to purchase pre-existing assets and hand-create only the simplest things. The amount of time saved will be well worth the cost.

The downside of using pre-made assets, however, is a lot of variability. Meshes can vary tremendously in poly counts, textures, etc; not to mention model formats (obj, fbx, gltf/glb, etc) which is a massive challenge itself. So I basically need a processing pipeline that can take a pre-made asset in any format (or at least the major ones) and turn it into something optimized and visually consistent with everything else with as little input from me as possible. Buying pre-made assets is moot if I have to spend just as much time adjusting the model to fit my particular needs.

This led me down a dizzying rabbit hole to the world of 3d software-stuff (? not sure what to call it) and so far I’ve come away with the following learnings:

• There are too many 3d formats! And no reliable means of converting between all of them, or even just the major formats. There’s assimp which seems ill-maintained, and you can use Blender too but it’s frankly heavy for the job. The best tool I’ve found so far is trimesh but afaik they don’t provide a command line tool or anything, so you need to have Python setup to use it. It’d be so nice to have something like ffmpeg but for 3d models. Instead I’ve found that specific conversion software, like obj2gltf or fbx2gltf are the most reliable.
• Formats aside, there’s so much variability already in meshes (welded vs unwelded, tris vs quads, messed up normals, bad UV mappings, water-tightness, etc) and thus so many opportunities for malformed meshes that will just gunk up any automated processing you have set up. Again, trimesh has so far seemed good at standardizing models to some degree such that they’ll then work for other mesh simplification programs.
• In terms of pre-made programs and libraries you often see just two extremes: extraordinarily expensive but good & robust (I’m guessing, I can’t afford to try them) geared towards big money-making studios (where licenses are generally in the tens of thousands per year) and 5 to 10 year old code for a random SIGGRAPH paper that seems so promising but isn’t written for general use and/or is an absolute nightmare to compile. There are things in between but they’re relatively rare. In contrast, for data science, for example, there is a massive wealth of libraries/software of different scales out there to use.
• Related to the previous point, I’ve found that where native Rust libraries exist for some of these they’re often unexpectedly lopsided. For example, the tobj crate can load .objs but it can’t save them. I don’t know enough about the format to say more–I’m sure there’s a good reason, but it just seems strange to me.
• The ecosystem is still very Windows-focused, given that Windows is still the dominant computer gaming platform. wine is a godsend here, at least for running .exes; so far I haven’t had much luck compiling Windows C++ programs (using msbuild or msvc or clang or whatever).

Reducing polys

There are a lot of different 3d model processing tasks but fortunately for me the only one I’m concerned with is “mesh simplification”, i.e. taking a high-poly mesh and turning it into a low-poly one that still resembles the original. Generally you will bake some of the high-poly mesh information as textures (e.g. a normal map) to help with this.

There’s a related task of creating an “imposter”, where we don’t modify the mesh itself but instead create a simple proxy polygon, e.g. an octahedron. You then pre-render the object from various perspectives and those are then projected onto the polygon. The idea here is if an object is going to be static and only viewed from a few angles, we don’t need to store/render all the geometry as we’re essentially looking at a 2d image.

Imposters are probably the right approach for set-piece elements and the like. They’re much easier for me conceptually and because of their simplicity are probably going to be more robust than actually mutating the mesh. There are plenty of Unity plugins that do this; I haven’t tried them yet but they’re all well-reviewed.

But for anything more complicated, especially things that need to move, imposters won’t be enough. We have to actually change the geometry to reduce the poly count.

I’ve seen a few different terms for this task; I’m not sure what the distinctions are between them, if any: simplification, decimation, remeshing, retopology, reduction, LOD generation, and probably more.

Reflecting the pattern above there are a few “pro” options for this task, namely Simplygon which seems to be the industry standard (and if I’m not mistaken is integrated into the Unreal Engine’s mesh simplification routines), evident in its whopping cost of $35,000 per year. They do have a free version but it’s very limited. There’s also InstaLOD which has a more generous free option, but I haven’t been able to download it and I think it excludes the use of the C++ API. When there are free offerings direct API access is usually behind the paywall or, when no free offering exist, behind the more expensive plan. For example, with Pixyz it’s $1,350/year for just the plugin and $2,450/year for the Python API.

So I haven’t tried any of these products but I wonder how much better they are in terms of raw mesh simplification quality. They probably have a bunch of other features important to giant studios like build system integration or support for more esoteric workflows/needs; that might be more of the selling point than how good their mesh simplification algorithm is. I wonder this partly because after looking through the free or cheaper options they mostly use the same approach: some variation of quadric mesh simplification. As I understand this basically involves computing a metric (the quadric metric) for each edge in the mesh, essentially ranking the edges according to how important they are (I’m waving my hands) and then deleting the least important edge. Rinse and repeat until you hit your target face/vert count.

Some examples that I believe use this approach are SeamAwareDecimater, FastQuadricMeshSimplification, MeshLab, and nanomesh. On my test meshes they all give similar results. It feels like this approach is better suited to preserving as much detail as possible when simplifying, which is great for going from high-poly to mid-poly, but when you start to get to the very low poly ranges (<3000 faces or so) the mesh just starts to degrade, have holes, etc.

This thread discusses some of the other shortcomings of this approach and is where I first encountered the term “silhouette protection”. With the quadric approach you eventually get to the point where you start deleting faces and start to lose essential elements of the model, until it becomes unrecognizable. Silhouette protection is meant to stop the algorithm before it reaches this point; or it may be a totally different simplification approach, I don’t really know.

One approach I thought might work is to take the convex hull of the mesh and iteratively introduce edges, essentially “shrink-wrapping” the hull around the original mesh until you reach the desired number of faces or some minimum error between the mesh and the hull. The downside with this approach is that I don’t think it’d work for meshes that have holes because a convex hull doesn’t take those into account (afaik).

I wasn’t able to find much pre-existing software that does this, just Semi Convex Hull for Mesh Simplification and Boundingmesh which uses “bounding convex decomposition”. I wasn’t able to get them to finish simplifying my heavier test meshes, they ran without terminating. However, I believe this approach is kind of how collider meshes are computed, but looking at this collider mesh generation plugin it seems like the approach (for at least the VHACD algorithm; see also and here) is to break down the mesh into different parts and then build the collider mesh from the convex hulls of those individual parts, thus the mesh is no longer contiguous. But this does get around the hole problem I mentioned, and I guess you could re-join the individual parts though.

So unfortunately no clear resolution yet, but there’s still more to try.

Hard to believe it’s been less than three weeks since the last update. I’ve managed to finish a lot even though I feel like I haven’t had much time to work on things (especially with Tears of the Kingdom out now).

Better character movement

I mainly was procrastinating because I needed to fix some issues with the AI/NavMeshAgent character movement and I had no idea where to start. NavMeshAgent is Unity’s built-in character pathfinding system, and it’s not bad. I’m using it with root motion (i.e. the animation drives movement rather than direct modification of the character’s world position), which makes working with a bit more complicated.

The problem was that when the character was running they would never be able to turn fast enough to match their computed path. So they would overshoot things and run into walls while making large turns. There is an angularSpeed setting but it didn’t seem to have any effect. I toyed with having root motion drive the rotation too, but it ended up being too complicated; and even so, the direct rotation looks fine.

What I ended up doing was manually controlling the character’s rotation, rather than letting NavMeshAgent do it:

_agent.updateRotation = false;

// ...

// Called in Update
private void SyncAnimatorAndAgent() {
    // ...

    // Handle turning ourselves, rather than
    // delegating it to the NavMeshAgent;
    // this allows us to have more control.
    // Turn faster if running.
    var step = maxTurnSpeed * Time.deltaTime * (shouldRun ? 2 : 1);
    Vector3 direction = _agent.steeringTarget - transform.position;
    direction.y = 0; // Ignore up/down
    var targetRotation = Quaternion.LookRotation(direction, Vector3.up);
    transform.rotation = Quaternion.RotateTowards(transform.rotation, targetRotation, step);

    // ...
}

This lets me change the rotation speed based on how fast the character is moving (i.e. running or walking). It also lets me have conditional behaviors based on the magnitude of required rotation. For example, if the character needs to turn 180deg, then I can have them turn before starting to move, which also looks a bit more natural.

There’s definitely a lot more tweaking to be done, but I’m happy with the results for now.

Changing the player character

One feature I wanted to have was for the player to swap out what character they’re controlling. Like all JRPGs I want party composition to be an important element of gameplay. This fortunately wasn’t too complicated, in part because of the character generation workflow I set up earlier (always nice when that work pays off!). Because all characters share essentially the same skeleton, I just need to swap to the new character’s skeleton (and model) and update a couple other parameters (how tall they are, etc).

The one tricky bit was rebinding the new skeleton to a plugin I’m using that controls where the character is looking. There’s no built-in way to do this rebinding. Fortunately after digging through the source code I managed to find a solution:

public void RefreshLooker() {
    // NOTE this assumes the path is always the same,
    // which it might be due to `clay`
    _looker.LeadBone = transform.Find("Model/Armature/Root/pelvis/spine_01/spine_02/spine_03/neck_01/head");
    _looker.RefreshLookBones();
    _looker.InitializeBaseVariables();
}

I did have to change RefreshLookBones from an internal to a public method though, so it’s not ideal.

Inventory UI

The biggest feature is the inventory UI, which is an RE-style spatial inventory system. I like it better than other inventory constraints, like a weight limit, though it can still be tedious to manage (need to find some good C# bin packing libraries to implement an auto-sort).

This was my first time making a substantial UI in Unity, using their UIElements system. It felt very weird to use what are essentially HTML and CSS (UXML and USS, respectively), and it took me some time to figure out how exactly I should structure things. The system needs some React or Solid-like framework to round it out.

My current approach is to avoid the UXML as much as possible and build the UI in C# instead (essentially creating “components” by inheriting VisualElement). Hopefully the UI demands remain simple enough that this remains viable.

The C# approach made it easier to design the inventory UI as a modal interface, i.e. different inputs change the UI’s mode, which makes it easier to ensure that there are less invalid states. The normal mode is just moving the inventory cursor around and entering other modes. For example, pressing the “move” button over an item in normal mode changes the mode to Moving, which then remaps inputs for that particular context (e.g. what was once the “use item” button is now the “place item” button). This feels like the cleanest, most extensible approach, but goodness does UI always take way more code than I anticipate.

I’m not going to include a screenshot of the inventory UI because it looks terrible, but it is passing all its tests!

Dice System

I played Citizen Sleeper and enjoyed it. The dice system there is great, and makes much more sense for the type of game I’m working on than the ones I mentioned in the last update. I’m definitely going to riff off of it for Fugue’s skill check system.

Other bits

• I also implemented a proper scene manager, which handles changing scenes and loading in the player character. Took longer than I expected, but seems to work well enough.
• I changed the verses script deserialization from JSON to YAML. The JSON deserialization code was a mess, and possibly kind of slow? I found a fast YAML library that vastly simplifies the script importer. YAML’s more pleasant to look at too.

Next steps

Two main things are on my mind:

I want to start implementing the dice/skill check system so I can finally test it. It’s the game’s core mechanic, so I need to make sure it makes sense before running too far ahead.

I want to use the Unity editor as little as possible. It’s kind of janky and just slow for me to work in. I don’t want to learn all of its shortcuts and wait for it to respond. The way I see it there are three primary activities I need the editor for:

1. Compiling C# and running tests
2. Building scenes
3. Building/testing the game

There may be a way around 1) but I have a feeling it would be very painful to set up, and perhaps not much of an improvement in terms of speeding up the development loop. There’s no way around 3), but it’s a relatively infrequent activity at this stage so I’m not worried about it.

I expect 2) will be where most of my Unity editor time will be after I’ve implemented the game’s core features. It’s unpleasant, especially in contrast to 1) where I’m mostly writing in my comfortable text editor and hopping to the editor only to compile and run tests. I would love to have something equivalent to my text editor but for building Unity scenes.

Fortunately this might actually be possible. Unity’s scenes are really just specialized YAML files, and I’ve already built part of a Rust parser for them in verses. In theory you could set up an external program to edit those YAML files. I’d need to be careful in scoping the tool, however. I don’t want to end up re-implementing a huge part of Unity’s editor.

For version 1 of umarell, which is what I’d call this tool, I’d probably just want to modify transform information (position, rotation, scale). I’d still have to import objects into the scene in Unity first (which is a relatively infrequent activity), to ensure that they’re properly initialized with all the right properties and what not. Then I’d pop over into umarell and position things.

I imagine the scene building workflow would be something like:

1. Build the static set elements in Blender.
2. Create a new scene in Unity and drop in all the objects that should be in the scene, and perhaps also attach any scripts/components they need.
3. Open the scene in umarell and start setting all the objects up.

I looked a bit into what the Rust ecosystem for something like is like, and this might be possible with rend3 and egui. It’s not a high priority at the moment–I need to finish getting all the systems down–but could be a fun project later.

I’ve been slowly chipping away at porting things to Unity, sketching out new game systems, and tweaking the verses script syntax.

verses Updates

For verses I’m constantly changing things to try and get the most concise syntax I can, without it becoming too hard to skim.

Two of the biggest changes are:

• Multiline remarks, which make it easier to write consecutive lines that are said by the same actor.
• Allowing conditionals to be inlined into a verse. Previously any conditional behavior had to be defined as a branching action, which would have to connect to a different verse entirely.

For the second change, the way it worked before was clunky. Say I want to have a character say something if foo==bar and then return back to the normal dialogue.

@root
Branch:
  - @next_verse
    If: foo==bar
  - @default_verse

@next_verse
[Character] Foo equals bar!?
Branch:
  - @default_verse

@default_verse
[Character] Continuing the convo...

This is a lot for what is essentially just a minor aside. The changed syntax introduces a new action, Aside, which groups actions under a condition, and then resumes the normal flow. So that script above would now be written:

@root
If: foo==bar
  [Character] Foo equals bar!?
[Character] Continuing the convo...

There are some other more minor improvements, like immediately specifying dialogue choice consequences/outcomes. Previously this would also be clunky, involving some variable being set by the script runner and then checking for this variable later on. Really roundabout and relies on things happening that aren’t defined in the script itself. Now it’s just a matter of:

[Character] Make a choice
  - A choice
    Outcomes:
      AddItem: SomeItem

The other change is better variable namespacing. Previously there was just “local” (local to the script) and “global” (set across the entire game). The scopes are now “script” (local to the script), “story” (local to the story the script is a part of, e.g. within the context of a mission), and “global”. This will hopefully make managing script variables easier and cleaner.

The last major verses update is that I actually got Unity scene parsing working well in Rust, and ended up stripping out Godot scene parsing entirely (not worth maintaining both).

Semi-related, I also switched entirely from vim to nvim/neovim and set up custom highlighting and in-editor parsing/validation for the game scripts:

Unity Updates

The Unity version of the game is more or less at parity with the Godot version now. I’ve also implemented a very rough first draft for several other game systems (time, missions, inventory, etc). For inventory I’m tentatively using an RE-style spatial inventory, since it feels like a better way to limit inventory than weight/encumbrance, and is visually more interesting to manage. Still need to build out the UI for it though.

Because I’m expecting development to take a long time I switched over away from the LTS version of Unity to the 2022 version. So far it’s been a better editor experience–less laggy, though the compilation loop still sucks.

In general my experience with Unity has been very positive. I’ve been able to structure my code in a better and more reliable way than with Godot. The renowned Scriptable Object talk was extremely helpful in designing an architecture that feels easy to build on. Unity’s component-based system is much nicer for decoupling, and having access to a more mature programming language is worth a lot (C# features like interfaces are very handy).

I’m still getting used to how Unity does things, like addressables and what not. There is a lot that’s clearly meant for very advanced, high-end games, and it’s hard for me to discern what’s overkill for me to adopt now and what I’ll regret not using further in the development process. But that’s just how these things go.

In general my focus now is on lowish-level framework stuff (currently trying to get character movement to work correctly). There are still many undetermined mechanics that will require playing around with, so trying to design things to flexibly accommodate different possibilities, or at least get a good foundation to experiment on. At some point soon I’ll cross over into actual gameplay and eventually writing (I hope).

For now the current near-term roadmap is:

• Get character movement working well
• Start building out more UI

Dice mechanics

I haven’t thought much on mechanics as I’ve been in implementation-mode for the past several weeks, but I have some more thoughts on missions, the energy mechanic (a stand-in for some kind of resource management mechanic). I find myself coming back to the Mario Party character-dice system and the system from Dicey Dungeons where you assign dice to slots. I like these better than regular D&D-style dice rolls because there’s some randomness but enough room to strategize so that you have more agency over outcomes.

While searching for the images above I came across a game called Slice & Dice where you construct the dice yourself!

Migrating to Unity

So the last post was about migrating to Godot 4 and now I’ve gone and started migrating to Unity. At the start of this project I was deciding between Unity and Godot and ended up going with Godot for a few reasons, a big one being that I had used Unity maybe 8 or so years ago (I was originally developing The Founder in Unity) and hated it. At the time I believe Unity’s UI support was basically non-existent and the game was rather UI-heavy, so it was a frustrating experience. I think the Unity Editor for Linux was also in beta, though I was probably using OSX at that point.

But recently I figured Unity was worth another look, since this is a 3D game and less UI-heavy, and surely things have improved in the interim years. And they have. The Unity Editor for Linux works (more on this below) and the Unity ecosystem is of course more mature and battle-tested.

Some other considerations:

• Console/closed-platform support. A big downside with open-source game engines like Godot and Bevy is that they can’t yet build for consoles, in part because these platforms require their integrations be closed-source (or something along those lines).
• Built-in animation retargeting. It’s been a struggle to figure out how to retarget animations for my generated characters. I was using the Auto-Rig Pro addon for Blender, which works great but wasn’t giving me the right root motion results in Godot. It seemed very complicated to debug. Unity, on the other hand, has a built-in animation retargeting for humanoid rigs that works great.
• C# is a more mature and strongly-typed language. Godot 4 improved a lot with GDScript but ultimately it still feels too in-development. It has typing support but many important types are lacking and it just doesn’t feel as solid as a proper strongly-typed language. C# isn’t the prettiest language to work with, but it feels like a good, stable foundation for a game.

And a couple other bonuses:

• I haven’t done much UI work just yet but Unity’s new UI Toolkit system is interesting…the styling and layout is basically CSS.
• Package management is also nice with openupm.

So far I don’t have much bad to say about my new Unity experience. The biggest issue is that the Editor still feels kind of janky in Linux; it’s rather slow and a bit buggy. It might still technically be in beta. I can’t, for example, drag and dock tabs into panels, which is annoying (see below for a workaround). Not sure if this is a limitation with my window manager or what. Godot’s Linux support on the other hand is amazing; their editor feels responsive and stable.

Setting up my text editor was tricky but it’s working alright now, except that I have to restart nvim to properly process new files (see this issue) and that the language server takes a long time to start.

In general the development loop feels slower than Godot, largely due to the increased compilation times. There are some ways to improve these times (mainly by essentially bundling your code into sub-packages with assembly definition files), but so far I haven’t noticed a major improvement (though it’s probably not apparent until your project gets quite big). It’s not the worst thing but it does make development drag a bit.

And a very minor gripe is the number of artifacts that Unity produces. Tons of .csproj files and other folders. Godot was really lean in this regard; I believe all these generated artifacts were confined to a hidden .import folder.

(I’m also stubbornly not doing the C# new-line curly brace thing)

I’m still getting familiar with most of Unity’s core concepts–how input handling works, how unit testing works, etc–and so far haven’t encountered any major road blocks. The documentation is ok, but I’ve still had to do a bunch of forum digging to figure out exact approaches to some problems.

I suppose one bit of weirdness is that there are two different UI systems available; one seems more appropriate for more static/simpler UIs (this is the CSS-like UI Toolkit system) and the other (soon-to-be legacy? idk) is a canvas-based UI system (closer to how UI works in Godot). The latter seems more appropriate for more dynamic interface elements–I’m using them for dialogue boxes primarily because they can use TextMeshPro which gives fine-grained control over text meshes. I think TextMeshPro is supposed to be integrated into the UI Toolkit? Maybe it is already? I don’t know.

I am a little sad to stop using Godot…it’s great and I’m excited to see where it goes in the next several years. It’s already amazing how full-featured it is–perhaps it could become a Blender equivalent for game development. If Unity and Godot were at closer feature parity (especially with the three benefits listed above) I’d prefer Godot. But for now Unity makes more sense.

clay

I ported the character generation system from hundun into its own Rust package, clay, so I can generate characters independently of hundun and via, for example, a script in bulk.

Not much interesting to say here, except maybe on how I bundled static assets into the Rust binary. The character generation part relies on several Blender Python scripts that I need to know the paths for (so I can call blender /path/to/the/python/script.py). The trouble is these files could be anywhere, and I don’t want to hardcode or constantly pass in the paths to these scripts.

What I do is package them (and other static assets needed, like brush images) with the compiled application using the include_dir crate. Then they can be extracted to a known location when needed:

/// An interface to run scripts with Blender.
/// Most of the character generation actually happens
/// in Blender with Python scripts. These Python scripts
/// are included in a kind of hacky way (see below).
/// This method means that if the Python scripts are edited
/// the Rust program needs to be re-compiled to include the
/// latest script versions.

use include_dir::{include_dir, Dir};
use std::{
    io::Error,
    path::{Path, PathBuf},
    fs::{create_dir, remove_dir_all},
    process::{Command, Stdio, ExitStatus}
};

const BLENDER_PATH: &str = "/usr/local/bin/blender";

// Where to extract the included Blender scripts when calling
// the `blender` command.
const EXTRACT_SCRIPTS_PATH: &str = "/tmp/clay-blender";

// Bundle the Blender scripts with the Rust binary.
// When needed we'll extract the scripts somewhere we can point to.
// This is a kind of hacky way to avoid juggling filepaths if this
// library is used elsewhere.
static BLENDER_SCRIPTS_DIR: Dir<'_> = include_dir!("$CARGO_MANIFEST_DIR/assets/blender");

/// Path to a file included in `BLENDER_SCRIPTS_DIR`.
pub fn bundled_file(path: &str) -> PathBuf {
    format!("{}/{}", EXTRACT_SCRIPTS_PATH, path).into()
}

/// Note: this assumes that `script` is bundled as parst of `BLENDER_SCRIPTS_DIR`.
pub fn blender(blendfile_path: &PathBuf, script: &str, env_vars: Vec<(&str, &str)>)
    -> Result<ExitStatus, Error> {
    // Extract the Blender scripts
    if Path::new(EXTRACT_SCRIPTS_PATH).is_dir() {
        let _ = remove_dir_all(EXTRACT_SCRIPTS_PATH);
    }
    create_dir(EXTRACT_SCRIPTS_PATH).unwrap();
    BLENDER_SCRIPTS_DIR.extract(EXTRACT_SCRIPTS_PATH).unwrap();

    let mut cmd = Command::new(BLENDER_PATH);
    cmd.args([
             "-b", &blendfile_path.to_string_lossy(),
             "--python", &bundled_file(script).to_string_lossy()
    ]);

    for (key, val) in env_vars {
        cmd.env(key, val);
    }
    let mut proc = cmd.stdout(Stdio::inherit())
        .stderr(Stdio::inherit())
        .spawn()
        .expect("blender command failed to start");

    let res = proc.wait();

    // Clean up extracted files
    let _ = remove_dir_all(EXTRACT_SCRIPTS_PATH);

    res
}

A very hacky way of setting the editor layout in Linux

I did manage to figure out a very hacky way of docking tabs in the end. You can export the current editor layout to a .wlt file, which is essentially just a YAML file (as far as I can tell, it’s the exact same format that Unity game scenes use). So say for example I want to dock the Test Runner to be in the same dock as the Inspector. I’d open the Test Runner–which opens in a new window by default–and then save the layout. Then I’d edit the .wlt file. The YAML file consists of individual subdocuments, separated like so:

--- !u!114 &1
(some yaml)
--- !u!114 &2
(some more yaml)
--- !u!114 &3
(etc)

The key here is these dividers preceded by ---. The number after & is the id (specifically, the fileID) of that component. Some of these represent entire windows (so in my case I’d have a main editor window and the smaller window spawned for the Test Runner), others represent docks (which I identify by their m_Panes property), and others represent the tabs themselves.

The gist is to look for the Test Runner tab (by searching for m_Text: Test Runner) and then getting the id of that component (say it’s 15). Then I look for the Inspector tab (searching for m_Text: Inspector) and get that id (say it’s 20). Then I look for a subdocument where {fileID: 20} is an item under m_Panes. That will be where the Inspector tab is currently docked. I just add another entry below it: {fileID: 15}.

I search for the document to other references of {fileID: 15} and then clear anything that references those parent ids, recursively, just to ensure that there aren’t multiple elements referring to the same component (i.e. deleting the dock that used to contain the Test Runner, and deleting the window that used to contain that dock).

Then save and load the layout in the editor.

Specifically for Linux. I’m using Ubuntu 22.04 and Unity 2021.3.22f1, with nvim-lspconfig.

This was a pretty heinous setup process, so I’m documenting this here to help others who want something similar. The actual setup process isn’t too bad, but figuring out all the individual steps was difficult. I came across a few guides but some were out of date, or were geared towards Windows, or just didn’t work for me.

I should also caveat that I’m writing this shortly after I got this setup working, so I may run into other issues with it once I start developing with it more. For example, there might be issues with new files. If I run into those problems and manage to solve them I’ll update this post.

Requirements:

• omnisharp-roslyn, I’m using v1.39.6.
• mono; but crucially not the one from the default package repos. You need to use their official repo; then install: apt install mono-devel mono-complete.
  • The default Ubuntu package repo version doesn’t include MSBuild so unless you use the official mono repo you’ll get errors like “Could not locate MSBuild instance to register with OmniSharp.” when running omnisharp-roslyn.

The process:

1. Download a release of omnisharp-roslyn (as mentioned above, I’m using v1.39.6). Extract it somewhere–for me, this was /opt/omnisharp-roslyn. The run file in that directory is what will start the LSP server.
2. Configure nvim. For me this all goes in ~/.vim/plugin/nvim-lsp.vim, but you can change that to match your own preference. I’m including the whole file but the key parts are what follows -- Omnisharp/C#/Unity. You must specify the path to the omnisharp-roslyn run script.

lua << EOF
local nvim_lsp = require('lspconfig')

-- Use an on_attach function to only map the following keys
-- after the language server attaches to the current buffer
local on_attach = function(client, bufnr)
  local function buf_set_keymap(...) vim.api.nvim_buf_set_keymap(bufnr, ...) end
  local function buf_set_option(...) vim.api.nvim_buf_set_option(bufnr, ...) end

  -- Omnicompletion
  buf_set_option('omnifunc', 'v:lua.vim.lsp.omnifunc')

  local opts = { noremap=true, silent=true }
  buf_set_keymap('n', 'gD', '<cmd>lua vim.lsp.buf.declaration()<CR>', opts)
  buf_set_keymap('n', 'gd', '<cmd>lua vim.lsp.buf.definition()<CR>', opts)
  buf_set_keymap('n', 'gi', '<cmd>lua vim.lsp.buf.implementation()<CR>', opts)
  buf_set_keymap('n', 'K', '<cmd>lua vim.lsp.buf.hover()<CR>', opts)
  buf_set_keymap('n', '<C-k>', '<cmd>lua vim.lsp.buf.signature_help()<CR>', opts)
  buf_set_keymap('n', '[d', '<cmd>lua vim.diagnostic.goto_prev()<CR>', opts)
  buf_set_keymap('n', ']d', '<cmd>lua vim.diagnostic.goto_next()<CR>', opts)
  buf_set_keymap('n', 'gR', '<cmd>lua vim.lsp.buf.references()<CR>', opts)
end

-- Omnisharp/C#/Unity
local pid = vim.fn.getpid()
local omnisharp_bin = "/opt/omnisharp-roslyn/run"
require'lspconfig'.omnisharp.setup{
    on_attach = on_attach,
    flags = {
      debounce_text_changes = 150,
    },
    cmd = { omnisharp_bin, "--languageserver" , "--hostPID", tostring(pid) };
}
EOF

3. Then we need to generate the .sln and .csproj files for our Unity project. There are two ways to do this:

   • Almost every other guide to this setup says you need to install Visual Studio Code to do this. This then requires that you go into your Unity project, go to Edit > Preferences > External Tools, then set Visual Studio Code to be your External Script Editor. Finally, check all the boxes for Generate .csproj files for:, then press Regenerate project files. This will work, and it’s what I tried first.
   • The alternative, which I found here, is to run /opt/Unity/2021.3.22f1/Editor/Unity -batchmode -nographics -logFile - -executeMethod UnityEditor.SyncVS.SyncSolution -projectPath . -quit in your project root folder (note that /opt/Unity/2021.3.22f1/Editor/Unity is just where I installed the Unity Editor, so change that to point to your location). This will also work, and appears to work without installing VSCode. The only annoying bit is that a project can only be opened by one instance of the editor at a time, so this won’t work if you already have your project open. Perhaps there’s a way to call it from within Unity?

4. Finally, one issue I had was that for some reason Assembly-CSharp.csproj wasn’t generated. This led to the Unity framework not being picked up by omnisharp, with errors like “The type or namespace name ‘UnityEngine’ could not be found”. This solved itself by creating a new C# file, e.g. Assets/Foo.cs and then refreshing the project files in the Unity Editor (which I guess causes Unity to compile the scripts and then generate this missing file). No idea if this is necessary though.

Lastly, the LSP server is slow to start up on my machine. Something like 30 seconds. For Rust and TypeScript development I use this plugin, which gives me progress on those LSP servers’ startups, but unfortunately it doesn’t yet work for omnisharp.

NB: Just a couple debugging tips if this doesn’t totally work for you: :LspInfo and :LspLog from within nvim can help you figure out what might be going wrong.

Gliss, and Migrating to Godot 4

Godot 4 was recently released and brings with it many improvements to GDScript (thankfully addressing most of my pain points with the language), better performance, and several other changes that I can’t yet appreciate. Because the game code is still pretty basic I figured it’d be worthwhile to just migrate to Godot 4 now. It ended up being a good opportunity to refactor some things now that I have a clearer idea of what systems need to be included.

Semi-related to this refactor: I’ve decided to first work on a smaller demo/prototype called Gliss (for glissando) to test out mechanic ideas and the overall development process. The hope is to flesh out all the game designs and systems and then Fugue will just a bigger version with few, if any, new systems & mechanics. My ideal outcome is that Gliss establishes the framework, and the expanding it into Fugue is mostly a matter of authoring more content–characters, locations, etc.

Overhauling the Sequence Editor (verses)

As I was starting to write out more sequence scripts I found the existing editor (below) to be clunky. And when I need to define new script actions (such as skill checks, more on that below), it requires a lot of lift to define the new frontend components and inputs. Just super unwieldy.

I revisited an idea which was to define a special plain-text format for sequence scripts. I never pursued it because I was daunted by the prospect of writing my own custom parser…but I had to do that anyway to parse Godot’s .tscn files, so what’s one more parser?

The process of writing the parser with nom actually wasn’t too bad. The trickiest/most frustrating bits were dealing with error handling (just haven’t fully grokked error handling in Rust in general) and handling recursive parsing (couldn’t figure out a good approach for that and ended up just putting a fixed limit on recursion depth). But otherwise once you get a handle on the combinators (especially with their indispensable guide) the whole effort becomes intuitive.

I also implemented script validation which checks for common issues like requesting nodes or entities that don’t exist in a given scene, or referencing files that don’t exist, or bad sequence script structure (orphaned nodes, invalid branching, etc), and even typos. The goal is to have some assurance that there will be minimal runtime sequence errors.

The end result is verses, a custom Rust crate/program for parsing and validating gliss/fugue sequence script files. This program can be used to parse scripts to JSON (to import them into Godot as custom resources), and the previous hundun sequence editor (pictured above) is now a relatively thin UI on top of it:

Now the script is just written in the editor on the left and the parsed sequence graph is displayed on the right. Validation happens live. Now the process of writing sequence scripts is less stop-and-go, more fluid than before. It also means that if I need to quickly edit a script file I can do it easily with a text editor.

The text editor itself is made with CodeMirror, which is an intimidatingly powerful editor library. Here I’ve set it up to have custom syntax highlighting and custom autocomplete, which lets me autocomplete, for example, actor names.

The Skill Check System

I began working out the skill check system–the raw skill check mechanic itself is very straightforward, just compare the skill level and difficulty and roll to see if you succeed. I designed the actual rolling algorithm to be visualizable, so you’re not just seeing your odds and then the result. Instead, a rough summary is that the skill difficulty sets a number of successful flips you have to achieve, and your skill level determines how many tries you have. So for a skill level of 3 you get 3 tries. Each try lasts until it fails, so it is possible to succeed at a challenge of difficulty 4, even with just a skill level of 3. The probability of a successful flip is tuned to produce the following overall skill check success probabilities (i.e. each flip is not 50/50):

This chart is kind of confusing, but each line represents a different skill level. For example, say the skill is “Lockpicking”. Say your skill level at that is 3 (s=3). You have about a 100% chance of succeeding at any skill check with a difficulty less than 3. You have a very good chance for difficulty of 3 and about a 60% chance for a difficulty of 4.

I’m hoping that the modifiers will be where this system gets more interesting. The modifiers themselves are just straightforward increases/decreases to skill levels, but I want them to be organized in a way that 1) requires interesting character build decisions (through skill progression) and 2) reflects a character’s beliefs about and experiences in the world (that is, characters don’t just mindlessly/mechanically get better at lockpicking; rather, as they get better it changes how they see the world; and how they see the world affects their proficiencies in certain skills).

I need to think more on 1), but the general idea is that each skill has one or two underlying “proficiencies” that are shared with other skills. For example the two proficiencies for “Lockpicking” might be “Hand-Eye Coordination” (which also underlies the “Medical” skill) and “Puzzle-Breaking” (which also underlies the “Digital Evasion” skill). At the 3rd and 5th skill levels you can pick which of the two proficiencies to take a bonus in (a more expansive build), or you can pick a perk (e.g. you get one free lockpicking attempt without using a lockpick, for a more specialized build). This isn’t all that different from typical skill systems.

Whereas 1) are intentional decisions the player makes, 2) reflects playstyle patterns, and so is more indirect. If a character frequently uses intimidation actions, or frequently witnesses them, they may pick up an “insight” of “Violence is the Answer”, which gives bonuses to violence-related skill checks and penalizes non-violent ones. If they are constantly lockpicking and hacking, they may pick up the “Security is a Fiction” insight, which buffs these skills, but the anxiety of this realization means they stress more easily (which connects to the Energy mechanic, which I’m still working out).

Refactoring chargen into clay

What I’m working on now is refactoring the character generation system (formerly chargen) into a separate crate called clay. This is to also streamline some things as with verses, e.g. make it easier to quickly edit characters and bulk generate large amounts of them. hundun will again be mostly just the UI on top and not handle the actual character generation logic.

Next steps

• Finish porting clay
• Figure out the character export/import into Godot workflow (running into some root motion issues here)
• Re-implement sequence script importing using verses
• Implement skill check mechanic for testing
• Continue developing the core mechanics (e.g. the Energy mechanic)
• Probably some other stuff I’m not remembering

Food in Mexico, Junji Ito’s new show, apocalyptic 90’s violence, and radioactive whetstones.

Food in Mexico

Kira and I just returned from Mexico (Oaxaca and Mexico City) for a wedding (which was wonderful) and I got to try many foods I’ve never had before. Some of the highlights:

Nanche but only in raspados (a kind of shaved ice) form, where the fruits themselves were suspended in a kind of syrup. The syrup tasted overwhelmingly like butterscotch candies, not so much like Werthers but like the kinds that would come in unlabeled gold reflective foil and be more of a translucent yellow than a solid creamy brown. The fruit itself had some of that butterscotch taste but also tasted a bit like an Asian pear mixed with haw.

Tepache is now fairly common as a canned beverage throughout bodegas throughout New York. I’ve only ever had it from a can. This version here had a much stronger molasses taste, and more of a pungent fermented flavor. It might have been a little alcoholic, which is probably absent in the canned versions.

Tejate, which is a maize-cacao concoction that also includes pixtle, which I’d never heard of before. Had a nice foamy top and was very easy to drink.

I have no idea what this herb is, but it was a garnish on top of a lime soda I’d ordered. It was bitter and tasted like a combination of artificial chocolate scent (scratch-and-sniff chocolate), artificial peach (like the Haribo gummies), and bubble gum (but like those small Bazooka bubble gums when the flavor is almost all gone and it’s getting difficult to chew). It was interesting at first but quickly became too overwhelming.

Another herb was pápalo, which came on top of a cemita we had. I don’t remember much about how it tasted except that it was kind of like cilantro, but stronger. Separately I also had chepiche as a garnish for a tlayuda, which tasted like an even stronger version of cilantro.

I didn’t get to try “Mexican yam” but they are unique-looking plants. This one kind of looks like a katamari. It’s hard to discern its silhouette against the backdrop of rocks so I outlined it.

Lastly, Kira signed us up for an amazing cooking class where we learned to make this delicious mole coloradito (among other things).

Junji Ito Maniac: Tales of the Macabre

I love Junji Ito’s work but this new anthology series missed the mark. Part of the horror of his work is his grievously detailed illustrations and how he frames the key moment of climactic terror. You’re drawn to linger on the page and absorb each stroke of his line work. But you can’t really linger in an anime, and so the impact of each horrific turn is totally dampened. And the stories often end very abruptly. I didn’t end up finishing it.

The only adaptation of Junji Ito’s that I’ve liked is World of Horror, which I talked about back in 2021. Considering this show, it might be because that game more closely emulates Ito’s style.

Apocalyptic Violence in the 90’s

Listening to Mother Country Radicals made me curious about the similar patchwork of radical activity in the 90s. I vaguely remember hearing about the Oklahoma City Bombing and references to Waco growing up but not really learning much about it. This 1999 masters thesis, “Political Violence in the United States: Apocalyptic Typologies of Left and Right Wing Political Groups and Their Violence through the Period 1990-1997” (by Gordon Daniel Green), gives an overview of the period from the framework of “apocalyptic” (aka millenarian) movements–i.e. movements that foresaw an impending major shift around which urgent action needed to happen. In particular the imminence of the shift called for violent action (as opposed to e.g. mass political campaigns), and these actions were carried out in small groups. On the left this typology includes animal rights groups (the Animal Liberation Front), environmental groups (Earth Liberation Front, Earth First!), and anti-industrial/technology groups (Deep Ecology, the Unabomber); on the right this includes militia/patriot groups, the Rescue Movement (anti-abortion activism, the Army of God), and end-times religious groups (the Branch Davidians).

These groups tend to see the world as Manichean (good vs evil) and zero-sum (any win for the other side is necessary a loss for their side; i.e. there can be no mutual gains or victories or compromise), and motivated not by personal gain but by “a higher cause”.

The author leaves out groups focused on racism and/or anti-Semitism because they “differ from the above groups in the desired outcomes of their actions…although believing in a coming apocalyptic Race War…[they] do not seek to change the beliefs, ideology, or actions of the rest of society. They seek only to change those of white or ‘Aryan’ members of society. … A racist cannot change a black man to white, nor change a Jew into a Gentile”, though of course there’s overlap between the beliefs of these groups and the ones analyzed here.

I was struck by how these movements reflect some of the movements of the present: conspiracies around “internationalists” (“globalists” being the preferred term today), panic around “federal tyranny” and gun restrictions, paranoia around secret world governments trying to bring in the antichrist (QAnon-like), false flags (belief that the Oklahoma City Bombing was “the equivalent of the Reichstag fire that brought Hitler to power. The government is often seen as either having prior knowledge of the bombing…or direct involvement”), and so on. And of course many if not all of the same environmental and animal welfare concerns remain. Though I can’t remember seeing any major millenarian cults with apocalyptic predictions lately. And the way these groups are organized probably have changed a lot, but it reading this it feels like those changes are of a degree than of a kind (e.g. how quickly and far-reaching something like QAnon can be with social media). One major difference may be the increase in violence motivated not by a higher cause but by feeling personally slighted (male lone-wolf shooters)…though they are connected in a way, and maybe this is the extreme end of the “small group” organizational form (with a group size of one).

Graphical/node-based programming prototype

I sketched out a node-based programming tool (not the first time I’ve tried this…), in part to implement a feature I always felt was lacking in other visual programming tools. This feature is inspired by the various Firefox Vim addons (can’t remember exactly where I first encountered the feature, it was either Pentadactyl, Vimperator, or Vimium).

In the screenshot above I have a port selected (“International navigation (memo item)”, highlighted in yellow). When a port is selected, candidate input ports are assigned a sequence of keystrokes to immediately connect the two ports. The keys are limited to asdf so you don’t have to move your hand from the home row to specify the target. Saves a lot of time of dragging your mouse and trying to target the correct port.

Radioactive whetstones

I was looking to pick up a new whetstone and found a couple reviewers notice that they were radioactive??

It sounds like it’s not something to worry about (and normal):

• Danger of Radiation from Whetstones
• Comparing whetstone’s when I stumbled across this

but I can’t find any more authoritative sources.

Chargen fixes and improvements

I’ve had to fix a few more issues with the character generation system, but also added some improvements.

One of the main problems was with the poly reduction (the Decimate modifier), which can make some interactions between clothing and the character body unpredictable. Basically when clothes is close to the body and you have the character in different animations/poses, sometimes the underlying body can clip through the clothes. If the clothes conform to the body’s geometry this is less of an issue because you won’t, for example, have pointy geometry that can poke through more rounded geometry.

The way MakeClothes handles this is with its delete groups, where you basically flag specific vertices to be deleted when wearing a particular clothing item. The underlying vertices are redundant because they’re covered by the clothes, and no vertices there means no clipping.

The Decimate modifier complicates this in two ways:

1. It may modify the body and clothing geometry in different ways, such that two surfaces that conformed are now shaped differently, so now the surface underneath may poke out through the surface above (consider that a smooth rounded surface has a higher poly count than a pointy pyramid-like surface; decimate converts these rounded surfaces into pointier ones.
2. It may modify the geometry such that delete groups end up deleting too much of the mesh.

So far these haven’t caused major problems; I’ve been able to solve them by adjusting delete groups or by slightly scaling up clothing/hair so there’s margin for pointy surfaces to stay concealed.

Aside from this I made a few minor quality-of-life improvements. I added a “Quick Preview” option which only converts the first frame of each character animation, cutting down character generation time by 30 seconds or so (really useful when tweaking things). I also added a way to generate clothing previews from within Blender and see them from within the character generator:

I also added a way to colorize clothing. Solid-colored clothes are made using a base forest green texture, (#1e3b2f), generated using the texture painting algorithm I described a few posts back:

In the UI you can pick a color and then that forest green is replaced in that texture:

This lets me squeeze a bit more mileage out of clothes meshes without needing to make different textures for every solid color I might want to use. The system is flexible enough that I can create more complicated textures that also use this system to re-colorize them, so long as they’re based around that same forest green.

And I finally figured out why my webpack builds were taking so long. They took anywhere from 5-10min for the initial build, now it’s just a few seconds¹.

Screen shader tweaking

I’m trying to settle on screen shader settings for the game. There is already some downsampling happening here (well, the max resolution is set to 720p, so pixel cruchiness is more visible when rendering to higher resolutions than that), and I also had the color depth reduced to 4 for awhile. It’s a nice visual effect but might get fatiguing, so I played around with some other settings.

Common settings across these renders: resolution=1, saturation=1.1, dithering=false, brightness=1.

(I did do some comparisons with dithering=true but this didn’t make much of a difference except introducing some artifacts at lower resolutions.)

I think I’ll probably stick to depth=6 (since it still introduces some interesting banding on the wall, but not too aggressive like it is with a color depth of 4) and contrast=1.2 for now. It may change as more environments and characters are created.

Test characters

I started making more use out of the character generation system to see if anything else needs improving. Part of this trial run required creating more clothing and hairstyles to try out. These characters are all randomly generated (except for the clothing/hair, which are added afterwards):

Next steps

I’ll likely leave the character generation system here for now, though I’ll need to make a lot more clothes and hairstyles eventually. The priority now is to figure out the story and game mechanics since those influence everything else moving forward.

At the end of the last post I mentioned some optimization considerations for Fugue’s character models. I managed to reduce the poly count of the human model a bit, but my quick-and-dirty approach (using Blender’s Decimate) messed up the mesh’s structure too much and caused some issues, so I’ll have to try a more manual method [Update: see the end of this post].

The other bit of optimization I started working on was reducing draw calls. A fully dressed character model has several different materials: one for the human (the skin texture) and then one for each item of clothing. I couldn’t find any clear information about how Godot handles draw calls and materials but generally more materials means more draw calls. So if I can merge the skin and clothes materials I should be able to reduce draw calls to as low as one per character.

I thought there’d be a Blender addon that does this but I surprisingly couldn’t find any. There are texture atlas baking addons which isn’t quite what I’m looking for–they assume that your UV maps neatly laid out, without overlaps, and they’re usually for baking environmental influence (e.g. lighting/shadows) into the texture (I just need to cut and rearrange the original texture data). Merging materials in the way I have in mind is tricky because each mesh has its own UV map and when you merge the meshes these UV maps all overlap:

(You can also see that there are some scaling issues, where the pants UV layout is now stretched–the original pants texture has a non-square aspect ratio.)

To solve this you need to scale/move the UV maps so they each have their own space, which then requires you to also move the underlying parts of their original textures to where their UV map regions were moved to so that the correct image data is still mapped to the correct faces.

In theory this isn’t too difficult–there are some tricky parts like placing the UV map regions (a concave bin packing problem) but everything else is straightforward if you have the UV map geometry. But actually implementing this was really rough–probably my most frustrating development experience in recent memory. I’m not even finished yet, so crossing my fingers that nothing else comes up. This is one of those problems that required many different approaches, and it’s possible that I’ll need to pull ideas from old abandoned ideas if I run into new obstacles…so I’ll try to document my thinking and attempts here.

The biggest struggle was with Blender itself. Blender is an amazing tool and its scripting capabilites are awesome, but they feel like an afterthought, especially for background mode/headless scripting. It has so many puzzling design decisions, many useful functions don’t have a Python API, and an inconsistent dependency on the UI. There are functions that behave differently depending on what the active window/area is or if you’re in EDIT mode vs OBJECT mode, and if you aren’t running the UI (i.e. running Blender in background mode) some of these functions don’t work at all. So of course the entire script I developed within the UI environment didn’t end up working when I switched to the background mode 🙃.

There are also some strange instabilities and limitations of headless Blender, like I can’t export UV layouts because “the GPU can’t be used in background mode”, but I’m able to render 3d scenes without any problem. I’m sure there’s a good reason. Similarly, for some reason loading JPGs and converting them to RGBA with Pillow segfaults from within the Blender environment but works just fine using Blender’s Python when not running Blender. Fortunately these problems always had a hacky workaround.

There also just isn’t a lot of information about scripting for headless Blender so it was difficult to find answers or discussion when I did run into problems.

The other difficult part was getting the UV map data I needed–in particular the geometry of each individual UV map region. What I’m calling a “UV map region” here isn’t quite the same as a “UV island”. Islands are sets of connected UVs and they can overlap. In the maps below the islands are outlined in blue:

In my case I care about contiguous regions of UV map, whether or not they are connected. So the UV map on the right with all the overlapping islands would be considered a single UV map region, and each island in the UV map on the left is also its own UV map region.

To identify these regions I needed two bits of information:

1. The polygon boundaries of each island
2. A mapping of UVs to the island they belong to

Surprisingly there is no built-in Python function to access UVs by island. There is a way to hack it though:

# Make sure the correct object is selected
# and that you're in edit mode
obj = bpy.data.objects['your-object-name']
bpy.context.view_layer.objects.active = obj
bpy.ops.object.mode_set(mode = 'EDIT')
bm = bmesh.from_edit_mesh(bpy.context.active_object.data)
uv_layers = bm.loops.layers.uv.verify()

# Select all UVs
# then generate seams from UV islands
bpy.ops.uv.select_all(action='SELECT')
bpy.ops.uv.seams_from_islands()

# Collect the island boundary edges here
edges = []
for f in bm.faces:
    for l in f.loops:
        if l.edge.seam or l.edge.is_boundary:
            a = l[uv_layers].uv
            b = l.link_loop_next[uv_layers].uv

            # Blender's texture coordinates are flipped
            # than most image processing;
            # i.e. top-left is (0, 1) rather than (0, 0).
            # So we reflect everything over y=0.5 (i.e. flip vertically)
            # to compensate.
            edges.append(((a.x, 1 - a.y), (b.x, 1 - b.y)))

This method relies on Blender’s built in “Seams from Islands” function, which marks the border edges of UV islands as a “seam”. Then we can iterate all edges of the mesh to find those that are marked as a seam.

This actually worked fine until I tried it on a UV map where the seams produced by this function didn’t actually match the island’s borders:

Here the islands, outlined in red, are just the openings of the pants. The rest of the island border isn’t selected.

I didn’t really understand why until I read more on how mesh geometry works in Blender. Meshes are composed of faces, and each face is composed of “loops”. A loop is a vertex and an edge (which connects to another vertex), though I use “edge” and “loop” interchangeably here. Each face has its own loops, even if those edges are shared with other faces. In the example below there are two faces joined by an edge. That edge is actually two edges, one for each face.

From looking at Blender’s source code (at uv_seams_from_islands_exec) I believe the way “Seams from Islands” works is that the UVs for each shared edges/loops are compared; if the UVs aren’t in the same locations then we know that those edges are separate in the UV map, and thus we have the border edge of an island. In the pants example above the parts that aren’t selected aren’t actually separated edges. It’s hard to tell from the image, but they are edges of joined faces that are folded over:

This can also be illustrated by using a graph-based approach for detecting seams:

import networkx as nx

# Same prep as above
bm = bmesh.from_edit_mesh(bpy.context.active_object.data)
uv_layers = bm.loops.layers.uv.verify()

# Collect all edges here
edges = []
for f in bm.faces:
    for l in f.loops:
        a = l[uv_layers].uv
        b = l.link_loop_next[uv_layers].uv
        edges.append(((a.x, 1 - a.y), (b.x, 1 - b.y)))

# UVs don't have any ids;
# we need to know when two UVs are actually the same UV.
# We do this by seeing if two UVs are within some small range;
# if so we consider them to be the same UV.
eps = 1e-6
uv_idx = []
def find_closest(uv: tuple[float, float]):
    x, y = uv
    for i, (x_, y_) in enumerate(uv_idx):
        x_diff = abs(x - x_)
        y_diff = abs(y - y_)
        if x_diff < eps and y_diff <eps:
            return i
    else:
        uv_idx.append(uv)
        return len(uv_idx) - 1

# Reconstruct the UV map geometry
# as a graph.
g = nx.Graph()
for a, b in edges:
    a_id = find_closest(a)
    b_id = find_closest(b)

    # Count how many edges connect
    # these two UVs
    if g.has_edge(a_id, b_id):
        edge = g.edges[a_id, b_id]
        edge['count'] += 1
    else:
        g.add_edge(a_id, b_id, count=1)

# UVs that are connected by only one edge
# are island borders
borders = []
for a_id, b_id, data in g.edges(data=True):
    if data['count'] == 1:
        borders.append((a_id, b_id))

This approach also works on the fact that island borders are separated edges. The general idea is this:

The connection between UVs C and D is not a border because two edges connect them–edge 3 and 5, belong to the left and right faces respectively. The other connections on the other hand have only one edge, and thus they form the island border.

This nice thing about this approach is it’s easy to identify which UVs belong to which islands. With the “Seams from Islands” approach I had a very crude method: just checking which UVs fall within which island polygons. This unfortunately doesn’t work with overlapping islands because the UVs could be assigned to any of the overlapping islands. With this graph-based approach islands are just the connected components of the graph:

uv_ids_to_islands = {}
for island_id, comp in enumerate(nx.connected_components(g)):
    for id in comp:
        uv_ids_to_islands[id] = island_id

However because this is basically the same idea as “Seams from Islands” it suffers from the same issues. The pants UV map would fail in the same way.

The approach I’m using now is less elegant (using graph structures just feels neat) but should work better (I hope):

from collections import defaultdict
from shapely.ops import unary_union
from shapely.geometry import Polygon
from shapely.validation import make_valid

# Same setup as above
bm = bmesh.from_edit_mesh(bpy.context.active_object.data)
uv_layers = bm.loops.layers.uv.verify()

# Here we collect all edges,
# but grouped into their parent faces
faces = []
for f in bm.faces:
    face = []
    for l in f.loops:
        a = l[uv_layers].uv
        b = l.link_loop_next[uv_layers].uv
        if not face:
            face.append((a.x, 1 - a.y))
        face.append((b.x, 1 - b.y))
    faces.append(Polygon(face))

# Group faces into their parent islands
island_faces = defaultdict(list)
for face in faces:
    # Since the UVs are all connected,
    # we can use any UV from the face to identify its parent island
    uv = face.exterior.coords[0]

    # `find_closest` has same definition as above
    # and we still use the graph-based approach
    # for mapping UVs to islands
    uv_id = find_closest(uv)
    island_id = uv_ids_to_islands[uv_id]

    # Fix face geometry as needed
    if not face.is_valid:
        face = make_valid(face)
    island_faces[island_id].append(face)

# Then merge the faces of each island to get
# the island borders/silhouette
island_shapes = []
for island_id, valid_faces in island_faces.items():
    merged = unary_union(valid_faces)
    island_shapes.append(merged)

Here I extract the geometry of every face of the mesh, rather than working with just the edges. I still use the graph-based approach to map faces to islands. Then each island’s faces are merged using shapely and the island’s borders are the exterior of this merged shape.

Here are the results of this approach, with the extracted borders in blue:

With the island borders extracted the subsequent steps are a bit easier, which is mainly identifying overlapping islands and merging them into one shape (what I was calling “UV map regions”) and cutting out the matching texture regions.

The other important piece is packing these UV map regions. I’m using nest2d, which is a Python interface to libnest2d, for bin packing. There were a couple challenges here too. I’m actually using a fork because the original version bizarrely leaves out a Python API to access the important packing information (translations and rotations of the packed polygons). I also had to expand the Python interface to support some additional libnest2d parameters to avoid packing overlaps. A final adjustment: libnest2d doesn’t support concave shapes so instead of directly using the UV map region polygons I use their convex hulls instead. So the resulting packs won’t be the most space-efficient, but that’s ok.

Here are a couple example results from the script:

The full code is available here.

The other component of this system is what handles the actual merging of the Blender objects and materials (merge.py in the code). This was more straightforward: grab the textures for each object’s material (assuming one main material), bin pack them into a single image, and update the UV maps accordingly. Then the packing system described here is applied to the merged object/texture to remove unnecessary image data and the final output is trimmed and/or scaled to be a square texture with dimensions that are powers of 2 (e.g. 512x512, 1024x1024, etc).

The resulting texture for the character I’m working with here:

There is some weird artifacting going on; notice the noisy speckled texture on parts of the face and one of the hands. I’m not really sure why that’s occurring, but it’s not especially noticeable, so I won’t worry about it for now.

Here we see the draw calls for this character reduced from 4 to 1:

Of course after all of this effort there had to be more problems🙃. The first was a lot of white/blank texture areas showing up in the character. I had thought that this might be an issue–basically extracted textures have no bleed, so if the UV maps aren’t exactly precise then they might cut into empty parts of the texture. The solution to this was to add a padding parameter that provides a few pixels of buffer around each texture clipping.

Yet even after that there was still one other problem. Though the Blender renders and in-browser GLTF preview look fine there is a weird seam that shows up in Godot:

Fortunately this wasn’t too difficult to solve. It was unlikely to be an issue with the exported GLTF itself because the in-browser GLTF preview didn’t have this issue. So it had to be something with Godot. For completeness I’ll go into the details.

When you import an asset into Godot it may process it in some way. For textures this can mean manipulating the images in ways to improve game performance. One such manipulation is the generation of “mipmaps”, which are scaled down versions of the texture to run when the object is at a smaller scale. The problem is that these mipmaps mean that the scaled down textures might not quite line up with the object’s UV maps, which causes texturing bleeding (thus the white line in the render above).

The actual modifications to the imported texture are controlled by default import settings you can define for your project:

That’s how it’s supposed to work, at least. These import settings aren’t respected when importing GLTFs, so I do it manually in my preview script:

# Ensure that mipmaps and filter are disabled
# for each material texture.
# This is because mipmaps can cause issues with bleed,
# i.e. blank parts of the texture bleed through to the UV map.
# For flag values see: <https://docs.godotengine.org/en/stable/classes/class_texture.html#enum-texture-flags>
for child in Util.find_children_of_type(instance, MeshInstance):
    for i in range(child.mesh.get_surface_count()):
        var mat = child.mesh.surface_get_material(i)
        mat.albedo_texture.flags = 0

This works, but then I lose the advantage of mipmaps. There is a clear visual difference between enabling/disabling mipmaps. When turned off the original resolution texture is displayed which can mean showing more detail (I suggest clicking the image to see the full resolution version):

Maybe it’s because I’m used to it now, but the mipmaps version looks better to me. It’s a bit softer and the eyes look better (though I think that’s because I maybe have too bright of a white for the eyeballs). I figured if the lower-res mipmaps version looks better I should just use a lower-res texture to begin with. So I tried downscaling the original 512x512 texture to 256x256 and 128x128 and they both look good. The 128x128 one looks the closest to the mipsmap version, but isn’t exactly the same (Godot may use a different downscaling algorithm; I used Pixeluvo to resize and they don’t say what algorithm they use). Most importantly this manually downscaled version doesn’t have the seam/bleed problem that Godot’s version does. One further issue here is that the 128x128 version does lose some detail–the earrings are much, much harder to see there, so that might be an argument for the 256x256 texture.

This is all a good example of the unexpected and massive diversions that happen with a project like this. I spent so much time and energy on this that could have been spent on more interesting parts of the game. And it’s totally possible that this all just premature optimization. I haven’t developed a game like this so I don’t have any intuition about what I should be focused on for performance. But I really want to avoid the scenario where I’ve generated all this content for the game and then I have to go back and redo everything because of some bad performance decision I made early on (like ignoring draw calls, poly counts, texture sizes, etc).

I don’t think this level of detail is interesting to anyone except me lol. I mostly chronicled this so I have some documentation to refer to when in 3 months I’ve completely forgotten everything and find another edge case that requires me to re-think the whole system (I hope that doesn’t happen, but you never know), but also because maybe there’s some really simple industry standard approach that I completely missed (though I tried to look for one!).

In the end this was one of those things where you’re dealing with two challenges at once: figuring out the high-level algorithm to solve the abstract problem you’re dealing with and figuring out how to implement it in unfamiliar and at times un-/underdocumented or baffingly-designed APIs/libraries. Never a pleasant experience, but I hope it’s worth it!

Update @ 12/9/22 14:30: Right after publishing this I realized that I don’t need to simplify the base human geometry, but only the final geometry that will be exported. Simplifying the base human geometry was hard because of the reasons I mentioned in the last post: meeting the constraints of MakeClothes (quads or tris, minimizing poles) and because simplifying the geometry gives MakeHumans fewer vertices to manipulate the human form with, leading to wonky results. What I’m doing now is throwing a Decimate modifier on as the very last step before exporting the model. This not only simplifies the base human but also any clothes that are attached. At this stage MakeHuman has already done its work, so the constraints of quads/tris and poles no longer apply.

For now I’m using Decimate with a ratio of 0.5, cutting about half of the polys:

Update @ 12/9/22 22:02: So there was another padding-related bug that was causing textures to overlap after packing them. Basically it came down to the fact that there are three different space systems that are used during the packing process: the UV space (where x and y are both in [0, 1]), the texture space (measured in pixels), and the packing space (which is just a scaled up version of the UV space so that coordinates can be integers). The padding is specified as pixels but I wasn’t properly converting them to the packing space, so I just had to implement the correct conversion.

Here are some images from the debugging process (I thought they looked nice):

After much chipping away the character generation system (“chargen”) is mostly complete!

As mentioned in the previous post the character generation system is driven by MakeHuman for the model generation, along with a custom texturing system for generating skin-level details.

This more or less reduces character creation to three distinct production tasks:

1. Creating clothes (and hair)
2. Creating new texture layers (e.g. tattoos, eyebrows, etc)
3. Creating animations

The process works like this:

1. Generate the skin-level details as a single texture.
2. Generate the model using MakeHuman
3. Dress the model using MakeHuman
4. Retarget and remap animations to the model
5. Export as a GLTF
6. Generate preview renders

For documentation purposes I’ll go through each part in detail below…but first, some screenshots (see the end for a video demo):

Generating the texture

This texture is composed of different layers to represent different features: there’s a base skintone layer, then additional layers of shading, e.g. some red in the cheeks and other high blood-flow areas, then other features like eyes, eyebrows, facial hair, close-cut hair, tattoos, and accessories/clothing items that don’t need to have separate geometry (e.g. earrings, socks).

These layers are added using different blend modes–e.g. some layers are simply overlaid and others are multiplied–so they mix in better with the base layers.

Some layers are used as masks (rather than being used directly) so that they can be filled in with different colors. For example eyes (irises), hair, and socks. In these cases the texture uses a base color (usually something close to black or a very dark grey) that is used as the “anchor” or “reference” color, and then the image is colorized to maintain the relative changes in color throughout. For example, a mask texture might have the base color #181818 but also include blacks and greys that are slightly darker/lighter, and want to change it to a red, say #ff0000. The image will be recolored such that pixels that were #181818 become #ff0000 and other pixels become a shade of red as well, maintaining the same relative lightness/darkness to the base color.

Generating the base skintone texture was tricky at first. Something that is probably obvious to digital/visual artists but that I had to learn is that solid colors don’t really look good or interesting (unless you’re going all-in on solid colors, like those blocky start-up illustration styles), so the original characters looked kind of bland with their perfectly uniform skintones. Creating a color field with interesting texture and variation is easy enough to do by hand in a digital painting application but took a bit of work to do procedurally. I considered trying to use Perlin noise and its ilk but I don’t have any intuition how to control noise well…so it would have taken me forever to get results I’d be happy with.

Instead the algorithm just roughly emulates how a person would produce such a texture…just using a paintbrush texture, painting a base layer of color, and then varying the color a bit and painting low-opacity random strokes on top:

I found that using LAB color space provided the best color variations that “vibrate” or “resonate” with the base color (I love the subtle pinks and greens and blues in the examples above).

This approach not only produces nice textures for the base skintone but also for hair.

Creating the other texture layers is straightforward–just paint onto a transparent image, using this reference image to know where things will show up:

The last bit worth mentioning here is performance/speed. There are a few very slow points in the character generation process and texture generation is one of them…sometimes. The image manipulation is happening in Rust, which is very fast if compiled in release mode. Otherwise it’s on the order of 10x slower. Because I’m still developing this content management tool I’m not often building in release mode since you lose access to the browser dev tools when doing so.

Aside on texture sizes: The input texture layers are mostly 2048x2048 (aside from some earlier ones which are 1024x1024). The actual skin texture output is 1024x1024, but I create a downscaled copy at 256x256. This is really small, and pixelation is definitely noticeable in the higher-res viewer but not noticeable in-game, given the size of characters and the downscaling screen shader effect (n.b. the screenshots here are using the 1024x1024 textures, not the 256x256 ones). Again, not really sure that texture size/memory use will be an issue, but I don’t need them to be bigger than this so might as well keep them small.

Generating the model

This step is the most straightforward since the bulk of the work is handled by MakeHuman. MakeHuman provides a number of parameters to control the body model, e.g. how much muscle there is in the upper arms, the shape of the face, etc. What I do is take a few higher-level parameters–gender, fat, muscle, height, age, and a few parameters that MakeHuman uses for more racialized features–and translate them into the more fine-grained parameters. These finer parameters are all random but parameterized by the higher-level ones, with the exception of some parameters such as facial parameters, which are totally random.

For example there’s a parameter called arms_{L,R}_lowerarm_muscle_decr_incr. The sampling distribution for this might find most of its density in (-1, 0) if the muscle slider is turned all the way down, and would be in (0, 1) if turned all the way up.

This step also applies the MakeHuman “game engine” rig to the model.

Dressing the model

The dressing of the model (applying clothes and hair) is also very straightforward since it’s again handled by MakeHuman. The part that’s more complicated is the production of the actual clothes and hair assets to be used.

This process is tricky in part because I’m just not a very experienced modeler, but the lowish-resolution look is more forgiving of that. The main reason it’s difficult is that MakeHuman/MakeClothes has a couple requirements that can be very hard to satisfy. The mesh must consist either entirely of quads or triangles–no mix allowed. In practice this would not be too bad–you can always convert a mesh entirely to triangles (using Blender’s Triangulate Faces), which as far as I know is what a game engine typically does anyways–were it not for a second MakeHuman/MakeClothes requirement that limits the maximum edges allowed for each vertex. It’s something like 8 edges max for any vertex (called “poles”). Going through and figuring out how to reconstruct your geometry to get rid of these poles is hard…some are easy, but others become weird geometry puzzles.

I haven’t been able to come up with a good foolproof method to avoid this problem. I try to stick to quads when modeling clothes/hair but sometimes you just forget and end up introducing a triangle or n-gon somewhere. Or I want to reduce the poly count so I use Blender’s Decimate modifier, which works amazing for reducing polys while preserving the general shape, but then introduces triangles and poles.

I did come up with one procedure for removing poles that has so far worked maybe 2/3 of the time, and is fortunately easy to script:

import bpy

bpy.ops.mesh.select_more()
bpy.ops.mesh.hide(unselected=True)
bpy.ops.mesh.select_less()
bpy.ops.mesh.delete(type='VERT')
bpy.ops.mesh.select_all(action='SELECT')
bpy.ops.mesh.edge_face_add()
bpy.ops.mesh.quads_convert_to_tris(quad_method='BEAUTY', ngon_method='BEAUTY')
bpy.ops.mesh.reveal()

What this does is it deletes the problematic vertex/pole, builds an n-gon face out of its surrounding vertices, and then converts that into triangles. Usually this creates edges that get rid of the pole completely, but it can also just move the problem to a different vertex.

I anticipate spending a lot of time making clothes/hair because that’s probably where the bulk of character variation will come from, so I hope I can figure out a more reliable solution soon.

The last thing to note here is that the grammar system I mentioned in the previous post is also implemented into the tool. Almost all the parameters depicted above can be randomized, including the character’s outfit, which can be randomized using an outfit grammar.

One funny thing is that I spent a bunch of time modeling hair and it’s given me a greater appreciation of the variety of hair styles and how much they change someone’s look. I’m sure it’s obvious to many people but not something I spent much time thinking about. Now I catch myself really studying the silhouettes of people’s hair lol.

Mapping animations onto the model

There isn’t a whole lot to say here since I haven’t figured out a animation/mocap process, so I’m just relying on Mixamo for now. The process of retargeting/remapping these animations to the character model is fortunately very easy thanks to a Blender addon called “Auto-Rig Pro”. I did have to create a custom remap config to go from Mixamo to MakeHuman’s game engine rig though, and it was a bit tricky to make sure that all animations were correctly exported (due to me not understanding how Blender’s actions and NLA editor work).

This step is also a hit for performance. The retargeting/remapping process just takes a lot of time: a few minutes for 3 animations, but this will vary on the keyframe length of each animation. Not sure that I can really do anything about that though.

Exporting the model to GLTF

This step is also pretty simple, just exporting the GLTF from Blender. The GLTF format is what Godot uses for importing the model.

The only thing I need to look into here is performance. It takes a ridiculous amount of time to export and absolutely shreds my CPU. I can barely do anything else while it’s exporting. I’m not sure what’s going on as it’s just a fairly low poly model that’s being exported.

Generating preview renders

The final step is generating preview renders to see what the model will actually look like. A couple of these renders are done in Blender but they aren’t as useful as the in-game renders, which let me see how the character looks with the game’s lighting conditions and screen shaders. This was also surprisingly easy to setup as Godot provides a convenient way to run the engine headlessly and take screenshots.

I was also able to use the same approach for previewing things in-game directly from Blender:

Fears and Steps for the Future

There’s still more work to be done. I haven’t fully finished integrating characters into Godot, so there may be other things that need to be fixed there. And I don’t have a great intuition about what poly counts I should be going for, so a great anxiety is that my poly counts will be too high and I’ll have to go back and redo the base character mesh and all the clothes/hair meshes. I don’t really anticipate this being a problem because poly count is (what I think is) low across the board and from what I’ve read poly counts are less likely to be bottlenecks nowadays. But you never know with these things…and I could probably safely lower the poly count of the base mesh, so I should probably do that earlier rather than later…

Update: Lowering the mesh poly count

After writing this I investigated lowering the poly count for the mesh. The original human mesh is around 3k tris–for reference, the Final Fantasy VIII Kiros model I was using for testing is around 750 tris, which was for the original PlayStation. Individual items of clothing so far ranging from 250-800 tris, though I could probably reduce those a fair amount more. Some of these clothing items delete the parts of the human mesh they cover, so they add less to the overall poly count.

Lowering the poly count was a difficult process: after applying Blender’s Decimate modifier there were several poles (the same issue mentioned above) that took a very, very long time to eliminate, but also because the poly reduction process and subsequent pole-editing messed up parts of the UV map which took awhile to fix (UV map editing in Blender is awful). I had to try and match the original UV map as much as possible or I’d need to re-create all the texture system layers, which I really did not want to do.

But I managed to produce a few lower-poly human meshes. The biggest reduction was down to about 750 tris. This lost too much detail, especially around the face. I tried a less extreme version that was about 1300-1400 tris, which was better, but having fewer vertices reduces the effectiveness of MakeHuman’s parameters though and made some parts too angular. Then I tried another version targeting the Decimate to the hands (which is the most detailed region after the face) and feet (which usually aren’t going to be visible and don’t need much detail), getting things down to about 2k tris while entirely preserving the face and preserving the original UV map as much as possible. A 33% reduction in polys isn’t bad!

In the higher resolution previews the hands are noticeably blockier:

But this is less apparent in-game:

For all I know this is a premature optimization and ~3k poly character models are fine. In theory if I ever need to switch the base human model I can just do so by batch re-generating all of the characters, but as the game develops it may become more complicated to do that. Hopefully I’ll have a better sense of what poly count works before that.

Small update: So something about the reduced poly mesh messes up the vertex mapping to the base MakeHuman mesh, I think. Basically vertices on the face/head are being deleted when vertices in the torso should be deleted. I may need to try a cleaner approach to reducing the poly count, that won’t mess up the underlying geometry as much…e.g. doing it by hand.

Update: Improving GLTF export times

The GLTF export times were around 10 minutes with 3 animations, which was way too slow. I was not exporting much–only the renderable/visible objects, which was just the lowish-poly human mesh and clothes. Everything else was hidden. But for some reason (which may be related to this issue) the GLTF exporter still processes hidden meshes, even if they aren’t to be included in the final GLTF. So the base MakeHuman mesh, which is pretty detailed, was taking up the bulk of the processing time. After deleting that mesh and some other objects (rather than just hiding them) the export time went down to about a minute. The character generation system does this exporting through a headless Blender script, but when exporting from the Blender UI it takes only ~15-16 seconds. No idea what could be the cause of that discrepancy.

Now the main bottleneck is the retargeting/remapping of animations, but again, I don’t think I can do much to speed that up.

The character generation system is starting to come together. There are still a few things to figure out, but I think I have the bulk of it figured out.

The character generation system is roughly broken down into four sub-problems:

1. Mesh generation: how will the actual geometry of a character be generated?
2. Texturing: how will the skin-level features (skin tone, eyes, eyebrows, nose, mouth, close-cut hair, facial hair, tattoos, scars, etc) of a character be generated?
3. Clothing/hair: how will additional geometry, like clothing and hair, be generated and mapped to the human mesh?
4. Rigging: how will the character’s skeleton be configured?

(A fifth sub-problem could be “animation” but I’ll handle that separately.)

A good system will encompass these sub-problems and also make it easy to:

• generate characters according to specific constraints through a single UI (eventually as a tab in the CMS)
• add or modify clothing, hair, etc
• generate a large amount of variation across characters

In summary, the whole system should be represented by a single configuration screen, and with a single press of a button I can produce a fully rigged and clothes character model. I’m not quite at that point yet but making good progress.

The system so far is all built around the open source MakeHuman, which makes the whole process much simpler. It provides a way to generate parameterized human meshes that supports the easy adding of components like clothing, and it has an addon for working directly within Blender. MakeHuman works by generating a “base mesh” which then can be used with other meshes (“proxies”) that map to the vertices of the base mesh. When the base mesh is transformed–either through animation or through a variety of different body shape/proportion parameters–these proxies are transformed too. Clothes are proxies, but so are “topologies” which replace the base mesh as the human mesh. This allows me to use a custom lower-poly mesh instead of the higher-resolution default mesh.

So MakeHuman takes care of mesh generation, and it provides a way to attach clothing and hair. The clothing and hair still need to be modeled individually, but this is less daunting a task as I figure I’ll only need to create a relatively small amount of clothing forms that each have several different textures. It may be easier to automate the generation of some of these textures, e.g. color variations. In the end this is not too different than your run-of-the-mill modeling and texturing; there are just a couple of extra steps to ensure that the clothes map on to the human mesh correctly.

MakeHuman also generates a rig for the mesh, so that sub-problem may be taken care of too. But because I haven’t figured out the animation process, I don’t know exactly if/how I’ll integrate the auto-generated mesh. For my test characters I’ve just been using Mixamo’s auto-rigger…so for now this one needs more work.

So that leaves texturing, or what I called “skin-level features”. These are features that don’t really have any volume to them, such as tattoos, scars, and undergarments and socks. This isn’t too difficult in theory: you just need to generate an image texture. The approach is to work in layers, and assemble a final texture by sampling/generating different layers for different features. So the bottom-most layer is the skintone, and on top of that you’d paste on layers of underwear, socks, eyes, nose, mouth, etc.

The face is of course very important here, and it’s the hardest to get right. I don’t yet have a good way of generating facial features. While the other parts (socks, undergarments, etc) can be generated by hand because they don’t require a ton of variation (e.g. I could probably get away with like 20 different pairs of socks), faces should be unique per character (PCs and NPCs alike). I would rather not have to create all of these by hand.

I’ve had some success using Stable Diffusion to generate faces to work from but it’s not consistent enough to automate (faces may not be head-on and so require some manual adjusting, for example). I think a parameterized generator might make the most sense here, where, for example, facial features are defined by bezier curves with constrained parameter ranges, and each face is just a sample of that parameter space. There could then be a pool of textures (for skin tones, lip color, eye color, etc) that are also sampled from to fill in the details.

For testing I just created the skin-level texture by hand, just so I could place a character into the game and see if it works visually:

And here is a comparison with the screen effects, without the dithering, and without the resolution downsampling:

The face definitely needs work but I feel ok–but not thrilled–about everything else. It does feel somewhere between the graphics from the original FF8 and its remaster (except for the face), which is sort of what I was going for. I think I need to sit with it for awhile, see how it feels as the rest of the game’s environments develop, and try different character models, clothing, etc. It’s at least a starting point–I feel a bit reassured that I have some method for generating decent-looking characters, one that could be modified if needed.

On this point: I’m kind of hoping that all the models and characters and so on just kind of work together visually, visually but not expecting that to be the case. I’m trying to design this character generation system so that I can make adjustments to e.g. textures, models and have those adjustments propagate through all existing character models. That gives me more latitude to experiment with the game’s visual aesthetic and makes me feel less like I’m committing to a particular one so early on.

This brings me to the actual generation system–everything up to this point is more about producing the assets that are then mix-and-matched to generate the actual characters. I don’t want to allow for totally random character generation because there are combinations that are atypical or implausible. With clothes, for example, people generally don’t wear a dress and pants at the same time, so I want to prevent this particular outfit from being generated (apologies if you do dress this way regularly). A context-free grammar (CFG) makes the most sense to me because it allows you to define set configurations that have variation, thus avoiding these problems of complete randomness.

With a CFG you will essentially define different “outfits”, where each component of the outfit can be drawn from a pre-defined list of options. Say for example I need to generate a lot of road workers. A simple CFG might look like:

RoadWorker:
    - HardHat
    - TShirt
    - HighVisVest
    - WorkPants
    - WorkBoots
HighVisVest:
    - YellowHighVisVest
    - OrangeHighVisVest
HardHat:
    - YellowHardHat
    - WhiteHardHat
    - HardHatWithLight
HardHatWithLight:
    - WhiteHardHatWithLight
    - YellowHardHatWithLight
TShirt:
    - RedShirt
    - BlueShirt
    - BlackShirt
WorkPants:
    - CarpenterPants
    - Jeans
WorkBoots:
    - BrownWorkBoots
    - BlackWorkBoots

A CFG is recursive in the sense that, if I want to create a RoadWorker, the program will see that HardHat itself can be expanded into different options. And then the program will see that one of those options, HardHatWithLight, can also be expanded into more options. So it will do this until it’s got all options, and sample from those.

Another feature to add is the ability to draw from the CFG with some fixed options. Like say I’m generating an audience for a group where everyone has to wear an orange beret; I can fix that option the program would only generate characters in an outfit which is allowed to include an orange beret.

Finally, every time a character is generated with the CFG, the resulting model will be saved with the combination of terms used to generate that character (in the case of a RoadWorker that might be YellowHardHat,BlueShirt,OrangeHighVisVest,...). This makes it easy to “regenerate” or update existing characters if one of the underlying models change. That way I can feel free to tweak the textures and models of clothing and other components without needing to manually update every character that I’ve generated so far.

In the near term this will probably all work via a Python script, but it would be amazing to be able to see changes in a character real-time. So a character might be generated through the CFG, but can be hand-adjusted afterwards, e.g. by swapping out that BlueShirt for a BlackShirt, or, if I go the bezier curve route for face features, by adjusting the eye shape for this character, etc. This might be feasible by calling MakeHuman and Blender via their Python interfaces, rendering the model, and then displaying the rendered image, but it sounds really complicated and janky. I’ll have to think on it some more.

It’s been awhile but I have been working on Fugue here and there. The biggest accomplishment is a more cohesive content management system, making it easier to keep most game-related content in one place and in a consistent structure. That way I don’t have to worry about misplacing things and I can build more automated processing tools without the whole development process becoming a total mess (I hope).

One of the CMS tools is a simplified wiki for the game world and mechanics. I’m working on copying over all my random scattered notes but this is so much easier to think with already.

I’ve also ported over the sequence editor. I’m not 100% certain that I’ll stick with Godot and so I want to be able to write all the action/dialogue sequences in a portable format. This saves sequences as a .seq file (which is really just a JSON file). Godot supports import plugins which are able to import custom filetypes into Godot resources. I’m doing the same with items (not pictured here), which as saved as .item (again, just a JSON file).

The actual integration with Godot is very hands-off. I just symlink the relevant CMS folders into my Godot project, and then the importer plugins handle the rest. That way I don’t have to mess around with copying over files, worrying if they’re the latest/correct version, etc. Whatever the CMS shows is what Godot sees.

Having a totally custom sequence editor gives a lot more control than the Godot addon I created. The downside is that validating actors/agents, scene objects, etc is difficult because that data is in Godot scene files (.tscn). These are plaintext but still requires parsing, so I wrote a .tscn parser which will pull out all the relevant entities that might be available to a sequence. I decided to use these extracted entities as suggestions for a sequence, not for validation. I thought that perhaps the workflow would be: write sequences in the CMS and then setup the Godot scene to match it, in which case validation is probably best handled from within Godot.

The rest is less interesting. There’s a visual reference manager, which is really just a fancy image viewer:

And I’ve ported over the texture manager/editor from before, with some snazzier features, including texture/image synthesis:

Outside of this CMS tool I’ve been experimenting with character modeling/generation, which is one of the last major (and most intimidating) content production questions (along with character animation–I’m waiting for the next release of FreeMoCap, and I’ve kind of accepted that object modeling will just happen by hand). I feel that I’m getting close to a good process. Hopefully I’ll figure it out soon.

After a few busy weeks I’ve found a bit of time to continue work on Fugue. One of the last major questions content production, after writing, textures, and music, is character animation (this still leaves object and character modeling as the other two problem areas). While I believe I can get away with lower-poly models and crunchier photo-textures, I don’t think that’s the case with low-quality animation–it’s too jarring. So I want to figure out a way to produce fairly good, realistic motion on the cheap.

There are a number of deep learning-based projects available for motion capture without a sophisticated tracking setup. Some are even monocular, requiring only one camera. There are some commercial offerings (such as deepmotion.com) but I want to see how far I can get with open source options. I’ll be able to modify those as I need and they’ll be easier to integrated into a more automated process than commercial options.

The open source projects are usually research projects, so they aren’t polished/are somewhat janky and probably don’t generalize very well. And their licenses often restrict usage to non-commercial purposes. For example, MocapNET, EasyMocap, and FrankMocap are all non-commercial uses only. I did find MotioNet which does allow commercial usage (under its BSD-2 license) and requires only one camera, so that was promising.

One alternative to the deep learning approach is to just use existing motion capture data and hope that covers all the animations I’d need. A great resource is the CMU Graphics Lab Motion Capture Database, which has generously been converted to .bvh by Bruce Hahne for easy usage in Blender. The collection encompasses 2,500 motions and is “free for all uses”. The range of motions is expansive enough (including things like pantomiming a dragon) that it’s possible it will have everything I need.

Still, I wanted to try out the deep learning approaches, in part because I was curious.

One note here is that these models typically output motions as .bvh files. These contain motion instructions addressed to a particular skeleton (where, for example, the left leg bone might be named LeftLeg). I used Mixamo’s auto-rigger to rig my character and the resulting skeleton has a different naming system. Fortunately there is a Blender addon, “BVH Retargeter”, that remaps a .bvh to a differently-named skeleton. It doesn’t include a mapping for Mixamo by default, but I set one up myself (available here, it goes into the known_rigs directory).

On this note, there is also this Deep-motion-editing project which has a much more sophisticated retargeter:

I don’t know yet if I’ll have a need for this, but good to know it’s there!

On to the tests:

I’m using a Kiros Seagill model (from FF8) for these tests.

Even though the MocapNET license is not what I need, I decided to try it anyways:

It looks ok, a little janky and all over the place though. And the hands aren’t animated.

MotioNet looked promising but unfortunately did not have very good output. The output pose is upside-down for some reason (this is a known issue), which seems like an easy enough fix, but the joint movement is stiff and incorrect.

The CMU motion looks great of course, as it’s actually captured properly. Again, the only concern here is that it doesn’t have a wide enough range of motions.

The last software I tried is FreeMoCap, which is still in very early stages of development, but there’s enough to try it out. It was quite a bit more complicated to set up as it works best with multiple cameras (they can still be fairly cheap, e.g. $20-30, webcams), and requires a charuco board for calibration, which I printed off at Kinko’s. That cost me about 30 to get it on poster board, but you can probably make something cheaper with an old cardboard box and printing on large-sized computer paper. In total I spent ~100 on equipment.

The most important thing is to get webcams that work for the size of your recording room, so get your full body in frame for all of them (which may require wide-angle cameras). Then you need to make sure that your charuco board is large enough that its patterns are clear on the webcams–the further you position the webcams, the larger the charuco board you’ll need and the lower the resolution you record at, the larger the charuco board you’ll need. Note that there’s also a resolution/frame-rate trade-off: when running 3 cameras at 1080p I get about 14-15fps, but I needed to run at that size for my charuco board to render clearly. And as another note, the FPS can be bottlenecked if you use a USB hub to run your cameras through (some of the cameras may not even work in that case); I ended up plugging each camera into its own port for best performance.

Getting the program to work was tricky, which wasn’t a surprise given the project is in an alpha state. I had to make a few changes to get it to run properly (mainly switching from multithreading to multiprocessing, since the threads were blocked on my system, and manually setting the FPS for my cameras, which for some reason would limit to 5FPS otherwise).

Below is an example of the program’s output. My recording environment is less than ideal (my camera setup was super janky, attached to books or shelves in a haphazard way), but the output looks decent. It’s jittery, and you’ll notice the pose skeleton and camera footage are swapped in the first and last videos. I’m not sure if that’s just a bug with this visualization or if it’s happening deeper in the program, in which case it may be why there’s that jitteriness and the skeleton angle is off.

The program can also output Blender files:

Here the issues are more apparent: the hands are especially all over the place. But even the limbs are too finicky. The demo video (above) has good limb motion, so maybe my setup is lacking (though the hands are also jittery).

FreeMoCap is a really promising project, but unfortunately it’s at too early of a stage to be consistent and reliable. For now I’ll probably develop the game using the CMU motion data, and then later, when I’m ready and FreeMoCap is likely much more mature I can go through and replace or refine with custom motions. Though at the rate development is going, there’s a good chance that FreeMoCap will be much further along by the time I’m ready to start really working on character animations!

This is a draft (completed in October 2020) of a piece that was never published. I’m posting it here for anyone who might be interested. The food system is a massive and important topic, and here I’ve only scratched the surface of one small part of it. Researching and writing this shifted my thinking on many topics–I hope others will also get something from it.

Limit Break

Nowhere in the world do we see limitless growth. In agriculture, as in the broader natural world, there are nutritional limits that prevent plants and the life that depends on them from growing unbounded. Two limiting nutrients of special importance are nitrogen and phosphorus. Together these form a bottleneck for agricultural systems, with implications not only for food and other agricultural products but also for key features of a post-carbon world, such as bio-based alternatives to fossil fuels and their derivatives and afforestation.

Nitrogen is everywhere, making almost 80% of the air that surrounds us, which is why at first glance its role as a limiter might seem strange. Unlike carbon dioxide, the nitrogen in the air is inaccessible to plants. There are a variety of ways it can be made available to plants, such as through certain “nitrogen-fixing” bacteria or, much less commonly, lightning strikes, but something has to convert nitrogen into its inorganic form (e.g. ammonia) for plants to make use of it¹².

Phosphorus, more commonly a limiting factor in tropical areas³, is not as abundant as nitrogen, circulating through the gradual weathering of rocks.

Though many cultures found ways to better conserve soil nutrients and modestly supplement the nutritional content of soils through manures or legumes (which host nitrogen-fixing bacteria)⁴, it wasn’t until the 19th century that a series of shifts occurred that effectively broke through these limits: the development of global fertilizer supply chains and the Haber-Bosch process for synthesizing nitrogen fertilizer at scale, named for Fritz Haber (sometimes called “the father of chemical warfare”) who developed the process and Carl Bosch who later improved it. This process is still the basis for essentially all synthetic fertilizer production today. Nowadays the amount of reactive nitrogen and phosphorus circulating is more than double that of the pre-industrial world⁵⁶⁷⁸.

These developments proliferated over the past century–use rates of nitrogen fertilizer increased over 40 times in the US over this period⁹–massively increasing global agricultural productivity, tripling agricultural value since 1970 to $2.6 trillion¹⁰. In the US synthetic fertilizer may be responsible for anywhere from 30 to 60% of yields, and even higher in the tropics¹¹¹². One of the most important consequences of the surpassing of these limits has been exponential population growth. Some 40% to half of the world population may owe their existence to the Haber-Bosch process².

This massive nutritional increase comes at a substantial social and environmental cost and, at least in its current form, is unsustainable. The breaking of these nutrient limits is a major contributor to the breaking of planetary boundaries¹³: a recent paper estimated that almost 50% of global food production violates planetary boundaries, with almost of half to that (25% overall) due nitrogen fertilizer¹⁴.

Nitrogen fertilizers are typically applied in excess⁷; for cereal grain production only about a third of applied nitrogen is actually taken up the crop, amounting to $90 billion of wasted nitrogen¹⁵, with some estimates ranging from half² to 80%⁵¹⁶ being wasted across all crops. Similarly, estimates suggest only 15 to 30% of phosphorus is taken up¹⁷¹⁸¹⁷. But a lot of this excess fertilizer leaches into water systems^213192021, causing eutrophication that has contributed to over 400 marine dead zones¹⁸. In the Gulf of Mexico such pollution from US agriculture costs the economy an estimated $1.4 billion annually by undermining fishing, recreation, and other marine activity²⁰²². This overflow also contributes to the flourishing of pathogens like the West Nile virus²³. Phosphorus fertilizers also contribute to heavy metal content in agriculture¹⁸, as much as 60% of cadmium in crops and soil²⁴²⁵¹⁷, some of which also makes it to water sources²⁵. In the US, agriculture is responsible for 70% of the total nitrogen and phosphorus pollution¹⁶.

This water pollution also has direct effects on human health, rendering drinking water toxic¹⁸⁹. Nitrate contamination is linked to many different diseases–blue baby syndrome, some cancers, and more. Violations of EPA limits of nitrate content in drinking water doubled from 650 in 1998 to 1,200 in 2008, while 2 million private wells are above the EPA-recommended standard²³. This pollution is expensive to treat and the cost is usually borne by the public. The cost of a treatment facility can be hundreds of millions of dollars; the estimated cost for agriculture’s share of the pollution is around $1.7 billion annually²³. The total cost of the environmental and health harm related to agricultural nitrogen in the US has been estimated at $157 billion per year²³.

Nitrogen fertilizer overapplication also contributes to agricultural greenhouse gas emissions, especially N2O¹⁹¹⁵² which is now the main contributor to ozone depletion⁹. The amount of these emissions is estimated to increase exponentially with increased fertilizer usage²⁶.

Other concerns regarding over-application–some also with any application of synthetic fertilizers–is their impact on soil health², such as soil acidification²⁷²⁰ and net losses in soil nitrogen over time (meaning synthetic fertilizers’ effectiveness decreases over time)¹⁵²⁸. While disputed, some argue that synthetic fertilizer also inhibits soil’s ability to sequester carbon^12282930. What is more accepted is that more traditional techniques of maintaining soil fertility, which are often abandoned with the introduction of synthetic fertilizer²⁷, improve carbon sequestration³¹³².

Of course, because the issue here is over-application–for example, China can use 30-50% less fertilizer and maintain the same yields and corn farmers in Minnesota reduced nitrogen use by 21% without a decrease in yield²⁰–then the solution should be straightforward: use less of it. Recommendations to avoid fertilizer overapplication are generally about better management, more context-specific application (e.g. depending on what’s already available in the soil and what the crop requires, carefully measuring out how much to apply)^{8273334203536375223}. This approach has been successful–in Denmark, for example, nitrogen fertilizer application decreased 52% since 1985 following the Nitrate Directive in 1987 which prescribes specific nitrogen management practices²⁰. Similarly, a simple one fertilizer management program based on color charts successfully reduced fertilizer N use by about 25%⁵. The US has also seen improved nitrogen fertilizer efficiency through education initiatives⁷, but stronger regulation regarding synthetic fertilizer application is typically met with fierce resistance from farm lobbying groups⁹.

Other recommendations look a bit more deeply into why farmers over-apply fertilizer to begin with, especially around risk and incentives. Crop insurance appears to have some effect on fertilizer application rates, though there isn’t consensus around their types and strength. One position argues that crop insurance programs encourage the planting of more input-intensive crops because they are better covered by the insurance, thus increasing fertilizer application; another argues that fertilizer over-application is a risk mitigation strategy (better to over-apply than under-apply and have a small harvest) that becomes redundant with crop insurance^382123342039. Another position argues that crop insurance encourages agricultural expansion onto lands that are otherwise ill-suited for it–with greater environmental consequences–because crop insurance mitigates the risk of poor yields⁴⁰. Some empirical studies have found that enrollment in crop insurance programs does increase the use of fertilizer³⁸⁴¹ and water⁴² and does influence land use and crop choice (e.g. towards more nutrient-demanding crops)⁴³, but others have found no effect or the opposite effect on fertilizer use⁴²²¹.

In the case of the USDA, applications for assistance require that the applicant demonstrate genuine attempts to increase yields, which generally means applying fertilizers and adopting other conventional agriculture techniques. Such requirements are implemented out of concerns of gaming the system⁴⁴. Because crop insurance is subsidized in the US, taxpayers essentially finance the consequent environmental destruction, while also suffering the consequences of water pollution, depletion of fish stocks, and more²³⁴⁴.

There are, however, environmental concerns beyond over-application, especially around the production of both nitrogen and phosphorus fertilizers.

Nitrogen fertilizer production–i.e. the Haber-Bosch process–is a large source of agriculture’s environmental footprint³². The process requires a hydrocarbon source, usually natural gas, which is combined with nitrogen from the air^45464735 and is also used to provide the required heat and energy. The natural gas is supposed to be used up by the process, such that methane emissions are minimal–but the actual emissions have been found to be over 145 times higher than reported⁴⁸.

The production process also emits significant amounts of CO2⁴⁸; in 2011, this accounted for 25 million tons of greenhouse gases in the US, some 14% of the emissions for the entire chemical industry⁴⁷. A huge chunk of agricultural emissions are actually attributable to fertilizer production due to these CO2 emissions - by one estimate, 30-33%⁴⁷.

There are other problems from fertilizer production, such as pollution contributing to an estimated 4,300 premature deaths per year⁹ and substantial energy demands, accounting for an estimated third of total energy used in crop production³⁵.

Phosphorus fertilizers have their own problems. The extraction and processing of the phosphate rock from which the fertilizer is produced come with the typical environmental issues of mining: water and air pollution, land degradation, erosion, toxic and occasionally radioactive byproducts (e.g. phosphogypsum)^{49505152531817}. However, compared to the nitrogen fertilizer production process, phosphorus fertilizer requires a relatively modest amount of energy (about five times less than nitrogen fertilizer)³⁵. Like nitrogen fertilizer, however, the process is dependent on the fossil fuel industry for inputs, namely sulfuric acid supplied as a byproduct from the oil and gas industry⁶. Both nitrogen and phosphorus fertilizer productions keep agriculture tightly linked to the fossil fuel industries we need to move beyond.

With phosphorus fertilizers one impending issue is phosphate rock’s finite availability. No equivalent to the Haber-Bosch process exists for phosphorus fertilizer⁵⁴, so it will continue to rely mostly on mined phosphate rock–about 96% of mined phosphate rock goes to fertilizer¹⁹⁵⁵ and this accounts for about 60% of phosphorus applied to agriculture (the rest is from recycled sources like manure)⁵⁵¹⁷.

As with other non-renewable resources, there are concerns around “peak phosphorus”⁵⁶⁵⁵¹⁷. This is an inevitability given that it is a finite resource, but of course when this will occur is difficult to predict as anticipating actual reserves of any mineral is hard and estimates change frequently and by large amounts and according to economic and technological circumstances^5756556. Estimates range from within a couple decades to hundreds of years¹⁷⁵⁶⁵⁵.

At present the US is a major producer of phosphate rock, primarily in Florida, but that supply is expected to be 60% depleted by 2030 and completely gone mid-century⁵⁶. China, another major producer, is expected to deplete in the near future as well⁵⁵.

The Metabolic Rift

All of these issues of the present pour out of a deeper history of fertilizer that represents a more fundamental shift in our relationship with nature. This “metabolic rift” is the sundering of agriculture from its local ecological context⁵⁸, and has at its core exploitation, colonialism, and imperialism⁵¹⁵⁹⁶⁰. The siphoning of fertility from the global periphery into Europe and its colonial descendants was foundational to pushing agricultural productivity beyond the limits of what the land could sustain.

Fertilizer itself was a common practice to maintain soil fertility while keeping it in production⁵⁹ but it took the form of night soil, ashes, bones, and other organic waste that were relatively local, often connecting rural and urban populations through substantial logistics systems⁶¹²¹⁷. But such methods could only maintain continuous productivity for so long; nutrient recycling is not perfect³²², and indeed soil depletion became an issue in industrializing nations as populations grew⁶. Alternative methods such as leaving land fallow or crop rotation did not fit the need for maximizing near-term productivity.

To resolve this dilemma, in the 19th century industrializing nations began sourcing fertilizer from distant places–mainly Peru, but also Chile, Ecuador, and Bolivia, in the form of guano and later nitrates^59176160. This “Guano Age” saw a tremendous transfer of wealth to British business interests that essentially controlled this trade⁶⁰⁶¹ as well as many deaths through horrendous slave-like working conditions of laborers from China and throughout Latin America⁶¹⁶²⁵⁹ and through multiple wars, including one following Spain’s seizures of guano-rich islands⁶⁰, a proxy war funded by the British so they could essentially annex Bolivian and Peruvian nitrate and guano producing regions⁶⁰⁶¹, and a civil war in Chile also funded by the British to prevent Chile’s nationalization of its fertilizer resources⁶⁰ (Gregory Cushman’s Guano and the opening of the Pacific world: a global ecological history provides a detailed look into this history).

Many Pacific Islands also saw their share of dispossession and violence. The US sought to secure its own guano deposits, passing the Guano Islands Act in 1856 to unilaterally allow US entrepreneurs to claim new guano islands as US territories⁶¹. Under this act 66 islands were claimed as US territory⁵⁹, with at least 8 still under claim or in dispute. Other colonial powers followed suit, compelling a flurry of claims throughout the Pacific⁵⁹ that devastated the islands unfortunate enough to be endowed with rich deposits. Two such islands, Banaba and Nauru, were key locations for the mining of phosphate rock. The Banabans, after years of violence and occupation, were eventually resettled after about 90% of their home island’s surface was left mined and the trees they relied on for food essentially gone^63646559. Nauru didn’t fare much better, with 80% of the surface left looking like a ‘moonscape’⁶⁶⁶⁴. While Nauru achieved independence in 1968, they are left in a state of economic despair⁶⁵.

The picture looks very different for the beneficiaries of this extractivism. The vast majority of the nutrient wealth extracted from these regions went to European and descendant countries such as the US, Australia, and New Zealand. European-style agriculture was unviable in Australia and New Zealand until imports of fertilizer derived from the Pacific islands turned each into exporters of meat and wool⁵⁹⁶⁵ . The fertilizer that resulted allowed the agricultural expansion that supported the industrial development of these countries⁶¹. Peru, like Nauru and Banaba, saw little benefit. With guano supplies basically depleted, Peru’s farmers became dependent on different nutrient supply chains: imported synthetic fertilizers⁵⁹.

For nitrogen fertilizer the most egregious cases of this imperialist exploitation came to an end with the Haber-Bosch process⁶¹, which as discussed above is not without its own significant problems and is still, with its reliance on fossil fuels, fundamentally extractivist. For other nutrients, especially phosphorus, this is not the case. While there Nauru and Banaba’s phosphate resources are basically depleted, North Africa is still a site of phosphate mining to this day. In particular, the case of Morocco and Western Sahara shows a clear through-line from fertilizer’s broader colonial history to its continuation in the present.

Under French control the region supplied phosphate for France’s agriculture⁶². Morocco continues to be the leading exporter of phosphate with the largest reserves⁶⁷, with estimates of up to 75%-85% of the world total reserves^68541857 . Yet some of these reserves that are counted as Morocco’s are not in fact in Morocco. They are in Western Sahara, what is sometimes called the “world’s last colony”⁶⁹⁷⁰. Morocco has occupied Western Saharan since 1975⁵⁴⁶⁹, with about 80% of the country under Moroccan rule⁷¹ under what is best described as a police state, with violence against and killings of activists, politically-motivated imprisonment, and other human rights violations^{54727374716970}, with hundreds of Sahrawi “disappeared” and tortured by the Moroccan government⁷¹⁷⁰. Most Sahrawi fled to Algeria, where they still mostly live in refugee camps^54716970.

This occupation is at least in part motivated by Western Sahara’s phosphate reserves, which is of uniquely high quality⁶⁹⁶⁷. Some 10% of Moroccan phosphate income comes from the Bou Craa mine in Western Sahara⁵⁴ and makes up most of the income Morocco gets from the region²⁴ (fishing is another big source⁷⁵⁷¹⁷⁰). As of 2015, Morocco has made an estimated profits of $4.27 billion from Western Saharan phosphate rock mines¹⁸.

The UN and the International Court of Justice both recognize Western Sahara’s right to self-determination⁷¹⁷⁰, and no country officially recognizes the Moroccan occupation as legitimate²⁴⁶⁹⁶⁷. Extraction of resources from an occupied territory is clear violation of international law⁶⁹, which determines that the Sahrawi people should have “permanent sovereignty over [their] resources”⁷⁰. Yet, over fifty years since this promise of self-determination, as countries continue to import Western Saharan exports⁶⁷²⁴, has not materialized in any meaningful way.

This is probably because several countries have a vested interest in Morocco. Morocco an important ally to US and the recipient of the most US foreign assistance in the region⁷¹⁶⁹, including financial support and training for the Moroccan military and its operations against the Polisario Front⁷¹⁶⁷, the Sahrawi nationalist movement. The US also helped plan the initial Moroccan invasion of Western Sahara; Kissinger feared communist activity in the area⁷¹⁶⁷. Morocco has suggested using its phosphate production as a lever for protecting its claim to Western Sahara–to pressure Russia, for example²⁴. In 1985, India recognized the independent Sahrawi state, the Sahrawi Arab Democratic Republic (SADR), but withdrew recognition in 2000 when they established a joint venture with the Morocco phosphate industry⁶⁹.

Fertilizer production elsewhere also depends on phosphate exported from the area. 50% of the Bou Craa mine’s output supplies fertilizer producers in North America, though in 2018 Canada-based Nutrien, the major North American importer, stopped accepting imports from the Bou Craa mine²⁴. Other major importing countries are India, New Zealand, and China²⁴. There have been some recent victories, however. In 2017, the Sahrawi Arab Democratic Republic successfully asserted a claim to a cargo of Bou Craa phosphate rock which was sold to New Zealand-based Ballance Agri-Nutrients²⁴.

As phosphate rock supplies dwindle elsewhere, production from places like Morocco become even more important. Some projections say that Morocco’s market share could increase to 80-90% by 2030¹⁸⁵⁵, effectively giving Morocco monopoly over the global phosphate market and a greater investment in maintaining control of Western Saharan deposits.

From all this it’s clear that the current use of synthetic and non-renewable fertilizers is unsustainable. Yet at the start of the millennium Vaclav Smil concluded that we are fated, at least for the next century, to depend on Haber-Bosch², and presumably the system of agriculture that is built on it. Have things changed in the intervening decades? Do we have better options now?

There are not many options for substituting phosphate rock (for phosphorus fertilizers the main approaches are using less or exploiting excess accumulated “legacy” phosphorus⁷⁶), so the focus here will be on nitrogen fertilizers. The main options can be roughly grouped into two categories. There are those which move away from synthetic inputs towards more “natural” approaches, such as organic farming systems, and those which more or less stick to the industrial-chemical model but substitute Haber-Bosch for different processes. Both of these feature into utopian visions of future food systems.

Fertilizers By Other Names

Natural Factories? Organic Systems

As alluded to above, the major differentiator of agricultural systems’ impacts is nutrient management and the nutrient bottleneck^{13773278792593}. Organic systems forgo synthetic fertilizers and other synthetic chemical inputs (though there are circumstances where some may be allowed). Beyond that there can be quite a lot of variety among specific organic systems. More traditional techniques such as intercropping can be used, but aren’t required for the organic label to apply⁷⁷.

In lieu of synthetic fertilizers, other fertilizers like manure are used. These can be less consistent than synthetic fertilizers, as they can have a lot of variety in nutritional composition⁸⁰ and also require additional microbial action before the nutrients are available to the plants (in synthetic fertilizers they’re immediately available, for better or worse)³⁵⁷⁸. But, in addition to the lower environmental footprint, they have other advantages over synthetic fertilizers. Whereas synthetic fertilizers can lower the pest resistance of plants–thus requiring pesticides⁸¹⁸²–organic fertilizers tend not to have this problem⁸³⁸⁴. They may also better contribute to soil carbon sequestration³¹¹³ and improve soil quality in other ways⁸⁵.

The benefits of organic fertilizers are somewhat tempered when taking into account the impacts of collection and transportation. A farm using organic fertilizer produced on-site is going to require less energy than one that ships organic fertilizer from some distant elsewhere. The reduced nutritional density of organic fertilizers also means that transport energy expenditure on a nutritional basis can be higher as well. But in general because no energy-intensive production process is required, the energy requirements are still substantially less than synthetic fertilizers³⁵.

The debate around the viability of wholesale replacement of conventional agriculture with organic systems inevitably ends up on the problem of the “yield gap”–that is, the difficulty organic systems have of matching the productivity of conventional systems.

Comparing organic and conventional agriculture systems is complicated. There are so many other context-specific factors that influence their relative performance, such as the particular crop, the growing environment, water availability, labor costs, and so on, and these factors may be related in zero-sum ways that by now are familiar: yield increases as N fertilizer use increases, but at cost of water pollution⁸⁶⁸⁷³. Comparisons may use a variety of crop specifically bred to maximize productivity under conventional systems, at the expense of traits which make them more productive under organic systems⁷⁷³². Comparisons can be further complicated by histories of colonialism and displacement, where the highest-quality land is captured by large-scale operations that also happen to utilize industrial agriculture techniques, thus further contributing to higher yields. Larger gaps are found when cereal crops are compared, but this might be because Green Revolution technologies have focused on improving cereal yields in particular⁷⁷. This is part of a larger trend: agricultural research largely focuses on techniques and technologies applicable in conventional systems, whereas organic and sustainable techniques are relatively underfunded⁶. The yield gap could be closed further with more research and funding.

One meta-analysis over several hundred agricultural systems–biased towards Western agricultural systems, including the most industrialized systems–concluded that organic systems fall short against conventional systems when compared on a yield basis; i.e. organic systems require more land, result in more eutrophication, and have similar greenhouse gas emissions per unit of food produced¹³³²⁸⁸. The yield gap ranged from a concerning 50% to a more modest ~90% of conventional output³²⁷⁷. Use of well-known techniques like cover cropping and intercropping can further close the gap¹³⁷⁷⁸⁰, but the gap could also widen under the effects of climate change⁸⁸.

When this yield gap is taken into account, many of the environmental benefits of organic agriculture are severely reduced to the point where conventional agriculture looks more environmentally friendly. The difference is usually attributed to the fact that organic methods require much more land to achieve the same output⁸⁸. Thus, for example, benefits in soil carbon sequestration are offset by the increase in deforestation from expansion of agricultural lands⁸⁹¹³⁸⁷. These comparisons, however, use life-cycle assessments (LCA) that are often biased towards the conventional conception of agriculture–that is, focused on agricultural outputs as products–failing to account for the very different conception from organic and other alternative forms that prioritize ecosystem health and have longer-term outlooks. For example, the effects of pesticides, though a key difference between organic and intensive systems, is often not considered⁹⁰. Land degradation is also generally not included in these assessments⁹⁰. On a long enough time scale the land degradation resulting from conventional agriculture undermines agricultural productivity (thus lowering these efficiency benefits from the LCA perspective)²⁹. We might also consider that, though perhaps industrial agriculture theoretically has lower land use requirements, the political economy of industrial agriculture is such that it will expand regardless of this land-use efficiency. Organic agriculture, as co-opted by existing agribusiness, is not itself necessarily immune to this.

Such results by no means indicate that conventional agriculture is “sustainable” (especially considering its dependency on finite resources such as phosphate rock) or that it’s “low-impact” (still requiring more energy than organic and with human health consequences from pesticides, water pollution, and so on)¹³. In any case, the prospect that industrial agriculture, which drives all the issues described above, is somehow better for the environment is a troubling if counterintuitive finding. Are there other newer approaches that resolve this dilemma? Ones that are both high-yielding and less harmful?

Take It Inside: Indoor Systems

One common proposal for improving agriculture–in terms of both yields and impacts–is to challenge its relationship to land. This set of agricultural systems fit neatly into narratives of high-tech progress, including hydroponics, aeroponics, aquaponics, and vertical farming. Hydroponics and aeroponics are two prevailing soil-less growing techniques, in which plants are grown out of water (in hydroponics and aquaponics) or nothing at all (aeroponics), and vertical farming can operate on either. These roughly all fall under the umbrella of indoor farming techniques (they don’t need to be indoor, but they almost always are), which also encompasses more “traditional” greenhouse approaches, but has a more potent contemporary imaginary in the form of extremely high yield per acre operations on otherwise non-arable land, with tightly-controlled environments and factory-like efficiency (some of these methods are grouped under PFALs: “plant factories with artificial lighting”⁹¹ or just “plant factories” as artificial lighting is usually a given).

Within the -ponic techniques is substantial variety: the exact method can vary (hydroponic, for example, includes “deep water culture” and “nutrient film technique” among others), as can specifics of the configuration (though these techniques are usually associated with artificial lighting, they can also use sunlight), and growing environment (higher energy requirements, for example, in a colder climate)⁹²¹³, and the specific crop grown, so generalizations can be a little tricky. But there is enough in common across these approaches that some comparisons are possible.

In terms of yields, hydroponic systems can have higher yields than conventional open-field systems⁹³¹⁶. In the context of urban agriculture, a substantial amount of a city’s food demand can be met with local rooftop farms: for example an estimated 77% of Bologna’s food demand is met in this way⁹⁴. A recent study on urban horticulture in the UK (Sheffield) suggests that the expansion of rooftop farms could help meet a substantial amount of the city’s fruit and vegetable needs⁹⁵.

With respect to fertilizer’s environmental impacts, these configurations feature closed water systems, meaning that water is used more effectively and recycled. This reduces both water usage and nutrient runoff and improves nutrient efficiency compared to conventional systems^{93969716989910091}. However, there is still potential for mismanagement and downstream pollution, as salt accumulation in the circulating nutrient solution still results in wastewater that must be properly disposed of⁹²⁹⁴.

A closed system can be more self-sustaining in other ways, such as using biomass waste to generate biogas to heat the greenhouse or using anaerobic digestion to produce fertilizer on-site⁹². Aquaponics generally avoids the use of synthetic fertilizers because fish waste provides nutrients^9810116, though this does not necessarily mean a complete independence from synthetic fertilizers: fish feed is still an external input, and some setups may require nutritional supplements¹⁶.

Advocates of these systems claim they don’t require pesticides, but indoor livestock farming was pitched on similar promises that haven’t panned out¹⁰². Avoiding pesticide use is even more crucial when these farms are in cities⁹⁴, so this claim warrants even more scrutiny than in other contexts.

These systems can have lower emissions⁹⁷⁹³, with most emissions coming from the facility structure (steel and concrete)⁹⁷⁹¹, other fixed infrastructure (e.g. pipes)^99939497, and energy use¹⁰⁰¹⁰³, though this varies with energy mix. One way to reduce emissions and other impacts from the facility structure is to re-use existing buildings, though this comes with penalties as the buildings are not tailored specifically for indoor agriculture⁹⁴¹⁰⁰.

Perhaps one of the most lauded benefits is versatility in location–indoor methods don’t require arable land¹⁰³. Indeed, they can require substantially less land even when considering the land use for energy sources. One paper estimates 1:3 ratio of greenhouse area to solar panel area and assumes “a conversion efficiency of 14% and an average daily solar radiation value of 6.5 kWh per square meter per year”¹⁰⁴. Using sunlight rather than artificial lighting is even better–one could imagine reclaiming the top floors of abandoned buildings to augment local food production.

From a financial perspective, these indoor systems tend to have high capital costs^{10310598106104}. This includes not only equipment costs, but also the land itself, which in urban centers are expensive compared to rural land, and operating costs¹⁰⁶–you pay for what is otherwise provided for free, e.g. sunlight. They tend to grow only leafy green and herbs for business model viability¹⁰⁵¹⁰², especially because conventional agriculture can still produce the same food for much more cheaply.

Even though the arability of the farm site itself is less important, other geographical aspects still are. For example, where water is particularly expensive (e.g. in Egypt where deep wells must be dug and maintained⁹⁸), the cost of starting an aquaponics farm is still quite expensive⁹⁸. Of course, there are situations where the high cost may make sense if it’s not possible to grow anything otherwise and other land uses don’t make sense¹⁰⁶¹⁰⁴.

The climate of the site is another important factor. Hydroponics systems can have tremendously more energy requirements compared to conventional agriculture, varying a lot by climate (cooler climates require more heating, for example), which can be disproportionate to yield increases^{10413107999216}. Lighting is the other big energy factor, but climate control is the cause for most of the variation between hydroponic systems^{1001049410316}. If the system is such that it has exposure to natural light (e.g. rooftop gardens), then the energy demand is of course further reduced.

In urban environments, these systems can have several additional benefits, depending on how tightly they’re integrated with existing urban processes^1039210091. The proximity to consumers can reduce impacts from packaging and transport⁹⁴⁹¹, as well as minimizing food loss as well⁹⁴¹⁰³. Integration with rainwater harvesting systems can reduce draw on city water supplies⁹⁴¹⁶, and wastewater systems can potentially be developed as nutrient solutions, rather than relying on synthetic fertilizers⁹². This could reduce the energy used in wastewater treatment, as the nutrients currently processed out to avoid eutrophication can be used to feed plants instead¹⁰⁸.

But the fact that indoor methods don’t require arable land is a little deceiving. Like conventional agriculture, arability still must be “imported” via inputs like synthetic fertilizers, so in some respects environmental impacts are merely dislocated from the site of production rather than altogether eliminated. Indoor methods are generally not designed to use organic fertilizers, as organic fertilizers are less consistent, less nutritionally dense, and often require some kind of microbial action to make the nutrients available to the plant (the exception here is aquaponics, which is designed around fish waste as fertilizer). For indoor systems, fertilizer production is still the major contributor to eutrophication^949397103. Even though less of this occurs at the site of use, plenty still occurs along the production and distribution chain.

So while these indoor approaches appear to have many benefits and may have a role to play in food production for a limited set of contexts¹⁰⁷ (where energy costs are favorable, renewable energy sources are used, the high capital requirements are manageable, and to supplement the diets of land-scarce urban centers), but we shouldn’t expect them to replace agriculture wholesale. And they do not necessarily lend themselves to socially-desirable models of production. One could imagine a near future in which insolvent smallholder farms on already marginal lands are displaced for investors to build towering vertical farms in their place.

With regards to fertilizer, they’re still held back by many of the problems with synthetic fertilizers–especially their production. Are we stuck with the Haber-Bosch process? Is there a cleaner way of producing them?

Other Routes to Nitrogen

Though the Haber-Bosch process is the dominant way of producing nitrogen fertilizer, there are a variety of potential alternatives¹⁰⁹¹¹⁰. These alternatives usually focus on Haber-Bosch’s use of fossil feedstock and high energy requirements, stemming from its need for very high temperatures (300-500C) and pressures (200-300atm)¹¹¹–“the single most energy-guzzling element of farming”¹¹². These intense demands contrast to the microbes that fix nitrogen under far less extreme temperatures and pressures due to nitrogenase enzymes¹¹¹.

Naturally, the promise of these nitrogenase enzymes has attracted research interest. One direction is focused on producing nitrogen fertilizer with solar power, at room temperature and normal pressure using these enzymes ¹¹²¹¹³, though at present the process is slower than the Haber-Bosch process¹¹⁴. There is also more recent research into methods using primarily water (for the hydrogen, that is normally supplied by fossil fuels in the Haber-Bosch process⁶¹¹¹), air, and electricity (“primarily” because the process may require expensive metals like palladium as a catalyst¹¹⁵) to create small fertilizer synthesizing devices. Other “solar” fertilizers are similarly produced from water and air but use solar energy directly, rather than through electricity. These also tend to produce more dilute fertilizers that can help avoid overapplication¹¹⁶. This could take the form of relatively small, low-maintenance devices–“artificial leaves”¹¹⁷–that are amendable to decentralized production^{116 111}. A decentralized model has several advantages, such as reducing transportation needs, which can make up a significant portion of fertilizer costs, especially for areas with poor infrastructure¹¹⁶. It may also offer an alternative to the existing capital-intensive, highly-concentrated world of fertilizer production, with its deeply entrenched interests and commitment to the Haber-Bosch status quo.

The American Farm Bureau Federation (AFBF), an agriculture industry lobbying group, is deeply committed to the fossil fuel-based method. The AFBF has a long history of working closely with the fossil fuel industry, supporting oil and gas domestic extraction (e.g. fracking), and even coal (which can be used as an input into fertilizer production⁴⁶, and common in Chinese ammonia production⁴⁷). This support has sometimes taken the form of outright climate change denial¹¹⁸. The Fertilizer Institute, a lobbying group for the fertilizer industry, is also closely connected to the fossil fuel industry, supporting expanded natural gas production, framing it explicitly as necessary for national food security and threatened by climate change policy¹¹⁹. This allegiance is likely only to grow stronger as pressure grows against the fossil fuel industry.

Questions of industry power aside, there is a lot of uncertainty about the viability of these alternative technologies^114111115. A handful of companies cropped up a few years ago trying to produce net-zero emissions fertilizer by replacing Haber-Bosch fossil feedstocks with biomass and switching to renewable energy¹²⁰, but none of the plants ever went through and the companies all went bankrupt¹²¹¹²². The available information seems to indicate mismanagement as a big contributor to these failures but it seems reasonable that the recent decline in natural gas prices–which has led to conventional nitrogen fertilizer production in the US to expand⁴⁷⁴⁸–is likely a large factor as well. The economics continue to favor fossil feedstocks.

Finally, changing the production process doesn’t necessarily help address many of the other downstream effects of nitrogen fertilizer. What room is there then for innovating on fertilizer? Is there something with the productivity of synthetic fertilizer but with less of the harm?

“A Sewer is a Mistake”

A handful of social reformers lamented this partition, begging their fellow citizens to bridge the expanding chasm between city and farm. In his 1862 novel Les Misérables, Victor Hugo wrote, “There is no guano comparable in fertility with the detritus of a capital. A great city is the most mighty of dung-makers. Certain success would attend the experiment of employing the city to manure the plain. If our gold is manure, our manure, on the other hand, is gold.”⁶¹

One option is to return to the older way of thinking: nutrients not as something to be produced but as something to be reused. The use of animal manure is a common example. If animal wastes can be used as fertilizer, then why not human waste too? 75 to 90% of nitrogen intake is excreted as urine², which is also responsible for more than half of both the phosphorus and potassium in domestic wastewater (an estimated 16% of all mined phosphorus passes through the wastewater system) while constituting less than 1% of its volume¹²³. In fact, recycling of human waste into fertilizer was once a key part of the agricultural nutrient cycle, until demand outpaced the recycled nutrients, more powerful fertilizers were introduced, and waste became seen as less of a resource and more of a problem (e.g. because of diseases) to be managed¹²⁴¹²⁵. Subsequent sanitation innovations responding to this conception of waste-as-problem such as flush toilets and more sophisticated sewage systems with the purpose of moving waste away from people also contributed to the disruption of these older “circular economies”¹⁷.

That such a system worked in the past means that we could try to restore it, and there is a renewed interest in recycling human waste into fertilizer¹²⁶. Phosphorus can be recovered from wastewater through a variety of means, such as using sewage sludge, which has been used to generate high yields⁸⁰. Or the nutrient-rich wastewater can be used directly as a medium to grow microalgae to treat the wastewater and harvest nitrogen and phosphorus to produce fertilizers¹²⁷. We excrete almost all of the phosphorus we ingest¹⁷, and it can be recovered from wastewater¹⁸¹²⁸. One study finds that some 40% of Australian phosphorus use can be fulfilled with a sewage recovery system⁵³. The value of waste takes other forms too. Biochar is another soil amendment that can be produced from sewage sludge, which does not work well as a nitrogen fertilizer but can have other benefits such as increasing soil carbon sequestration and preventing nutrient leaching^1298012129.

Unfortunately the valorization of wastewater for fertilizer is not without problems. The removal of nitrogen and phosphorus are both energy-intensive¹²³ and the scalability of these systems are still uncertain¹⁷.

One key complication is waste contamination. Even in the past nightsoil required treatment to be safe², and now we face a substantially more diverse set of chemicals, including pathogens, pharmaceuticals, cosmetics, and heavy metals from automotive and industrial runoff^13078123. This is true even of animal wastes. Manure from concentrated animal feeding operations (CAFOs, i.e. “factory farms”), which, though massive in volume and thus an ample supply of nutrients, suffers from these problems of contamination¹³¹. This question of contamination has meant that these human waste-derived fertilizers aren’t yet permitted for organic farms³². However, processes like vermicomposting, which uses earthworms to digest waste into more nutritious–potentially increasing some mineral concentration (phosphorus, for example) by up to 120% percent¹²⁷, can reduce the content of some of these contaminants¹³⁰⁸⁰. However, the economics are again unfavorable: mining phosphate rock is cheaper than recovered phosphorus.

Reusing excreted nutrients rather than dumping them into the environment is a clear improvement to our current system, but the biggest limitation is that no nutrient recycling process will be able to recover all of the incoming nutrients. We couldn’t rely on it alone: even a high-efficiency waste nutrient recycling system will still need to be supplemented.

Little Helpers: Biofertilizers

There is a class of fertilizers called “smart fertilizers” which usually refer to some slow-release mechanism so that nutrients are released more synchronously with plant uptake (i.e. released as plants need, instead of all at once). The mismatch of nutrient availability and plant need is a large factor in why fertilizer runoff and N2O emissions are as bad as they are; if plants used more of the nutrients then less would make it to waterways¹²¹³². This category encompasses a number of different methods, including organic wastes¹², using slow-release casing, and biofertilizers.

Biofertilizers are fertilizers based on microbes; for example, nitrogen biofertilizers take advantage of existing nitrogen-fixing microorganisms such as cyanobacteria^{133134135136137}. Like other fertilizers, they increase yields^135136130138, and like the naturally-occurring instances of these microbes, biofertilizers may provide additional benefits such as pest protection^{133139135136140137}, reduced soil erosion¹⁴¹¹³⁷, additional growth beyond just that from the increased nutrient supply^136130140137, and additional carbon sequestration¹³⁷. The production process can involve fermentation (e.g. with yeast¹³⁶¹⁴⁰) or vermicomposting (i.e. the use of worms) from agricultural waste or wastewater^13613080. These processes are accessible even in conditions of low capital (they are substantially cheaper than synthetic fertilizers¹⁴²¹⁴⁰) or without additional energy sources.

The use of such “plant probiotics”¹³⁵ is not itself new^13514180143–for example, use of manures also introduces these microbes¹³⁶¹⁴¹ and some are already commercialized on a small scale¹³⁵. Environmental concerns around the Haber-Bosch process and other extractive fertilizer production processes, along with advances in biotechnology¹³⁶, are drawing a renewed interest in commercializing the technology on a larger scale. There are however issues with scaling this to an industrial model: for example, transport and storage of living organisms is more complicated than relatively inert chemical fertilizers^13313680.

However, again these biofertilizers are being developed within the context of the prevailing agrochemical industrial regime; companies like Bayer (who purchased Monsanto in 2018) and Nutrien (a $34-billion fertilizer company and second-largest producer of nitrogen fertilizer in the world¹⁴⁴) are partnering with startups developing this biofertilizer technology, which is also focusing more on developing novel microbes rather than adapting existing ones¹³⁹¹⁴⁵.

The homemade biofertilizer Super Magro is one example of the more promising route biofertilizers could take. It consists of a simple, low-cost fermentation process and was released as an open source recipe for other farmers to use freely and augment¹⁴²¹⁴⁶.

What do we do?

Comparisons of these options are usually stated in terms of crop yield. Though some are still in development, these alternatives generally have yields that are worse than or only comparable to the prevailing synthetic fertilizer regime. There are cases where lower yields might be acceptable, such as with hydroponic urban agriculture where no food would be grown otherwise¹⁰². But in general, when viewed through crop yields, many analyses find these alternatives to be more environmentally damaging because of their lower yields.

Generally the emissions of agriculture are tied to yield¹⁴⁷³³; systems that are more intensive and produce higher yields end up having lower emissions per output. For example, organic systems have lower yield per hectare so they require more land to produce the same amount of food that an industrial farm would, which then leads to more deforestation and greater carbon emissions³³. The worrying trade off between feeding people and reducing environmental impact seems to dissipate: industrial agriculture surprisingly gives us the best of both worlds. But this doesn’t make any sense–we know that approach to agriculture has substantial harms.

This apparent paradox becomes resolvable by examining its key assumption: that feeding everyone necessarily requires absolutely maximizing agricultural productivity, which at present means widespread synthetic fertilizer use. This is the mainstream framing of the problem, advocated by organizations like the Bill and Melinda Gates Foundation¹⁴⁸. That is, the core cause of inadequate nutrition and starvation is taken to be that we aren’t growing enough food. Therefore we need powerful synthetic fertilizers to help us grow enough.

But are we really not growing enough food?

Food Waste

If we focus on food production we end up overlooking another important aspect: food waste. Consensus on exactly how much food is wasted is difficult because of lack of comprehensive data; estimates usually rely on surveys, looking through trash, or inferring through some other model. What is defined as food waste also varies. It can be simply the food that could have been eaten but is not (some definitions require that the food be eaten by a person, e.g. not a pet). or include food eaten in excess of nutritional requirements¹⁴⁹¹⁵⁰, or include that can no longer be sold, even if it still edible¹²⁵, or still further be complicated by cultural differences, where food in one culinary tradition is considered waste in another¹²⁵.

What is consistent across these definitions and data is that there is a lot of food wastage. Estimates of food waste range from 20% to 50%¹⁴⁹¹⁵¹ (the wide variation is due to the differing definitions of food waste and outdated or lack of comprehensive data¹⁵⁰), and appears to be growing¹⁵², even as agricultural output also grows¹⁵³. In North America and Europe alone, where most of the waste occurs (twice as much as in Sub-Saharan Africa and South and Southeast Asia), this waste could “feed the world’s hungry three times over”¹⁴⁹.

Accounting for food waste dramatically increases the impact of each calorie of food consumed. Food waste accounts for as much as 38% of energy in the food system¹⁵³; in the US this amounts to at least 2% of overall annual energy consumption¹⁵⁴. This translates to a large amount of carbon emissions: in the UK, some 17 million tonnes CO2 emissions are attributable to food waste¹⁵⁵; globally this figures to about 8 to 10% of GHG emissions (25-30% of food production’s emissions)¹⁵⁶. These impacts include not only the emissions from production, but also that food waste that ends up in landfill emits CO2 and methane¹⁵². For the UK, an estimated half of waste emissions comes from food waste¹⁴⁹. Clearly if our concern is the environmental impact of food production, then food waste represents wasted harm. Reducing food waste not only reduces the impact per calorie from our existing food system, but makes a transition to a less harmful system more feasible in general because less overall output is required⁸⁸.

Food waste is, on one end, usually framed as a technical problem for developing countries, and on the other, a behavioral one for developed countries. In developing countries, most food waste is attributed to lack of adequate technology and infrastructure. This might include poor transportation infrastructure that leads to food lost in transit, lack of well-sealed or cold storage leading to increased spoilage or infestation, crude harvesting equipment that leaves edible material behind^149157125. In wealthier countries, more food waste occurs closer to the consumer^155157150149, in part because improved production, storage, and distribution infrastructure reduces the share of waste at earlier points in the chain but also because higher incomes entail more wasteful habits¹⁵⁸, such as an avoidance of “ugly” produce^157159150158, excessive portion sizes¹⁵⁷, and poor meal planning¹⁵⁷¹⁵⁰. Confusing, inconsistent, or outright arbitrary best-by/best-before/expiration date language is also an issue^157160159, as are promotions encouraging larger purchases. There may also be legal obstacles preventing the distribution of excess food. The spatial organization of the global food system also drives waste: the more sprawled out a food system is, the farther food has to travel, which means more opportunities for food to be lost or spoiled along the way¹⁵⁰.

With this understanding the policy recommendations are usually straightforward. For developing countries solutions focus on technology and infrastructure, which can be as simple as sealed storage drums¹⁵⁰. Developed countries food waste policies focus around consumer education and behavior change^155149157125 and removing hurdles to distributing surplus food¹⁵⁶. For example, in France there is legislation from 2016 that fines groceries from throwing away food that’s still good, which has resulted in more food donations and less food waste¹⁶¹. There are also community-based solidarity groups such as Food Not Bombs that try to ensure that food is being used to feed people.

But this analysis and its consequent policy solutions overlook how endemic these problems are to the political economy of food production–not just technical or behavioral problems–and how developing and developed food waste is deeply interconnected, where food waste is in fact structurally rational.

For example, consider the avoidance of “ugly” produce. It is not only consumer preference at point of purchase that leaves “ugly” produce behind but the decisions of retailers as well. For example, Walmart sets standards about carrot straightness and color, causing one supplier to discard as much as 30% of its carrots that do not meet these standards¹⁵³. The EU used to have regulations specifying standards around the curvature of bananas¹²⁵ among many other standards regarding other produce¹⁵⁹. The cost of such standards are borne by smaller farmers, especially those in the Global South who have no influence over such decisions¹⁵⁹ and often can’t afford the more sophisticated harvesting technologies that reduce bruising¹⁵⁰. In these cases the absence of demand leaves food to spoil¹²⁵.

Yet, despite the narrative that producer waste is less of an issue in wealthier countries, similar dynamics are found. In California, for example, on average 24% of perfectly good food is left on fields unharvested¹⁶². Crop varieties, labor costs, and environmental factors all play a role, but so too do consumer preferences (leaving blemished food behind) and market prices–when prices are low, more may be left unharvested.¹⁶²

Questions around food production really obscure two issues at play here: one is the question of whether or not we produce enough food to meet everyone’s dietary needs, and another is whether or not that produced food is accessible to everyone. Looking more closely, our food production system blows past the first issue, and the problems of hunger and malnutrition have much less to do with sheer agricultural productivity and more to do with how food is production is governed and how food is distributed^77883163164. The need for the environmentally-damaging acceleration of agricultural output becomes murkier.

The Contradiction of Cheap Food

You could argue that productivity and access are actually closely linked. According to the laws of supply and demand, increasing productivity increases access because it will drive the prices of food down. The massive productivity increases in agriculture over the past century has seen an equally massive decrease in food prices, with many food commodities at 67-87% of their 1900 price¹⁵⁸. It’s hard to dispute that food prices are an important factor in food security; rising prices of food are not ever really thought of as a good sign precisely because it points to a decreased access to food.

With the hopes of avoiding high food prices and creating caloric abundance countries then would have an interest in developing agricultural productivity as much as possible, if not the farmers themselves. Yet then a crisis of overproduction occurs, in which a massive surplus of food does indeed drive prices down, but to the point where farmers are unable to recoup the costs of production.

A great deal of US agricultural policy over the past century is precisely about managing this overproduction. In the US, during downward prices and overproduction in the lead up to the Great Depression, the US had “breadlines knee-deep in wheat”¹⁶⁵. The US government paid farmers leave land idle during the Great Depression to avoid overproduction and consequent price drops. Controversy around the policy in 1949–framed as paying farmers to not work–led to a different system establishing price floors, guaranteeing farmers a minimum price for their product, thus encouraging them to maximize use of their land¹⁶⁶. This also meant that these downstream industries paid the farmers’ incomes–and included conservation incentives which discouraged intensified exploitation of more and more land¹⁶⁷. Some farmers support a return to these policies^168169167, with a recognition that the drastically low prices contemporary subsidies enable also undermine the livelihoods of farmers in other countries–because cheap US exports set the price for other countries’ agricultural products and enable a dependency on imports to the detriment of self-sufficient agriculture, which then allows agribusiness to buy up unused land to produce cash crops for export¹⁷⁰¹⁷¹.

One of the consequences of overproduction is a squeeze where farmers can’t recoup their costs of production. In the US, the 1970 price of corn was about $8 a bushel in today’s dollars. Today it’s about $4 a bushel. At the same time, the price of fertilizers have doubled¹⁷². Trends like this mean that, for many crops, prices have been below production costs for decades¹⁶⁹. The subsidies which are meant to compensate for the lost income from these lower prices do not adequately fill the gap, with farm incomes–with subsidies included–seeing large declines¹⁶⁹. US farm income has been decreasing¹⁷³, down to 50% of 2013 income in 2018¹⁷⁴. In Canada, farmer market net income—income after subtracting government payments, has been negative at least as of 2010¹⁷⁵.

Debt has long played a big role in agriculture, helping to smooth out the staggered nature of farm expenditure and income–many expenses are upfront, but income isn’t realized until after the harvest¹⁷⁶. This difference is especially large in input-intensive agriculture, with the amount of chemicals used. A bad harvest can disrupt this cycle, leading to falling behind on loans and jeopardizing future access to capital¹⁷⁶.

Against the backdrop of this price-cost squeeze, this debt can only accumulate. In the US, farm debt has been increasing since 2013¹⁷⁴. At the same time the business of farm loans is expanding–the fertilizer company Nutrien expanded its farm loan program threefold to $6 billion¹⁷⁷, further profiting off of a problem their expensive input regime created in the first place. Because incomes have been declining, farm loan delinquencies have hit a 9 year high¹⁷⁸.

At the same time, these low prices, which are encouraged by subsidies and crop insurance guarantee an income to farmers, are to the benefit of downstream industries, such as livestock and snack/beverage/food processing companies, lowering the costs of their inputs. Effectively, taxpayers provide discounts for these businesses¹⁶⁹, saving them some $3.9 billion per year^179180168. With these low prices, the corn for a box of Kellog’s Corn Flakes–which makes up 88% of the product–represents only 2.4% of the retail price¹⁸¹. These downstream companies–as well as the input companies, i.e. those that provide fertilizer, chemicals, seeds, machinery, etc–have seen record profits¹⁸² and provided massive payouts to shareholders:

As revealed in a recent report conducted by TUC and the High Pay Center on the behaviors of FTSE100 companies in the last 5 years, listed retailers paid over £2bn to shareholders in 2018, despite none of them being Living Wage Foundation accredited, while listed Food & drinks companies paid almost £14bn – more than they made in net profit (£12.7bn). To put that into perspective, just a tenth of this shareholder pay-out is enough to raise the wages of 1.9 million agriculture workers around the world to a living wage, for example those who produce food at the origin of the food chains that they coordinate as lead firms.¹⁸³

In addition to cheaper inputs, overproduction gives these downstream companies more discretion over what they will and won’t accept. For example, companies adopt stricter quality standards¹⁵⁹ and retailers can return whatever they don’t want or fails to sell in time, at no cost¹⁵⁰, making farming even riskier than before.

For the farmer in this trap, there appear to only be a handful of options. One standard practice is to suppress the wages of farm workers, who are already among the most precarious group of workers. In the US, the H-2A visa system is used to keep wages low for migrant labor, to the point where a quarter to almost a third of such families are below the federal poverty line¹⁸⁴¹⁸⁵ and lowers wages for domestic farm labor as well¹⁸⁶.

Another option is to expand markets for food. The problem with food is that it is relatively inelastic in demand; but one argument sees food waste against a backdrop of increasing food consumption in the US, with marketing that tries to enlarge the American appetite to offload an oversupply of cheap food¹⁵²¹⁵³. Farmers themselves do not typically have a role in this expansion; and most marketed food is in the form of snacks and processed foods of that nature; i.e. products of companies downstream from farming.

The other is to increase production. The low prices mean that productivity and volume is all the more important to make a living, which encourages environmentally harmful but output maximizing processes and the expansion of agricultural land¹⁶⁷¹⁶⁵. This only makes the problem worse–as farmers attempt to stay afloat by accelerating production, the markets for their crops end up flooded, further driving prices down.

The other is to overproduce to hedge against small harvests but withhold excess to keep prices high¹⁵⁹. This is why we see for example perfectly good food left unharvested to rot. This behavior seems unreasonable–it is in many ways–but is rational, considering the farmer’s position. When dealing with large retailers, the situation is such that if they fail to supply the agreed amount, they are dropped as a supplier¹⁵⁰¹⁵⁹, so the rational move is to overproduce just in case.

All of this is made easier to endure or even profit from with scale. The high costs of inputs and low prices of output are especially felt by smaller farms. We see this in US farming: almost 80% of small family farms’ operating profit margin are in the “high risk level”. In contrast, large-scale farms tend to be at a “low risk level”¹⁷³⁴⁴. Larger farms gain better access to loans and better prices¹⁸⁷. The current subsidy system exacerbates this, disproportionately benefiting larger farms who aren’t struggling in the price-cost squeeze^441662361188 and also tend to be more environmentally destructive²⁹, even when accounting for the larger cropland¹⁷³. Numbers range from 75-85% of farm subsidies going to only 10-15% of farms¹⁸⁹³⁹, and the 2018 Farm Bill has not really altered this dynamic¹⁹⁰. The advantages of highly-capitalized, larger farms contributed to the consolidation of farming into fewer bigger farms¹⁸⁷. In the US some 3 million farms went out of business in the postwar period¹⁹¹.

Even though dealing with agricultural overproduction has been a chronic problem of US agriculture, it came to have its usefulness in achieving US geopolitical goals. Harriet Friedmann¹⁷¹ and Philip McMichael¹⁶⁴ outline a succession of “food regimes” where agricultural power is applied to capitalist expansion. Following World War II, the US instrumentalized its agricultural surplus by directing it towards contributing to European reconstruction and using food aid to slow the spread of communism^28171164. Later, finding outlets for this surplus itself became a geopolitical goal. For example, developing markets for agricultural products through World Bank and IMF structural adjustment programs and WTO free-trade agreements in the Global South^19228164193 . The development of such markets is itself partly accomplished by the surplus of US agricultural overproduction, which undermines local agriculture with low prices and disrupting food security. Rather than developing food self-sufficiency, the focus is on economic development to increase the affordability of imported food¹⁹⁴¹⁷¹. In a strangely circular way, agriculture is no longer for growing food, but for growing commodities with which to buy food.

Part of these economic development programs is the consolidation of productive land and its re-orientation towards cash crops, further undermining food security¹⁹⁵. This is the case in Zimbabwe, where the areas with the highest agricultural productivity also have the worst chronic malnutrition¹⁹⁶. At the same time, the use of non-tariff barriers, such as sanitary and phytosanitary measures, effectively allow exporting nations to protect their own markets and make it so that only highly-capitalized operations can absorb the costs these additional measures require¹⁹⁷.

Figures like Thomas Sankara saw how food aid was exploitative and that self-sufficiency the route that preserved autonomy:

However, major organizations like the UN’s FAO, World Bank, and the IMF and foundations like the Gates Foundation continue to press that low productivity is the main problem with food security and thus synthetic fertilizer, high-yielding varieties, and other Green Revolution technologies need greater levels of adoption in developing countries ¹⁹⁸¹⁹⁹.

Fanning the Flames

Mexico

The Green Revolution was first coordinated as a development strategy in Mexico. David Barkin describes how these programs saw an increase in dependency on food imports²⁰⁰²⁸, even though Mexico has all it needs for food self-sufficiency²⁰¹. Even worse, the expansion of the agricultural industry is in parallel with decreasing food consumption. These programs of productivity increases are driven with a focus on increasing export-oriented crops, crops for markets, rather than for improving self-sufficiency²⁰¹¹⁸⁸. Achieving the maximum productivity of a land isn’t possible without the capital required for the purchase of synthetic fertilizer, pesticides, high-yielding variety seeds, and so on. Poorer farmers are displaced by those with that necessary capital, instead needing to find off-farm work to buy the imported food^{61201187202188}.

This process of undermining self-sufficiency has the consequence of creating new markets for countries to sell their food surpluses to, and facilitates the process of proletarianization, creating a pool of cheap labor²⁰¹¹⁹⁸. It also serves other geopolitical goals as well. Barkin quotes one American diplomat commenting on Micronesia, who plainly put it:

If we examine our effects on the island culture, we see they have a real case. Before the U. S. arrived, the natives were self-sufficient, they picked their food off the trees or fished. Now, since we have them hooked on consumer goods, they’d starve without a can opener. Some of the radical independence leaders want to reverse this and develop the old self-sufficiency. We can’t blame them. However, we can’t leave. We need our military bases there. We have no choice. As some of my friends in Washington say, you’ve got to grab them by the balls, their minds and hearts will follow ²⁰¹

Viewed this way, the Green Revolution in Mexico has a continuity with the colonization of Latin America, in which colonists monopolized the most fertile land, away from their function of sustaining the local population and instead towards producing profitable crops for export. Those who used to farm those lands are marginalized–if they weren’t outright killed–towards less productive lands with greater water scarcity¹⁹⁸.

Sub-Saharan Africa

Sub-Saharan Africa (SSA) is constant target for Green Revolution development²⁰³. Food imports in SSA have increased for decades²⁰², and part of this is blamed on low productivity due to low fertilizer usage²⁰⁴⁸ and high levels of soil degradation and erosion²⁰⁵. Fertilizer is especially expensive in SSA, in large part due to high transportation costs²⁰⁴ (some of this might be addressed by recent development of domestic production capacity²⁰⁶) ; farmers there can pay as much as five times as a farmer in Europe¹⁸¹⁷. Because of the overapplication of fertilizers elsewhere, and the accompanying environmental harm, some call for increasing fertilizer use in SSA with a corresponding or larger decrease elsewhere (for example, 588kg nitrogen fertilizer/ha/year in northern China vs 7kg in Kenya)²⁰⁷.

The soil degradation which supposedly necessitates synthetic fertilizer in SSA is often attributed to poor management, a form of development rhetoric that frames peasants as ignorant about how best to work their land^{198188195205199}. More recent degradation looks to have more to do with climatic and biophysical factors²⁰⁵, but the farming of soil eroded lands has more to do with a process of marginalization–the original farmers had developed their methods of effective soil conservation²⁰⁵¹⁹⁶–where larger landholders, European colonists, and transnational corporations monopolize the highest quality land and displace other farmers to smaller and smaller holdings, where the land has to be worked harder to meet the same needs^188187196. In some cases, it is these transnational companies that overwork the soil, moving on to other fertile land once it’s been exhausted¹⁸⁸.

The increased adoption of synthetic fertilizers might only accelerate this soil degradation. One study looked at Zambia’s input subsidy program and found that the introduction of fertilizer subsidy programs does in fact increase fertilizer usage, which subsequently leads to a reduction in traditional low- or no-input fertility-management methods such as fallowing and intercropping and an increase in continuous cropping–that is, a shift towards more intensive agriculture that disrupts natural cycles of soil restoration^208209210. This shift in turn contributes to reduced levels of soil organic matter, which is important for maintaining soil health, which decreases the soil quality. Though there was a short-term boost in productivity, the resulting soil degradation could lead to a net decline in the long run which can only be delayed by further increasing the usage of expensive fertilizer²⁰⁹.

A review of eighty studies on fertilizer subsidies in Africa found that these subsidies are more likely to go to households with larger landholdings, greater wealth, and/or connections to the government, even if programs are ostensibly deliberately designed to avoid such outcomes²⁰⁷²¹¹. Fertilizer use stops once subsidies are withdrawn, so farmers are left with degraded soil and unable to afford the now necessary inputs to stabilize yields. Ultimately, productivity increases peter out after a few years, and there are only negligible effects on improving self-sufficiency, reducing import dependency, and alleviating poverty²¹¹.

These subsidies are expensive: African governments devote a large part of their budgets to these input subsidies, some $1 billion a year²⁰⁹²⁰³. For Zambia, as much as 35% of the country’s agricultural budget was going towards fertilizer subsidies²⁰⁷. At one point, 9% of the Malawi’s entire budget went to these subsidies¹⁹⁵. This is more or less a transfer of public funds to fertilizer producers like Bayer/Monsanto¹⁹⁵²⁰³.

As high as these subsidies are, they can’t compete with those in wealthier countries. OECD countries provide subsidies to their own farmers of about $1 billion per day²¹², and such financial support encompassed 30% of all farm receipts from 2003-2005¹⁵³.

This is compounded by the fact that Green Revolution technologies often come as a package. Specific varieties of crops are necessary to achieve the full yield improvements of synthetic fertilizer use (this varieties are usually intellectual property), and increased sensitivity to pests from fertilizer user necessitates the use of pesticides^20883213. Higher productivity can also require more water, beyond what the water cycle normally provides, so irrigation also increases⁶¹.

In the case of Malawi, the suggested alternative to fertilizer subsidies is addressing underlying issues:

policies aimed at relieving the key constraints that our model tries to capture—such as land, credit and labor availability—might go further towards addressing the bottlenecks to on-farm intensification that smallholders in Malawi encounter, and could lead to the adoption of better on-farm practices that reverse soil degradation and improve the long-term viability of small-holder agriculture.²¹⁰

Interestingly, many countries in SSA are agricultural exporters while being net food importers. That is, SSA’s dependency on food imports is in part due to agricultural land and resources being directed for export-oriented production (in a form reminiscent of their colonial histories²¹⁴) rather than supplying the country’s food needs–“even a small substitution in their agricultural export products into food would eliminate their deficits”¹⁹³.

This push towards agricultural production for export is in part driven by this need for income to purchase food imports and in part to manage national debts. The 1990s, Malawi, under the advisement of USAID and the tobacco industry, re-oriented production from food crops to tobacco crops with the goal of paying off its debt, with part of the rationale being that revenue from tobacco exports would be plenty to purchase the food needed to account for reduced production. The consequence, however, was that the ability for Malawian farmers to feed themselves was no longer dependent on their own agricultural ability, but rather on market prices for tobacco–which fell by 37% in 2009, and made it difficult for farmers to feed themselves¹⁹⁵.

Sometimes this shift to export-oriented cash crops is further frustrated by the surpluses of major producers. Where some countries had advantages in producing things like sugar, the rise of processed foods meant fewer people were consuming sugar directly, instead mediated by processed foods–and as such, that opened up the possibility of sugar substitutes, i.e. HFCS, which then undermined countries with the comparative advantage for sugar production, as corn was overproduced and very cheap. Thus these countries were in a double-bind: food import dependency plus declining revenues from crop exports¹⁷¹.

These dynamics appear elsewhere too. In Nepal, though fertilizer subsidies do tend to increase crop yields, environmental problems are exacerbated²¹⁵, and the subsidies may disproportionately benefit farmers who are more connected to the state and/or wealthier to begin with²¹⁵²⁰⁴. In India, initial bumper crops from subsidized fertilizer application then gave way to crop failures and soil depletion²⁰⁸²¹⁶, saw a move away from traditional techniques like crop rotation towards continuous cultivation of export-oriented crops²⁰⁸, and increased stratification between farmers who could afford the Green Revolution technologies and those who could not afford to stay on their land²⁰⁸¹⁸⁸. Vandana Shiva describes deeper social shifts these technologies fomented:

The shift from internal to externally purchased inputs did not merely change ecological processes of agriculture. It also changed the structure of social and political relationships, from those based on mutual (though asymmetric) obligations—within the village to relations of each cultivator directly with banks, seed and fertilizer agencies, food procurement agencies, and electricity and irrigation organisations.²⁰⁸

Hand to Mouth

Content warning: There is some mention of suicide in this section.

This high-input and market-dependent form of agriculture puts farmers into a double-bind: they are both exposed to the impacts of fertilizer prices on their own production costs but also on the costs of their food²¹⁷¹⁸. One study finds that increasing fertilizer prices would decrease crop yields by up to 13%²¹⁸, reducing income.

In addition to the typical weather and climatic risks of agriculture, they are now exposed to the volatility of export restrictions, financial speculation, as well as the weather and climactic risks of wherever they import food from^219220217! Risk has magnified, rather than diffused, through this system^221217219195.

Arrangements such as contract farming integrate smaller farmers into global agricultural industrial supply chains through arrangements that ensure buyers (with substantially more market power, who do not need contribute to the costs of production, also ways around labor laws) bear little of this risk²⁸¹⁹⁵, much like the increasing use of contract workers throughout other industries.

Scalability becomes absolutely vital to generate enough revenue to remain solvent. Farmers with lower yields–i.e. subsistence farmers and those with less access to the capital necessary for scaling up–are the ones at the greatest risk^17832222. Agricultural finance recognizes this, compounding the effect by giving larger farms cheaper loans, along with other advantages such as more opportunities to exploit cheap labor¹⁸⁷. The trap is intensified due to the declining effectiveness of inputs, which have the effect of degrading the soil²¹⁶. The general necessity of absolutely maximizing output for a piece of land encourages the extractivist model of agriculture, in which the soil is “mined”, with nutrients pulled out faster than they can be naturally replaced²⁸; sometimes even faster than they can be “unnaturally” replaced. In India, farmers require three times the fertilizer to achieve the same yields as they used to, while the prices for those crops remain the same or decline²¹⁶. Insolvency is basically inevitable.

When farmers try to re-integrate traditional practices or develop systems parallel to conventional inputs, they are shut down. In Zimbabwe, farmers could not sell their own seeds as alternatives to the engineered hybrids–the practice was made illegal¹⁹⁶.

What are the consequences of this precarity and lack of control over one’s life? One may be an increase in farmer suicides²²³–farming is the profession most at-risk for suicide²²⁴. The farmers most likely to commit suicide are those who have marginal landholdings, focus on cash crops, and are indebted, which are exacerbated by all of the dynamics described above^223208199225.

Another consequence of this precarity is the increasing concentration of agricultural land and production.

Land Grabbing

Farmers, now unable to support themselves on the land, have no choice but to sell it. Agricultural land concentrates into fewer and fewer hands, contiguous with the dispossessions that occurred with colonization. Here, however, these dispossessions result from a mix of economic factors, driven by overproduction, and legal means, with post-colonial states, under the direction of development organizations like the US’s Millennium Challenge Corporation, often facilitating the process by unilaterally replacing traditional land right systems with ones more amenable to foreign investment and ownership^195201226.

More recently foreign acquisition of agricultural land, called “land grabs”, have attracted a lot of attention. These acquisitions may be about food, water, and energy security^227228229, just about expanding agribusiness amidst rising demand for biofuels and other cash crops²²⁸, and for financial speculation¹⁹⁵²³⁰, faciliated by technologies that make their agricultural development easier to manage remotely as financial assets²³¹. The countries where these land grabs are most common in recent years are Sub-Saharan Africa and South Asia²³². In Ethiopia, some 10% of the country’s agricultural land has been handed to foreign investors for commodity crops for export¹⁹⁵. The World Bank and FAO has framed these as “win-win” opportunities for economic development²³³²³⁴ . In Sudan this export-oriented appropriation of land saw higher food prices, monopolized water usage, and resistance from the local population that was met with violence²²⁶.

Sometimes farmers aren’t displaced but are actively involved in the investment process, seeking capital to stay afloat²³¹. Otherwise farmers are displaced¹⁸⁷²²⁶: for example, a British firm acquired 9,000 ha of land in Tanzania, potentially displacing over 11,000 people for biodiesel production²³⁵. The newly managed farms may be highly automated so that there isn’t even much promise of any employment²²⁹, so these displaced people constitute a “surplus” population often relegated to slums¹⁸⁷. Foreign workers are brought in to work the land with relatively little interaction with the local population²³²; the land is meant to operate, it seems, as a virtual extension of the country that owns it.

This is also occurring throughout the global North, where agricultural productivity tends to be higher and property rights are less risky for investors²³¹, though the Food Security is National Security Act was introduced in 2017 to prevent foreign control of US agricultural land²³⁶. In the EU, millions of farms were lost over the past decades¹⁹⁵. In the US, there is the original and ongoing dispossession of indigenous lands and subsequent and ongoing dispossession of black farmers’ land^23723044. US agriculture is already highly concentrated: some 3% of farms in the US are responsible for 50% of the value of all agricultural production²³⁸⁴⁴.

Often this land remains idle. For example, biofuel production slowed when oil became cheap again, and it wasn’t worth developing the land²³⁹.

Back to Biomass

Agricultural products encompass much more than just food; they are also produced as fuel, pharmaceuticals, bioplastics, and so on. Even within the category of “food” there are agricultural products for direct human consumption, those that are used as animal feed, and that used for food processing (e.g. snacks, beverages, oils). These categories are not clear cut either: corn may be grown for any of these, but corn itself is a heterogeneous crop and individual corn varieties are not necessarily fungible. Corn grown for animal feed, for instance, will be starchier than that grown for direct human consumption. Cane or beet sugar, though something we use in cooking directly, is more commonly an input into the snack and beverage industry (where it hasn’t been out competed by corn-derived high fructose corn syrup). The same is true for other such “flex crops”, which have a variety of possible end-uses.

The sustainability of food production and its ability to actually feed people then is deeply connected to these alternative uses of agricultural capacity. For example, expanded demand for biofuels may cause food prices to rise (30% of grain price increases during the 2007-2008 global price food spike is attributed to biofuel demand²⁴⁰) and encourage deforestation from agricultural expansion ^241242217. Here food’s market value dictates whether or not it’s grown instead of potentially more profitable alternatives. But there are many things we can do to reduce the necessity of fuels; we cannot reduce our need for food to live.

Agroecology and Technology

What alternative is there to all of this? The commodification of food–seeing it as something to be produced and sold like factory goods–drives a lot of the dynamics described above. Agroecology that sees food not as something to be manufactured and agriculture not as a factory-like process but as a scaffolding for relationships and a source of autonomy.

Agroecology has some overlap with practices like organic and regenerative agriculture, at least in terms of focusing on environmentally-friendlier techniques, but where it differs is its commitment to social and economic issues outside of the specific production process (sometimes contrasted with a “input substitution” approach to agriculture¹⁹¹). An organic or a regenerative farm’s efforts at sustainable or remediative production might be hampered by market demands, for example²⁴³. That’s sort of reflective of organic agriculture’s trajectory. As Julie Gutman describes in Agrarian Dreams: The Paradox of Organic Farming in California, organic agriculture originally encompassed more radical notions of food system change but was over time shaped to be focused on very specific aspects of the production process and integrated into the existing agribusiness industry²⁴⁴. Agroecology is not itself immune to this kind of co-optation, as a recent report on “junk agroecology” describes²⁴⁵.

Peter Rosset and Miguel Altieri outline agroecological production principles:

The production system must (1) reduce energy and resource use and regulate the overall energy input so that the output:input ratio is high; (2) reduce nutrient losses by effectively containing leaching, runoff, and erosion, and improve nutrient recycling through the use of legumes, organic manure and compost, and other effective recycling mechanisms; (3) encourage local production of food items adapted to the natural and socioeconomic setting; (4) sustain desired net output by preserving natural resources (by minimizing soil degradation); and (5) reduce costs and increase the efficiency and economic viability of small and medium-sized farms, thereby promoting a diverse, potentially resilient agricultural system.¹⁹¹

Actual techniques can encompass many things, such as re-integrating rural-urban nutrient cycles²⁴⁶, using natural fertilizers and nitrogen-fixing cover crops like legumes, soil-conserving and soil-building practices, ecologically-rooted pest management practices, and so on¹⁹¹. These have both environmental and economic benefits. For example: a system integrating perennial native prairie plants can reduce fertilizer use and runoff, saving more than $850 million per year in costs and damage²³, and can increase yields by improving soil health, such as with the use of “fertilizer trees” as in parts of Africa (a practice called “evergreen agriculture”)²⁴⁷²⁴⁸. These techniques can be used while considering their social and political effects. George Washington Carver, for example, advocated the use of legumes (such as peanuts) both as a way of maintaining soil health but also as part of a program for economic and political autonomy²⁴⁹. Agroecological soil management practices can improve fertility without needing to resort to subsidized inputs²⁵⁰, helping break the debt cycle. In general, these approaches have been found to improve crop yields while reducing inputs²⁵⁰.

The acroecology movement is led by peasants and peasant organizations like the Landless Workers Movement (MST), the Zapatista Army for National Liberation (ELZN), and La Via Campesina, with networks encompassing thousands and thousands of small producers across continents^28251252. Peasants are a powerful force; despite centuries of dispossession, over half of the world’s arable land is still under their ownership²⁸, and, despite industrial agriculture’s dominance in our food system, are major producers of food.

Smallholders (farmers with <2ha of land) produce about a third of the world food supply (measured as calories) on about a quarter of the agricultural area (close to half of the food supply if farms <5ha of land are included)²⁵³. What this implies is that smallholders already have productivity that out pace larger-scale operations, even on land that may be marginal. Further research and development of agroecological, low-input methods have the possibility to increase this number even more. However, funding tends to disappear for researchers developing these techniques²¹²²⁵⁴. One example is that, following a $1.7 million donation to Iowa State University by the Koch Foundation (the Koch brothers own Koch Fertilizer), the university’s Aldo Leopold Center for Sustainable Agriculture had its funding pulled after 30 years²⁵⁵. Despite this, agroecological innovation happens outside of formal research settings through peer-to-peer knowledge exchange and development. Unlike the agricultural input industry, which is top-down and where products are pushed onto farmers who have essentially no input in their design, agroecological techniques are often co-created by peasants and refined to their own specific contexts²⁸.

Agroecology is sometimes characterized as a romanticization of labor-intensive subsistence farming²⁵⁶. A lot of labor-saving technologies basically work by substituting human labor with fossil fuels. If we look at the energy efficiency of these technologies, we see a different picture: about five times energy (fossil fuels, mostly) go into one kg of cereal in industrialized countries vs farmers in Africa, with more drastic differences on a crop-by-crop comparison, such as 33 times energy in the US to produce one kilo of maize as a traditional farmer in Mexico¹⁹⁵. There are similar estimates for industrial food production more generally, 3kcal fossil fuel for 1kcal food¹⁵², with estimates as high as 7kcal for 1kcal¹⁵³ or 10kcal²¹³²⁵⁷. That the industrial-style of agriculture (input-heavy) is a net negative in terms of energy has been demonstrated since at least 1960s; which is characterized in terms of crops that are bred to maximize productivity in terms of the valuable parts of the crop (e.g. what is edible) at the expense of other functions, such as pest resistance, which are instead externalized to, for example, pesticides²⁵⁸.

Similarly, agroecological methods have been characterized as equivalent to traditional subsistence practices or the movement as anti-technology²⁵⁹. It may be more appropriate to say that it has a different understanding of technology, measuring it not only by how useful it is in the field, but by what impacts it has on relationships. Does it create a dependency on finite resources and empower large companies? Does it involve buying into a system of debt²⁵⁴? “Technology” is also often conflated with industrial, large-scale, and energy-intensive processes, but traditional labor-saving practices like coppicing²⁶⁰or the ongoing development of perennial grains²⁶¹ can also be considered technologies of sorts. The previously mentioned fertilizer tree system is itself an outcome of research starting in the 1980’s on nitrogen fertilization alternatives, conducted with the participation of farmers²⁴⁸, offering an example of productive agroecological research. The Out of the Woods collective also describe “cyborg ecology”, where technology adoption should not be out of some binary commitment to a constructed purity around traditional techniques or modern technologies but out of evaluating what works without sacrificing ecological and social priorities²⁵¹. The distributed production of solar fertilizers mentioned much earlier might fit this criteria.

Cuba is sometimes pointed to as an example of what agroecological practices could make possible on a larger scale. Cuban agriculture was once more akin to industrial, Green Revolution agriculture found elsewhere, supported by trade deals with the Soviet Union which supplied the necessary inputs: fossil fuels, fertilizers, machinery, etc.¹⁴³²⁵² When the Soviet Union collapsed, these inputs were suddenly unavailable, resulting in GDP and calorie consumption each plummeting by one-third^{262263222264163}. Ongoing US blockades prevented dumping of cheap food into the Cuban market, further forcing Cuba to develop its own self-sufficiency in the absence of these inputs²²².

Through this “Special Period”, land reform redistributed over 1 million hectares of state-owned land to a variety of cooperatives and individuals^{262263143252264} (under usufruct terms, so most of the land is still state-owned¹⁶³). Cuba went on to achieve the highest per capita food growth rates in Latin America²⁵², facilitated by the land reform and the adoption and peer-to-peer transmission of agroecological techniques via peasant movements such as Campesino-a-Campesino^25222214328–the latter of which was ongoing, but provided an opportunity to expand after the forced withdrawal of the industrial regime¹⁶³. Over 65% of the country’s food is produced by peasant family farms on 35% of the total arable land.¹⁶³Cuban urban gardens, organopónicos, also a legacy of the post-Soviet “Special Period”, produce as much as 70% of the fresh vegetables consumed in cities²⁶²²⁶³ and 50% of the produce for the entire country¹⁶³. Lower levels of dissolved nitrogen in the island’s rivers, compared to the US’s Mississippi River Basin, is speculated to be a consequence of this shift in farming practices²⁶⁵.

This is not to say Cuban agriculture is fully agroecological–there are still conventional farms (usually state-run) and varying adoption of agroecological techniques¹⁶³. There is still some dependency on oil for agriculture, though those who have adopted agroecology are more resilient to its absence, such as from US sanctions this past year²⁶⁶.

Cuba’s food system still has other issues, such as with food waste, which is comparable to rates elsewhere, and generally attributed to infrastructural inadequacies rather than the overproduction and market-driven causes in the global north¹⁶³. Yet this still results in inadequate food for many.

Pressure from agribusiness entering into Cuba²⁶⁷, especially with opening Cuba-US relations (for example: a Cargill-associated “US Agriculture Coalition for Cuba” seeks to “to lift the embargo in order to liberalize trade and investment and reestablish Cuba as a market for US products”¹⁶³), and tensions within the country with some leaders advocating for higher-input/more conventional systems²⁶³. Changing relationships with the US may be the largest determinant of how Cuban agriculture changes in the future, with fears that it may become more focused on export for organic food markets and less on supplying domestic needs¹⁶³.

Cuba’s dependency on expensive food imports has also increased¹⁴³ which has made expanded self-sufficiency a priority. Laws were passed in 2008 and 2012 to enact a further distribution of 1.7 million hectares of land to over 200,000 farmers, with roughly an additional 900,00 hectares up for distribution in 2017¹⁶³. The impacts so far are unclear – the overall amount of food imports is contested, with one figure placing it at 40%¹⁶³. Hurricanes increase Cuba’s dependency on food imports, though the agroecological farms tend to recover more quickly than monoculture farms²⁶³¹⁶³. Though Cuba’s food system is not perfect, it shows that an agroecological system is worthwhile and achievable.

Conclusion: Healing the Rift

The focus here has been on food, but these issues will become more pressing as the economy–if we are to adequately address climate change–shifts back to one with greater dependence on biomass for materials. Consider that a tremendous amount of materials are petroleum-derived or otherwise produced from non-renewable resources²⁴⁶–including those that industrial agriculture rely on²⁸. Switching to bioplastics, for instance, can have greater environmental impacts and greenhouse gas emissions than petroleum-based plastics due to land use change²⁶⁸.

Climate change will lead to an aggregate decrease in agricultural productivity^26919510, increasing the number and scale of droughts and heatwaves²⁷⁰ and eroding and submerging productive land²⁷¹. Food prices would rise and possibly encourage an influx of investors and speculation. We could see climate change and its effects on food production used to increase pressure of technocratic development authorities to act with a stronger mandate to implement Green Revolution-style programs and strengthen the narrative of peasants using land unproductively and as justification for land grabs. Climate change is already having an effect on displacing farmers²²⁸.

The key cause for these issues with conventional agriculture stem from the fact that it is not designed to fulfill a need—i.e. sustenance and nutrition, though it is often spoken as such in policy and popular discourse—but rather, it is designed to produce and sell commodities. There is a huge dissonance between the refrain of FAO population projections and the need to increase food production to feed them, when the current system of tremendous output fails to feed even the people we have now.

A report from Nature notes that, “[p]roducing enough food for the world’s population in 2050 will be easy.”²¹². The report goes on to note, the more urgent challenges are sustainable productivity and, more importantly, poverty–poverty that is itself produced by this productivist agricultural regime. As the authors note:

The 2008 food crisis, which pushed around 100 million people into hunger, was not so much a result of a food shortage as of a market volatility — with causes going far beyond supply and demand — that sent prices through the roof and sparked riots in several countries.²¹²

A recent paper similarly suggests that almost 11 billion people can be adequately fed within planetary boundaries, but it requires a radical restructuring of global agriculture¹⁴.

One way to support this restructuring is to return wealth that has been extracted from these post-colonial nations and invest it in agricultural infrastructure that support greater food autonomy. It can mean increasing public research for agroecological methods and land reform, such as redistributing land back to those who have stewarded it for generations¹⁹⁸. It requires moving on from the concept of food security–market access to food–to food sovereignty: producing food for need, prioritizing agroecological methods, local production and control^195272153, and restoring more context-appropriate traditional crops where they have been displaced by commodity crops¹⁹⁶.

The problems of fertilizer are inseparable from the broader political and economic problems of food production. Throughout history people have recognized how a community’s control over their own means of living is a prerequisite for flourishing and self-determination. As Vann R. Newkirk II notes in his “The Great Land Robbery”:

There’s a reason the fabled promise that spread among freedmen after the Civil War was not a check, a job, or a refundable tax credit, but 40 acres of farmland to call home.

Modeling demonic divinities

Growing up I was very fond of Buddhist art and I would make an effort to visit various Buddhist temples whenever my family would go to China to see my grandparents. I found the paintings and sculptures of terrifying figures to be strangely calming, and only later did I learn that these fearsome beings are in fact benevolent, wrathful “protectors”.

While visiting Kira’s parents last week I had the opportunity to read through the catalogue for one of the Rubin Museum’s opening exhibitions, “Demonic Divine” which was exactly on this topic. I’ve wanted to find some way to draw from this tradition of “divine demons” for Fugue and reading through the catalogue helped me figure out how I could do that.

Without going into too much detail, one fundamental Fugue world-element is the phenomenon of “hauntings”, which are mental blocks and fears that characters experience. The character progression system is based around confronting and resolving these hauntings (perhaps in addition to a simpler experience-based system, not really sure yet). The way I’ve pictured this is that progression happens along a path (the character’s “道”/“way” in a sense) and these hauntings are literal roadblocks, represented by these demons, along this path. So a key question is how exactly does this process of resolving hauntings¹ work?

One other feature I want to include is an in-game game (tentatively called “Gem”), sort of like Triple Triad in Final Fantasy VIII. Including something like that lets you do a lot of interesting things: explore its cultural significance, show how different places and people interpret the game (perhaps with different house rules and variations), explore all the infrastructure that comes up around a relatively simple game once it becomes a major phenomenon (rules committees, judges, player associations, its role in national conflict, etc), and in general be a means of showing different value systems, thought processes, philosophies, etc, like Azad in Iain M. Banks’ The Player of Games. I’m still working out Gem’s design, but so far it’s like some combination of chess, checkers, and Chinese checkers, with the variability of a card game–i.e. many different pieces with different abilities, and players can build a “deck” of pieces to play with like they might build a deck in Magic: The Gathering.

My current thinking is that to get through a demonic roadblock you challenge the demon to a game of Gem, in a “chess with death” style. The demon itself is represented as a piece on the game board that you need to remove, sort of like the king in chess. However the goal of the game is not necessarily to “defeat” the demon. Each demon represents a mental block/fear that has a complementary virtue; the point is not to excise the demon from your mind but to recognize its value as an ally. In the context of Gem, this means capturing the demon instead of destroying it. If I design Gem right, then capturing the demon will be comparatively difficult to destroying it. If you’re able to capture it (maybe “convert” is a better term) then you have access to that piece for future games of Gem and perhaps some kind of bonus “blessing” for the character outside of Gem.

I’ve been working on modeling the first demon, based on Chitipati. Chitipati is two skeletons depicted with a frame of flames which acts as a memento mori and more generally represent endless change:

The first part I tried to model is the flame halo. Similar features are common in wrathful diety iconography so this model would be useful for other demons too. I made several attempts at modeling these flames using only Blender’s geometry nodes with the hope of generating them procedurally, but they never came out well (maybe because of my inexperience with geometry nodes). In the end I hand-modeled five flames and use geometry nodes to distribute them in an arch, which I’m happy with:

When I moved on to modeling Chitipati I started with a fair amount of detail. It did not look very good, so I gave it another shot from scratch using a rougher approach, and it looked a lot better (though the hands look TERRIBLE…I’ll give them more attention later maybe, to organize them as particular mudras). Relying on textures for detail rather than capturing it in the mesh makes a lot more sense for this game, especially if I’m using photo-textures. And because I’m rendering out at a lower resolution, finer mesh detail won’t really show up anyways.

A side note: I’m modeling this demon as only one skeleton, rather than Chitipati’s two.

This test render’s resolution is high enough where the lack of mesh detail is apparent (pretty boxy looking in some parts). When rendered in-game the low detail looks more appropriate:

Infilling and seamless tiling with GIMP

One idea for the flame halo was to use content-aware fill or a similar infilling algorithm to generate a full image of the flame background and then some how turn that into a 3d model. That route didn’t really work–I couldn’t make it look good; it just looked flat and janky–but I did find an infilling plugin for GIMP called Resynthesizer that is really promising. It didn’t work very well for the flame frame (perhaps because it’s an illustration or because there isn’t a lot of content to reference for the infilling) but it works much better with texture photography. Here’s an example using an image of some overgrowth:

After applying the Resynthesis plugin (using Filters > Map > Resynthesize) and checking “Make horizontally tileable” and “Make vertically tileable”:

That result looks great, but it wasn’t actually seamlessly tileable. Fortunately there is another filter, Filters > Map > Tile Seamless, that handles that:

It looks really good tiled:

This texture has a lot going on so it may be an easier example. If you look closely at the seamless tile version you can see some ghost leaves overlapping other leaves, which might be more noticeable in a sparser texture.

It’s more apparent in these textures:

You can see some mushiness/blurriness/choppiness in the patterns from the overlapping areas. It’s not terrible, especially for textures you won’t look closely at and in the context of the game’s downsampled resolution. Again, part of the game’s aesthetic is about giving a big margin of error for quick-and-dirty assets, whether through low poly modeling or iffy automated tools.

When I have more time I want to see about integrating this directly into the Texture Editor from the last post; it would be nice to not have to open up GIMP every time to process images this way.

As a side note, it’s been awhile since I’ve used GIMP and I’m impressed by this latest version! Feels like it had a big glow up like Blender did.

For the past year I worked on Half-Earth Socialism, an online game accompanying the book of the same name by Drew Pendergrass and Troy Vettese (Verso 2022). The game launched at the beginning of May; you can play it here. This post will make more sense after you’ve played the game!

This post is adapted from a talk I gave at Trust (who organized the project) and goes a bit into the design and development process of the game.

Genesis

Trust was approached early in 2021 about developing a website to accompany the forthcoming book Half-Earth Socialism, which would be published a little more than a year later. To very, very briefly summarize the book (the book itself is a quick read so I encourage you to give it a look!): the authors focus on land use as the central variable of concern for the health of the planet (hence the name “Half-Earth” socialism, building off of E.O. Wilson’s idea of the same name) and emphasize the need for rational democratic planning to make decisions around the future of the world. For example: how much land should be devoted to energy production, and how much to food?

Democratic planning requires some way for people to meaningfully engage with plans: to understand them, evaluate them, and make their own plans. The original proposal for the site was based around a linear programming calculator where people could play with the parameters of a model that Drew wrote. I imagined it as something akin to Chris Crawford’s Balance of the Planet (1990), where the player is similarly adjusting parameters of a global model to influence planetary health.

In this original version you could choose your energy mix, energy use levels, meat consumption, and so on. The model would figure out the allocation of land, emissions, and so on that were required. Using this model you can easily compare results and see, for example, that veganism opens up quite a lot of land for energy production. But part of the book’s appeal is the vignettes throughout that imagine what life might be like under Half-Earth socialism or what it might be like were it not to happen. We wanted to take this model and build a richer, more narrative experience around it to reflect some of the feeling those vignettes evoked: the pacing of such a world, people’s concerns and values, and the social fabric of their lives.

Reading Group

Before any development began we first participated in a reading group, organized by Chiara Di Leone, covering topics like: socialist cybernetics, cybernetic planning, complex systems management interface design, climate modeling, and games that we found interesting or related to the these topics. These helped us coalesce on a set of mechanics, design elements, constraints, and feelings to develop the game around.

Two games that we looked closely at were 11 Bit Studio’s Frostpunk and Nerial’s Reigns. Frostpunk’s gameplay is centered around one primary variable (heat) and is filled with many brutal, no-win policy decisions around managing morale. The game is really well organized around its primary variable–it’s very easy to see how heat is distributed, what’s producing it, how it decays, and so on, so even though the decisions can be difficult, you’re seldom disoriented or confused about what your priorities are. The game is, however, very dark and depressing. In contrast to Half-Earth Socialism’s more utopian outlook, Frostpunk is a never-ending crisis. We knew that we wanted our game’s arc to be different: the beginning is a difficult struggle to get through, but if you do well the game opens up into a world better than the pre-crisis past. There’s light on the other side!

We knew early on that we wanted the game to be a web game, keeping in line with the original website idea and making sure that it’s easy to access. If you’re making a web game then you really need to consider mobile usage. We expected that most people would share the game on Twitter and thus others would likely encounter it on their phones and want to give it a try right there. That assumption’s held: the majority of plays so far have been on mobile resolutions. Reigns was a main inspiration of how you could make a simple yet deeply engaging game for mobile. It has one main interaction–swiping left or right–but it’s enough to support a great branching narrative. It’s also very flexible in terms of time commitment: you can play a session for a few minutes or an hour if you want. That’s something we wanted to replicate in our game–players could finish a run in 5 to 10 minutes, do something else, and come back and try a different approach later. That isn’t how things turned out (people have spent 30 minutes just reading the starting cards) and in the end the scope of our game was just too complex to really reproduce these elements of Reigns.

From the whole reading group process came a few orienting values:

• Accuracy: Most games prioritize a fun and engaging experience, and often systems that have real-world correlates are simplified in service of this priority. But because of the nature of the book and the weight of the subject, we wanted to prioritize rigor and accuracy a bit more than a typical game would. This was a huge challenge because representing something as complicated as the planet and human economic activity requires a great deal of simplification regardless of commitment to accuracy, and with the complexity of the planet and the global economy, details matter a lot. And often details that don’t seem to matter end up becoming quite important later on.
• Amusing: Climate change and ecological disaster are already very weighty topics without us needing to exaggerate it. We wanted to bring some levity to the game so that the player doesn’t feel pummeled by depressing thing after depressing thing (although it does kind of happen because of the subject matter). So using dialogue, character design, etc, as outlets to lower the “seriousness” of the game and leave plenty of space for the content to do that.
• Expressive: Another goal was for the game to play like a political compass quiz, in a way. We had a few player “builds” (above) in mind that made play feel expressive.
• Quick: Shorter games that you play in rapid succession, to try different things. I think originally we wanted a session to be 5-10min…but sessions last far longer now. It takes at least 5 minutes just to soak in the game’s starting content!

As mentioned above, take a cue from the book and convey the feeling of living under HES through dialogue and other narrative elements.

Development proceeded in roughly five categories:

• Story: the main beats of the game and game dialogue.
• Art Direction: character design, graphics, game feel, and so on.
• Game Design: the game’s primary mechanics and interactions.
• Legibility: the game’s information design, how players access/read the information they need to make decisions and how they make those decisions.
• Technical Requirements: various constraints around the game’s technical infrastructure and architecture

Story

We wanted to give the game world a richer feel through events, which give us opportunities to world-build and develop character personalities. They’re also one main way players get feedback about their plans. More importantly they allow us to represent way more than we could with the core game model alone, which focuses on a relatively limited set of variables (water usage, land usage, electricity production, fuel production, etc). With a very simple probability system we can represent a much wider range of events like mass coral bleaching events or cultural changes like a new cuisine trend of eating invasive jellyfish, without needing to make the model itself much more complex.

Of course, the events we have in the game pale in comparison to all the possible events one might think of for the future. They allow us to represent more detail than we could otherwise but open up an overwhelming amount of things we could represent. It’s one of those areas where we had to just stop adding things at some point, even though we probably could have added hundreds more.

Technical Requirements

Mobile and web support were the primary technical constraints, and those came with their usual challenges (mostly cross-browser compatibility). The more interesting technical requirements were related to the models and data that we needed.

Early on we knew we wanted to have some kind of climate model running, but climate models are usually huge, requiring supercomputers long periods to run. Drew suggested Hector, which is a “simple climate model”. It runs quickly on commodity hardware, but of course lacks the depth and detail of its massive counterparts. From a technical standpoint I didn’t really want to have a server crunching a climate model for several players at once, even if it’s a relatively simple one. We managed to get it to run directly in the browser so that each player runs their own model independently.

We have some other models running too:

• a biome model, not running on mobile but on desktop, which colors the world according to temperature and precipitation changes over time.
• a linear programming model for determining production resource allocation and guided assistance in planning. This ended up being reduced to a much simpler form because there weren’t models available for the browser that handled the kind of optimization we needed.

The other major set of requirements were data requirements. For processes there were input requirements per unit output, impacts (e.g. CO2 emissions) per unit output, and current global process mixes. We also needed current emissions, current biodiversity loss, population projections aggregated for each region, global per-capita demand for each of our outputs, estimate per-capita impacts based on regional income levels, and impacts and inputs for different sectors/industries.

It’s often very difficult to track down good numbers for these. Data may be available at regional or national levels, but not globally. Or there may be a lot of variability in estimates. Some technologies like vertical farming and cellular agriculture are very new, so there are only estimates for very limited cases, if at all. For example with vertical farming we could only find a couple sources that on a one or two crops published by vertical farming startups (so very unclear how trustworthy the numbers are), and instead relied more on values from more general indoor greenhouse farming.

Art Direction

The game’s art direction is mostly Son La’s department but I’ll briefly mention my two main contributions.

The first is the globe’s design, which took an embarrassingly long time (I’m really inexperienced with shaders). Our main aesthetic reference was a sort of retro-computing, so we played a lot with poorer color representation (dithering) and lower resolutions (pixelation).

The final version drew from this graphic that was produced for an article about the book:

And here’s the final version:

One major element of the game’s feel was the imagery we used to represent projects, processes, and events. We had a very small team, and it was really only Son La creating visual content (and also working on the UI development). We ended up with 238 events, 123 projects, 29 processes, 9 industries, and 20 regions, all of which needed images, so ~420 images total. There was no way we could create all of that on our own, so we looked at CC-licensed and public domain imagery. The problem with sourcing images that way is that they vary a lot in quality and style, and they kind of just look like digital photos. We played around with ways to process them so that they were more interesting and consistent and ended up with the following line:

Legibility

There is a lot to consider when assembling your plan and so a major challenge was making information available and clear to the player. All throughout the interface are tooltips and “factors” cards that breakdown what’s contributing to whatever variable you clicked on. The hope is that whatever info you need to make a decision is available quickly, but it still is and feels like a lot of information!

Game Design

The most challenging part of the project was the game design. There were a lot of different things we wanted to communicate, different feelings we wanted to evoke, and so that led us down a few different design paths. But the biggest difficult was that the game is meant to somewhat accurately represent a set of very complex systems–we wanted some legitimacy and rhetorical weight behind what happens in the game. This commitment to accuracy of complexity is directly at odds with making an entertaining and accessible game. Most games are not complex in terms of their mechanics. Even games that are very deep do not have to be complex, and they often aren’t. Truly complex systems make for unfun games because they in their very nature inscrutable, so they become a very frustrating experience. You never quite know why something is happening: is it because of something you did, is it because of something you didn’t do, or did it have nothing to do with you at all? Part of what makes a game fun is learning its rules and systems, and one reasonably expects consistency, predictability, and legibility in how the game responds to your actions. Complex systems don’t care.

Balancing was also very difficult to do with such a complex game. We couldn’t anticipate all the paths or strategies a player might try. Ultimately we hoped to avoid players finding a strategy that works in the game that wouldn’t work in the real world because of some detail we left unmodeled, but it would require a lot more testing to have some assurance that we succeeded.

I’ll briefly describe two sets of ideas we had that I really liked but got cut for one reason or another.

Regional System

I remember “space” being a big concern. Given the book’s focus on land use, how we deal with space is really important. But it came in conflict with priorities like accuracy. Games might abstract space to make it more manageable, like using larger units of space which then reduces the level of detail you can represent spatially; or they might limit their focus e.g. to a single region or map or level. The earth is really big. We can’t really represent it in great detail without just blasting the player with stuff to manage. So we could limit the player’s focus at a given time to a single region. That’s basically where this design came from.

One bonus was the opportunity for more visual feedback/eye candy: you could zoom into a region and see wildlife return, to make your impacts on the world feel rewarding and more obvious.

But there was a more important game design element to this regional system. We wanted to avoid was the “god view” in games, where the player unilaterally makes decisions of all kinds, which is contra the democratic planning that’s emphasized in the book. It’s very challenging to design a game about global planning without making it a god game! Games are often fantasies of control…representing democracy in a game very difficult because it can disrupt that fantasy and make the game a frustrating experience. It’s interesting how certain kinds of friction are expected in a game, like a hard boss fight being difficult, but others, like having your decisions questioned or ignored, are not.

I don’t think we succeeded in avoiding the god view. The parliamentary system is supposed to represent that to some degree, and you can be ousted from power if you’re too unpopular. But that’s not quite the same as democratic planning. One iteration of this regional system idea had a greater and more autonomous role for individual regions (in the current game they mostly just exist to spatialize the game a bit, but they don’t really do much on their own). The player would set targets and maybe some specific policies/projects but regions would go and figure out how to achieve those targets on their own. There would be more bargaining with regions to accept targets or to achieve them in a particular way.

A further iteration on this idea–which was definitely out of reach given our constraints–was a multiplayer regional system. Different people play different regions, and perhaps elect one player to be a global planner for a term. The global planner mediates regional relationships: regions have to negotiate with one another, like if I’m representing East Asia I want North America to reduce their energy usage in exchange for reducing my coal usage or something.

Turn-based System

The other major concept was a turn-based game, similar to Into the Breach. Each turn is some fixed time amount, and you have a preview of everything that will happen in the next turn or next n turns. For example: this natural gas plant will emit this much methane next turn, this patch of permafrost has 50% chance of melting by next turn, this hurricane will move left 2 tiles next turn, etc. It doesn’t really work for the global scale, since you can’t deal with things like an individual power plant, but I still think you could make a fun and interesting game this way if you could get away with more simplifications. And it’s not clear what amount of time a turn should represent. The hurricane movement, for example, requires a much shorter turn time, but other decisions like building new power plants are better suited to turns of a year or more. Similarly, it’s not clear what the spatial resolution should be. How many hexagons should the globe be divided into? A hurricane and a power plant are on two different scales.

One advantage of this design was that scientific uncertainty can more easily be a bigger part of the game. That natural gas plant might emit this much methane, but you don’t really know without better sensors. That 50% estimate for that patch of permafrost is based on your best climate models, but if you invest in improving them and training more climate scientists you’d have a better estimate. In this way the benefits of good long-term planning manifest as “powerups” in a sense that make the shorter game loop easier.

Card System

We ended up settling on a card-based system. It felt more familiar and easier conceptually, giving players something to hold on to while we barraged them with other new information to absorb. Cards also provide a convenient way to compartmentalize “abilities” and a give players a discrete object to think with. Basically all player actions are expressed through manipulating cards in some way.

There are still a lot of ways to use cards. One idea was to have your plan be something like equipment slots in an RPG, like Diablo (on the left here). But instead of a head slot for your helmet you’d have a concrete slot for your concrete production technology, and another one for your transport policy, and so on.

This is kind of how it works in Civilization 6 (on the right). This form ended up being too limiting because there are many projects we wanted to include that don’t fit neatly into an existing slot, thus requiring many single-purpose slots, and just making the whole thing clunky and confusing.

Above are a couple other ideas—on the left we have something based around a deck. One idea was that your “plan” is a deck of cards that you assemble and use to react to events, under the idea that a good plan prepares you for the future. So if your scientists are telling you a major heatwave is likely, you’d have a mass cooling center policy prepared to respond to that heatwave, if it does happen.

This ended up not really working because it feels like busywork for the player—you’re making two redundant decisions, the decision to establish the cooling centers, and the decision to use them when the heatwaves occurs. In trying to minimize extra actions, we’d assume that if you have the cooling centers in place, you’d want to use them.

Above is one of our prototypes for card interactions. This one is a Reigns-like interface, with four directions instead of two. We ditched it because as I mentioned the Reigns reactive playstyle didn’t quite fit what we needed.

We settled on this “scanning” interaction, which has a retro computing punchcard-like vibe. Conceptually this is like the deck idea without the redundancy: in a way you’re scanning cards to add them to your deck (plan) so they make bad future events less likely.

We also wanted to add a political aspect to the game, in part to make gameplay more interesting (the player can’t just do everything they want to do–to lessen the “god view” problem) and also add some drama and an opportunity to develop the world through some strong personalities. They also give players some scaffolding to develop playstyles. There’s more clear guidance on what an accelerationist might want, for example, to nudge the player towards using those cards. The parliament system was also conceived to bring some of the “democratic” side of “democratic planning”, but I don’t think we succeeded in that. It’s my main regret of the game. I feel that we would have needed to design the game very differently for that to have worked, and at that point it was too late to make such major changes.

Tooling

One last thing I want to show off is the Half-Earth editor, where all the content (dialogue, events, regions, projects, processes, and so on) and model parameters are written. Whenever I embark on a project like this, one of the first things I do (once the main architecture/schemas are sorted out) is build content authoring tools. It makes later work much quicker, makes it easier to experiment with the content, helps me think through more ideas and possible conflicts in that level of the design, and makes it much easier to bring on others (like writers/researchers Lucy Chinen and Spencer Roberts) to contribute without needing to muck about in the code.

Final note

Half-Earth Socialism was one of the bigger projects I’ve worked on, and I’m proud of what we accomplished with such a small team. Our ultimate ambition was way beyond our capacity and resources, but we managed to achieve quite a lot of it. The game was made possible because of these people:

Wow, it’s been almost a year since I last updated this blog.

I haven’t had time to work on Fugue until a month or so ago. Since then I’ve been chipping away at more tooling. Once the core game mechanics/systems are in place, I’m expecting that most of the time will be spent creating content for the game: writing, modeling, environment design, etc. So I’m building out tooling and figuring out strategies to streamline all of these processes.

Godot makes it easy to develop editor plugins that integrate relatively seamlessly. It’s not without its challenges and frustrations but those are more to do with Godot in general than specifically about their plugin development process (see below).

Writing

The game will play out mostly through characters saying and doing things, and these actions need to be specified in a way where I don’t need to meticulously program each one. Previously the game’s narrative elements used “Dialogue” as the main organizing element, focusing on spoken dialogue, and let me write “scripts” of spoken dialogue lines with a playback system to have the appropriate characters say their lines in order. That ended up being too limiting because I want to write not only dialogue but to specify various actions/stage directions to write scripts that basically describe entire scenes, including character movement and animation, sound and environmental cues, and so on. So I restructured that whole system around “Sequence” as the main organizing element, with “Dialogue” as a sub-component.

A Sequence is composed of “Actions”, which include dialogue lines, choice prompts, animation triggers, game variable setting, character movement and rotation, etc. At the time of writing the following actions are available:

• Line (L): A line of dialogue, spoken by a single Actor.
• Decision (%): A set of choices that the player must choose from.
• VoiceOver (V): A line of voice-over dialogue. The difference between this and Line is that it does not require the speaking actor to be present and shows in a fixed position on screen.
• Prompt (?): Basically a combination of Line and Decision. A line of dialogue is displayed with the decision’s choices.
• Pause (#): A blocking pause in the sequence
• SetVar (=): Set a state variable to the specified value (strings only). There are a number of targets
  • Global: Set it on the global state
  • Sequence: Set it on the local state (local to the current sequence). These values persist through multiple executions of the same sequence (i.e. they aren’t reset whenever the sequence is run again).
  • An Actor: Set it on the local state (local to a specific actor).
• PlayAnimation (>): Play an animation with the specified for the specified Actor
• MoveTo (->): Move the Actor to the specified target
  • You can use this to “bounce” the player character if they enter somewhere they aren’t supposed to.
• LookAt (@): Have the Actor look at the specified target
• ToggleNode (N): Toggle the visibility of the specified node. Can fade it in/out (but looks quite janky)
• RepositionNode (>N): Move a node to the position and rotation of the specified target. This happens instantaneously…so you could use it for teleportation; but more likely you’d use it to rearrange a scene while it’s faded out.
• TogglePortal (P): Enable/disable the specified portal.
• AddItem (+): Add an item to the player’s inventory
• PlaySound ())): Play a sound
• Parable (~): Start or end a Parable (Quest)
• Fade (F): Complete fade the scene in or out
• ChangeScene (>S): Change the scene. Because sequences are associated with one scene, this will end the sequence!

Sequences may be triggered in one of three ways: the player interacting with an object (such as talking to an NPC), the player entering a zone/area, or they automatically start when a scene loads (“ambient” sequences).

A “Sequence Script” is a graph of two types of nodes: “Verses”, which are lists of actions, and “Forks”, which include one or more “Branches” that each have a set of conditions. If a branch’s conditions are true then its child verses are executed.

Sequences are associated with only one scene. Generally multiple sequences will be related in some way: the might be part of the same narrative arc, for example. So Sequences can be further organized into “Stories” which is basically just a grouping of Sequences, without any significant additional functionality.

The Sequence and Story Editors both make it very easy to quickly sketch out scripts and later refine them. They both have built-in validators to ensure that scripts are correctly specified, i.e. they don’t refer to any objects that aren’t in the scene, aren’t missing any required data, etc.

Sequences and Stories are just stored as relatively simple JSON so they can be further processed/analyzed outside of Godot easily.

I expect that as the game’s writing and development continues more actions will be needed. But for now this set has been comprehensive enough.

Textures

Finding source images that fit my licensing requirements and then editing them into textures is a very tedious process. I built a web tool that makes it much easier to find public domain and CC source images (vastly simplified by Openverse), cut out clippings from them and pack those clippings into textures or generate seamless textures by wrapping and blending their edges. It tracks where the clips were sourced from so that attribution is much easier to manage.

Music

I’m not at the point where I’ve given a ton of thought to the game’s music, but I do have one tool, dust, that I developed to help sketch out musical ideas. I didn’t develop it specifically for this game but it’ll be useful here too. It’s a chord progression generator/jammer that outputs MIDI so it can be used as an input to most DAWs (tested with Bitwig Studio and Live 11). It helps to get around blank-canvas-syndrome by giving you a chord base to start working with.

Miscellaneous

Items

I’ve started working on the item system, which is very simple at the moment (and hopefully will stay that way). To manage the items I created an Item Editor, which, though a lot simpler than the Sequence Editor, is just as useful.

Blender scripts and templates

Blender’s also been nice to work with because of its support for Python scripts. It’s a little clunky to get things integrated, but can be powerful once you’re going. In my case I’m mostly using a “quick export” script that helps me avoid the tedious work of keeping exported files organized (navigating to the correct folder, setting the filename, etc) and double-checking my export settings are correct. In the case of items, which require a static icon to show in the UI, the export script automatically exports a properly-cropped render of the item to the item icons folder so I don’t have to bother with that at all.

Another small but immensely helpful thing is having a specific template for Fugue modeling work, with materials, cameras, and what not preconfigured. My material settings change very infrequently; I’m usually just swapping out textures, so this saves me a lot of time configuring materials over and over again.

Dialogue Layout System

Not really a tool, but something I’ve been refining for awhile now. This is the system that determines where dialogue boxes are placed on screen. Many games have a fixed dialogue box, e.g. at the center bottom of the screen, but I want it to feel more spatial, especially as there won’t be any voiced lines in the game (too expensive/too much work/difficult to change and iterate on) so there won’t be any 3d audio to offer that auditory depth.

As far as I know there is no easy or reliable way to layout rectangles in a 2d space to guarantee that there are no overlaps. Not only should there be no overlaps, but each dialogue box should be reasonably close to its “host” (the actor that’s speaking) so that it’s clear who’s speaking. I have a reasonable expectation/constraint for myself that something like five at most actors should be speaking at once and the game has a minimum viewport size to ensure there’s a reasonable amount of space. That is, I’m not expecting that overlaps will be impossible, only that they are unlikely given these constraints.

The approach I’m using now is using a fixed set of anchors for each object and a quadtree to detect collisions. We just try placing a box at one of an object’s anchors, and if it collides with an existing dialogue box or object, try the next anchor.

As you can see from the prototype above (on-screen objects are beige, off-screen objects are grey, and dialogue boxes are black), it’s not perfect. The box 8 at the top overlaps a bit with object 2 – this is due to how dialogue boxes for off-screen objects are handled, which could be refined, but I’m treating as an acceptable edge case for now.

Another shortcoming is how 2d bounding boxes are calculated from 3d objects. Basically I compute the bounding rectangular prism around the object and project that to 2d space. Depending on the shape of the object that may work well or it may end up placing an anchor far-ish from the object’s mesh. You can kind of see it in the screenshot below, the “I’m on the move” dialogue box is meant to accompany the smaller NPC, but it’s kind of far away. Tighter bounding boxes might be possible but I’m worried about the overhead they’d require. Something to look more into.

Unit Testing

Godot doesn’t have its own unit testing framework but there are two popular third-party options: gdUnit3 and Gut. They both seem fairly powerful but Gut felt a bit clunky and I couldn’t get gdUnit3 to work properly (compile errors, which I chalk up to Godot’s weird stochastic-feeling nature, more on that below). I ended up writing my own very simple testing framework instead. It lacks basically all of the advanced features present in other testing frameworks (spies, mocks, etc), but for my needs it’s working great.

Things not covered here

There are still a few key content areas that I don’t have a good approach for:

• Character animation. This is something I’m not very good at and a huge factor in the visual quality of the game. Crunchy textures and low-poly models are a lot more forgiving than terrible animations. There are now deep learning motion capture tools that might work with a commodity web camera, but I haven’t tried them yet so I don’t know if their output is good and what the workflow is like.
• Mapping textures. Taking a texture and then adjusting a model’s UV map so that it doesn’t look warped is also really, really tedious. No idea how to streamline that.
• Object modeling. This is harder to streamline/automate because there’s so much variation. Some categories like buildings and plants could be streamlined through procedural generation via Blender’s geometry node system. Fortunately I enjoy modeling so I don’t really mind doing this, it’ll just be very time consuming. One more general possibility is to figure out a decent processing pipeline for taking free models and converting them into an polygon count that matches everything else. But finding an approach that is robust enough seems unlikely.
• Character modeling. To make the world feel lively I’d like to have many, many background characters and a fair amount of more important NPCs. This might be doable with some kind of procedural/parameter character variation system (i.e. creating a few key archetype models, then having a script to vary some vertices, scales, etc) alongside with a procedural texture generation system (for clothing, etc). Again, this might be doable with Blender’s geometry node system.

Thoughts on working with Godot

I’ve spent a far amount of time with Godot while working on all of this. My only reference point is Unity, which was very unpleasant to work with. Everything felt so fragile. Small changes can break tons of other things, with no easy way to undo the damage. Kind of like how in Microsoft Word adding a space can mess up your whole document’s layout.

Godot has overall felt better than this, but it still has a similar, if reduced, fragility. I’ve felt discouraged to experiment with new ideas out of the fear that I will just break a bunch of existing code/scenes and have to manually fix everything. I’ve found that even just opening a scene file can alter its contents–not yet in a way that has caused me trouble, but it’s still very different than other programming work I’ve done, where things really only change if you change them. It’s like trying to build a house on moving ground. Version control is a bit of a safety blanket but its effectiveness depends on what changes I can revert to.

GDScript has been a surprisingly pleasant language. It still lacks many features I’d like like sets, first class functions, and iterators (all of which I believe are coming in Godot 4) but usually those haven’t been an issue. What has been very frustrating is how Godot parses/compiles scripts. If you have a syntax error in one file it ends up breaking a bunch of other files that depend on it (which is to be expected) but it reports these issues as obscure errors that don’t point to the originating error. I’ll be inundated with messages like The class "YourClass" couldn't be fully loaded (script error or cyclic dependency). (it will not point you to what this error might be) or mysterious errors like Cannot get class '_'. repeating several times. Then it requires a painful process of trying to figure out where the syntax error actually is by opening scripts one-by-one until I stumble upon it.

This is less likely to happen if you’re using Godot’s built-in script editor because you’re more likely to catch the syntax error before it causes too much trouble. However Godot’s built-in editor is really lacking, mainly because you can only have one script open at a time and if you need to edit multiple files at once it requires a very tedious process of manually jumping between files one at a time. So I use an external editor, which does have a language server integration with Godot–so it does catch syntax errors, but sometimes I don’t catch them in time, and then these dizzying cascading errors happen.

I’ve also noticed that sometimes there will be compile errors that are fixed by reloading the project. It feels like these happen because of an unusual (sometimes seemingly random) parse order for scripts, like classes are found as undeclared when in fact they are. I haven’t looked into it too much.

That all being said, Godot has been wonderful to work with overall. These frustrating experiences are infrequent, and it sounds like many of them are being addressed in Godot 4. I’ve enjoyed it way more than Unity and it’s an amazing privilege to have access to such a powerful open-source project!

This week: low-density ancient urbanism, an extraction-free Atacama Desert, AI and climate change, and Wakanda.

The real urban jungle: how ancient societies reimagined what cities could be, Patrick Roberts

Most cities around the world, while still considerably varied, seem to have comparable density, layout, and land-use. The ancient tropical cities of the Khmer and Classic Maya empires were more sprawled and interspersed with agricultural plots and “forest gardens”. In general they sound more integrated with the environment–perhaps easier because unlike the cities that would come later they were not yet inundated with toxic industry. Whether or not the impact of urban living shakes out to be an environmental net positive or negative is something I’ve meant to read more about but yet haven’t gotten around to. But these examples of different forms of urban life are nice to think about, even if they occurred under tremendously different circumstances.

Lithium Landscapes: From Abstract Imaginaries to Deep Time and Multi-Scalar Topologies, Samir Bhowmik

A reflection on the vast timescales that give rise to lithium and the earthbound terraforming (human extraction of) lithium gives rise to. I especially like this painting:

Portuguese artist Mafalda Paiva’s painting Lithium Fields, which depicts the Atacama Desert reminds us of what could have been—a paradise instead of an extractive landscape. In the artist’s vision, “the salt flats hum with a preternatural vibrancy, an effect produced by the exaggerated density of species and radically foreshortened topography.”

These Are The Startups Applying AI To Tackle Climate Change, Rob Toews

I’m skeptical that AI can be applied to substantially help mitigate climate change. There are specific applications that seem to be useful, e.g. around power efficiency, the development of new materials, etc. It probably has a role to play but hard to say whether it will be much more than other technologies treated with far less fervor–this article breathlessly states: “Artificial intelligence is the most powerful tool that humanity has at its disposal in the twenty-first century.”

I wonder if the wide proliferation of AI that many of its proponents either foretell or actively try to engineer through their own companies/investments will net out positive or negative in terms of carbon, given the intense energy requirements to train the largest, most sophisticated models. If AI is commodified and so as unnecessarily ubiquitous as microchips then perhaps it’ll come out net positive emissions.

This list of companies claiming to use AI against climate change is interesting, especially because it reveals what it means to publications like Forbes to “tackle” climate change. Several of the companies are focused on identifying climate risks or managing climate insurance for businesses. A great deal of time is spent companies applying AI to carbon offsets, which looks more like companies bending over backwards to legitimize a completely ineffective/actively harmful way of “addressing” climate change rather than actually working to reduce their emissions. The “solution” to offsets sounds like more surveillance infrastructure.

Wakanda

I’m not a big fan of the MCU, but the creation of a cinematic universe opens up a lot of storytelling and character possibilities so I reluctantly follow along every so often to see what they’re doing with that capacity. I wish it were some other more interesting, less indulgently militaristic fictional universe that had the resources to do something like that. Instead we get Marvel and soon the Mattel Cinematic Universe.

Ever since I found Wakanda listed as a US free trade partner I wondered how Wakanda might relate to other countries and how people outside perceive it. While reading Doomwar I came across this off-handed mention from Shuri:

I’m so curious about the terms of that loan. The first Black Panther movie wrestled a bit with Wakanda’s responsibility towards its neighboring nations or developing nations (as it used to posture as) but if I recall it never really answered that question. I haven’t watched Black Panther recently so forgive any misremembering, but I believe Killmonger was just defeated without any satisfying resolution to his analysis/proposals about spreading Wakanda’s wealth. IIRC he proposed distributing vibranium for uprisings across the world–even if T’Challa disagreed with violent uprisings, maybe the spreading of that wealth is worth considering. Although given what Shuri says here, maybe that’s exactly what they’re doing. But again, I wonder on what terms?

I wonder if in some ways the now-opened Wakanda is perceived in similar ways to how China is perceived in our world. I don’t know what it takes to qualify as a “superpower” but I imagine Wakanda checks off many of those boxes. Surely many people in the MCU pin their hopes on Wakanda to usurp the planetary hegemons…sadly, they’ll probably be disappointed.

This week: Comfort with contradictions, anime and Gen Z

Dialetheism and Eastern Philosophy

We recently finished watching The Good Place, which was very sweet. Overall I enjoyed it, but there was one plot hole that I was disappointed to see left unaddressed, I’ll leave it in the footnotes to avoid spoilers¹.

The other disappointment is the show’s heavy (maybe exclusive) reliance on Western philosophy² I don’t really blame the show for this because it’s just an enormous problem in how philosophy is taught in general. Something else that would have been nice to include is a reckoning with the extraordinary racism of Kant (see the previous link for some examples).

That’s all to say that after the show I wanted to read more on non-Western ethical systems. This interview introduced me to the idea of “paraconsistency” and I got sidetracked into reading about dialetheism, which is the view that statements can be both true and false (in orthodox logics statements can only be one or the other; such inconsistencies “explode”, i.e. entail everything to be true). The simplest example is the Liar’s Paradox, which in one sentence is “this statement is false”. My understanding is that under dialetheism this statement is not a paradox and to be rejected but accepted as-is.

As the SEP entry on dialetheism explains, dialetheism shows up a lot in Eastern philosophies (as well as in Western philosophy: dialectics, for example). The Daodejing and Zhuangzi are riddled with them. “The Way of the Dialetheist: Contradictions in Buddhism” (Yasuo, Garfield, Priest) gives an overview of how dialetheism shows up in Buddhism, describe some interpretative possibilities: are they presented as literal/true, or are they used as rhetorical/pedagogical tools for insight (as in Zen)?

My own understanding is usually that these apparent contradictions are not actually the case–for example, a common form is to say that thing X has property Y and Z such that those simultaneously having two properties is inconsistent/impossible, which I often take to mean “thing X” refers to our term for X and not X itself; and that in fact that term refers to two things that are different. For example, “race is real and not real” could refer to how race is not “real” as in it is an immutable part of the universe but is still “real” as social reality–as something that feels total and inescapable in our lives.

But this interpretation doesn’t work in many cases. When Buddha says “the ultimate truth is that there is no ultimate truth”, that seems to be more of a straight-up contradiction.

Welcome to Planet Egirl, Cecilia D’Anastasio

I definitely feel old given how much more I’ve been reading about subcultures through publications like Wired rather than experiencing them. But I’m struck by how much anime has influenced Gen Z and how visible/trendy it is, whereas for my generation it was an almost clandestine activity. Ariel brought up a good question: is this a popularization of Japanophilic culture or is anime being separated out? Anime production is still more or less the exclusive domain of Japan as far as I’m aware.