[IndieDev] Balancing Access vs. Experience; Where Should a Game Dev Draw the Line?

I think I've finally hit upon a subjective enough issue that I'm really not sure what the right solution is. The problem in a nutshell:

Player wants to play Eon Altar, a game where their phone connects to their PC. However, they're at University, or they otherwise don't control the router/network they're connected to, and the PC/phone can't see each other, and therefore they cannot currently play.

The game heavily relies on the network being really fast (low latency), and going through the Internet could possibly make the experience extremely poor. So we--the developers--when designing the game, decided that it should only work over local area network (LAN), which clearly prevents our player from playing.

So here's the catch: do we create a solution where they can directly type in an IP Address on their phone to connect to the PC, at the risk that we have way less control over the experience, and the latency might make the game unplayable (and result in an unhappy player, and thus bad reviews/word-of-mouth); or do we just leave it be and tell the user, "Sorry, no dice?" (and result in an unhappy player, and thus bad reviews/word-of-mouth).

We know how the experience will degrade: we saw this at PAX Prime, with 60,000+ techno-savvy geeks all on our mobile/wifi devices. Despite having our own routers, the radio interference made the game frustrating. Your movement marker wouldn't react cleanly or smoothly and dialogue responses would sometimes hang briefly. Pressing and holding for free-running was frustrating because it would take 200ms - 500ms for your character to start running, and stopping also took that long, so you'd overshoot, constantly. It all made the game feel really unresponsive--because, well, in the context of a extremely high latency network, it really was unresponsive.

There are in-between options here. We could bury the Connect Directly to IP Address under a help menu as a last resort kind of thing, with big warning signs on it saying the game wasn't really designed for over-Internet play, but is that really helpful? Are we still going to get slagged, even with a warning message?

There's also a maintenance cost and a support cost. If the Direct to IP doesn't work for whatever reason, we (and by 'we' I mean 'I' as I'm the technical guy) will have another thing to possibly have to troubleshoot remotely via forum/email, but on the other hand, it is a workaround tool we could have built into the game. However, it means that I likely won't be able to fix some other unknown subset of bugs/features because I'll be spending my time implementing this feature.

But having players who can't play the game is frustrating on our end, too. We want people to play Eon Altar, and anything that gets in the way of that--short of not having a capable mobile device, sorry, but that's required as part of the vision of our product--should be fixed with prejudice if possible.

So, do we, or don't we? #IndieDev, #EonAltar

[IndieDev] The Omniscient Programmer

We've been in Early Access for Eon Altar for a good 4.5 months now, and as we get closer to releasing, it hasn't been an easy transition ramping down. I've been the Lead Programmer since April 2015, and recently I've been the only professional programmer on the team. Even when we had a team of 3 programmers, it necessitated a very generalist approach as we just haven't had the number of staff to allow for deep specialization. It's helped that some of our non-programmers know some programming (our lead designer and our animator specifically), but they've not been entirely self-sufficient in that arena--and nor would I expect them to be any more than I'd be expected to be self-sufficient building an animation.

However, today, as the primary technical person, I am expected to know, well, everything technical.

It's a crazy challenge, but an awesome one as well, especially on a game as relatively complex as Eon Altar is. Compare that to my time at Microsoft where I could afford to go really deep on a single problem at a time, like designing and implementing a replacement for an old, inefficient proprietary database; or syncing two file sync engines to the same cloud endpoint. Granted, those are both incredibly complex and nuanced areas, but I had months, even years, to really get everything correct. Eon Altar, our timeline, feature set, and low staff numbers didn't allow for that amount of depth or complexity.

With almost no one aside from the Internet to fall back on at this point excepting a few very specific cases, I need to understand the entire codebase, and be able to make changes within it with confidence.


Being a Programmer

At the lowest layer, this coincides with the sort of things most professional programmers should know at some level: coding and debugging, memory management, algorithmic complexity, language features (C++ vs. C# vs. Javascript, for example), performance profiling, optimization (space and/or time), multi-threading, networking, testing. If one were to compare programming to another job, like a civil engineer, this would basically be "how to build a basic building with infrastructure that won't fall down under the stresses of the natural world".

Nothing too special, though having extra knowledge of how code looks and performs when it's compiled; how different hardware such as graphics cards, processors, RAM, and hard disks changes performance characteristics; how the operating system you're working in handles memory and disk writes; how your garbage collector works if you have one; and so on can help make your code even better--and part of why coding for consoles which largely have set hardware capabilities is actually pretty appealing.


Being a Unity3D Game Programmer

To be a game programmer specifically, and being the only programmer, it means I must understand how every aspect of games works, including rendering, sound, physics, time, animation, AI, pathfinding, cameras, and runtime asset management, just to name a few major systems. We can't have one programmer working on rendering and animation and a different programmer working on AI and pathfinding.

That's where a system like Unity3D or Unreal Engine both come into play, because all of those things are written for you. But you're still building on top of the tools provided, which means you still need to understand how they work, and fairly intimately in many cases--especially when someone decides that the built-in pathfinding tech just isn't good enough.

Some basic camera math using characters on-screen and the camera frustum to calculate the distance the camera needs to be to include all characters on the screen.

An example here would be Unity3D gives us basic camera tech. We don't need to write the code that tells Unity3D how to render a screen based on a camera and a bunch of polygons and textures that the camera can see. But I do need to understand how that works to be able to debug rendering issues, or build a camera controller system, or use cameras to create projectors.

Using our civil engineer example from before, this might be analogous to, "How to build a skyscraper specifically, and all the extra aspects that go into it," like ensuring it doesn't collapse under its own weight, can handle winds at high altitudes, and keeping upper floors cool because heat rises.


Being an Eon Altar Programmer

Then for Eon Altar specific features, I need to understand the level design tools we've created or imported from the asset store and how they interact with everything else, like state machine scripts which control many in-game sequences; character data; dialogue; level loading; UI; Controller-Central Game networking; layered music system; saving and loading games; logging and collecting crash data; analytics; game versioning; combat/action systems; tutorials; lore; camera cage to keep players on screen; combat and power AI; platform-specific features; and I'm probably missing a few other systems.

And then a lot of those systems interact with each other. Save games requires knowledge of player and level state, as does reconnecting controller devices. Game versioning is important to do robustly so we can prevent bad builds or old builds of the game and controllers from connecting to each other. Combat and power AI hooks into character data and the combat/action systems. Camera cage and camera controller both know about player movement and can guide/control it at occasions. Level design tools can inform pretty much everything on the list.

Interestingly enough, being the programmer makes some of that easier to an extent. Despite not writing the original version of some of those features--I didn't write the dialogue system or UI, or the underpinning of the combat/actions, or the sound systems, and while I've had a strong hand in many other systems, I wasn't always the lone programmer on them--because I have intimate knowledge now due to iterations and bug-fixing, writing other features such as save games and reconnect actually went pretty smoothly as I already have all the details I need to know. In a bigger team, I'd have to rely on others understanding what I want to do and getting their feedback on how to integrate with their features.

I'm stretching the civil engineer analogy pretty hard at this point, but this might be analogous to, "How to design extra specialized infrastructure systems within the skyscraper." Maybe I should've picked a different job to use as an example...


Being the Technical Person

Of course, it doesn't end with that. I am the technology gatekeeper, which means I'm also in charge of creating, developing, and maintaining the technological infrastructure surrounding the game, including automating build systems for all platforms; unit testing; patch scheduling; deploying to Steam, iOS, and Google Play; debugging Windows, OSX, iOS, and Android; keeping abreast of changes to the Unity3D platform and keeping it up to date; code repository management, including creating release branches, development branches, and bailing out coworkers who get in trouble with it; differentiating asset management for the exe based on platform (as both our Controller and Central game use the same codebase/project); and of course, technical support for our customers.

Technical support so far involves a lot of debugging customer networks from afar. Are all their devices actually on the same network? Can their iPhone or Droid even see their PC in any app, not just ours? Is something like a firewall eating UDP packets? Why are Zenfone 2s specifically freezing?

We've had a few other bugs that our logging has helped us nail, along with good descriptions from the folks in the forums as well. Add to that automated crash/unhandled exception logging for iOS, Android, and PC (and I had to write something cloud-side for the PC from scratch, which was an interesting exercise), and a lot of what I do today is trying to suss out wtf went wrong at any given moment.

For our civil engineer, this might be equivalent to organizing delivery of materials and workers, scheduling when parts of the building would go up, orchestrating repairs, and answering the phone when someone's toilet got clogged.


Bigger Teams Than Ours

If you were to look at a bigger team, like at EA, the build engineering and deployment would generally be another department entirely, and you'd have specialists on your team for handling platform specifics, or at least mobile vs. desktop. Tech support would still be done by those who owned the breaking features.

You'd probably have different programmers for game logic versus designer tools versus your engine programmers, who'd be handling rendering/sound/asset management/networking, and often would specialize in one or two of those sub-areas. And the more programmers you can afford, the more complex those features could be. A lot of our features are pretty simple, because we literally did not have the manpower to ship anything more complex.

While programmers make up a small portion of most teams (content creators generally being the lion's share), you're still looking at anywhere from 6 - 18 programmers for a major game like an RPG, and usually for many years. Mass Effect, a triple-A RPG, had about 45 in-house programmers over the course of it's development according to the credits. Probably not all at the same time, mind, and Mass Effect is certainly far more complex than Eon Altar (especially since we didn't have to create the engine from scratch), but that gives an idea of team size. Tales of Zestiria seemed to have 10 in-house programmers, and 10 from a contracted studio based on their credit scene. Heck, Super Mario RPG had 5 programmers.

Which all is to say we definitely would never have been able to make Eon Altar without an engine like Unity3D, but even with an engine like Frostbite, Dragon Age: Inquisition still has well over 60 programmers listed in their credits.


Holy Moly So Much Stuff

So how does one keep all this stuff in their head? Well, to be fair, you kinda don't. I hinted at it earlier, but the Internet is a great resource to allow you to make it once you've faked it to a degree. You really do need your fundamentals down pat and largely internalized, but once you start getting into systems such as physics, you kinda just need to understand vaguely how it likes to store information, and how it works on a frame-to-frame basis, and you can usually either puzzle the rest out when you need it, or look up the specifics. Game specific features, thankfully, are documented and/or in code.

A lot of tech support is going, "Oh, crap, why is it broken? Okay, let's try a few general tests to see if it's a problem I understand. Nope? To the Internet!" Build engineering? I knew I had to link a bunch of stuff together, and knew that I could do it in a bash script, so it became a question of, "Let's go learn bash scripting for OSX to tie all this together!" One day I'd like to learn more about Jenkins if I'm going to be doing more automation, as that seems to be the industry goto for that sort of thing based on further research.

Behind every great programmer is a self-conscious person making it all up as they go, desperately hoping not to be found out.

— Justin Searls (@searls) January 7, 2016

My university degree provided me a mathematics background as well as some classes in graphics, physics, and cameras on top of a lot of the "being a programmer" stuff, but Unity3D itself I learned on the job. I've learned a fair bit about software lifecycle management from my time at Microsoft, as well as a lot of other engineering skills that you just don't learn in school. Everything else I've picked up effectively by osmosis, or research when required. And so much research, all the time!

What makes programming so wonderful and terrifying is that every day you're pretty much learning something new: about the tech you're using; about some pattern or technique you've "discovered" or read about; about some security flaw that you now have to patch in your own code; about how to perform some task that you didn't even realize you needed to perform the day before; about some new tech you need to integrate into your product.

The danger in generalizing, of course, is not being able to dive deep enough when a large organization requires more specialized expertise. It also means that it's quite possible for you to know just enough to get yourself into trouble, but not out of it. But so far that hasn't happened to me, knock on wood and all that.

Eon Altar has been a wonderful project to work on, and I've learned so much doing it, both from my fellow programmers when we still had them, and a lot just by going through the motions of building features and debugging issues. We're not done yet, but it's definitely been worthwhile.
#IndieDev, #EonAltar, #Programming

[Minecraft] Automate All the Things!

It's been a while since I last posted; nearly a month. December was busy, between a trip to Toronto to see family, running tech support for Eon Altar, doing some apartment hunting, and some money hunting, it left me with little drive for much else.

However, January is here and I figured, let's toss something up on the Interwebs!

And what better than a video tour of my Minecraft base? A few folks on Twitter were curious to see, so I made a video of the different contraptions I built--generally using other people's tutorials, but I'm starting to get better at redstone free-hand so hopefully I'll start building things of my own soon!

Farms/contraptions you'll see in the video include: a semi-automatic cocoa bean farm, automatic cactus farm, item sorting system, automatic golem (iron/poppies) farm, automatic sugarcane farm, automatic pumpkin and melon farms, semi-automatic wheat/potato/carrot farm, semi-automatic cow cooker, automatic smelters, and a villager generation/trading hut.

I still have a few more farms to set up long-term: an automatic slime farm, a mob spawner sky-farm for sweet sweet overworld mob drops, and a gold/XP farm in the Nether. If I was really saucy, I could set up an automatic witch hut, but my Villagers provide me with plenty of redstone and glowstone, so there's not much point. But once the other farms are done, I'll be well on my way to having as many materials as I could ever want and I can build, build, build!

Without further ado, the video!

#Minecraft, #Redstone, #Video

Game Design and Player Behaviour

There's a really great theorycrafting discussion going on in the WoW Twittersphere that was kicked off by a post that challenges how guide writers present information as "the only choice". Basically, given two choices that are fairly similar mathematically, by changing how you present said information, the general community opinion can swing significantly.

To anybody who studies communications and/or marketing, this fact should come as no surprise. But the more interesting aspect of the discussion was what Celestalon--a technical game designer on WoW--posed to the community at large:

Discussion Topic for the Day: What can we (Blizzard) do to help solve the problem described in this article? https://t.co/Wv7H0xwAWI

— Celestalon (@Celestalon) December 10, 2015

Many of the responses that came out of that boiled down to, "You can't, it's the community's problem." Which is a frustratingly unhelpful answer, and possibly flat out incorrect.

Game design, you see, absolutely influences player behaviour, in-game and out. There seems to be two schools of thought on the matter: design the game and let players interact with it and each other as they will; or design the game specifically to encourage players to behave in certain manners. When said like that, it sounds pretty clinical, but it's really not in practice.


Real-Life Examples

In World of Warcraft, and many other MMOs, to do group content you used to have to find and build a group yourself. You spent time (often a lot of time), gathered people, and hopefully managed to get through the content okay. Good players built up a reputation with other players on a server, and jerk players generally got shunned.

Enter LFD in the Wrath of the Lich King era, which took players from any server and threw them together in a random group to play the same group content. Less control over your group in exchange for a high level of convenience. This absolutely changed how players behaved. More people did group content because of convenience, but things like personal reputation mattered less because the pool of people to group with was orders of magnitude larger.

This was a case where the system was designed, but the influence on player behaviour was either not thought totally through, or deemed okay in balance with what the game gained from having such a tool available.

FFXIV evolved the concept keeping player behaviour in mind with further systems like handing out extra currency and experience to parties with a newbie in them, and commendations to hand out to others for whatever you wanted--good behaviour, great guide, awesome player, the rare dragoon that didn't die in the fire--and as such has helped FFXIV's community be nicer players (even if they aren't generally better players, but that's a different discussion).

In Eon Altar, we design around a very specific player experience--the couch co-op slightly competitive but mostly cooperative experience. Players play together as a group, but the game story and mechanics all subtly (or not-so-subtly) nudge players into competitive play styles. Friendly fire for AoE, real-time looting system where trading with each other is important, character agendas and secret quests/missions that may conflict with each other. Playing Eon Altar with friends is a very different experience than say, playing Goldeneye or Final Fantasy Crystal Chronicles with friends, despite all of them having heavy focus on local multiplayer.

The Eon Altar example I use very specifically because it shows that game mechanics and meta player behaviour are absolutely intrinsically linked. You can't just design something and say, "players will be players, yo." Can you imagine if we released Eon Altar in its current state online? Without having players be local and therefore with local social norms to enforce acceptable behaviour, the game would be troll city. That's not to say we don't have ideas, but it's clear because of that player behaviour component, the game would not translate well as-is to pure online play.


Simultaneous Insufficient Info and Info Overload

For Celestalon's issue of cookie cutter builds and people just wanting/requiring/taking a given choice suggested from the community, there isn't an easy answer. Some people like choices, others just want to be told, "this is the best option," because they don't want to choose or can't fathom the choices available.

But just because it's not easy doesn't mean it's not a worthwhile exercise. MMOs like WoW are complex beasts, and so much information isn't in the base game itself that gets built by the community at large. Other times some of the information is in the base game, but without any context from the designers and so the players are expected to come up with ideas themselves, to which a large subset turn to the community to make those decisions for them.

Some of the problems are that they're imminently calculable. When someone can say, "this talent or ability does 20% more DPS than that one," it's a no brainer, and a robust theorycrafting community will discover these quite quickly. Some of the problems are that the theorycrafting community gets something wrong, or right but heavily caveated and the caveat gets lost, and then you have this strange state where the community as a whole has a gospel that's factually incorrect.

But at the end of the day, you're in a group, and that group requires a specific minimum level of performance, therefore between peer pressure and mathematics, there's a heavy push towards conformity and the "correct" choice. If the designers won't--or can't--provide the information towards the players or even make some of their choices for them if the player opts into it, then the player will seek that information elsewhere. To be fair, even if the designers provided that information, the player might seek that information elsewhere, but is there anything the designers can do to reduce that need or requirement?


What to do?

If we're spit-balling, perhaps remove talents entirely, or making talents generally utility-only perhaps so there isn't a correct answer? Maybe provide default options--all passives because a player who doesn't want to think about the game will probably not play at a level sufficiently to use active talents to their maximum effectiveness.

If the designers can help solve or ameliorate this issue, it's a huge win for them because it means they're a step closer to interesting choices (or they've basically cut them from the game and no longer need to spend design time on them). But leaving it to the community to solve by itself probably won't solve anything.

Rather, they should use game design to help address the issue, and Celestalon's question probably stems from recognizing that. Heck, if they can "solve" it, you bet many other games would take note and follow suit. 

Rohan at Blessing of Kings has an interesting take on whether it's an issue to be "solved" at all.
#WorldOfWarcraft, #GameDesign

[WoW] Artifact Weapons as Alternate Advancement

There's a little consternation in the community on Twitter at least that Artifact Weapon advancement is ultimately dull. Eventually you'll get everything, and it won't even take that long.

@Drewbobmiller Not multiple patches. Most likely between 1 and 3 months, but we haven't finalized tuning on it yet.

— WarcraftDevs (@WarcraftDevs) December 1, 2015

As far as I can fathom, there's two primary goals about Artifact Weapons that Blizzard wants to achieve:

1. They want an alternate advancement method that goes beyond hitting level 110.
2. They want to reward people for playing a specific spec (or just playing their character in general).

When Blizzard moved to the current talent system--which works better at end game in my opinion than the old talents did--they lost that zing when you ding. Even if it didn't mean much in the grand scheme of things, choosing where you put your talent point as you leveled felt good. It made you feel like you had direct input on the growth of your character.

At the end of the day with respecs it actually didn't matter. At all, really. But it gave you a sense of steady progress and the illusion of choice, which might be almost as good as actually having choice.

Blizzard tried to return that to us in the form of Draenor Perks, but because they were random, it didn't quite have the same oomph. So with Artifact Weapons, Blizzard returns us to the benefits of the old talent system, but not tying it to levels means we can continue advancing beyond 110, adding longevity to the advancement system.

@ShamanGio You will eventually be able to unlock the entire tree.

— WarcraftDevs (@WarcraftDevs) December 1, 2015

Of course, in modern day gaming, there's no way they could lock you forever into specific choices, so you'll be able to unlock the entire tree eventually. This means that short of world first raiding, there probably isn't much "game" to deciding what you want first outside of, "Oooh, that sounds cool!" There's still an open question of respeccing your Artifact Weapon as you level it, but I'd be shocked if you couldn't reset the point distribution on it somehow.

If you're looking at Artifact Weapons as a way to add choices to your character and interesting Theorycrafting differences between paths, you're barking up the wrong tree, as that's clearly not what Blizzard intends for the Artifacts based on the info they've handed out. And once Legion is over? They can just bake the Artifacts into the specs baseline (or bits and pieces of them) and drop them for WoW 8.0.

The part that doesn't fit this is, of course, relics. This is where Theroycrafters will get to play mathmaker and determine which traits are the greatest for adding extra levels to. Of course, we might not get that much choice in the matter; do we know yet if relics will increase specific traits? Or subsets of traits that we can choose from? But it's a minor amount of customization, throwing TCs a bone if you will.

Overall, it's a nice alternative advancement system, but it's not nearly as complex as some folks wanted. But that's why we (ostensibly) have talents anyway. #WorldOfWarcraft, #Artifacts, #Legion

"Do I Sound Gay?" Documentary; Thoughts and Reactions

Today I'm going blogging off-roading for a bit because I watched an interesting documentary that I feel I need to unpack and analyze.

The documentary, "Do I Sound Gay?" has been on my radar for quite some time, and I was excited to hear it was on Netflix. After seeing it, it was definitely good, but also rather difficult to watch at the same time. Aside from a couple of content warnings--news footage where they show a gay kid getting assaulted, and a couple of censored porn scenes as examples of hyper-masculinity in gay culture--David's story was also largely my story.


The Gay Voice

There's a definite "gay voice" that acts as a flag or a tell for many people. Sometimes erroneously, but fairly often it's accurate. Listening to someone, it's easy to hear certain affectations that sound "gay"--at least in North American culture, more on that later. I'm damn sure I sound gay, given I've been asked a few times when streaming if I was (the answer is yes, duh).

In the documentary, speech academics talk about certain tells, such as elongated 'S' sounds; overly-enounced 'T' or 'K' sounds; the voice tone sliding up into a sentence, or ending up when more masculine voices end down. All of which now I can clearly hear in my own speech in this Archimonde N kill video (surprise, we downed Archimonde last night! hooray!).

Of course, there's the question of why do I care? And is it really a bad thing?


Self-Hate, or Valid Concerns?

It's easy to dismiss the documentary early on as David being a self-hating gay--many of the reviewers do, and I know I was tempted to at the beginning as well. But at the same time, how much of that is rooted firmly in trying to pass as straight, to hide, not draw attention to oneself? When you spent most of your early life hiding in fear of getting beat up, or even killed, hiding is a pretty sensible maneuver.

Into adulthood, I know it's something that I've been concerned about when giving presentations to my peers. Are the other programmers going to take me seriously if I sound like a queer? Early in my career some of them evinced homophobic attitudes, but thankfully later in my career it hasn't seemed to much matter, which is a huge confidence booster. But these are totally valid questions to ask oneself, especially given research into female speech affectations being a liability in the corporate environment.

And then there's the fact that my voice apparently didn't always sound "gay." Mirroring David's experience, sometime during University my voice became more nasal, higher pitched. More from my throat and nose than from my chest. I knew this because my sister commented on it, wondering aloud if my singing voice would be different when my speaking voice had changed. I apparently never noticed the change, but then again it happened gradually for me, but "suddenly" for my sister.

Was it because I was mimicking the gay peers I generally didn't have growing up? Was I flaunting my sexuality as out and proud now that I didn't have to hide, over-compensating and camping it up? Where did that voice come from?

It's not so much a matter of, "Oh god, why did this happen to me?" as it is an intellectual curiosity at this stage of my life, but there were times when I was younger where I wish I could have code switched to a "straight masculine" voice more easily.


Fascinating Introspection

Something the documentary didn't touch on but I thought was an interesting bit of info is the whole speaking from the throat/nose vs. the chest. Both times I was in Australia, the standard way to talk in Sydney and Melbourne is from the throat it seemed--or at least it was easier to mimic the accent by speaking from the throat--and most voices tended to be higher than they are in North America. The documentary pays lip-service to this by having a couple of interviews with other language speakers, but never sits down to make a serious comparison, so I don't know if it's really extensible outside of this continent.

Overall, it was a fascinating documentary. One which tackles head on historical stereotypes, cultural contexts, and even the dangers that the gay voice can represent. Something a little different, but something pretty interesting in my personal opinion.
#Documentary #Sexuality #Gay

[IndieDev/EonAltar] 3200% Physics Performance Increase

There are certain numbers you pretty much never see in programming. You're often pretty pleased when you can get an algorithm performing 25% - 50% faster. Once or twice in your career, you might nail a mind-boggling number like 3200%. Not a typo.

Green wireframes indicate colliders.

In Eon Altar, we use physics colliders for just about every type of interaction you can think of. Actors and the movement cage to keep them on-screen; turning environment rendering occluders on/off; determining when actors are in melee range of other actors; quest triggers; snapping the movement marker to interactable objects; raycasting the movement marker to the ground; occluding ranged attacks; when to change the camera angle/position; and so on.

My list of systems that had physics colliders, and what they interacted with.

With the sheer number of things we use colliders for, the physics system can get really bogged down really quickly. Whereas the engine can cull geometry that's not visible in an active viewport--for example, when a statue goes off screen, the engine isn't spending time and resources rendering that statue: there's no point--that's not the case for physics objects. If the statue has any physics components, they must be calculated. Things off screen absolutely can and should still run in the physics simulation as they can possibly affect things on screen.

Colliders that don't touch other colliders, or don't move, don't cost much to maintain. However, the more contact points colliders have, the more expensive the calculations get. In Eon Altar, before optimizations, an average level would have upwards of 2000 active colliders, with over 8000 contact points every frame. This bogged the system down to about 10 FPS on a good machine. It dipped to 2 - 5 FPS on crappy machines.

This is a small piece of E1S2, The Portal Obstinate. You can see the actors, plus colliders all over the place.

So the end result was clear, we needed to reduce the number of active colliders at any given moment, and when colliders were active, we needed to reduce the number of contact points. Believe it or not, reducing the contact points was actually the easier of the two tasks.

Unity has a pretty handy feature where you can tell it to only have colliders on specific layers contact colliders on other specific layers. For example, the UI colliders need never interact with say, camera colliders because those two systems never interact with each other. Below you can see the default setting in Unity: everything colliding with everything else.

This is the default, and the default is dumb.

This is clearly not what we wanted. I told my boss, Ed, that I needed 3 days of alone time to break down the system and re-design it. At the end of the third day, if I hadn't received the kiss of True Love from my prince, I'd turn back into a mermaid.

However, making it system-wide that only certain layers interact with other layers was easy, once I figured out what layers had to interact with what other layers (which was seriously the hard part, as it required a lot of thinking/designing/researching code). Add to that code to ensure certain components are on specific layers to make them designer-proof--because you don't want to rely on the designers (or programmers for that matter) remembering "rules", that path leads to bugs and heartbreak--and boom, you've significantly reduced the number of contact points even for colliders that are overlapping, as they may not contact at all if their layers don't interact.

Interaction diagram. Note that some layers interact with themselves, but many don't. One layer may interact with a second layer, but the second layer may not interact with the first, which is a subtle but powerful feature for reducing collision calls.

A much more sparse physics layer setup. This is slightly evolved from the above diagram, so they may not match 1:1.

That alone netted us an 800% - 1600% increase in frame rate, depending on the level.

The second trick was to create "Encounter Zones". Giant colliders that turned a bunch of objects in the hierarchy on or off when the party encountered them. The idea being we don't need a bunch of AI or colliders running if the party is nowhere near them. So everything from enemies to traps to loot to interaction points get grouped up in these Encounter Zones, and when the party gets close enough, voila, everything appears!

This caused some other issues, like having to create a method to slowly load NPCs over time (because initializing an actor is really expensive, and if you initialize 15 actors in a single frame, that frame is going to last on the order of seconds), but for the most part it did the job rather admirably. 

Adding in that gave us another 200% - 400% performance increase depending on the level, so in theory, in the worst levels, we could have been looking at upwards of a 6400% total framerate increase, but in practice I measured a 3200% increase in some of the worst parts of some levels.


Conclusion

Granted, these are things we probably should have implemented off the bat, so it's not quite as impressive as I'm making it out to be, but that's sort of the breaks when you only had 2 - 3 programmers making an indie game and way too much to do to keep other people like design unblocked.

Sadly, these sorts of optimizations took a backseat to feature work until design was all, "WTF the game runs so slow!" and engineering was all, "Whelp, time to put features on hold/cut them and fix this issue instead."

Not that the situation was anyone's fault per se, but the reason we got into situations like that was partly because we had not enough programmers and not enough time to finish infrastructure versus feature work, and partly that nobody on the team at the time had shipped a product from scratch, so we didn't necessarily know we'd hit these situations--though I'll be honest, the physics overhaul had been on my list for months and it was just a matter of waiting until it was unbearable such that I could tell folks to leave me alone while I worked my magic. It also ended up being easier, I think, that most of the systems had been implemented and I could holistically solve the system instead of doing it piecemeal over time.

But at the end of the day, it got fixed up, and it's super solid now. Programmers have to remember that new components with colliders need to have their layers thought about (and we have a wiki with guidelines and instructions), but designers can just use the tools and it all does the right thing, which is pretty awesome.
#IndieDev #EonAltar #GameDevelopment