Blizzard and the Mythical Man Month; Or Why WoD is a Transitional Expansion

Alternative Chat had an interesting tweet the other day which spurred a conversation as they are wont to do, trying to figure out "what's wrong with Warlords." It's no secret that a vocal portion of the player base is pretty unhappy with the dearth of max level solo content in WoW currently.

So, yeah I worked out what's wrong with Warlords. A systematic failure to actually produce consistently playable content outside raiding.

— Licensed to Alt (@AlternativeChat) July 25, 2015

The first half of Mists of Pandaria can be seen as sort of the closest to what the WoW team want to achieve: rapid content releases. Having a non-raid patch leap-frogged with a raid patch worked really well, not to mention that each patch was pretty meaty. 5.1 brought in a slew of new dailies, story missions, and scenarios. 5.2, Isle of Thunder, Dinosaur Island, and Throne of Thunder. 5.3 was happy invasion land in the Barrens, and while a little thinner content-wise, was still engaging (plus the Brawler's Guild). 5.4 brought Siege of Orgrimmar and Timeless Isle. Content-wise, by most accounts it almost came too quickly, leaving us with over a year of empty space between Mists and Warlords.

Compare that to Draenor, and post-leveling content has been extremely sparse. Twitter integration, and a selfie camera which "was a fun side project for one designer and one programmer, coming in on the weekends to make a cool toy to go with Twitter integration." Tanaan Jungle, which I actually quite enjoy and has provided a good framework over Timeless Isle in my humble opinion. Timewalking, which most of the tech existed back in Mists. Finally, Hellfire Citadel, which--aside from problematic story moments (as usual)--has been an excellent raid.

So what's the deal? Why is Warlords flailing where Mists actually turned out quite well (ignoring the Lost Year)?


The Mythical Man Month

I postulate that this is the transition expansion. Blizzard is ramping content creation up: they mentioned that they increased the team size by 40% at the end of Mists, and were working on another 40% increase in size from there. Basically, they've nearly doubled the size of their team. I'll talk mostly from the programming angle, but to a large extent this still applies to other disciplines.

At my old job, we tripled the size of our team overnight. Went from about 20 developers to about 60. I was in charge of educating them and bringing them up to speed, so I can tell you with first-hand experience that this is an immense task, and comes at (an extreme) cost to general productivity for the team as a whole. And it has little to do with how good or bad the developers are in general. In fact, almost every developer I worked with when we brought on the new folks were incredibly smart, passionate, and knowledgeable people.

No, the issue with exploding your team size is much more basic than strictly skill sets, and yet far more nuanced than mathematics can describe.

There's a book, sometimes known as the "Bible of Software Engineering," called The Mythical Man-Month. This book is basically required reading for project managers (and if it's not, it should be). The basic premise of the book is the following:

Adding manpower to a late software product will make it later. -- Brooks' Law

The book itself goes into the myriad reasons why this holds true, but the primary reasons are it takes time for new employees to get up to speed, and more employees means an exponential increase in lines of communication


Onboarding New or Transferred Employees

By far, the biggest short term cost is getting all these new folks up to speed. Each team has their own processes, their own tools, their own hierarchy, their own codebase. Established employees know the history of the product, and they know the bugs, quirks, and odd spots in the code. An established programmer might know intuitively that if they change something in the animation engine, for example, that they may need to double check the sound because of how those two systems might be intertwined in that particular product.

Think of it this way: let's say you were brought on to write a book set in the World of Warcraft universe just after the events of Warcraft III, but you knew little about it aside from, "I've heard of Warcraft." Sure, you know how to write a decent story, you know English, and are decent at self-editing. So you have the skill set, but you don't have the context.

You'd need to research what lore already existed around the subject, you'd need to make sure what you wrote could sit in the timeline correctly without accidentally introducing minor bugs (like getting Jaina's eye colour wrong), or worse, accidentally introducing things where you'd have to retcon entire chunks of the story to make fit. The Warcraft lore universe is massive. And you don't really know your way around.

So instead, you get the basics from someone who knows everything inside and out, and you pester some folks for questions because you're still not sure if you got the details correct. The experienced employees take time out of their day to fix your mistakes and guide you through your work, so not only are you really slow to start with, but your work impacts the efficiency of the experienced employees.

This is absolutely true for programming, art, or design, among other disciplines as well. For programming in particular, one spends a lot of time coaching the new/transferred programmers. That's not to say the time is wasted. Eventually those newbies will become productive team members on their own, but adding them to the team can and will reduce that team's throughput in the short term.

And note, short term can mean anywhere from 3 months to a year. Depends on the complexity of the product. Even experienced, senior programmers coming into an existing codebase can take upwards of 3 months before they're productive enough to work without someone established looking over their shoulder (or bailing them out of trouble). It's not that they aren't smart or accomplished; it's that they just don't know the source material well enough yet to not accidentally goof up without realizing it yet.


Doing the Math

If we assume that for each new employee, it reduces existing team throughput for an established employee for an average of 50% for the first 3 months (which in practice is a conservative estimate, from what I've seen), then having a team increase in size by 40% means you're looking at reducing overall individual throughput by about 20%. And no, these new employees aren't contributing enough to overcome that throughput loss, at least not in the first few months. They're often making mistakes and bugs that the more established employees need to compensate for (aka negative contributions).

And of course, from there it's not like it spikes to 100% efficiency, either. You'll have a gradual ramp back to 100% efficiency, but ideally you'll be at greater than 100% relative to the old team size eventually.

But it's never that easy. First of all, it's only the most experienced transferred employees who'll ramp up in only 3 months. But often, they're pretty independent by that time. No, generally for the average programmer it seemed that about 5 - 6 months was required before they could do work largely independently (assuming they're doing stuff that isn't isolated). For really fresh out of school employees (brand spanking new), ramp-up takes about a year, usually. Sure, they'll make features, but they'll be pretty bug-ridden and still need a lot of guidance.

The other gotcha here is that often the established employees doing the education will be the ones with the most experience in the code and/or product, which means that you're actually taking a bigger than 20% overall hit.

Let's say a programmer with 10 years of experience is about four times as productive as a programmer with 2 years of experience. Interestingly, Brooks mused that a good programmer was 5 to 10 times more productive than a mediocre programmer, and that fits in my experiences as well. If you're using your super productive people to teach the new folks incoming, then that 20% hit might actually be closer to 40% because not every employee is equal.

So by adding a bunch of folks, Blizzard would have tanked their individual throughput anywhere from 20% to 40%, depending. And they did it twice by doing two major hiring waves (because chances are, the people who did onboarding to begin with probably were tapped to do it again the second time).

That all doesn't take into account the re-structures Blizzard would've had to perform--that many new/transferred people means new managers, new communication structures, and so on. Those alone generally tank productivity as the rank and file try to figure out what's going on and educating their new leads on what they've been doing. It also doesn't take into account the standard "forming; storming; norming" that would occur when introducing that many new people onto the team, which would also exacerbate the ramp-up time.


The Lost Year of Mists

So when can we as consumers expect the fruits of Blizzard's labours here? It takes a while for content to work through the pipeline. While it looks like a bunch of stuff happened last second on the PTR just before Warlords was released, you can bet most of that work was planned months in advance (and then iterated on later). Art assets had to be completed, game systems programmed, server tech vetted, the entire thing QA'd. We knew they were working on Tanaan as "early" as before Warlords launched, so what we have today was started probably at least 8 months before the patch was released.

The lost year of Mists was quite likely half a write-off from a production perspective. I don't know when they'd have done the second surge of new employees, but it would've reset the clock on the productivity dip.

Which would explain why Warlords felt so anemic. Leveling was amazing, Garrisons was a lot of "content", but most of it is random/programmatic. It likely was more programmers and designers than art--which makes it pretty attractive as the programmers probably needed something to do while everyone else was building the Draenor continent.

But the daily hubs? Even Celestalon Watcher said they were half-assed. Flight wasn't a thing, so that'd have reduced the load on the art/design team for building the continent (which they're paying the deferred cost now, hence why flight is delayed). No scenarios. Proving Grounds literally just got adapted to the scaling tech.
Edit: Whoops, Watcher == Ion, Celestalon == Chadd.

The Next Expansion

But we know they were already working on the next expansion even before Warlords shipped, so I think we'll probably see an expansion announcement at Gamescom. We'll also likely see a beta server ready to go by Blizzcon in my opinion.

Honestly, software takes a long time to make, and organizations take a long time to readjust--especially ones as large as the World of Warcraft team. So while people may be getting antsy about Blizzard's ideals of an annual expansion at Gamescom in 2013, they're probably only now getting to the point where the hiring and training they've done will become visible to us consumers.

Blizzard lost a lot of ground, and Warlords likely suffered because of these huge organizational changes. But the engine should be primed and I'm looking forward to seeing what the next expansion brings. Blizzard has a lot to (re)prove, and hopefully they'll be up to the task. #WoW, #GameDesign, #Expansions