When keeping it Agile goes horribly wrong

Disclaimer: I don’t like being Agile. I also don’t like complaining. Too much negativity is a bad thing, and in some cases, it can be even worse than being Agile. I subscribe to the simple philosophy that time spent whining can be better spent improving the situation. Given that I’m not always successful at following my own principles and given that this article is (mostly) facetious, I’ve decided to compromise: The following essay is a tongue in cheek commentary about why Agile sucks. It makes little to no attempt to come up with something better (perhaps because much like democracy, Agile is the worst methodology, except for all the other methodologies tried before it). So without further ado here is my anti Agile manifesto:

Once upon a time, there was this development methodology called Waterfall. It sucked and everyone wrote huge essays ripping it to pieces. It was such an easy target because of how ludicrous it was. Anyone who has ever worked on a project that had complexity greater than that of a “Hello World” application knows that getting all the requirements and design right on the first try, let alone the implementation, is a delusional dream. In some ways, its not even really fair to criticize the Waterfall model anymore. Its like beating a dead horse. Why bother doing it? There’s no challenge. Well, people still like to berate the Waterfall model because it serves as a useful strawman, a foil against which they can hock their new and improved development methodologies. The thing is, a wise man named Fred Brooks once said that there is no silver bullet. Software complexity spans across multiple layers, covering architectural distances that are orders of magnitude apart. So long as humans are writing the code, the building of software is always going to be an imperfect process. Any methodologies should merely be guidelines, not strict rules defining orthodoxy that constrict and bind the builders. Anyone promising the one true way to software development must be regarded with skepticism.

Yet over the years, new false prophets in software engineering delivered sermons promising untold productivity and awesomeness. The most popular of these was from the cult of Agile, which quickly became a full blown religion, complete with its own church and accompanying dogma. A methodology that was supposed to be about “individuals and interactions over process and tools” is now bogged down by the process it sought to avoid. Instead of circumventing tools by shuffling sticky notes around on a whiteboard, software tools designed to simulate sticky notes shuffled around on a whiteboard were adopted. Heretics who complained about how it was a bit too much were castigated and excoriated by the church. Cowed into submission, they would dutifully complete their two week sprints while complaining about how stupid it all was.

Among the front line of enforcers in the Agile army are the scrum-masters. They ensure strict adherence to the doctrine and processes of Agile. Straying from the path is punished by meetings on improving the process improvement process. Scrummasters come in two flavors: the full time scrummaster and the hapless developer who is asked to do double duty writing code and managing the sprint. The former has no technical skills and if they have no domain knowledge either then its best to run. Run and don’t look back. Apparently you can get certified as a scrummaster, and once you have been ordained, you are supposed to be able to provide value to a project just because you are familiar with Agile. I’ve seen this in action and the results aren’t pretty.

I remember at one meeting while the developers were having a technical discussion, the scrummaster stared on helplessly and asked everyone to please use the “As an X, I want Y, so that Z” format so as to provide the proper context. He was promptly ignored for being useless. Later, said scrummaster tried to randomize the team by asking the developers to retroactively edit the completed stories in a sprint to be tagged under a different release. Presumably this is because the scrummaster had no idea how to use Microsoft Team Foundation Suite. Such a scrummaster is dead weight. The problem is exacerbated because the scrummaster has no real power or control. The product owner/manager controls what features goes in the milestone. The developers not only don’t respect the scrummaster, they don’t report to him either. They report to their dev managers. The takeaway here is don’t hire full time scrummasters, but if you must, make sure they have a few years of domain knowledge. The developer/scrummaster hybrid role is not much better and is an unholy abomination. Developers are typically overloaded and randomized enough as it is. Having to manage a sprint on top of it while being invited to yet even more useless meetings is a major pain in the ass.

The meetings all revolve around completing the sprint. Sprints are typically two weeks and are comprised of stories, the basic unit of work. Stories are the building blocks of epics. Unlike the epic of Gilglamesh, which detail the exploits of the legendary warrior king of Uruk and his partner in crime Enkidu, these epics are typically mundane high level overviews of new features that need to be built. Depending on how orthodox the particular Agile church you belong to is, the stories need to be in the famous “As an X, I want Y, so that Z” format. For example, “As a user, I want this super duper awesome feature that shoots pew pew laser beams, so that I feel awesome and give more money to the company”. Or: “As a developer, I want to drink wine out of the bottle, so I can dull the pain of shuffling sticky notes around a whiteboard every day during my daily standup meetings”.

If the stories are not written correctly, they are rewritten or removed from the sprint in the grooming meetings, where the wheat is culled from the chaff. These are supposed to be highly efficient meetings, but I’ve seen these degenerate into pointless semantic meetings where people argue about the definition of “done” and what the proper “acceptance criteria” for a story is. These typically spin off yet more banal meetings about vocabulary words where there is much weeping and gnashing of teeth. Once the stories have been groomed, then the sprint planning meeting can begin. The stories in the sprint are then “sized”, and scope is adjusted accordingly. “Size” is just an arbitrary metric measuring how much work a story is. It can either be measured in terms of complexity (How difficult a task is) or time (How long it will take). The two are not necessarily the same. On an increasing scale of 0 to 5 points, banging your head against the wall 1000 times would be a 1 point story based on complexity, but a 5 point story based on time. These meetings can drag on forever if huge arguments ensue about how large a story is. The good news is that most people just want to get the meeting over with. As the sprint planning drags on, most people’s spirits are broken and they just simply stop caring enough to even bother arguing.

Once the sprint is finally complete, there is the sprint demo. A lot of the times this is to non technical stakeholders or customers. While demos are great if you’re working on a UI and have something flashy to show off, webservices, messaging queues, build scripts, database work, and the other sort of behind the scenes stuff that makes everything work don’t lend themselves to exciting presentations. Its pretty awkward but there are workarounds such as demoing only the UI and hiding any XML, database queries, and command line interfaces. Following the demo there is usually a sprint review. This is a meeting where everyone looks at the sprint burndown chart, a graph that tracks the number of remaining story points in the sprint over time. A perfect burndown is a straight line with a negative slope that reaches 0 at the end of day 14. In reality, the burndown charts resembles a drunk person stumbling around the park. Sometimes the line plateaus for days before small and erratic stepwise increases and decreases eventually reach a number that not quite zero. During the review meeting, everyone discusses ways to make the burndown chart be prettier in the future:
“Perhaps instead of sizing based on time or complexity, we can size based on the position of the moon relative to the earth and the sun?”
“How about consulting the oracle bones instead?”
“How about basing it on how many developers are weeping when they read the story?”
Regardless of whatever zany ideas are proposed, the resulting burndown never even remotely resembles a straight downward line.

Closely related but subtly different from the sprint review is the sprint retro, where the team reflects back on the sprint. On a dysfunctional team, this is mostly for catharsis as team members yell about what went wrong. Typically, this results in a set of action items with clearly identified owners. This is actually a great exercise in team improvement, but your mileage may vary. Those who feel empowered and accountable for the project at large will actually follow up on the action items. Those who are shackled by bureaucracy and red tape will shrug their shoulders, send out a few passive aggressive emails, and call it a day. On teams that are well run, the retro is mostly just a formality.

Throughout the entirety of the sprint are the daily standup meetings. This is where people talk about what they’re working on. Each person usually says something along the lines of: “Yesterday I worked on this stuff, today I’m going to work on this other stuff. No blockers. Sunny outside with a small chance of showers.” These tend to be tedious. To be fair though, they can provide some modicum of value by making sure the entire team is on the same page and aware of any potential risks of blocking issues. Then again, simply having a team that talks to one another accomplishes the exact same thing. The potential danger lurking in the standup meetings is having a weak scrummaster. Its up to the scrummaster to make sure that people don’t take up too much time reporting their status, ramble on, go off topic, or get into a huge debate with others. Any kind of serious issues should be taken offline and discussed after the meeting. If the scrummaster is not strict about this, the standup meetings will start to drag on and become a major source of pain for everyone involved. The problem grows exponentially worse the larger the size of the team, as though daily standup drama is directly proportional to how many people are unlucky enough to attend.

On paper, Agile isn’t completely terrible. There’s a lot of good ideas. The problem is when it is actually put into practice. In that regard, it is quite similar to both Waterfall and Communism. While both are ideas that sound great on paper, they fail miserably in the real world. Agile, an anti-process process of writing software, becomes bogged down in process. Managers obsessed with metrics start demanding more granular detail on story size estimates, more information on what business value a particular story provides, and more visibility into the sprint progress (typically accomplished by asking for daily status reports). Soon things get so ridiculous that something as simple as fixing a typo in a variable name requires a story, approval from business, and a triage meeting before the scrummaster moves it into the sprint. Those on a dysfunctional team will hide behind process like a government worker hides behind red tape to not get anything done. They thrive in an environment like this, feeding off the death of team morale.

At this point in my life, I’d rather do everything ad hoc. There has to be a certain amount of flexibility in the software development process. Sometimes the requirements will not be clear, and they will rarely be perfect. Nor will the requirements be set in stone. Unfortunately, requirements do change, and sometimes at the last possible minute. It sucks, but the solution to this isn’t to add yet even more processes and meetings to prevent this from happening. Minimizing volatility is a good goal, but the team should eschew the rigidity of a predefined process and feel empowered to make their own decisions. Agile helps foster communication among team members. But if the team is filled with toxic people who can’t play well together , no methodology will ever solve that. If the team is filled with competent people who have good rapport and open lines of communication, then Agile is not really necessary. I once worked on a team that had a hard ship date in two months. For those two months we kept the daily standup but eliminated the overhead of grooming, planning, and retros, and simply did what needed to be done. This requires a certain level of proactivity and an ability to handle the almost chaotic nature of such freedom, but it was a refreshing refrain from the suffocating nature of Agile.

The TLDR cliff notes is, teams should follow the spirit of Agile rather than enforcing it to the letter. Rigidity leads to bureaucracy, which defeats the whole purpose of Agile in the first place. Agile was meant to be a lightweight methodology designed to ship code quickly and frequently. The keyword here is lightweight. The frequent grooming, design, planning, demos, and retros are there to foster effective communication, not to stifle people with the tedium of it all.