Over the last two years, I have observed RUP and Scrum processes in action and I can make some comparisons. And they have more in common than you might think. In particular, the factors that cause them to succeed or fail are very similar.
In a document heavy process, requirements are gathered and documented. Systems are designed, coded, and tested. And the tests are traced back to the requirements to ensure they are met. The Rational Unified Process (RUP) is one of these. It has evolved a lot to embrace iterative approaches, which is an improvement. These more traditional approaches can work. We did a pilot with RUP, and it was successful. But it took more time and resources than was expected. Based on this success, we started a large project, and it was failing to deliver on time. Developers were frustrated, so they introduced Scrum as an alternative.
In an agile process, requirements are gathered in the form of stories, which are not complete in the sense of “all the details are there.” But they are a promise to have a conversation. Agile methods depend upon an ecosystem of supporting practices like providing dedicated users, colocated with developers, to engage in a frequent collaboration, providing feedback early and often. Developers deliver often so that there is something on which to gather feedback. We did this for a small portion of the project, and that portion succeeded.
But what happens when you switch to a so-called “agile” method, complete with brief stories (promises to have a conversation), yet you don’t assign enough users to have the conversation? Or you don’t assign enough business analysts to help the users express their complex requirements coherently? We tried to do that for the rest of the project, and it failed. We got poor requirements, and it left the impression that agile methods don’t work.
What works is having the right users, assisted by skilled business analysts, to provide requirements that are good enough that developers can produce a system that is usable. I have come to really value skilled business analysts, even in an agile context. If the project is small, then perhaps a skilled developer can play the BA role. But on a large, multi-system project, or on a multi-project team, keeping up with requirements and architecture are just too much for someone who is focussed on coding. And most users are not good at expressing their requirements in a consistent and simple way. Eliciting the essential requriements is an art, as is resolving ambiguous requirements through reason, persuasion, and negotiation.
So here we had four case studies: One RUP pilot that was over schedule and over budget, but completed and well liked by the users. Several agile pilots that were successful. They took longer than expected. In all these cases, is the delivery at fault or the expectations? And we have one large project that failed, whether it was run under RUP or Agile approaches. Interesting. Small succeeded, large failed. On closer examination, it is clear why the large project failed. It exceeded the resources available. RUP and Scrum configure resources differently. They have different rythms and styles. But neither method can succeed if the scope of work overwhelms your ability to define scope. Also, neither approach can succeed if the scope exceeds the resources. And you need a balanced team of cross-functional skills (devs, testers, analysts). If your team is having consistent problems with requirements, and you do not have a business analyst, then hiring another programmer will probably just aggravate the problem.
If you bite of more than you can chew, you will choke. It doesn’t matter how you try to chew it. The only thing that will save you is to spit it out, cut it up, and eat it one small bite at a time. And you need all your teeth for chewing, not just the canine teeth (the pointy ones).