Big Modeling Up Front (BMUF) Anti-Pattern

Synonyms

• Big design up front (BDUF). BDUF is commonly used to refer to this approach in the XP community.
• Big requirements up front (BRUF). BRUF and BDUF typically go hand-in-hand.

Motivation

BMUF is a popular approach because people:

1. Mistakenly compare software development to civil engineering. A common analogy is to compare software models to architectural diagrams for a bridge or building. Unfortunately the analogy isn’t an accurate one: software is more malleable than concrete making it easier and much less expensive to change your mind part way through the effort; The material (app servers, operating systems, “¦) used to build software isn’t as well understood as the material to build bridges (steel, concrete, “¦) making it much more difficult to accurately plan up front; Incremental delivery often doesn’t make much sense when you’re building a bridge, so what if the foundation has been set but the rest of the bridge isn’t in place, although incremental delivery of software is clearly possible and as I argue above the norm. Much better analogies are comparing software developers with chefs or comparing software development with putting on a play (as described in the wonderful book Artful Making).
2. Get motivated to do so by the business. Your business stakeholders often demand “accurate” estimates and schedules to be developed early in the lifecycle so that they can determine whether they want to fund the initiative. This motivates IT people to model in detail up front to support the creation of those detailed plans, plans which often prove to be wildly inaccurate in practice. This inaccuracy is the result of our stakeholders being unable to tell us what they actually want, this is human nature, so as a result the quality of our estimates are only as good as the quality of the information the they’re based upon. We need to educate the business on the inherent challenges associated with fixed price projects and provide them with better strategies for funding agile teams.
3. Think comprehensive requirements documentation means something. There is a common misconception among stakeholders that you can fully specify a system before constructing it. Yes, you can in fact do so, but as you see in Examining the “Big Requirements Up Front (BRUF)” Approach this leads to incredible amounts of wastage even if you end up “successfully” delivering software. In other instances, development teams often misunderstand, or outright ignore, parts or all of the requirements specification anyway.
4. Don’t know any better. Many organizations have spent the last thirty years putting a well-defined, non-agile, and very serial process in place that encourages BMUF. The U.S. Federal Government is a classic example of such an organization, often requiring extensive requirements documents to be created before allowing any other form of development to occur. Yet, as this site shows, you do in fact have other options.
5. Are overly specialized. When you staff your team with specialists, they’ll perform their specialist tasks very well whether it’s appropriate or not. When you have modeling specialists on the team they’ll do a lot of modeling, and likely write a lot of documentation, because that’s what they’re good at. The end result is that you’ll get far more documentation than you actually need — you pay for it, you get it. What you really want are generalizing specialists who have the skills and abilities to do just enough modeling, coding, and so on.
6. Believe programmers don’t have the skills to model themselves. This is a self-fulfilling prophecy. If you hire overly specialized people, then set up an environment where one group of specialists hands off work products (e.g models) to other groups of specialists, then you pretty much guarantee that they won’t learn a wider range of skills.
7. They got sidetracked by traditional data professionals. Unfortunately many within the data community believe that you require a serial approach to design, particularly when it comes to databases. This belief is the result of either not understanding evolutionary development or some misguided need to identify the “one truth above all else”. Evolutionary database design techniques such as agile data modeling, database refactoring, and database regression testing work incredibly well in practice.

Impact

There are several significant problems with a BMUF approach:

1. It reduces your willingness to embrace evolving requirements. The more that you invest in detailed models early in an initiative, the more decisions that you will make based on those details, and the harder it will be to update your work if change does occur. This quickly demotivates you to embrace change. But, the reality is that change happens. Yes, it is clearly possible to create comprehensive requirements documents before allowing construction to begin, and you can clearly build a system that fulfills those requirements, but is it what you really want? As I discuss in Agile Requirements Modeling stakeholders’ understanding of the system and their environment changes, their priorities change, the environment in which they do business changes, all motivating changes to requirements. A BMUF approach may ensure that you get the system that you ask for but it virtually guarantees that you won’t get what you actually need. The interesting thing is that the 2008 Project Success Survey found that when given the choice business stakeholders overwhelmingly prefer solutions which meet their actual needs, not solutions built to spec.
2. It increases the chance of poor decisions. This isn’t intuitive at first, but the reality is that at the beginning of an initiative is when you are most ignorant about the situation that you face. Yes, gathering more information can help a bit, but what usually happens is that you start going down an inappropriate path and simply gather more information which backs up your poor choice. We know that people aren’t very good at defining up front what they want, but that they are reasonably good at providing feedback and homing in on a solution via an evolutionary approach. The implication is that although creating detailed models early on provides a “scientific facade”, all it really does is motivate you to make and then commit to decisions based on what we know to be poor quality information. As lean software development tells us, we are much better advised to defer committing to decisions until the point where we actually need to make them as that puts us in a position where we’ll have the best possible information upon which to base that decision.
3. It increases the chance your vision will be ignored. By doing too much modeling up front, then handing it off to the coders, you pretty much remove all of the joy out of development for them. Is it any wonder that developers often choose to ignore the models they’re provided, or to “improve upon” the ideas contained in them?
4. It decreases morale. If you treat programmers like “coding monkeys” who can’t be trusted to make important technical decisions, you’re effectively telling them that they are low-skilled and unimportant. The end result is that your good developers will leave your organization, and that a significant percentage of your remaining developers will in fact be the low-skilled people that you’re apparently striving to attract. Doesn’t sound like a great strategy to me.

The Solution

You still need to do some initial envisioning up front, including both requirements envisioning and architectural envisioning. You’re still doing some initial up front modeling, it’s just that you are doing so in an effective and agile manner. You should create very slim, high-level models early on which overview the scope of the effort and identify a likely architectural strategy. Then model storm just in time to get the details when you need them. With this approach you retain the advantages of modeling, to think something through, without suffering from inflexibility or over-documentation.

I suspect that the best thing that can be said about BMUF approaches is that it provides a context which justifies the employment of modeling professionals. I’m not sure that is what your organization is actually hoping to achieve.

Acknowledgements

I’d like to thank Huet Landry for his feedback concerning this article.