Agile Architecture Envisioning: An Agile Core Practice
- When should you do initial agile architectural modeling?
- Why should you do some initial agile architecture modeling?
- What should you model initially?
- What modeling tools should you use?
- How much modeling do you actually need to do?
- Why do you need to do less initial architectural modeling than you think?
- Parting thoughts
Agile Model Driven Development (AMDD), see Figure 1, explicitly includes an initial architectural modeling effort during initial envisioning of an agile initiative (what some processes might call the sprint 0, Inception, or Initiation phase). Initial architecture modeling is particularly important for scaling agile software development techniques to large, complex, or globally distributed development (GDD) efforts.
Some people will tell you that you don’t need to do any initial architecture modeling at all. However, my experience is that doing some initial architectural modeling in an agile manner offers several benefits:
- Improved productivity. You can think through some of the critical technical issues facing your team and potentially avoid going down fruitless technical paths.
- Reduced technical risk. Your team gains the advantage of having a guiding vision without the disadvantage of having to overbuild your system – just because you’ve modeled it doesn’t mean you have to build it.
- Reduced development time. Initial agile architecture modeling enables you to make better cost and time estimates, two pieces of information which management will want.
- Improved communication. Having a high-level architecture model helps you to communicate what you think you’re going to build and how you think that you’ll build it, two more critical pieces of information desired by management.
- Scaling agile software development. Your initial architecture model will be a key work product in any “agile at scale” efforts because it provides the technical direction required by sub-teams to define and guide their efforts.
- Improved team organization. Effective teams are organized around the architecture or line of business, not around job function. As you scale to larger and/or distributed teams the sub-teams should each be responsible for one or more sub-systems — you don’t want to organize your sub-teams around job function (e.g. an architecture team, a development team, a testing team, …) because that requires you to increase the documentation and bureaucracy overhead which in turn increases risk, cost, and time to value.
Early in the initiative you need to have at least a general idea of how you’re going to build the system. Is it a Python application? A .Net application? Java? Something else? To do this the developers on the team will get together in a room, often around a whiteboard, discuss and then sketch out a potential architecture for the system. This modeling work is based on, and performed in parallel to, your initial high-level requirements modeling efforts. Your architecture will evolve over time so it does not need to be very detailed yet (it just needs to be good enough for now), and very little documentation (if any) needs to be written.
When I’m doing initial agile architecture modeling I’ll typically focus on high-level free-form diagrams which overview how we think we’ll build the system. I will typically focus on:
Often some form of technology stack diagram (Figure 2) or deployment diagram will do (Figure 3). These diagrams are useful because they depict the major software and hardware components and how they interact at a high level. This includes legacy assets, including legacy databases and legacy systems, which may need to be analyzed in greater detail later in the initiative. Right now your goal should be to identify these assets and perhaps discuss their viability as sources of functionality and/or data for your system.
You will also be identifying technical constraints at this point in time. For example, although you would love to work with the latest version of DB2, unfortunately your corporate database standard is Oracle (as you see in Figure 2) and you’re therefore constrained in that architectural choice.
Another common diagram to create is a user interface (UI) navigation or UI-flow diagram, see Figure 4, to explore how you will architect the UI of your system by exploring the flow between major UI elements, including both screens/pages and reports. This is critical to your system’s success because the user interface is the system to your stakeholders. Not the technology, not the data, and not the really cool frameworks that you’re working with. If you do not architect the user interface effectively you run the risk that you will build a system that your stakeholders aren’t interested in working with.
Part of your initial architectural modeling efforts, particularly for a business application, will likely include the development of high-level domain model as you see in Figure 5. This model should be very slim, capturing the main business entities and the relationships between them. Some people consider this type of model to be an initial requirements model instead of an initial architecture model — it doesn’t really matter because as you see in Figure 1 both of these initial modeling efforts are performed in parallel anyway. Figure 5 depicts an example using UML data modeling notation (you can use any notation that you like when agile data modeling, I prefer UML). The initial domain model will be used to help guide both the physical data model as well as the class design, potentially captured via a UML class diagram. I will often create this type of diagram using a whiteboard initially and then later I might transfer it into a drawing tool if the effort of doing so provides sufficient value.
Finally, another common architectural model that I sometimes consider capturing are change caseswhich are potential architecture-level requirements which your system may need to support. Figure 6 provides examples of two change cases, the describing a potential a technology change and the second a potential business change. Change cases allow you to test the long-term viability of your architecture without requiring you to overbuild your system because you can think through the impact of likely changes to ensure yourself that your system will still work.
|Change case: Registration will occur completely via the Internet.
Likelihood: Medium likelihood within two to three years, very likely within ten years.
Impact: Unknown. Although registration will be available online starting in September, we currently expect less than one quarter of registrations to be made via the Internet this year. Response time will be an issue during the peak use periods, which are the two weeks prior to the beginning of classes each term, as well as the first week of classes.
Change case: The university will open a new campus.
Likelihood: Certain. It has been announced that a new campus will be opened in two years across town.
Impact: Large. Students will be able to register in classes at either campus. Some instructors will teach at both campuses. Some departments, such as Computer Science and Philosophy, are slated to move their entire programs to the new campus. The likelihood is great that most students will want to schedule courses at only one of the two campuses, so we will need to make this easy to support.
Agile modelers will use the simplest tool which will get the job done. As you can see in Figure 2 and Figure 4 I’ve found whiteboard diagrams typically suffice for initial architecture models. Your goal is to think through the critical technical issues at this point in the lifecycle, not create pretty diagrams. There’s nothing wrong with using more sophisticated drawing tools, as you can see in Figure 3 and Figure 5, just ensure that the investment you’re putting into the effort is worth it. It all depends on the situation that you’re working in. Furthermore, paper-based tools are also quite common. For example I would be tempted to initially capture the change cases of Figure 6 on index cards and only capture them using a more sophisticated tool if appropriate.
Let’s consider several common situations that you team might find itself in, and then reflect upon how much architectural modeling would be appropriate in that situation. These scenarios are:
- Your team is working with known technologies that it has worked with before. This is the simplest situation you can find yourself in and at most I suspect that you need to create a single whiteboard sketch to ensure that everyone is working to the same vision, and frankly you likely don’t even need that. For example, if this is the tenth web-based application that your team has built using Websphere and DB2, how much architectural modeling do you really need to do?
- You have little or no experience with the technologies. Traditionalists will often tell you that to reduce your risk that you should model everything in detail, but if you step back and think about it for a minute, this actually increases your risk. Does it really make sense to do a lot of detailed modeling when you don’t really know what you’re doing? No, all that could possibly achieve is that you waste a lot of time, write a lot of documentation which will likely be ignored by the developers because they’ll know pretty quickly that you don’t know what you’re talking about, it give you false confidence that you know what you’re doing, and it will likely give management a false sense of confidence that the team has a solid architecture plan in place. What you really want to do is do a little bit of modeling to help you identify what you don’t know and, if possible, you want to invite people on to your team who do have the experience that you’re missing. Then you need to gain some hands-on experience with the actual technologies themselves via architectural prototyping/spiking, and then based on that experience evolve your architectural models so that they reflect what actually works. In other words, let your architecture emerge.
- Somewhere in the middle. This is where you typically find yourself: perhaps you have experience with some but not all of the technologies rr perhaps your team is experienced with the technologies but simply hasn’t used them all together at once. In middle ground situations such as this it makes sense to do a lot more modeling initially (perhaps even several days) so as to identify potential risks and likely strategies to mitigate them. Modeling makes sense in this situation because it enables you to define a shared technical vision within the team and to potentially think through major issues. In other words, some initial architectural modeling will offer value to your team.
Figure 7 depicts the value of modeling, showing that models reach their point of maximal value when they are barely good enough, and for initial architecture envisioning this is typically after several days or a week or so. Many people are initially shocked by this claim, but when you think back to all the teams that you’ve ever been on, how many of them, if you were allowed to, could you have modeled a pretty good initial architectural strategy within a week by putting a few smart people together in a room with a lot of whiteboards? I’ve asked this question of several thousand people now and only been told about a handful of teams where this wasn’t true. A lot of people thought that they had examples where it didn’t work out, but their problems almost always revolved around not being able to get people who knew what they were doing or logistical problems of getting the people together. The people that did need more than a week were in situations where the team was distributed so they needed to invest more time to identify the subsystems/components and the interfaces to them so that they could then organize subteams around implementing those subsystems or subcomponents (a practice called API First or Conway’s Law). Or they were working on life critical systems (or death critical systems in the case of a couple of military projects), or were in situations where a significant investment in software or hardware was required.
Traditional modelers assume that it is desirable to model the problem and solution domain in detail at the beginning of an initiative. I refer to this as “big modeling up front (BMUF)“, something which encompasses “big requirements up front (BRUF)” where you create a detailed requirements specification early in the lifecycle and “big design up front (BDUF)” where you create a detailed requirements specification early in the lifecycle. These practices make sense when you’re a modeling specialist, and sure enough traditional modelers always have seem to have good reasons for why BMUF is desirable. Table 1 summarizes the common arguments for doing detailed architecture models early in an initiative and argues why they’re wrong.
|We can think the technical details through up front||This may be true of very simple systems, but for even moderately complex systems we cannot hope to think through the details because of the various nuances of each technology, changing usage patterns of the components, and changing technologies themselves. It may appear that everything in your architectural model is going to work, but until you prove it with code and then act on what you learn you’ll never know for sure. The best architectures evolve over time, they’re not defined up front.|
|We need to identify the best technical strategy||Yes, you need to identify a good strategy but that doesn’t mean you need to write a lot of detailed documentation. With a BDUF approach you’re being asked to make serious technical decisions when you know the least about the problem domain, increasing the chance that you’ll choose the wrong path. Lean/agile approaches instead advise you to make these sorts of decisions at the last possible moment when you can safely make them, instead of at the beginning.|
|We need to build the technical foundation first||You need a technical vision to work towards and good communication/collaboration between the people on the team. A detailed architecture/design motivates you to overbuild your system — it’s in the design, so you’d better build it, whether the stakeholders really need it or not. How many times have you seen teams spend the first six months building great frameworks which will support all future needs, only to discover that a lot of the framework isn’t needed at all? Unless you get the design perfectly right, which is very difficult to do in practice, you’ll end up building far more than you need to.|
|We need to specify the design for the coders||Coders are really smart people. They may need some guidance and leadership, but they very likely don’t need to be told in detail what to do. The fact is that you put a yourself at risk when you present a detailed architecture to coders. Some coders may choose not to follow the design in the belief that they know better (and perhaps they do). Worse yet, the more modeling and documentation that you do, the greater the chance that’ll you’ll commit to the documented approach because of the investment that you’ve made in the work. This occurs even when the design is known to be questionable because it’s often too much effort to rework the design approach once you’ve started down a specific path.|
|Architecting software is just like architecting a bridge||The bridge analogy generally goes like this: “Building a bridge is a very complex thing, just like building software. Bridge builders create complex blueprints up front and then work to those plans, therefore we should do the same thing too.” If someone doesn’t have direct experience building bridges and building software then shouldn’t we question their assumptions? I know some physical architects, and they work in a far more creative and evolutionary manner than you might think. To see a world-class architect in action, I highly suggest watching the documentary “The Sketches of Frank Gehry”. I can very safely say that Gehry is an agile modeler, not a traditional one. So, perhaps the bridge analogy does hold, but not in the way that traditionalists think.|
|The architecture should be defined in detail just as the requirements are||Have you ever tried to fully specify your design when you are decorating a room in your home? No matter how much effort you put into it, you’ll always discover that the couch is at the wrong angle, that the wall art doesn’t work with the area rug, and so on. You always have to adjust your design and sometimes even make major changes. If you can’t do something as simple as specify up front the decoration of a room, what makes you think you can do the same for a system which is many orders of magnitude more complex? People don’t know what they want; at best they may understand their goals/intent. People are very good at saying what they don’t like, and providing suggestions for possible improvements. This implies the need to take an evolutionary approach where you obtain feedback on a regular basis, not a serial, documentation-based approach. This assumes of course that you actually want to build a system which reflects the needs of its stakeholders. The bottom line is that requirements change throughout an initiative and you need to accept this and act accordingly.|
|Detailed documentation and models add value||This argument might be partly true, although writing comprehensive documentation early in the lifecycle is the wrong time to do it. Agile Modeling recommends that you write documentation only when the information being captured has stabilized AND if someone understands and is willing to cover the total cost of ownership (TCO) of that documentation. You don’t want to document speculative things, and documenting requirements and proposed designs early in the lifecycle are clearly speculative, because speculative things have a tendency to change and thereby increase your TCO for that documentation.|
|There’s no proof that evolutionary techniques work||Actually, yes there is. When people ask for “irrefutable evidence” what they’re often saying is that they don’t want to move away from their preferred way of working. Traditional approaches were adopted with far less evidence than what we have today supporting evolutionary approaches, and to be quite blunt I think that the traditional track record speaks for itself — the evidence, if you’re willing to accept it, clearly points away from BMUF.|
You should strive to do just enough modeling for the situation at hand and no more. The implication is that there is no one correct answer for how much modeling each team will do, that instead it will vary from team to team. My philosophy is that repeatable results, in this case having a shared architectural vision within the team, are far more important than following a repeatable process that ensures that each team does the same amount of modeling each time.
In later iterations both your initial requirements and your initial architect models will need to evolve as you learn more, but for now the goal is to get something that is just barely good enough so that your team can get going. In subsequent releases you may decide to shorten iteration 0 to several days, several hours, or even remove it completely as your situation dictates. The secret is to keep things simple. You don’t need to model a lot of detail, you simply need to model enough. If you’re writing use cases this may mean that point-form notes are good enough. If you’re domain modeling a whiteboard sketch or collection of CRC cards is likely good enough. For your architecture a whiteboard sketch overviewing how the system will be built end-to-end is good enough. I cannot say this enough: your goal is to build a shared understanding, it isn’t to write detailed documentation. A critical success factor is to use inclusive modeling techniques which enable active stakeholder participation.
Many traditional developers will struggle with an agile approach to initial modeling because for years they’ve been told they need to define comprehensive models early on. Agile software development isn’t serial, it’s iterative and incremental (evolutionary). With an evolutionary approach detailed modeling is done just in time (JIT) during development iterations in model storming sessions.