Understanding XP through FDD

tesugen's picture

In July I decided to explore the fundamentals of FDD by comparing it to XP. It turned into an attempt to formulate XP using the ETVX format, which proved to be very fruitful for the understanding of XP.

The blog entry is titled Understanding XP through FDD, and I'd appreciate any comments or corrections as I am sure there are points where I haven't understood FDD fully.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

XP and FDD - In part a response to tesugen

Peter

Interesting post!

There are a number of ways in which FDD and XP are similar, and many of these are just good practice. There are also a number of ways in which FDD and XP are fundamentally different. In part because they address different problems. You rightly identify FDD as a process, while XP is more a set of principles. This has important implications for their applicability.

You are correct in saying XP has only 2 processes (I find it hard to think of Explore as a process in its own right!). I also think the Planning Game process is mis-named and should be called Manage Requirements.

I'd like to look at how FDD and XP are similar even when apparently different, and then explore how they address different classes of problems. Finally I briefly explore what you might categorize as a theoretical difference.

BTW, I am strongly of the opinion that there is no one-size-fits-all solution to organizing software development. This is because the problems we have to solve are often very different. There are circumstances where I would use FDD and circumstances where I would use XP. It depends on the problems I have to solve and the constraints under which I have to solve them. There are also circumstances where I would use neither!

Both FDD and XP incorporate best practices established and well understood for quite a while. A good example is that FDD uses inspections to remove defects and improve quality, while XP uses pair programming. To me, they are both variants on the same thing! Pair programming is process of continuous inspections (while FDD uses discrete inspections). I am convinced both work (and this has been known for at least 20 years). The difference is which is more effective/efficient. I am not aware of empirical evidence comparing the two, but its really the only way to determine which is superior. But I stress both work and produce very similar results.

One way in which FDD and XP address different problems is in testing. XP formalizes the requirement you write the test first, whereas FDD has little to say on the subject of testing and certainly does not mandate writing the tests first. While I am a convert to automated testing, I am unconvinced of the benefits of writing tests before writing the code and I am not aware of anyone has demonstrated significant benefits.

So how do XP and FDD address different problems!

Perhaps the most important way is scale. You cannot organize a large project the same way you can organize a small one! XP works best when you have a small project (less than ten people). This is because of the interaction problem - in any system where people interact as peers, as the number of people increases linearly, then number of interactions increases logarithmically. It is well understood that any peer-to-peer system breaks down at over seven to nine people. The *only* way out of this problem is process and division of responsibilities. IMHO FDD works best with medium size projects - say ten to fifty people. I should point out that many XP proponents would dispute this. In my defence I direct you to this paper http://www.xpuniverse.com/2001/pdfs/EP205.pdf which points out that in a fifty person XP project, lack of an overall view of the system was a serious problem. Note this helps explain one way in which XP and FDD differ - in class ownership. XP says all classes are collectively owned, while FDD has class owners. I strongly suspect collective class ownership doesn't scale and the article above tends to confirm this!

A second way in which software development can vary is the extent to which the requirements are understood upfront, and have been agreed and articulated. While customers are reluctant to admit this, it is a frequent problem. There are several reasons. One is that different parts of the business have different priorities that often conflict. Another is that introducing a new system necessarily changes the problems the business needs to address (if it didnt there wouldnt be any point to the system). XP assumes that requirements are progressively exposed throughout the project and handled through progressive refactoring. In contrast FDD exposes the requirements upfront.

To summarize the XP position - You can always refactor incrementally! In effect you can incrementally increase the scope of your software indefinitely without significant issues. However, I have seen what I call the scope-discontinuity problem on a lot of software projects. It is best explained by anecdote. You have a set of requirements and design a solution (in an OO project an object model). As time passes new requirements are discovered (exposed). Your solution gracefully incorporates these new requirements, until without warning there is one requirement that your solution can not support. Your solution is broken and major refactoring is required. In contrast FDD exposes the full scope of your software upfront. This is one of the major consequencies of the first FDD process - Develop an Overall Model. The overall model defines the scope of the software upfront. This effectively avoids the scope-discontinuity problem, but at some cost in incorporating new requirements.

The third important difference between XP and FDD I'd like to highlight is the nature of the commitments they make. XP is (IMO) a variant of RAD (Rapid Application Development). RAD and therefore XP commits to deliver in specified timeframes, i.e. when, but makes no commitment on what is delivered (The Planning Game - not part of the original XP - is designed to solve this problem). Whereas FDD endeavours to define what is delivered as well as when it is delivered early in the project. This is the purpose of the startup phase (the first three FDD processes) Jeff has described in his latest newsletter.

Another way of explaining this is that there are three primary variants in software development - scope, time and resources. FDD attempts to fix all three. Process 1 - Develop an Overall Model is the first step in fixing scope. You can view it as fixing coarse-grained scope. FDD process 2 - Build a feature list, fills in the details of the scope. Process 3 - Plan by Feature attempts to fix time and resources. So by the end of the FDD startup phase, all three are fixed. In contrast XP assumes fixed resources (or makes no attempt to plan resources) and commits to frequent releases. However, the XP Planning Game is continually feeding new requirements into the development process. So scope is continually changing. A good argument can be made that the XP approach optimizes time and resources, while handling changing requirements (and priorities). The downside is that it makes the customer's planning harder because of limited visibility on future deliverables. I have a rather flippant stock answer to the question - When will the system be finished?, which is - When you run out of money! This answer contains a hard truth that is difficult for customers to grasp, that there are always more requirements for any system, irrespective of the time and resources spent to-date. There is a lot to be said for getting the customer to make the hard choices about requirements as early as possible. This is one of FDD's biggest advantages over XP!

I am an advocate of release oriented software development. There is much to recommend it. Not least the customer sees frequent tangible progress (a good way to ensure your project doesn't get cancelled Smiling), but there are limits to the businesses capacity to implement new releases. Both XP and FDD deliver frequent new releases. I think XP is more suitable when new releases are made of an existing and in production system, especially where the business impact (i.e. the need for business changes) is small. XP also works well when developing relatively small new systems with minmal impact on existing business processes, such as some websites. I think FDD is more suitable when developing a new system of significant size, or a new software release that significantly impacts the business, because of FDD's superior visibility on deliverables and dates (for reasons explained above).

BTW, I think both XP and FDD would both be improved by a release process, but I'm straying off-topic!

Finally the theoretical difference! I would characterize this as - What is the best way to achieve the optimum underlying model for the system? While I call this a theoretical difference it has important practical consequencies!

Implicit in the FDD approach is that it is possible to create a reliable and robust domain object model from a set of relatively high-level requirements. Peter Coad and Jeff deLuca go further and say that software applications share common business domain patterns and these patterns can be re-used to produce reliable robust domain models (Implementation object models are domain models plus implementation patterns). See the Java Modelling in Color Book! I know from my own experience that robust domain models can be produced early in the development process. This has the important consequence that large scale refactoring is not required at the detailed design and development stage (assuming you got the domain model and implementation patterns right!). In contrast XP has no explicit domain model, and as previously described relies on continuous refactoring to get the right object model. Which leaves me with two key questions! How reliable is XP in getting the right object model?, and How expensive is the refactoring along the way?

As always, the opinions expressed are mine alone!

Phil Bradley

Jeff De Luca's picture

Testing and Scale

Testing is most definitely a part of FDD. It is a mandatory step in the Build By Feature process. It is part of the exit criteria for that process.

It is a common misconception, that FDD cares less about testing than other methods. Well, there aren't that many mandatory steps in FDD but testing is one of them. You must test. It is fundamental and in FDD a feature is not finished until it passes all test.

FDD does design before code and test. XP doesn't - thus since XP writes tests before coding, by default it stands out more (i.e. it's in the way it is described that makes things seem different).

Regarding scale, lack of an overall view of a system is a major concern at scale. Fred Brooks wrote about this in Mythical Man Month where he said that experiences with big systems show the brute-force approach to scaling up to be costly, slow, inefficient and to produce systems that are not conceptually integrated.

This is one of the reasons (but not the only reason) for the chief programmer led feature teams in FDD as a chief-programmer (surgical team) organisation offers a way to get the product integrity of few minds and the total productivity of many helpers, with radically reduced communication paths.

Regarding project/team sizes, I have not seen an upper-bound of only 50 for FDD. I believe, strongly, that it can scale far beyond that. The structures behind this I have used on a 250 person project. At the other extreme, it can also very successfully scale below 10. Again, we have much empirical evidence for this. It is down around 3-4 people where FDD needs tweaking. For a team size that small, you think differently about the assignments of classes as you tend to assign related classes as chunks to a developer. That is, somewhere more functionally or package oriented.

Jeff

Polarization and Middle Ground

Phil,

I have to say that I agree with comments entirely. I really agree that the agile community has sought to polarize the debate into agile versus non-agile where agile is defined as something pretty similar to XP and everything else is seen as heavyweight and traditional.

This is common technique in political science because it forces people to make a choice - "which side are on?"

In Alistair Cockburn's 2002 book, Agile Software Development, he claims that XP does not scale beyond 12 people. I know that the core XP crowd disagree with this view.

My view of FDD, having run 13 projects with it in the last 4 years, is that it becomes truly effective at 10 people. Well below 10 people e.g. 4 people it is problematic and cumbersome.

I believe that FDD does represent a good pragmatic choice for many businesses who need some variant on the traditional quality process of define the scope first, then estimate the schedule and budget. The "trust me I'm a developer" and "just give me some money and a month and I'll be back" doesn't cut it in many organizations.

I believe that you will find chapter 32 of my book interesting in this regard.

David
--
David J. Anderson
http://www.uidesign.net/
The Webzine for Interaction Designers

Ontology mapping

Hi Tesugen,

You have provided an interesting perspective, epscially for one such as myself who has only read about XP and not actually worked on a proper XP project.

I think the ETVX framework is a very powerful way of describing processes. It could also be seen as an ontology, into which we can map processes and compare them. I think there may need to be another layer to compare how each handle the standard 'lifecycle' phases/activities of requirements, development, etc.

One thing that struck me about your article was how you noted it was difficult to "fit" XP into the ETVX framework. This is at odds with the vast majority of processes, and may very well be intentional, due to the "extreme" nature of XP. It does however make meaningful comparisons that much harder.

You mention also that XP seems to be about practices and values just as much as it is about processes. FDD certainly has many practices and values, and shares many with XP. Unfortunately they have not all been formally documented, which may give some the wrong impression. But I certainly like many of the values and practices that I've read about XP.

In so far as FDD subscribes to the Agile Manifesto, it takes on the values it espouses. It may happen that the "instantiation" of these somewhat "abstract" values differ when embedded in FDD versus XP. That in itself would be a worthwile topic for further investigation...

You mention: "But for XP, I feel that the tasks for the Plan and Develop processes, to a greater extent would be tasks that you execute non-sequentially and multiple times."

Isn't this equivalent to FDD's #4 Design by Feature and #5 Build by Feature? These phases are highly iterative, and so would correspond strongly to those in XP.

The main differences (as I see it) would be that in XP, the activities are driven by the tests while in FDD it is driven by features (derived from functional requirements). But since I see testing as a means of validating that the product meets the functional requirements, there must be an implicit design that the test cases embody..

The other big difference is that in FDD, within processes #4 and #5, there are quality gates at each stage. This means that code or design inspections, held with the Chief Programmer (CP) and the Developers, are held to review the work product. These can iterate too - ie. if issues are identified during a design review, it goes around again before coding. This makes sense, given the well-known rule that the later a change occurs in the project lifecycle, the more expensive it is to implement.

"When reading about FDD, you sometimes get the feeling that it is very close to XP. For example, features in FDD and user stories in XP have much in common." I thought that XP's stories were more like Use Cases? If so, that's fairly different to FDD Features. I'm not a big fan of Use Cases, but I don't really know how XP treats requirements (with indifference perhaps? Eye-wink).

"I find it appealing although I think that they do too much modeling before starting programming." This is something that comes up a lot. It would appear that XP is in many ways "anti-design". But FDD doesn't advocate a complete and detailed design in Develop an Overall Model. That model in #1 is mainly concerned with getting the shape right and identifying the significant classes, and the relationships between them. The model is fleshed out in #4 Design by Feature, and only then during these iterations of design-build do you actually do low-level design.

So maybe XP cannot sensibly be mapped into the ETVX framework after all. I don't know it well enough to say; perhaps at a finer level of granularity we could. For example: the Entry for a Plan cycle might be that you have a story ready to go. Then the Task is to write the tests that validate the story has been implemented. The Validation may be that all tests pass, and the Exit is a complete set of tests that correspond to a story, ready to implement. Does that make any sense?

(It may seem linear, but it is inherently iterative, and certainly is in FDD also.)

There are many interesting points here worth pursuing...

regards,

:: Gavin

http://antonym.org