Design Debt

Introduction

Traditional design/bid/build projects are plagued by a problem, a problem that's not unique to architecture. The problem is that there's too much work to do and not enough time to do it. Work that needs to be done takes longer than expected or doesn't get done at all. The result: design debt. If you haven't heard of design debt before, it's because, until now, we haven't had ways to describe and measure it accurately. This paper introduces design debt in the context of architectural projects. It shows what design debt is, how it accumulates, how to measure it and how to manage it.

What is Design Debt?

Design debt [1] is work that remains unfinished after a deadline. It's work you wanted to complete but didn't. You just submitted a set of contract documents to the building department, but didn't have time to include window details. Completing the window details and other related work you didn't do to get the building permit is design debt.

Design debt usually has to be paid off after the deadline, in a subsequent project phase. You'll have to complete the window details before the project goes out to bid. Otherwise, you'll have to explain your intent later, by responding to questions or RFIs during construction.

Paying back design debt can be painful. The building department might reject your drawings because important pieces are missing. The contractor might request a change order for additional work because a detail wasn't worked out. Delayed work might take longer to finish because original team members are no longer available. You often pay design debt back with interest.

Design debt tends to demoralize project team members. The work should be done, but there's more to do. We feel uneasy. The larger the design debt, the more uneasy we feel.

XPM's task-based practices make design debt visible and measurable, early in the project when you can do something about it.

XPM Refresher

In XPM, we break a project into phases, then into tasks. We create a task card for each task. We then estimate the number of points required to complete each task and add it to the card [2]. Each point represents an ideal hour, or the time it would take if we could work on the task without interruptions. Then, each iteration, which is typically a week long, we choose the highest priority tasks and finish them. The number of tasks we choose depends on the team's velocity, which we track each iteration. If our velocity is 20 points per iteration, we choose and expect to complete 20 points of work each week.

Measuring Design Debt

Using XPM, we can tell, after the first week or two, whether we'll be able to meet our deadline or not. For example, with 120 points to complete, and a velocity of 20 points per week, we'll need 6 weeks to finish a schematic design package. If we have only 5 weeks, we'll want to change the schedule, reduce the scope, add more staff -- or accept 20 points of design debt. In other words, to measure potential design debt, just count the points you don't expect to complete.

To measure actual design debt after you've met a deadline, just collect and total the points on the remaining task cards.

Measuring design debt helps you make projections and tradeoffs, as you'll see later.

Types of Design Debt

Design debt is represented by a stack of task cards. Each card includes an estimate of points required to complete it. We measure design debt by counting the points in the stack. There are other distinguishing characteristics of design debt that help us describe and classify it.

Potential vs. Actual. Design debt can be either potential or actual. If you estimate a project at 100 points and know you can only complete 90 by deadline, you have 10 points of potential design debt. You may be able to eliminate the potential design debt by extending the schedule, adding more staff, or reducing the project scope. After you've met your deadline, count the points on the remaining task cards to determine your actual design debt. You'll have to pay off the actual design debt in the next project phase or deal with it some other way.

Hidden. Most design debt, whether actual or potential, is visible and can be measured because it's represented by task cards. Some potential design debt, however, might remain hidden until discovered. Hidden design debt is represented by tasks that you must do but haven't noticed yet. For example, you might be required to prepare a cost estimate, but neglect to make a task card. You might show an assembly on a drawing, but fail to specify it. Someone might update a floor plan, but forget to update the ceiling plan. Hidden design debt always shows up sometime in the future, usually during a review process or finally, during construction.

Cumulative. Cumulative design debt accumulates consistently each iteration. It's caused by understaffing. Debt accumulates as long as the project remains understaffed. For example, if a project is estimated at 1000 points and scheduled for 10 weeks, we need to complete 100 points per week. If the team's velocity is only 90 points per week, then we'll accumulate 10 points of design debt each week. The design debt is built-in. Cumulative debt is worse than it seems. If we continue with a velocity of 90 points per week, after three weeks, we'll have to do 104 points each week for the remainder of the project to clear the design debt. If we wait until six weeks, we'll have to do 115 points. And if we wait till the end, we'll have to do 190 points the last week.

Design debt can also be classified by the types of tasks included.

Elastic vs. Inelastic. Elastic tasks are difficult to estimate. We don't know how long they'll take because we learn more about the task as we work on it. An elastic task might take one hour or ten. Most design tasks are elastic. So are tasks that require selection. You can select the first thing you see or spend hours doing additional research. Inelastic tasks are 'cut and dried'. There is no question about when an inelastic task is done. You know the task to create a drawing index is done when all the sheets have been added to the index. We try to avoid accumulating elastic design debt.

High-Value vs. Low-Value. There are two types of high-value tasks: high-reward and high-penalty. High-reward tasks are those that return a higher than normal payback. They're tasks that we learn from, usually via feedback, rather than those that simply move the project forward. A code review is a high-reward task because it tells us whether we have to change our design to meet code requirements. Presenting a schematic design at a meeting to get feedback from the client is also a high-reward task. High-penalty tasks are those that must be done because the penalty for not doing them is so high we can't defer them. For example, if drawings won't be accepted by a building department without a wet signature, then adding a wet-signature is a high-penalty task. Low-value, or 'soft' tasks, on the other hand, are less important. They're the 'nice to do' tasks. We try to complete high-value tasks early in the project and defer low-value tasks until later. We want all our design debt to be soft. Some soft design debt might even be healthy. It indicates that we've thought of all required tasks and deliberately put off the least important.

Generative. A generative task is one that generates more tasks. Design reviews, coordination reviews, and other types of reviews are generative. The review itself can be reasonably estimated, but the number of new tasks generated by the review can't. Meetings that generate action items are generative tasks too. We try to schedule generative tasks well before a deadline so we can complete them as well as the tasks they generate.

Time-Sensitive. Time-sensitive, or deadline tasks need to be done by a specific time. Like preparing for a meeting. If you don't do a time-sensitive task on time, you have to deal with the consequences. In many cases, once the time has passed, the task is no longer required. All tasks are ultimately time-sensitive, but here we're talking about tasks that must be completed within a specific pre-deadline iteration. Time-sensitive tasks come up frequently and are dealt with as they arise, so they rarely contribute to actual design debt.

Causes of Design Debt

There are three interrelated variables under your control on an architectural project: schedule, scope, and staff [4]. Changing one affects the other two. If scope increases, you can increase the schedule or increase staff to compensate. If you reduce the schedule, you can reduce scope or increase staff to compensate. Failure to adjust the variables appropriately leads to design debt. We'll look at each variable as a cause of design debt independently, but keep in mind that they're interrelated. We'll first cover schedule (not enough time), then scope (too much work), then staff (not enough staff). Then we'll dig deeper.

Not Enough Time

Design debt occurs on projects constrained by time. With unlimited time you can work as long as you have to. You won't accumulate design debt. Once you commit to a fixed schedule, you risk accumulating design debt. The shorter the schedule relative to the amount of work required, the more likely you are to build up design debt.

In XPM, we estimate the time it will take to complete each task, knowing that our initial estimates are likely to change. If our estimates are consistently low, we'll probably accumulate design debt. For example, we estimate a project at 1000 points. It really requires 1200. If we staff and schedule based on the 1000-point estimate, we've just built in 200 points of design debt. Of course, we don't know that at the beginning of the project.

Too Much Work

We staff and schedule a project based on scope. We have to estimate how much work will be involved. If we guess low -- if there's more work required than we estimated -- we'll accumulate design debt, unless we make adjustments as the project progresses.

The amount of work on an XPM project is represented by a set of task cards (or a list of tasks in a spreadsheet). We try to think of every task required to complete the project and add it to the list. If we forget an important time-consuming task, our estimate will be wrong. If we base our staff and schedule on a low estimate, we'll end up with design debt. Here's a simplified example. We estimate a three-week project at 90 points and allocate staff accordingly. We forget to include the finish schedule, which takes 10 points. We can put off the finish schedule until later, or do it now and put off a lower priority task. Either way, assuming we're unable to make other adjustments, we accumulate 10 points of design debt.

Not Enough Staff

Design debt is caused by understaffing. If you could select just the right staff for a project, adding and removing team members as needed, you could theoretically meet any realistic schedule. In the real world, we're constrained by staff. We select a project team and stick with it, often throughout the project. This isn't as bad as it seems, because team members readily adapt to changing conditions and requirements. But understaffing does lead to design debt, so let's talk about it.

Sometimes we understaff projects intentionally, because we have no other alternative. Perhaps the right people aren't available and we can't find or afford to hire more. Assuming the schedule and scope can't be changed, we'll have to accept design debt -- cumulative design debt, the worst kind.

Some managers understaff intentionally, hoping that somehow things will work out to their benefit. In this case, I'm reminded of a quote from Hal Macomber: "hope is not a project strategy".

Many architectural firms schedule and staff a project based on fees available. The intent is to limit the work to match the fee. However, this leads to design debt if the fee doesn't correspond to the work required. For example, take a project with an $8,000 budget for schematic design. If the average billing rate is $100/hour, you can spend 80 hours to finish the schematics. You schedule one person for two weeks. The problem shows up when you create a release plan and discover that there's more work required than you anticipated. Say the release plan shows you need 100 hours to complete the schematics, not 80 as you originally estimated. Unless you make adjustments to staff, schedule, or scope, you'll have 20 hours of design debt to deal with. Instead of basing schedules on fees, base them on the actual amount of work involved. Use task-based, not fee-based scheduling.

Underlying Causes

More often than not, we don't understaff, over-schedule, or accept more work than we can do intentionally. We do it because we've misestimated something. Usually that means overestimating what we can do or underestimating what must be done.

Overestimating What We Can Do

We typically staff and schedule a project based on the amount of work we think the team can do. If we think we can do more than we actually can -- if we overestimate what we can do -- we're likely to accumulate design debt. For example, say a project requires 1000 points to complete. You estimate the team's velocity at 100 points per week, so you set a 10 week schedule. If the team's velocity is actually only 90 points per week, you accumulate 10 points of potential design debt per week.

How do you accurately estimate how much work a project team can do? For an ongoing project or an experienced team transitioning to a new project, it's fairly easy: Just look at the team's velocity from prior iterations. Velocity tends to remain stable. If we did 100 points last iteration, we're likely to do 100 points this iteration. It's harder to estimate velocity for a newly-formed team, but you can determine a team's velocity after the first few iterations. If possible, delay making time commitments until a team's velocity has been established.

Underestimating What Must Be Done

There are two ways you can underestimate what must be done. You can underestimate the scope, that is the amount of work itself, or you can underestimate the time it will take to do the work. I'll talk about ways to handle these problems as I talk about avoiding design debt.

Avoiding Design Debt

Assuming you have the requisite staff, if you schedule and staff a project based on accurate estimates, you won't accumulate design debt. You'll need to estimate the project's scope, that is, how much work is involved. You'll need to estimate the time it takes to do the work. And you'll need to know how much work the team can do in a given time period. I've already discussed how to use a team's prior velocity to estimate how much work the team can do, so now I'll cover the last two items: estimating scope and time.

Estimating Scope and Time Together

The most reliable and easiest way to estimate both work and time is to use prior projects for reference. If the current project is similar to a prior one, you can base your estimates on prior tasks and time. You may not even have to break the project into tasks at this stage. If schematics on a prior project were 200 points [3], schematics on the current project should be too, assuming the projects really are similar. Otherwise, use task cards from prior projects selectively. Just choose tasks from prior projects that apply to the current one and use or modify the estimates as you see fit.

Estimating Scope

If you don't have prior projects to refer to, you have to determine the amount of work required by other means. Using XPM, that means breaking the project into phases and listing every task you can think of for each phase. I've discussed ways to do this in a paper called Release Planning Tips. I've extracted some examples:

Review your contracts. Include every deliverable and activity in your contracts, both described and implied. Look for tasks in the Owner-Architect Agreement, Consultant Contracts, Construction Contract, General and Supplemental Conditions, and the Specifications. At the same time, make sure your contracts clearly define the expectations of all parties concerned. Clarify if necessary. Use common sense to reveal implied tasks. For example, if you need to obtain someone's approval, you'll have to prepare something, present it in a meeting, get feedback, make modifications based on the feedback, have another meeting, and perhaps even repeat the process several times.

Use checklists. If you don't have prior projects to refer to, list tasks based on checklists, or, as I call them, task lists. Even incomplete lists can be useful.

Collaborate. More people provide more perspectives. Collaboration may take more time, but if your schedule and reputation depends on an accurate list of tasks, the additional time spent will be worthwhile.

Add contingencies. Experience helps, but even with experience, you can't foresee every task that will be required on an architectural project. If you must set a fixed schedule, add contingencies for unanticipated tasks.

Estimating Time

After you've determined how much work is required, you have to determine how much time the work will take. Here are some suggestions:

Use prior projects. I mentioned this above, but want to re-emphasize the value of prior projects. Even if you didn't use XPM and don't have task cards on which to base your time estimates, you still might be able to extract valuable time-related information from prior project reports.

Break large tasks into smaller ones. Smaller tasks are easier to estimate. Your estimates might be wrong, but high estimates will tend to offset low ones. What's important, perhaps even more than the estimates, is to include every possible task you can think of.

Collaborate. I also mentioned this above, but task estimates based on consensus are more likely to be accurate than those done by someone working alone. A solo person might introduce a consistently high or low bias.

Too high? You may be shocked when the sum of your individual task estimates is higher than expected. Verify your estimates if they seem low, compare them with prior projects, but don't change them without a good reason.

Commonly Overlooked (and Underestimated) Tasks

Managing Design Debt

If you can't prevent design debt by modifying the scope, adding more staff, or extending the schedule, you'll have to manage it. Here are some suggestions:

Do high-value tasks early. Each iteration, as you review the remaining task cards, select the highest value tasks first. Things like elastic tasks, generative tasks, and high-reward tasks. The remaining tasks will be low priority. Some of them may never get done. Some may never need to get done. Make sure the most valuable tasks get done first.

Defer low-value tasks. Being even more proactive, if a task can be put off until later, defer it. Put the deferred task cards in a separate pile so you don't have to look at them each planning meeting. If they're important, they'll come up again.

Use task cards. In XPM, we write a card for every task and don't do any task unless there's a card for it. This prevents scope creep very effectively, whether from in-house or outside pressures. Since we select and discuss each task in planning meetings, everyone has an opportunity to question the task. Is it part of our contract or an additional service? Is this task really required or is it 'gilding the lily'?

Scrutinize each task. In planning meetings, we read and discuss each selected task aloud. We discuss it to scrutinize it. Is this task really required? Can we accomplish the task objective a better way? Is there a way to do this task quickly? Have we done something similar before? Can we put it off till later?

Define each task. As part of our task discussions, we define what 'done' means for each task. It usually means the task and related tasks are done. If a task calls for a wall to be moved, it usually means that the floor and ceiling plans are updated, the interior elevations changed, area calculations revised, and consulting engineers notified so they can change their designs. But it might mean just move the wall and get feedback from the client. That's why we discuss it. For each task, we do what needs to be done, and nothing more.

Timebox elastic tasks. Keep elastic tasks from spiraling out of control by timeboxing them. Limit the time spent to an hour or two. Then evaluate the results.

Do frequent reviews. Don't wait until the deadline. Reviews provide feedback. They uncover incomplete work and generate additional tasks. We want this information sooner rather than later. In XPM, someone reviews and integrates the output of each task as soon as it's finished.

Keep the same project team. Over time, a team that works together gets more efficient. Team members develop shortcuts for communicating and learn how to work effectively with each other. They maintain a vast collection of knowledge about the project. They know who did what and who knows what. Adding and removing team members disrupts the team's balance and slows them down.

Improve via retrospectives. In XPM, we hold weekly retrospectives to discuss issues and find ways to improve. Is design debt increasing? Did a task take longer than expected? What can we do about it? How can we improve?

Each sheet is a task card. Near the end of a project, where there are one or more review sets showing work to be done, we use each sheet as a task card. A team member selects a sheet (or specification section) and makes all the corrections required, collaborating with other team members as required.

Overtime. Having the project team work overtime is always an option to manage design debt. However, overtime should be used sparingly and as a last resort. It's ineffective for working off large amounts of design debt and leads to staff burnout. Overtime may be required to get the last few tasks done before a deadline, but canceling non-project tasks, like staff meetings, can be just as effective.

Fast-track. Fast-tracking means doing only what's required to meet the next deadline and no more (unless you have extra time). It's an extreme method of selecting the most important tasks first. For example, if the next deadline is to get a building permit, do only what's required to get the permit. While waiting for the building permit, add additional items so you can bid the project. I haven't tried this suggestion and there's plenty of risk involved, so I wouldn't even consider it unless there were no other alternatives.

Notes

[1] The debt metaphor was initially conceived to describe a condition where software developed over time becomes more and more complex unless efforts are made to constantly simplify it by refactoring. As the code becomes more complex, it's harder to add to it because each time you add something, you risk breaking something created earlier. For example, see:

In architecture, almost all debt design-related, so I use the terms 'design debt' rather than 'technical debt'.

[2] We usually create a spreadsheet to generate task cards during the initial planning stages. We list the tasks in one column and points in another. This makes it easier to calculate totals and make revisions. Later we print the cards using a word processor's mail-merge function.

[3] When a task is done, the person who completed it initials the task card and adds the actual number of points to the card, adjacent to the estimate. We use the actual points for estimating future projects.

[4] This is a variation on the 'project triangle' (scope, time, and cost, with quality in the center) for traditional projects. I consider quality a function of scope, with scope represented by a set of task cards. Scope, in other words, is not scope as defined by contract -- it's much more detailed and comprehensive than that. I use 'schedule' instead of 'time', and measure it in days or iterations. Instead of 'cost', I use 'staff' which comprises most of the variable cost of an architectural project. Staff implies both the number and composition of the project team. I also prefer the terms 'schedule, scope and staff' for their alliterative value.

Printer-Friendly Version

Click here for a version of this page that's more suitable for printing. After printing, click the back-arrow of your browser to return to the original format with an index alongside.

Feedback Wanted

I welcome your comments and suggestions. See contact us for more information.

Document Information