90+ Best Project Management Hacks 2019
Working on a larger-than-one-person development project means working on a team, and working on a team means being managed. So learning something about project management from both sides is an essential part of learning software development. This blog explains the best 90+ Best Project Management Hacks that used in 2019.
Project management is an involved and complicated set of tasks. We’ll restrict ourselves to several tasks that will impact you as a developer the most:
Traditional project managers usually take on a great deal of responsibility. They are responsible for managing scope, cost, estimation, schedule, quality, personnel, communication, risk, and more.
However, in an agile project, the entire team is usually responsible for managing the project. If there is a separate project manager, that person is largely responsible for the following:
Making sure the team has the resources it needs
Ensuring that the team adheres to the agile values and principles
Shielding the team from outside interruptions
In particular, the agile project manager doesn’t manage the day-to-day operations of the team. That’s up to the team itself. The objective is to make sure there is no delay in management decision-making.
Project planning is forever. By that, I mean that project planning continues throughout the entire duration of the project. “The Plan” is never really set in stone, because things in a typical software project are usually in constant flux.
In those projects that are using a plan-driven process model, a project plan is an actual document, written by the project manager, that is approved and signed off on by the development team and by upper management.
It is, in effect, a contract, albeit a rolling one, of what the team is going to do and how they’re going to do it. It says how the project will be managed, and in the most extreme plan-driven projects, even states how and when the document itself will be modified.
What’s in the project plan? Generally, a project plan consists of the following seven parts:
Introduction and explanation of the project
Hardware, software, and human resource requirements
Task list and size and effort estimates
Project monitoring and reporting mechanisms, collectively known as project oversight
Not all of these are necessary for all projects or project methodologies. In particular, plan-driven projects will use all of them, whereas agile projects may use a few on a single page.
A project plan is a great tool for setting down what you think you’re doing, an outline of how it will be done, and how you plan on executing the outline. The problem with a project plan is that it’s static. Once it’s written and signed off on, upper management thinks the project will run exactly as stated in the plan. But the reality of the project often thwarts the plan.
As with many other parts of project management, agile methodologies do project planning differently.
An agile project plan is feature-based
is organized into iterations.
is owned by the team, not the project manager.
The project organization section of the plan contains the following three things:
How you’re going to organize the team
What process model the project will be using
How will the project be run on a day-to-day basis
If you’re working with an experienced team, all this is already known to everyone, so your project organization section can be “We’ll do what we usually do.”
However, this section is a necessity for brand-new projects and inexperienced teams, because the organization section gives you something to hang your hat on when you start the actual project work.
Agile projects simplify the three items just mentioned. Nearly all agile teams are self-organizing. Because they’re small and because one of the agile principles is the idea of common code ownership, agile developers don’t segregate themselves based on what part of the code they’re working on or on a particular skill set.
Agile developers share code, share testing, and share expertise. Like all software developers, those on an agile project are constantly learning new things and improving their skills.
No matter what agile process model is being used—XP, Scrum, Crystal, feature-driven, and so on—all agile projects are iterative, use short development cycles, and produce running code with more features at the end of each iteration. Simple.
Different agile projects will run differently on a day-to-day basis. But pretty much all of them include daily stand-up meetings, at least daily integrations, and some form of shared programming—for example, pair programming.
Whether the project is using a time-boxed methodology or not, iterations are typically short, and developers spend quite a bit of time writing and executing unit tests.
In the risk analysis section, you need to think about the bad things. What can possibly go wrong with this project? What’s the worst that could happen? What will we do if it does? Some risks to watch out for include the following:
Schedule slips: That task that you estimated would take three days has just taken three weeks. In a plan-driven project, this can be an issue if you don’t have regular status meetings. Waiting three weeks to tell your boss that you’re late is always worse than telling them that you’ll be late as soon as you know it.
Don’t put off delivering bad news. In an agile project, this is unlikely because most agile projects have a daily status meeting. That way, schedule slips are noticed almost immediately, and corrective action can take place quickly.
Defect rate is excessive:
Your testing is finding lots of bugs. What do you do—continue to add new features or stop to fix the bugs? Again, this can be a real issue in a project where integration builds happen according to a fixed schedule—say, once a week.
In a project where integrations happen every day or every time a new feature is added, you can keep up with defects more easily.
In either case, if you’re experiencing a high defect rate, the best thing to do is to stop, take a look around, and find the root cause of the defects before adding more functionality. This can be very hard to do from a project management standpoint, but you’ll thank yourself in the end.
What you’re doing isn’t what the customer wanted. This classic problem is the result of the fact that customers and developers live in two different worlds.
The customer lives in the application domain where they understand from a user’s perspective what they want the product to do. The developer understands from a technical perspective of how the product will work.
Occasionally, these worlds intersect and that’s good; but lots of times they don’t and that’s where you get a misunderstanding of requirements. The best ways to avoid this situation are to have the customer on site as often as possible and to produce deliverable products as often as possible.
New features, altered features, deleted features . . . will the misery never end? Requirements churn is probably the largest single reason for missed delivery dates, high defect rates, and project failure. Churn happens when the customer (or your own marketing folks, or the development team itself) continues to change requirements while development is underway.
It leads to massive amounts of rework in the code, retesting of baselines, and delay after delay. Managing requirements is the single most important job of the project manager. In a plan-driven process, a change control board (CCB) examines each new requirement and decides whether to add it to the list of features to be implemented.
There may be a member of the development team on the CCB, but that’s not required, so the danger here is that the CCB adds new features without understanding all the scheduling and effort ramifications.
In agile processes, the development team keeps control of the prioritized requirements list (called the product backlog in Scrum), and only adjusts the list at set points in the project—after iterations in XP, and after each sprint in Scrum.
Turnover: Your most experienced developer decides to join a startup three weeks before product delivery. The best way to reduce turnover is to (1) give your developers interesting work, (2) have them work in a pleasant environment, and (3) give them control over their own schedules. Oddly enough, money is not one of the top motivators for software developers.
This doesn’t mean they don’t want to get paid well, but it does mean that throwing more money at them in order to get them to work harder or to keep them from leaving doesn’t generally work. And if despite your best efforts, your best developer does leave, you just have to move on. Trust me, it won’t be the end of the world.
The best way to mitigate the effect of turnover is to spread the knowledge of the project around all the members of the development team.
Principles like common code ownership and techniques like pair programming work to invest all the team members in the product and spread the knowledge of the code across the entire team. One of the best blogs on managing and keeping software developers is Peopleware by Tom DeMarco.
Once you’ve got a list of the risks to your project, you need to address each one and talk about two things: avoidance and mitigation. For each risk, think about how you can avoid it.
Build slack into your schedule, do constant code reviews, freeze requirements early, do frequent releases, require pair programming so you spread around the knowledge of the code, and the like.
Then you need to think about what you’ll do if the worst-case scenario does happen; this is mitigation. Remove features from a release, stop work on new features and do a bug hunt, negotiate new features into a future release, and so on. If a risk becomes a reality, you’ll have to do something about it; it’s better to have planned what you’ll do beforehand.
Once you address avoidance and mitigation, you’ll have a plan on how to handle your identifiable risks. This doesn’t completely let you off the hook, because you’re bound to miss risks; but the experience of addressing the risks you do come up with will enable you to better handle new ones that surprise you during the project.
If your project is using an iterative process model, it’s a good idea to revisit your risks after every iteration and see which ones have changed, identify any new ones, and remove any that can no longer happen.
This section is a piece of cake. How many people do you need for the project? Do they all need to start at once, or can their starting dates on the project be staggered as phases are initiated? How many computers do you need? What software will you be using for development? What development environment do you need?
Is everyone trained in that environment? What support software and hardware do you need? Yes, you do need a configuration management system and a stand-alone build machine, no matter which process model you’re using.
The platform you’re targeting and the application domain you’re working in answer many of these resource questions for you. That’s the easy part. Questions about team size, start dates, and phases of the project will likely not be able to be answered until you do the first cut at effort estimation and scheduling.
The first step toward a project schedule is seeing what you’ll be doing and figuring out how long each step will take. This is the classic chicken-egg problem. You can’t really do estimation until you have a fairly detailed decomposition of the features or user stories into tasks.
But your manager always wants effort estimates and schedule data before you start doing the design. Resist this. Make design your top priority once you’ve got some idea of the requirements.
If you select a small set of high priority requirements and then design a solution for that feature set, then you can do an effort estimation of that iteration. Don’t worry that the requirements might change—they will. You need a detailed decomposition of features into implementable tasks before you can do effort estimation.
Don’t ever believe anyone who tells you, “That feature will take six months to do.” That is a wild-assed guess (WAG) and bears little to no relation to reality. You just can’t estimate something that big. The best you can do is to say, “I was once on a team that implemented a feature like that in six months.” And even that only helps a little.
You’ve got to get your work broken down into tasks that are no more than about a week in duration. One or two days is a much better bet. Even better, never do estimation in any unit except person-hours.
That way you’ll be more tempted to work with small increments of hours, and you’ll break your larger tasks down into smaller ones that you may actually know how to do. Once you have a believable list of tasks, you can start doing size and then effort estimation.
Size always needs to come first, because you just can’t figure out how long something will take until you have an idea of how big it is. Size can be several things, depending on your work breakdown and your development model; functional modules, number of classes, number of methods, number of function points, number of object points, or that old standby, uncommented lines of code.
Actually, no matter what you initially measure size in, you’ll end up with estimates in terms of KLOC—thousands of uncommented lines of code.
There are several techniques for getting effort estimates: COCOMO II, function point analysis, and the Delphi method are just three. All, however, depend on being able to count things in your design. The estimation mantra is: size first, the effort and cost estimates, finally schedule.
All other things being equal, the Delphi method is a quick and relatively efficient estimation technique. Here’s one way it can work: find three of your most senior developers.
These are the folks who’ve got the most experience and who should, therefore, be able to give you a good guess. Then give them the task breakdown (assuming they weren’t already involved in doing the initial breakdown—the ideal situation).
Then ask them to give you three numbers for each task: the shortest amount of time it should take, the longest amount of time it should take, and the “normal” amount of time it should take, all in person-hours.
Once you have these numbers, add them all up, the shortest together, the longest together, and the “normal” together and take the mean.
Those are your estimates for each task—the averages of the best guess by your best developers for each task. You then use those average values for each task as the official (for now) effort estimate and proceed to create a schedule.
Finally, you should have the right people—the developers who will do the work—do all the estimates for the project. Managers should never do development estimates. Even if a manager has been a developer in the past, unless they’re deeply involved in the actual development work, they shouldn’t be in the business of doing development estimates.
Once you have estimates of the tasks in your first release or iteration and have people resource estimates, you can create a schedule. There are several things to take into account before you can look at that spiffy Gantt chart with the nice black diamond that marks the release date. Here’s one possible list:
Get your developers to tell you the dependencies between tasks. There will be some tasks that can’t start before others finish. There may be tasks that can start once others are half-finished. There will be some that can all start together. You need to know because the task dependencies will push out your delivery date.
Figure out what your duty cycle is. Out of each eight-hour day, how many hours do your developers actually do development? You need to remember that reading mail, attending meetings, doing code reviews, taking breaks, going to the bathroom, and lunch all eat up time. You can’t assume that an eight-hour task will be done in a single day.
Realistically, out of each eight-hour day, two to four hours are eaten up with other stuff, so your duty cycle can be as low as 50%—four hours a day. Duty cycles can also vary based on corporate culture, so you need to figure out what yours is before you start to the schedule.
Take weekends, vacations, sick days, training, and slack into account when you’re making the schedule. If your senior developer has a task on the critical path of your project, you probably need to know that they’re taking that three-week vacation in May.
You can’t schedule a developer to work on two tasks at the same time. Most project-scheduling software won’t let you do this by default, but most of them will let you override that.
Don’t. You’ll be tempted to do this so that your schedule doesn’t push out past whatever deadline your manager or marketing team wants, but resist the temptation. You’ll only have to change the schedule when you miss the date anyway.
Finally, use project-scheduling software to make your schedule. You don’t have to do this—just using a simple spreadsheet technique like the one proposed by Elon Spolsky can work for small projects.
But using real project-management software like Microsoft Project, Fast Track Scheduling, Basecamp, or Merlin for plan-driven projects and web applications like Jira, Pivotal Tracker, or Trello for agile projects provides lots of features that make keeping the schedule up to date much easier.
The big thing that project-management software can do that your spreadsheet can’t is track dependencies.
Elon doesn't understand how Microsoft Project is useful in this; in fact, he says, “I’ve found that with software, the dependencies are so obvious that it’s just not worth the effort to formally keep track of them.”
This might be true for small projects, but when your team gets to be 10 developers or larger and you’re working on 100 or more tasks, knowing something about the dependencies of your project can help manage who’s working on what, and when.
This knowledge is even more critical for agile methodologies like Scrum. In Scrum projects—where you’re using short time-boxed sprints and you need to know the priorities of each user story in the product backlog and how they relate to each other.
And you must have good estimates for how long each task will take and you need to track the progress of a number of tasks in the sprint backlog every day— knowing dependencies can be the difference between making your sprint and not.
Elonis right that tools like Microsoft Project are overkill for many projects, so for those, you can use a spreadsheet approach that just lists the features and tasks you can see right now (see Table ); but project-management software sure is handy to have around when you need it.
Table. Spolsky’s Painless Schedule (with the Velocity Addition)
Task Priority 2
Orig Est 3
Curr Est 4
Spolsky’s painless schedule lists the following seven columns that should be in every schedule:
Tasks within the feature
The Priority of the task
The Original Estimate (in person-hours)
The Current Estimate (in person-hours)
The Elapsed Time worked on the task (in person-hours)
The Remaining Time on the task (also in person-hours)
Elon correctly emphasizes that tasks need to be fine-grained and small in terms of effort. Otherwise, as noted previously, your estimates will most likely be wildly off. He also suggests that each developer either have a separate spreadsheet or, as shown here, you add the eighth column with the developer assigned to the task.
Having all the tasks on the same sheet makes it more crowded, but easier to see all the tasks at once. Though not exactly “painless,” this method of keeping a schedule is useful for smaller projects with a fairly limited number of tasks.
[Note: You can free download the complete Office 365 and Office 2019 com setup Guide for here]
I suggest adding a ninth column to measure the velocity of each task. Velocity is a term from XP6 and is defined as the estimated effort of a task, divided by the actual effort.
In our case, we’d use the Original Estimate of the task and the Elapsed Time. If you overestimate your task, your velocity will be greater than one (your task took less time than you originally thought); if you underestimate, it will be less than one (the task took you longer than you originally thought). Ideally, velocity should be 1.0, but that hardly ever happens.
It’s worth noting that humans are really bad at estimating time. To get around this problem, some project-management tools have you create an estimate using magnitude instead.
The simplest type lets you do small, medium, or large for estimates, whereas others (for example, Pivotal Tracker and Trello) let you give each task some number of points on a scale—say, 1, 2, 4, or 8 points for each task.
This makes it easy to total up the number of points the team is trying to implement in each time-boxed iteration and over time gives you the average numbers of points a team can implement per iteration.
The reason for using velocity is to give each developer and the project manager an idea of how accurate the developer’s estimates are and to help do a better job of estimating next time.
Ideally, as a developer gains experience, their velocity will approach 1.0 on each task. Alternatively, if a developer’s velocity jumps around a lot (one task is 0.6, another is 1.8, a third is 1.2), then a crash course in estimation techniques might be appropriate.
In my experience, a new developer’s velocity will start out gyrating wildly, with most values well under 1.0; the new developer is overly optimistic. But as time goes along velocities will settle into a range centered on 1.0, maybe from 0.85 to 1.15. As a developer gains a history, the project manager can then start to depend more on their estimates, and the schedules will be more accurate.
Project oversight is what happens once you’ve got a schedule and your project is underway. It will typically include a number of review meetings for things like the overall architecture, design, and code.
Once your project begins, the work needs to be managed. How this happens depends on the process you’re using. But regardless of the process, you need to manage the schedule, manage the developers, manage the process itself, and above all, manage your manager.
A manager’s technique is critical to keeping a project on schedule. Fear is not a motivator. Appealing to professional pride is, though. If your boss doesn’t support you, you’re doomed.
Without creative, supportive management, you’re doomed. If your people aren’t happy, you don’t have hope. Treat your developers as humans, not resources. Supporting your team and keeping them insulated from distractions is your number one job. Remember, projects are cooperative, social events.
Status Reviews and Presentations
Status reviews and presentations are an inescapable part of any project. The bigger the project, the more formal the review. Remember that reporting status doesn’t fix problems and that generally, upper management doesn’t like hearing about problems.
Tough. When you give a project status report, just tell them where your project is and where it’s going during the period before the next status report.
Don’t embellish and don’t make excuses; be honest about problems and where you are in the schedule. Just providing good news is usually bad for your reputation; something will go wrong at some point, so it’s best to report it and get it out of the way right away.
You must communicate bad news about the project as soon as possible. That’s the best way to mitigate the problem and get others involved in helping to find a solution.
When giving a presentation, be it a status review or a technical presentation, make sure you know your audience. Set your presentation to the level of the audience and keep the purpose of your presentation in front of you and them at all times. PowerPoint is ubiquitous in the industry, so learn to use it effectively. Keep your PowerPoint presentations short and to the point.
Avoid cramming your slides with lots of bullet points. Don’t make your bullet points complete sentences, mostly because you’ll be tempted to read them. This is the kiss of death for two reasons: it takes too long and takes attention away from what you’re actually saying. And it insults the audience. Surely they know how to read?
Your bullet points should be talking points that you can then expand upon. This lets your audience focus on you, the speaker, rather than the slides. When you’re constructing a PowerPoint presentation, use as few words as you can.
Inevitably, you’ll introduce defects (errors) into your program. Defects don’t just appear; developers put them there. As a developer, your aim is two-fold:
Introduce as few defects as possible into the code you write.
Find as many of them as you can before releasing the code.
Note, By the way, I’m deliberately not using the word bug here, because it sounds both inoffensive and cute. Defects are neither. They are errors in your code that you put there.
Despite your best efforts, though, you will release code with defects in it. It’s just inevitable. For a program of any size, there are just too many possible paths through the program (called a combinatorial explosion) and too many different ways to introduce bad data for there not to be defects.
Your objective is to release with as few defects as possible and to make those defects ones that don’t really impact the product or its performance.
To make this a reality, most development organizations have a set of defect levels they use to characterize just how bad a defect really is. These defect levels are generally built into a defect tracking system that allows developers and project managers to see how many and how severe the current defects are. One set of levels looks like the following:
\1.\ Fatal: Either this defect causes the product to crash, or a fundamental piece of functionality doesn’t work (for example, you can’t save files in your new word processor).
\2.\ Severe: A major piece of functionality doesn’t work, and there’s no workaround for it that the user can perform (such as Cut and Paste don’t work at all).
\3.\ Serious: A piece of functionality doesn’t work, but there is a workaround for it that the customer can perform (for example, the keyboard shortcuts for Cut and Paste don’t work, but the pull-down menus do).
\4.\ Annoying: There’s a minor defect or error in the documentation that may annoy the user, but doesn’t affect how the program works (say, Paste is always spelled Patse).
\5.\ New feature request: This isn’t a defect, but a request for the product to do something new (as in, the word processor should have a speech-to-text feature built in).
Whenever you find a defect in a piece of code, you’ll file a defect report in the defect tracking system (to keep track of how many defects you’re finding, what types they are, and how severe they are) and you’ll characterize the defect by severity level. When the developers are fixing defects, they start at level 1 and work their way down.
In nearly all organizations, no product can release with known level 1 or level 2 defects in it. Most organizations also try their best to remove all the level 3 defects as well.
Most development teams will do a retrospective after every project. A retrospective, as the name implies, is an opportunity to reflect on the project just completed and answer a few questions. Typically, the questions will be like the following:
What went right? Did our process work the way we anticipated? Did we meet our schedule? Did we implement all the features required by the customer?
What went wrong? Why did we have so many defects? Why did we need to work 60-hour weeks for the last month of the project?
What process issues came up? Did we follow our process? If not, what parts were problematic?
What do we need to fix for next time? Given the answers to the preceding questions, what do we need to fix in our process, work habits, or environment for the next project?
Who’s responsible for the fixes? Does someone have to be responsible for the changes to our process—who is it? (Don’t make it a manager; the development team should own the process).
In a plan-driven project, the retrospective is typically held either after the product release or after each major iteration of the product. In an agile project, a retrospective is always held after every iteration. So, for example, in Scrum, a retrospective is held after every sprint.
So where do we end up? We’ve gone through the general parts of managing projects, and I’ve presented some alternative ways of doing project management.
The most important ideas to consider are that the developers should own the process and management should be supportive and listen to the developers— particularly where schedules and estimates are concerned—and be the buffer between the developers and the world.
If you can work in an organization where those things are true, be a happy camper because you’ll be able to write great code.
The hardest single part of building a software system is deciding what to build. No other part of the conceptual work is as difficult in establishing the detailed technical requirements, including the interfaces to people, to machines, and to other software systems. No other part of the work so cripples the results if done wrong.
No other part is more difficult to rectify later. Therefore, the most important function that the software builder performs for the client is the iterative extraction and refinement of the product requirements.
Before you start coding – yes, before you start coding—you need to know what it is you’re going to build. That’s what requirements are: a list of stuff you have to implement in order to create your terrific program.
Most developers hate requirements. Really, all we’d like to do is sit down and start coding. All of us have that super-programmer mentality: “Just give me the problem and I can sit down and design and code it on the fly.”
Not! If you want to be a productive developer, make fewer errors, and come up with a good, clean design, you need requirements. And the more detailed they are, the better. A good set of requirements tells you just what the program is supposed to do. It gives you the scaffolding around which you’ll hang your design.
You’ll do requirements anyway—it’s one of those steps in a standard development life cycle that you can’t avoid—but if you don’t make room for it in your project, you won’t create a great program.
Being intentional about requirements forces you to think about the details of the program and lets you listen to the users so you have a better idea of what they really want. So let’s talk about the requirements.
What Types of Requirements Are We Talking About
We’re really talking about functional requirements, the list of features the user will see and be able to use when they fire up your program. These are the “black box” requirements that show the external behavior of your program. As far as the user is concerned these are the only requirements that matter.
In a plan-driven process, the output of this activity of identifying requirements is a functional specification of what the software system is supposed to do. For an agile process, the output is a set of user stories that define the product backlog.
During the course of uncovering requirements for your project you’ll usually see four different types:
user requirements, domain requirements, non-functional requirements, and non-requirements.
User requirements are nearly always expressed in natural language. They are the details of what the user expects to see as they use the program. They also include descriptions of screen layouts, dialog boxes, and menus. Any interaction element in the program should be described in the user requirements. For example:
Logging into the system: When Gloria clicks the Login button on the main page, a Login dialog box appears in the middle of the screen. The Login dialog must contain two text boxes, labeled “Username” and “Password.” There must also be two buttons in the dialog box, labeled “Submit” and “Cancel.”
If at any time Gloria clicks the Cancel button, the dialog box shall disappear and she will be taken back to the previous screen. In normal usage, she will click in the Username text box and type in her username, and then click in (or tab to) the Password text box and type in her password.
The text typed in the Password text box must be hidden. Once Gloria is finished typing in her username and password she must click the Submit button. If she has entered a correct username/password combination she will then be taken to the main menu page.
If Gloria’s username/password combination is incorrect, an “Invalid username or password, please try again” message shall appear in the dialog box, the text boxes shall be cleared, and she will be given the opportunity to log in again.
As seen in this section, you can express user requirements as scenarios, and as detailed screen-by-screen descriptions. Remember to use pictures as much as you can when you’re doing user requirements.
If your program is web-based, you can create lots of quick and dirty HTML pages to show the user. If it’s not web-based, use a drawing program to create pictures of what the user will see, or just draw them by hand on paper.
These are requirements that are imposed on you by the application domain of the program. If you’re writing a new version of an income tax program, you will be constrained by the latest IRS regulations.
A general ledger program will have to abide by the latest edition of the Generally Accepted Accounting Principles (GAAP), and a smartphone will need to implement the latest Global System for Mobile communication (GSM) protocols.
You don’t need to write down all these requirements, just refer to them. A set of detailed domain requirements gives the developers information they will need during the design of the program.
Domain requirements are usually considered “middle layer” software because they are the heart of the application, below the user interface and above the operating system, networking, or database software.
A lot of domain requirements will get implemented as separate classes and libraries with their own APIs. Users are concerned with domain requirements only insofar as they affect the user requirements.
Non-functional requirements are constraints on the services and functions of the program and also expectations about performance. They can include target platform specifications, timing constraints, performance requirements, memory usage requirements, file access privileges, security requirements, response times, the minimum number of transactions per second, and so on.
These are usually requirements that may not be visible to the user, but that does affect the user experience. An example of this type of requirement is that your web page must load and display within three seconds.
These are the things you’re not going to do. You will need to communicate this to the customer because after laying out what the program will do, the most important thing to do in the requirements phase is managed expectations.
One of the worst phrases a customer can utter at that final demo before you release is, “But I thought it was going to do . . .” You need to tell all the stakeholders in a project what the program is going to do and also what it’s not going to do.
In particular, you need to let them know that there are requirements that won’t be implemented—at least not in the current release. “Only one countdown timer may run at a time.” “There will not be a defrost cycle that allows defrosting modes to be selected by food type.”
It’s likely that your customer won’t read this section, but at least you can point to it when they ask. Be careful, though, because the number of things that your program will not do is nearly infinite.
Requirements Gathering in a Plan-Driven Project
A functional specification describes what the program will do entirely from the user’s perspective. It doesn’t care how the software is implemented. It talks about the features of the program and specifies screens, menus, dialogs, and the like. Think of it as a badly written user manual. The second kind of spec can be called a technical specification.
The technical specification describes the internal implementation details of the program. That is, it talks about data structures, algorithms used, database models, choice of programming language, and so on. We’re not going to talk about technical specs, just functional specs.
They don’t write functional specs. So there! I’m off the hook.” Well, in fact, agile methodologies do write functional specifications. They’re just in a different format from the 300-page single-spaced requirements document that some plan-driven methodologies require.
XP requires that, together with the customer representative or product owner, you write user stories that lay out what the program will do. That’s a spec. We will discuss how agile methodologies do requirements later in this blog.
The important part and the idea behind this entire blog is to write down what your program is supposed to do before you start coding.
But I Don’t Like Writing
A standard argument made by software developers is that they can’t write. Nonsense! Everyone can learn to write functional specs. But writing is work. You have to get in there and practice writing before you’ll be any good at it.
If you’re still in school (be it undergrad or graduate school), take a course in writing, one where you’ve got to write essays or journal entries or stories or poetry every single week. You should also have to read other works critically; reading other people’s writing, whether good or bad, is a great way to learn how to write better.
Functional requirements should always be written in a natural language. Why? Well, it’s the Sapir-Whorf linguistic relativity hypothesis, don’t you know?
In a nutshell, language not only determines what you do say, it determines what you can say (and think). That is, the language you use determines what kinds of thoughts you are able to have; it tends to constrain your thinking processes, and thus what you can think about and how you express your thoughts.
If the language doesn’t have room for certain kinds of thoughts, you are much less likely to think them. Natural languages are much more expressive and varied than programming languages, so you want to write requirements and do your designs in natural languages and save the programming languages for implementation later.
Whether you believe the Sapir-Whorf hypothesis or not, it’s nearly always a good idea to develop your functional requirements in a natural language so you don’t get bogged down in the syntactic and semantic details of a programming language before you need to.
This doesn’t mean you can’t think about implementation while you’re doing the functional requirements (you will trust me), but just shunt those thoughts over into a “technical note” sidebar of your specification or a completely separate document. You might also look at Kenneth Iverson’s Turing Award lecture, “Notation as a Tool of Thought,” for a similar discussion.
Outline of a Functional Specification
Every functional specification is different, just as every software development project is different. So take this outline with a grain of salt and just use the parts that apply to your project. Lots of the ideas here are from Spolsky. Every function specification should have the elements discussed in the following sections.
This is your executive summary. A paragraph or at most two of what the program is supposed to do. “This program runs your microwave oven. It interfaces to a keypad and an LCD display that provides user input and output functionality. Its functions are limited to those that a standard microwave would have, with the addition of single buttons for pizza and coffee reheating.
It also will run a time of day clock and a stand-alone countdown timer. It doesn’t control the light. It has a safety interlock that will prevent the microwave from starting if the door is open.”
You should always put in a statement right at the beginning that “This specification isn’t done yet. If you think something is missing or wrong, just sent me an email.”
That helps keep all the marketing guys off your back and lets you file new feature requests in your mail trash bin. Lots of people will put a big, black DRAFT in the header or footer of the document.
That can work as well, but folks tend to ignore it. Some people will use a big DRAFT watermark on their specs so that every page has the word embedded in the text.
That doesn’t stop people from yelling at you either. At some point, your disclaimer should change to something like “This specification is as complete as it will be for this release. If you think something is missing or wrong, just sent an email to the author and we’ll consider it for the next release.”
Somebody needs to be responsible for the functional specification. Not a committee, not the development team, one person. This is usually either the development manager or the project manager, depending on how your company sets up development projects. There are pros and cons to all the different organizational arrangements.
If the development manager (the person to whom the developers report) is in charge of the functional spec, then that person is usually up to speed on all the technical aspects of the project.
That’s good. On the other hand, if your boss writes the functional spec, it might be harder to tell them that there’s something wrong with the specification, or that you don’t agree with the design.
Also, development managers were probably developers at one time and so they may not have the people skills (read: charm and schmoozing skills) necessary to talk to marketing, the customer, documentation, testing, and so on.
If your company uses project managers that are in charge of the specification, design, and schedule but don’t have developers directly reporting to them, then you run the risk of getting someone that isn’t as technically astute as a former developer.
On the other hand, these folks can usually charm the socks off the other teams, so negotiations are a lot smoother. Project managers need to have some technical skills and to be very good at getting all the stakeholders to reach consensus on the contents of the functional specification.
Scenarios of Typical Usage
These are the actual requirements. A great way to get customers to respond to your requirements list is to present several scenarios of typical usage of the program to them as part of the specification. This has a couple of advantages:
First, if you write the scenarios as if they’re user stories, the customer is more likely to read them.
Second, customers are more likely to understand what you’re doing and come up with ideas for things you’ve missed or gotten wrong. This is always a good thing, because the more customer input you get early in the process, the more likely you’ll actually create something they want.
In many agile methodologies, including XP, user stories are often written like scenarios. In XP, the customer is part of the project team, so you get constant feedback on user stories and daily program builds.
In Scrum, the customer isn’t required to be part of the project team, but they are strongly encouraged to keep in close contact with the team. Also in Scrum, shorter sprint lengths allow the customer to see working versions of the product more often.
In the Unified Modeling Language (UML, see www.uml.org), there is an entire notation used to create use cases (another word for scenarios). But as already discussed, nothing beats natural language for describing usage scenarios.
Once you’ve written a couple of scenarios, you’ll have a much better idea of how your program will flow, and what screens, dialog boxes, menus, and so on you’ll need.
This lets you go through each one of those screens and flesh out the details of how they’re laid out, what buttons, text boxes, icons, graphics, and so on they’ll have, and what other screens they connect to. Use pictures!
A picture of a screen or a dialog box is worth way more than a thousand words. It gives the reader something to react to and it gets them thinking about program flow and user interface issues.
When you first write the functional specification, there will be one or two things you don’t know. That’s okay. Just put them in the “Open Issues” section.
Then every time you meet with the customer, point to this section and try to get answers. Some of these questions will move to requirements sections and some will end up in the “Non-requirements” section, after you get those answers. By the end of the project, though, this section should be empty. If it’s not, well, you’ve got issues that will haunt you.
Design and New Feature Ideas
If you’re like most developers, you’ll be trying to design and code the program in your head all the time you’re doing your requirements gathering and analysis.
That’s just what developers do. The two types of notes developers and project managers typically create are technical notes containing design or coding ideas for developers, and marketing notes containing feature ideas for the marketing folks and the customer.
So, to keep from forgetting the design and implementation ideas you have during the requirements phase, write a separate notebook. This notebook is just a separate document that contains a note for later. Ideally, it’s a document that’s shared with the entire team.
Finally, as your project proceeds through development, new requirements and features will surface. This always happens. But if you want to keep to a schedule and deliver a working product, you just can’t implement everything that will come up.
If you want your requirements to be up to date, you need a place to put all the tasks you will do later. That’s what a “Backlog” document is for—all the requirements you’re going to consider for the next release of the product.
This does a couple of good things for you. It tells the customer you haven’t forgotten these features, and that by moving them to the next release you are committed to delivering the current release as close to the published schedule as possible.
And it tells the developers that you’re not out of control and that the project has a good shot at being done with high quality and on time. For more information on backlogs, take a l atook any of the Scrum agile methodology descriptions.
One More Thing
One more thing about the functional specification: don’t obsess. Chances are, you’ll do a good job of picking out requirements and writing them down in the functional spec, but it won’t be as detailed as you like and it won’t be complete. Don’t worry. The only time a functional specification is complete is when you ship the release.
Don’t spend time trying to get every single detail correct; don’t spend time trying to tease every requirement out of your customer. It just won’t happen. Set a time limit, do your best, and let it go. You don’t want to have a bunch of developers sitting around twiddling their thumbs with nothing to do, waiting for the spec, do you?
Requirements Gathering in an Agile Project
First things first. In an agile development project, there is no functional specification. That’s because agile developers recognize from the beginning that the requirements will change and so they should embrace change and defer making decisions about requirements and design as long as possible.
Also, because in an agile project the customer is an integral part of the team, the agile developers also know that they can get immediate feedback on feature implementations and they can get timely updates on requirements from the customer.
This doesn’t necessarily make the process of gathering requirements any easier, but it gives everyone more confidence that the current set of requirements is the right set.
For most agile methodologies the key idea in requirements gathering is the user story. The user story is just that—a description of some feature or scenario that the customer wants to execute in order to get some type of work done.
The classic way to describe the contents of a user story is to say, “As a <role>, I want to do <action>, so that <reason/benefit>.” By expressing a user story this way, you get to the who, what, and why of the requirement.
The Three Cs
A user story has three fundamental components, expressed by Ron Jeffries in 2001: the card, the conversation, and the confirmation.
All user stories are written on cards. A card can be a Post-It note, an index card, or a larger piece of paper. But it’s nearly always a physical thing.
Although the card contains the text of the story “As a <role> I want to <action> so that <benefit/result>,” it’s really an invitation to a collaborative conversation about what the story really means and what the user really wants. Note that the card isn’t very detailed. It usually just contains an outline of the story.
It’s a placeholder for the real requirement that will be subsequently hashed out. Stakeholders can write on the card, put estimates on it, questions, and so forth.
The conversation about a user story takes place between all the important stakeholders in the project, the product owner or user, the development team, the testers, marketing folks, and maybe others.
This is a substantive discussion about what the product owner really wants from the story. The conversation is ideally held in person, and the discussion will include more details about the story, possibly estimates of size, and an estimate of the relative priority of the story.
The conversation may include breaking the original story into two or more smaller stories if the initial estimates indicate the effort to implement the story may be too large.
The last component of a user story is confirmation. The user or product owner provides this information in the form of acceptance criteria for the story. The acceptance criteria are usually written on the back of the card as a short bullet list.
These criteria will end up being acceptance tests that the product owner will use to confirm that the implementation of the story is acceptable to the user.
The best way to create acceptance tests is for the product owner to generate examples of the story in use and then for the development team to automate the examples. That way, the product owner can execute the acceptance tests on the delivered feature and confirm whether the implementation works or not.
INVEST in Stories
Note that the three components of a user story just mentioned don’t tell us all about the story. A lot of the details will come out of the conversation and the confirmation.
They also don’t tell us anything about the quality of the story and what makes a good user story. Bill Wake, in a classic blog post, laid out the characteristics of a good user story using the acronym INVEST.
The idea here is that your user stories should be independent of each other. That way a single user story (which, depending on the agile process you’re using, may take several iterations to fully implement) can be scheduled and implemented separately from any other user story.
Wake gives the example of a multi-layered cake. If you take a slice out of the cake, you can eat (implement) just that one slice, independently of any other.
This may not be completely possible—think of things like the radio software in a mobile phone. In order to fully test part of the user interface embodied in a user story, you may have to have all the radio software working first. We’ll see this same idea later on as loose coupling in object-oriented design.
A good story leaves room for the parties involved to negotiate the details of its implementation; it provides the essence of the requirement. The story is not so specific that it reads like a contract. Rather it provides the developer with a goal and allows the owner and the developer room to create an interesting and workable implementation.
A good user story must be valuable to the customer. It must describe a feature or a service that the customer wants. Because user stories will be scheduled and implemented in a development iteration, they must add value to the product when the iteration is complete.
In addition, if the team decides a user story is too large and must be split into two or more stories, each of them must provide value to the customer. This idea gives the development team guidance on how to split stories—based on value to the customer, not on technology.
User stories must be able to be estimated. Estimation is critical to the product owner so that they can assign a relative priority to the story. This characteristic also allows the team to focus on the size of the story. Stories that are too large can’t really be estimated and so should be candidates for splitting. Estimation is part of the negotiation in the conversation.
It means to look at both the size of the story and the possible effort involved in implementing it. If a team is having trouble estimating the size or the effort of a particular story, it may be that the story is too large and needs to be split. Estimation is also the first step in decomposing the story into implementable tasks.
As implied above, good user stories should be small. Ideally, a user story should be implementable in a single sprint or iteration. That will allow the development team to decompose the story into a number of small tasks—ideally of eight person-hours or less worth of effort.
It also speaks to the Value characteristic because if it’s small, the story will add value for the customer at the end of a single iteration.
One way to manage story sizes is to not care about them if the story a is low priority. As the story moves up in the product backlog, it will become more important, and at some point it will be important enough to do a detailed estimate of its effort. If large enough, (longer than a single iteration), you should consider splitting it into smaller stories.
Good stories must be testable. This harkens back to the plan-driven idea of traceability. Once implemented, one should be able to trace a feature back through the implementation and design and into the original requirements. In agile, this practice is usually implemented using test-driven development (TDD) and acceptance criteria.
The product owner writes the acceptance criteria, and the developers will implement the unit tests that will confirm that a task is correct. The acceptance tests the product owner will use confirm that the user story is correctly implemented. This is typically the definition of Done in an agile environment.
If a user story is written and the product owner is unsure or unclear about how to write the acceptance criteria, this may mean that the story details are unclear and the story conversation should be restarted to clear up any confusion.
The Testable characteristic is also a way for the development team to include non-functional requirements (performance, response time, usability, and so on) as things to be tested.
At this point, the total number of user stories generated by the product owner and agreed upon by the development team is added to a list of the total number of things that need to be done to create the product. This is known as the product backlog.
Where a plan-driven process team will have a long, detailed document—the functional specification—an agile product team will have a stack of cards that end up defining the product. This stack of cards is only preliminary, though.
As the development process moves along, an agile product team will be adding, removing, and dividing user story cards constantly. The product owner will be the primary person doing this job, and it’s the product owner’s job to decide when the product is done and should be released. There may still be cards left in the stack when a product is released. Those cards are destined for the next product effort.
Once the team agrees on a set of user stories and adds them to the product backlog, the developers must then plan for the next iteration or sprint.
This planning includes taking each user story and breaking it down into a set of implementable tasks, each of which is easy to estimate with effort requiring a relatively short amount of developer time. In short, tasks are the work to be done in order to implement user stories.
This work of decomposing stories into tasks is the work of the development team, not the product owner. Generally the entire team will do this exercise as part of planning the next iteration or sprint. The estimates for the tasks are added up until the amount of effort reaches the amount of time available to the team in the next iteration.
In Scrum, tasks can be assigned point values based on their perceived difficulty. The more difficult the task, the more points are assigned. The team will add up the points for the high-priority tasks until the point value reaches the team’s average velocity.
These tasks are then presented to the product owner, who either approves the list of work to be done or suggests changes by changing the priorities of stories in the product backlog. Eventually, everyone agrees on the stories, and tasks for the iteration and work can begin.
In order to perform this decomposition and effort estimation, the team must be able to identify tasks within stories and write them out on cards so they can be put on a task or kanban board. Each task must meet certain goals, characterized by the acronym SMART.
Where user stories can be general and leave room for interpretation and negotiation, the tasks created from them need to be specific so they can be implemented. This will likely uncover hidden requirements in the stories.
These requirements can be incorporated into the tasks or they may require the creation of a new user story that will then get put in the product backlog. In any event, tasks should be as specific as possible, including details about data structures and user interfaces if necessary.
The key idea here is that the team needs to know when all the requirements for the task have been completed. That is, when is the task done? Each team will have a different definition of done, but it should include things like “the feature works as listed on the task,” “all the unit tests pass,” and “the code has been reviewed and integrated.”
The task must be something that a developer can do within the timeframe of the iteration or sprint. The developer must also have the skill set necessary to achieve the task. This doesn’t mean the developer can’t ask for help. This goal can also integrate well with the idea of pair programming, which will spread the required skill set across two developers.
This goal ties in with the Valuable story component discussed earlier. With respect to tasks, this means that the task must do something that makes progress towards the creation of the user story implementation. It should add value to the iteration for the customer.
This goal means that the task, as estimated, can be finished within the iteration or sprint. If the task turns out to be harder than expected, the team is expected to divide it into two or more tasks and estimate them separately.
The other goal implied here is that the total number of points assigned to the tasks included in the iteration is doable within the team’s average velocity.
As the tasks are created from user stories and estimated, they’re added to the sprint/iteration backlog as things to do in the next sprint or iteration.
Depending on the agile methodology being used, once an agreed-upon number of tasks are added to the backlog, the number of tasks in the backlog may or may not be changed while the sprint or iteration is underway.
In Scrum, no more tasks may be added, except by the developers themselves, for the duration of the sprint. Any new work that’s discovered must be added to the product backlog instead.
Most software engineering texts use the phrase requirements elicitation to talk about the process of getting your users to tell you what they want. Hunt and Thomas, in their blog The Pragmatic Programmer, use the much more descriptive phrase requirements digging to emphasize the point that what you’re really doing is digging for all those requirements that your customer doesn’t know they want yet.
Hunt and Thomas also make the terrific distinction between requirements, policies, and implementations as a way to illustrate the requirements digging process. For example, “The system must let the user choose a loan term” is a nice succinct requirement.
It says that there’s something you have to do. It isn’t specific enough for implementation yet, but it tells the developer something concrete that must be built.
“Loan terms must be between 6 months and 30 years” isn’t a requirement, although it kind of looks like one. This statement is an example of a business policy.
When statements like this are presented to developers as requirements, they have a tendency to hard-code the statement in the program. Wrong, wrong, wrong. Policies like this can change, so you need to be very careful about putting business policies in your requirements.
It’s almost always the case that you need to implement a more general version of the business policy than is stated. The real requirement is probably something like “Loan terms are of finite length but the length of the loan will vary by type of loan.” This tells you that you probably need to build a table-driven subsystem to handle this feature.
That way, the loan term for a particular type of loan can be changed by making a single change in a data table, and the code doesn’t need to change at all.
“The user must be able to select a loan term using a drop-down list box” isn’t a requirement either, although, again, it may look like one. This is only a requirement if the customer absolutely must have a drop-down menu to choose their loan term. Otherwise, this is an example of the implementation that the customer would like to see, and it may not be a requirement.
As Hunt and Thomas state in their blog, “It’s important to discover the underlying reason why users do a particular thing, rather than just the way they currently do it.
At the end of the day, your development has to solve their business problem, not just meet their stated requirements. Documenting the reasons behind requirements will give your team invaluable information when making daily implementation decisions.”
Why Requirements Digging Is Hard
There are several reasons why pulling requirements out of your customer is a really hard exercise. We’ll look at a few of these.
Problems of Scope
Lots of times the boundaries of what your program is supposed to do are fuzzy. This can be because of several things. The program may be part of a larger system, and the integration of the parts is ill-defined.
The customer may not have thought through exactly what they want the program to do, so they start throwing out all sorts of ideas, many of which may not even apply to the problem at hand. Finally, the customer may have dropped into implementation-land and be providing unnecessary levels of detail.
It takes lots of patience, discipline, repeatedly saying the word no, and repeatedly asking, “Why does this need to be part of the program?” in order to overcome problems of scope. The scope is directly related to requirements creep, so beware.
Problems of Understanding
Let’s face it: the customer and you as the developer speak different languages. Your customer is the domain expert and they speak the domain language (accounts receivable, accounts payable, reconciliation, general ledger, and so on).
You speak the design and implementation language (class, object, method, use case, recursion, activation record, and the like). This is usually worse than an American in Paris; at least there, both sides can pantomime their needs and figure things out. With problems of domain understanding, the best you can usually do is order drinks together.
There are usually two ways to overcome problems of understanding. The first is to have someone in the middle who has lived in both worlds and who can translate between the two.
Some companies have folks called system engineers or technical marketers who fulfill this role. These folks have done development and have also worked the customer side of things, so they can speak both languages. Good system engineers are worth their weight in user stories.
The second way to promote understanding is to have the customer as part of the development team. This is the approach taken by some agile methodologies, notably XP. When the customer is part of the development team, you get to talk to them every day, ask them questions, and teach them technical stuff.
Both sides benefit. And because the on-site customer sees intermediate product builds as soon as they pop out of your build machine, you get immediate feedback. Win, win, and win.
Problems of Volatility
Things change. This is by far the hardest part of requirements gathering and analysis and the biggest reason why schedules slip. You can’t do anything about it. Get used to it. As Kent Beck says, “Embrace change.” What you can do is manage change. Create a backlog of new features that get added as they arrive.
In the Scrum methodology, new requirements are always added to the product backlog—they’re not added to the current sprint backlog.
This allows the current sprint to proceed normally, and the requirements are all reviewed at the end of the sprint. Another way to manage change is to push the decision onto the user. Give the user a choice: “If we implement this new feature it will add six weeks to the schedule. Do you still want it?”
Alternatively: “If you want to keep to the original schedule, we can only implement and test one of A, B, or C. You pick the one you want most.” This is one of the things that the agile folks mean by courage10; sometimes you have to take a stand and choose what’s best for the project as a whole.
From a developer’s perspective, non-technical problems with requirements are the worst ones you will see. In fact, these are problems developers should never see; their managers should shield them from non-technical problems. Non-technical requirements problems are fundamentally political.
Examples abound. One group of customers in an organization has a different view of the program requirements than another group. Or worse, one manager has a different view than another manager. The program being developed will reduce the influence of one department by automating a function where they used to be the sole source of expertise.
The program will distribute data processing across several departments where it was once centralized in a single department. The list goes on and on. The best advice for non-technical problems is to run away—quickly. Let your vice-president deal with it; that’s why they’re paid the big bucks.
Analyzing the Requirements
Once you’ve written down a set of requirements, you need to make sure that these are the right requirements for the program; you need to analyze them. Analysis has three basic parts.
First, you must categorize the requirements and organize them into related areas. This will help the designers a lot.
Second, you—or better yet, the customer—need to prioritize them. This is critical because you won’t be able to implement all the requirements in the first product release (trust me, you won’t). So, this prioritized list will be what you’ll use to set the contents of each interim release.
Lastly, you need to examine each requirement in relation to all the others to make sure they fit into a coherent whole. Ask yourself a series of questions:
\1.\ Is each requirement consistent with the overall project objective? If your program is supposed to sell your users blogs, it doesn’t also have to compute their golf handicap.
\2.\ Is this requirement really necessary? Have you added something that can be removed without impairing the essential functionality of the program? If your first release is supposed to allow users to buy blogs, you probably don’t need to also allow them to buy sailboats.
\3.\ Is this requirement testable? This is probably the most important question when you’re doing requirements analysis. If you can’t figure out how to test a requirement, then you can’t know that you’ve implemented it correctly or that you’re finished. All requirements must be testable, or else they’re not requirements. In most agile methodologies, the rule is to write the test first, then write the code.
\4.\ Is this requirement doable in the technical environment you’ve got to work in? This question normally applies to those non-functional requirements mentioned previously. Are your requirements feasible given the particular target platform or set of hardware constraints you must work under for this project?
For example, if your target platform is a Macintosh running macOS, a requirement that the DirectX graphics library be used is not doable because DirectX is a Windows-only library.
\5.\ Is this requirement unambiguous? Your requirements need to be as precise as possible (so that they are testable), because as sure as you’re sitting here reading this, someone will misinterpret an ambiguous requirement, and you’ll discover the error the day after you ship. Your requirements should never contain the words or may.