The Kevin Test for Successful Projects

One of my personality ticks is constantly asking the question, “Why do projects fail?” Too many times to count, I’ve trolled the internet and tried to absorb the lessons learned by countless before me. I’ve had my fair share of flops and I can say with confidence, in hindsight, it’s always easier to spot the cause of a failed project after it fails. However, when you are in the middle of a project, it’s not so easy. It’s like the fog of war. You are so entrenched in the present that by the time you’re convinced the project is going to fail, it usually already has.

Several years ago, while searching for the answer to the ultimate question, I stumbled on a post by Joel Spolsky entitled “The Joel Test: 12 Steps to Better Code.” Written back in August of 2000, this test was basically a simple, objective survey designed to help developers build good software. Over the years, I have found myself going back to Joel’s test to see how my current software development practices stack up to the questions that he outlines. Even after a decade, I’ve never scored a perfect score; but I’ve found that, as long as I maintain a high percentage of affirmative answers, the quality of my software was high.

The Joel Test
  1. Do you use source control?
  2. Can you make a build in one step?
  3. Do you make daily builds?
  4. Do you have a bug database?
  5. Do you fix bugs before writing new code?
  6. Do you have an up-to-date schedule?
  7. Do you have a spec?
  8.  Do programmers have quiet working conditions?
  9. Do you use the best tools money can buy?
  10. Do you have testers?
  11. Do new candidates write code during their interview?
  12. Do you do hallway usability testing?

In October 2011, Rands (aka Michael Lopp) followed up the “Joel Test” with a post entitled “The Rands Test”. In this test, Rands asks 10 subjective survey questions that are designed to help you gain perspective on how well your team functions as a group. Unfortunately, I’ve never scored a perfect score on The Rands Test either; but that’s not the point. The point, on both tests, is to try to maintain as many positive point answers to the survey questions as possible while simultaneously trying to improve the areas where your responses are negative points. Having a strong score on The Rands Test means you have a strong team.

The Rands Test
  1. Do you have a consistent 1:1 where you talk about topics other than status?
  2. Do you have a consistent team meeting?
  3. Are handwritten status reports delivered weekly via email?
  4. Are you comfortable saying NO to your boss?
  5. Can you explain the strategy of your company to a stranger?
  6. Can you tell me with some accuracy the state of the business? (Or could you go to someone / somewhere and figure it out right now?)
  7. Is there a regular meeting where the guy/gal in charge gets up in front of everyone and tells you what he/she is thinking? Are you buying it?
  8. Can you explain your career trajectory? Can your boss?
  9. Do you have well-defined and protected time to be strategic?
  10. Are you actively killing the Grapevine?

These two tests are great if you want to build good software or good teams. However, what if we combine good team behaviors and good software development behaviors together and come up with a test for good project behaviors. While I can’t possibly concede that I’m in any league with Joel Spolsky or Rands, I do venture to take what I have learned from them and try my hand at what I’ll dub “The Kevin Test” for successful projects. All credit however should go to the aforementioned because “The Kevin Test” shamelessly takes the two tests, combines them, and tries to come up with a litmus test for successful software development projects. My contribution merely applies and expands many of their ideas to the concept of projects, which lies between the regions of strong teams and strong software.

The Kevin Test
  1. Do you have a backlog of requirements or specs?
  2. Do you keep a list of bugs, issues, and scope changes?
  3. Do you fix problems before starting new work?
  4. Do you have a schedule for when work is supposed to be done?
  5. Do you feel that the team members can speak freely and frankly?
  6. Do you have testers (other than the developers)?
  7. Do you think you can communicate what you are building to a stranger?
  8. Do you know what you are supposed to do next?
  9. Do you have too many concurrent tasks?
  10. Do you conduct retrospectives?
Do you have a backlog of requirements or specs?

It’s surprisingly easy to attend a meeting, and immediately go back to your desk and start creating the solution. On extremely simple projects you might get lucky by skipping the backlog, but on more complex projects you are taking a huge risk. Creating a backlog of the requirements is the first step in showing that more thought was put into the project than merely attending a meeting. The backlog should contain known specs and their status, whether they get implemented or not. Once you have a backlog, questions about scope, completed work, and remaining effort suddenly become trivial to answer (“It’s in the backlog”). Additionally, the backlog can serve as a tool for estimating the effort for similar projects.

Do you keep a list of bugs, issues, and scope changes?

As people test or use your solution, they will come up with bugs, issues, or changes to scope. Some of these you will fix and others you will ignore. Keeping a log of these details and the decisions made about them, provides a historical record, and eliminates scope misunderstandings. This project artifact, along with the requirements backlog, is a roadmap that tells how the project got to where it is.

Do you fix problems before starting new work?

Ignoring problems with the assumption that you will get to them later is technical debt. Also, unfinished or faulty work can spread to other parts of your project. These unresolved issues tarnish the perceived quality of the overall product. In addition, fixing problems before starting new work reduces context switching and focuses attention on current tasks.

Do you have a schedule for when work is supposed to be done?

All too often, I hear people give estimates like “We should be done in about three weeks.” Unfortunately, what typically happens is most of the work ends up getting done, in a rush, just before it’s due. If a setback is encountered, the due date gets blown with little to no time left to react. However, if you set multiple short term goals, you will have a continuous stream of due dates. Consequently, instead of having most of the work performed at the end, work will be performed throughout the project schedule. There is no better incentive to start working now than having a deadline that is tomorrow instead of in three weeks. If a setback occurs early, you’ll likely have more time to make adjustments or at least notify stakeholders about the problem. Daily SCRUM meetings and weekly sprints are perfect examples of setting short term goals.

Do you feel that the team members can speak freely and frankly?

The balance of authority between stakeholders and team members cannot be lopsided. The weaker represented group, stakeholders or team members, needs to have an equal voice. If there is not a healthy debate in meetings, then problems or issues with the project are concealed. The “Yes Man Phenomenon” ensues and emotional investment in the project is replaced by apathy.

Do you have testers (other than the developers)?

Asking a software developer to test their own code is like asking an Algebra student to grade their own paper. If a mistake is made, how will it get noticed? Developers don’t think and behave like testers. The order in which inputs are collected and entered may not match the order in which users experience these workflows. Suppose a customer doesn’t know the shipping zip code, should you prevent them from placing the order? Developers and sales staff might have different opinions. The point is what’s valuable and important to users is different than what’s valuable and important to developers.

Do you think you can communicate what you are building to a stranger?

If you can’t describe the value and basic function to a stranger then it will be difficult to estimate build complexity or timeframe. If you don’t understand the underlying problem you are trying to solve, then you won’t know if you were successful at solving it. You don’t have to be a domain expert, but you do need to know enough about the problem to participate in an intelligent conversation.

Do you know what you are supposed to do next?

One of the benefits of SCRUM, when done correctly, is that anyone can know, or determine, the status of the project at any time. Built into the SCRUM process is a backlog that clearly defines what has been completed, what is being worked on, and what is remaining. If any member is unsure about what they should be doing or where the project stands, then all they have to do is consult the project backlog. Meetings to discuss “where we are on the project” are either unnecessary, or are the result of poor project planning. Consult the backlog to see what’s next if there’s slack time between completed work and the next meeting.

Do you have too many concurrent tasks?

One of the biggest project velocity killers is context switching caused by multitasking. Unlike computers, human brains are not well equipped for multithreading conscious tasks. Every time you context switch, there is a significant warm-up period of low productivity. When working a problem, it takes a while to remember where you were, and where you were trying to go. If you are doing too much context switching, your whole day can be a string of task warm-ups with little work getting accomplished.

Do you conduct retrospectives?

In the words of George Santayana, “Those who cannot remember the past are condemned to repeat it.” The point of this meeting is to communicate what’s working and what’s not. The discussions should be encouraging but frank, and liberal with positive reinforcement. Generally, this meeting attendance should be kept small; no one likes their flaws to be in the public spotlight. It’s important to note that “retrospectives” is plural. That is, multiple retrospectives should occur throughout a project not just at the end; if you’re only conducting a retrospective at the end of a project, you’re probably conducting an autopsy. Mid-project retrospectives can be an opportunity to make corrective changes before a project fails. Of all the things you can learn on a project, the retrospective can be the most valuable. Unfortunately, retrospectives are often skipped or done poorly.

It’s not critical to have a perfect score on “The Joel Test”, “The Rands Test”, and “The Kevin Test”. Rather, focus on maintaining good form and positive answers to most of the questions while simultaneously working to improve those elements that are lacking. Don’t pick too many problem areas to address at the same time; otherwise, you might fall into the context switching trap. Focus on the problems that are most likely to cause a project to fail, such as not having a backlog. Over time, good project management form will become habitual and projects not using it will feel unnatural. As projects succeed, there will be a natural tendency to repeat the behaviors that contributed to the project’s success.