Discipline When You Need It Most
In the locker room for the University of Michigan football team there is the following quote:
Under pressure you will not automatically rise to the occasion and become a hero. You will revert to the level of your training effort and perform at that level.
Or something like that.
At least, that's what I've heard. I've never been in their locker room. But I remember hearing something about that on TV once and it really stuck with me.
If you are a football guy, this is the explanation for all those long hours of drills. The mantra, "You play the way you practice," exists for this reason: So that the level of your training will be sufficient to carry you through the crisis.
If you are a software person, this applies to you also. "Playing the way you practice" means having the discipline to follow your process all the time, even when you don't want to or feel pressure not to. In fact, especially when you don't want to or feel pressure not to.
The entire reason you have a process is to help you make decisions in hard times. If you are a software company that never has deadlines, never has commitments to meet, never has quality standards by which you are measured, then you probably don't need a process. You probably don't have customers, either. But if that doesn't apply to you, then you probably have a process, or need one.
Here are some examples of software development crises that are mitigated through process:
- Sales just told a big potential customer that your software contains a key feature that it doesn't actually have.
- A big partnership opportunity is available to your company if you will drop your current plan of record and instead implement a bunch of custom functionality.
- An executive just installed the beta of the next release of your product, and found out that you cut that executive's pet feature that his cousin really wanted.
- You are so sure the fix you just made couldn't possibly break the build, you are tempted to just check it in and go home instead of waiting for the test suite to run.
- You're nearing shipment of your product, so you are tempted to forego a new unit test for the bug you just fixed and just ship the thing.
- Your colleague or boss pressures you to just put in a "fix" for the product issue they found, even though the product is technically working as designed.
What should you do in each of these scenarios? I'll tell you:
You should follow your process.
Your process should tell you how to handle each of these situations, and it probably does. Scrum, for example, has built-in mechanisms for handling each of these scenarios.
The challenge is that when the pressure is on, you, along with every other normal human being, naturally feel tempted to cave in. The pressure of the situation causes you to panic. You don't make your decisions rationally and objectively. You make them based upon emotion instead, which is not generally the best guide.
When the pressure is on, it is more important than ever to follow your process. Trust it to help you make these and other decisions correctly. When you feel pressured most to abandon your process, that's precisely when it is most important to follow it.
Which is why you should always follow your process. During normal operations, your process may seem like it is oppressive or costly, and it may be. But if you follow your process when you don't have to, you will have the level of training you need to follow it when it will save you.
Guest Post: How to Neglect a Product to Death
By: Dan Reese
My friend and former co-worker Matt Ryan recently commented on the impending death of Novell Forge. A message on the Novell Forge site confirms that Novell will be shutting things down soon. I can corroborate some of Matt's story as I was on the Forge team for about a year during its heyday.
But what really interests me about the situation are the implied instructions on how to neglect a successful product into a slow death that can be blamed on the product itself.
- Avoid rewarding or recognizing any of the people involved. Even better, reward someone else.
- Do not feed its success. Withhold funding, staffing and career growth opportunities.
- Provide poor support. Delay fixing problems.
- Set unrealistic goals and expectations. Blame the product or team for failure.
- Find excuses to kill the project. Focus on the negative in all meetings with executives.
This was a particular worry at Mozy when it was acquired by EMC. They promised that EMC did not want to be the lumbering elephant that accidentally squashed its shiny new purchase. And it was true. But we worried about accidental squashing anyway.
It has been two years, and I have seen projects and executives come and go. Though I do not believe it was intentional, at times it felt like Mozy was being neglected. But the core of Mozy has remained strong and continues to grow.
Based on my experience, here is what to do to keep a successful product moving forward:
- Execute anyway. Deliver a quality product in the face of neglect.
There is precious little a neglected production team can do other than produce. I heard something once I have always remembered: nothing succeeds like success. It is much harder to produce in the face of neglect, but it is also nearly impossible to ignore or argue with.
Mozy is clearly not perfect, but despite occasional neglect it continues to provides a valuable, profitable service. Working with smart people helps. Working for smart people helps. Working with people you like helps. Working on something you care about helps. Working with cool technology helps. But over time, delivering a useful, profitable product is what matters.
(This guest post is originally authored by Dan Reese and reprinted by permission. It is subject to the copyright terms at the original location.)
Fudging Complete
Are you guilty of this?
If you have kids, I bet they do it. You tell one of your kids to mow the lawn or do the dishes. After a while you see them playing XBox or watching TV. You ask, "Did you mow the lawn/finish the dishes?" "Yeah, I'm done," they claim. Then you go to check out the handiwork, only to find that there's a good 100 square feet of lawn that simply got skipped, or that all the dirty pots are still sitting on the countertop.
Don't get so high and mighty: You probably did it when you were a kid, too.
Apparently, as we grow up, we change. Some people don't, I guess. Some people continue to knowingly lie about this stuff even as adults (e.g., presidents of the USA). But most of us change a bit. Perhaps we don't like getting caught. Or we don't like the idea that we failed to do something we said we'd do.
The problem is, even though most of us change, it doesn't mean we are all delivering what we committed to.
If you've been given a task, completing the task means completing the work pertaining to that task. All of it. Not "the bulk" of it, or "the majority" of it, or "everything except a few odds and ends." It means the whole thing.
That's what Fudging Complete is. It's when, either because you are afraid of the consequences of not finishing, or because you don't like to admit failure, or because you don't want to lie, you try to convey a sense of "complete" when you aren't really complete. You redefine "done" to mean "mostly done". Or you change the definition of the task so that it matches the work that has been done, even though now the task doesn't match its intended purpose.
It's when you knowingly check in code that hasn't been fully tested in order to meet your feature complete date. You know the feature isn't actually done, because you haven't completely tested it. Or maybe you've done some preliminary testing on it, but you haven't written your unit tests. You know if you do what you should, you will miss the date, and that looks bad. So instead you can make your date by checking in today. You are knowingly incurring test debt, but at least you made your date. But you aren't really complete.
It's when you are at the end of a sprint and you take a look at the committed items, and notice that all but one of the committed items are done, and report your sprint status as having met your commitments anyway. In the footnotes you make note of the fact that the sprint is complete, except for one or two small tasks that didn't get finished, but other than that the sprint is complete. But you aren't really complete.
If you do this, you are Fudging Complete. You are like the movie rated PG-13 for "partial nudity." What does it mean to be partially nude? You are either nude or you aren't! A movie either has nudity in it or it doesn't.
Of course, "partial nudity" technically makes sense and is in fact quite common. I am partially nude as I type this post: My forearms are fully exposed. But the term doesn't really mean what it implies. Saying a movie contains "partial nudity" is simply one person's way of trying to deceive you about what is really in the movie. It doesn't have nudity! It only has partial nudity!
Fudging Complete has the same effect. Telling your manager that your project is complete, except for one or two loose ends, conveys the wrong sense to your manager. It is technically possible to be partially complete, but saying it doesn't convey the right message. It tries to convince your manager that you are complete, and you aren't.
When we do our work, we need to approach it the way a basketball player approaches shooting a basket. When you're playing basketball, whether you will take the shot depends on a number of things: Whether you actually have the ball, whether you are near enough to your basket that you think you can make the shot, whether anyone on your team is open closer to the basket, whether you are open to take the shot, etc. However, you know as you shoot the shot that there's a chance you will miss. Missing the shot is no big deal; every basketball player misses shots. If you miss, it is not that big of a deal. You quickly regroup and get ready for your next opportunity to shoot.
Working on software projects needs to be like this. It is okay to miss. Everyone misses. Sometimes software engineers are afraid to fail, so I don't call it "failing" — I call it "missing." If you miss, learn from your mistake and do better next time.
What you shouldn't do, however, is be dishonest about missing. Everyone misses. You are not perfect. Good teams and companies and managers know you will miss and will appreciate you being frank with them about what happened, instead of trying to redefine what completion means to make yourself look good. Don't do that. Don't fudge complete.
Discipline as a Prerequisite to Critical Code Contribution
Merriam-Webster's dictionary defines discipline as: "..."
Nah. Just kidding.
Most software teams I've worked on have had some degree of discipline around what code changes go into the codebase. This tends to take the form of:
- Any change should be associated with a defect or feature that is documented somewhere.
- Commit small, isolated, related changes, all associated with a single defect or feature, that can be easily rolled back.
- Test changes before committing them; avoid breaking the build at all costs.
That last one seems most obvious, but it is so critical that it has to be stated anyway. Think of it this way: If you work for me on an already functioning product, I would rather have you do NO WORK AT ALL than to break the build.
Of course, I would not pay you to do no work at all, either. But breaking the build is taking the product in the opposite direction from where we want it to go.
Some time ago I worked on a software team* where a team member had a problem with this kind of discipline. Details don't matter too much, but the net effect was that whenever this person made a commit, the rest of the team would tense up and hold their breath as they integrated the updates. This person had broken the build enough that his/her commits weren't trusted by the rest of the team. And even if the code would compile and run, you still didn't trust the changes because this person would often sprinkle in little changes hither and yon that were not tied to a documented defect or feature, and often wouldn't even call out the changes in the commit message.
(As a slight distraction here, I'll point out how frustrating this is when the team member works in the same office as you, as cited above. When that person works remotely, it becomes MUCH MUCH WORSE.)
I discussed this with my boss one day who wasn't sure what to do. We had a person on the team who was obviously very talented, but who was just not following good practice despite repeated requests to do so. I offered up the suggestion that I'm restating here, which is basically the following:
Disciplined software engineering should be a prerequisite to having the rights to make critical code contributions.
In other words, no matter how talented a programmer is, they shouldn't be allowed to contribute code to critical parts of the product if they do not exhibit sufficient discipline to be there. So what are the "critical parts" and what is the "sufficient discipline"? The critical parts are the parts that, if they are broken, the software doesn't perform it's primary function. For a product like Firefox, the "Help" dialog is almost certainly not critical, but the HTML parsing engine definitely is. The sufficient discipline is what I outlined above, but primarily centered around not breaking the build.
So to summarize, what it means is, someone who continually breaks the build should be disallowed to commit changes directly to the critical parts of the product until they earn that right back by adopting the discipline required. Of course, if you are that free-spirited maverick, there's a pretty strong incentive now to change your approach. You may not be paid to be disciplined, but you are paid to deliver software, and if you can't deliver software because your approach is irresponsible, this should encourage you to get on track.
Some teams choose to approach this by building elaborate systems that enforce proper behavior. I could build a complex checkin proxy that automatically merges my changes with the latest changes in the repository, builds the product, and then runs the tests (there are tests, right?), and then only makes my commit once all of that passes acceptably. That's nice and all, I suppose. But it seems like a lot of infrastructure, overhead, and process to impose upon the whole team, when to me it seems reasonable to expect that professional software engineers have enough discipline to do this on their own.
So if you aren't doing this now, how do you go about changing?
Well, you've gotta have a suite of automated tests in order to do any of this - a "broken" build is defined as one that does not pass the automated tests. If you do not have any automated tests, it is time for you to repent, my son. I do not have time to go into this here, but you must confess your grievous sin and atone by writing an automated test and adding it to your build process. Just one test is enough to get started - but it MUST be available as a part of the normal build process. Then every time a build passes the test suite but is, in fact, broken, update your test suite.
Now that you have automated tests, insist that your developers do the following before they commit:
- Update.
- Build.
- Test.
See? It's easy. If the tests pass, they can commit.
Now it is time for you to be disciplined. Committing code that does not pass the tests is a naughty thing to do. Do not let your developers be naughty. Take immediate action. I think I'd take the following approach:
- First time is a meeting with the boss. Go over the process. Make sure they understand the process and what happened. Personally, it seems like it would be tough to misunderstand, but hey, things happen. Don't be presumptuous or condescending. This meeting alone might be enough to get most people in line.
- If the problem continues, require peer review before checkin.
- If the problem continues, another meeting. This time, express that all of their current assignments have been reassigned to other members of the team, and they are being reassigned work that is not critical to the functionality of the project. Let them know that you are concerned about the impact this will have on their performance, because they are expected to contribute at a higher level than what their new assignments will allow. They have to demonstrate changed behavior and commitment to discipline in order to earn the right to do the important work again.
- If the problem continues after that, it is probably performance-improvement-plan time. Or mutual-agreement-that-they-find-a-different-employer time.
Actions Speak Louder Than Code
It took me a while, but I finally settled into my routine and got to where I'm reading my RSS feeds most days again. I was going through the posts of the past month or so, since the job change, and ran across this article on the "Making Good Software" blog about things that keep someone from being a good software engineer, outside of (and often in spite of) an ability to engineer software.
I'll summarize here. It isn't my intent to plagiarize; if you are remotely interested go read the article. Here are the things:
- Lack of discipline
- Big ego
- Poor communication
- Forgetting the customer
- Lack of proper work prioritization
I have known many of these people during my career. Indeed, I was one of them. I remember coming to Novell from IBM almost ten years ago. I thought I was pretty hot stuff and I made sure my team knew it. In fact, I actually said (this is embarrassing to admit) on more than one occasion, "There are people who know C++ better than I do, but I haven't met any of them." My ego surely made me hard to work with. It definitely was a cause of friction between myself and my management chain, and ended up being a (deserved) source of frustration and difficulty for me, until I recognized the problem and started working to address it.
I'm pretty ashamed of having behaved that way back then. I hope I'm better than that today. I guess recognizing the weakness is a good first step. Fortunately for me, back then I was on a really great team with a lot of very capable, patient, and talented engineers that waited for me to learn from my mistakes and to grant them the mutual respect they deserved. I consider myself pretty fortunate to have been able to learn from them what real software engineering is about.
Over my career I've had to work with people like this from time to time, software engineers that manifest one or more of these traits. Sometimes these guys are pretty talented technically. I've felt sorry for them as I've observed, realizing that these weaknesses are going to hold their career back until they recognize them and work to overcome them. No amount of programming prowess will compensate for it. And what's even worse is, often because these people have the personality issues they have, you don't get anywhere by trying to bring these weaknesses to their attention; they are often unreceptive to this type of feedback. Like I said, you just have to wait until they recognize it themselves.
I can imagine being in a performance review with someone like this, having them explain to me all the technical awesome they did, and me replying, "Your poor soft skills are shouting so loudly that I cannot hear your technical awesomeness." Or, as I said in the title, actions speak louder than code.
I really believe this is true. To write software professionally, of course you must have technical ability; however, this is a necessary but not sufficient condition for greatness. The best software engineers I've had the fortune to work with in my career, past and present, not only had awesome technical ability but did not exhibit weakness in these areas. And I'll tell you what: Those teams are wonderful teams to be a part of. Those teams create strong. uplifting work environments and are able to deliver great products that meet customer demand.
Another way to say this is, in order to be a good software engineer, you must first be a good employee.
In fact, I'll tell you how important I think this is. The ability to mitigate or eliminate these defects from a software engineer's persona is so important to me that, if I had my own company and were making the hiring decisions, I would not hire a candidate that I knew had these problems, no matter how incredible their technical ability.
A person with these weaknesses is really only suited to be set to the side to work on a special side R&D project where interaction with other employees is limited, and they don't have to interact with customers at all. Problem is, those kind of projects are either a) strategically important to the long-term future of the company, or b) of little to no real value, or c) a combination, often high potential value but with a lot of inherent risk that causes the real value to be low. If the project is strategically important or of high value, do you really want to reward the biggest jerk in your company by giving him the highest profile assignment, leaving your best engineers to maintain the legacy project? Wouldn't you want to have someone working on that high profile assignment that knows how to collaborate with others and assemble all the best ideas to solve the problem the best way, even if that solution isn't his/her own? Contrariwise, if the project is of little real value or has so much risk that it offsets the real value, why even do it at all?
Nope. In my company, if I were ever to have one, I wouldn't hire or keep an employee who had these weaknesses and was not committed to addressing them. I've seen the difference, both in morale and productivity, between teams where they don't have these problems and teams that do.
