Ok, so you’re working at a non-Agile shop, possibly even a shop with bad project management. How do you get them to being Agile if you’re not the CEO or VP of Engineering? By doing something I call:
CrockPot Agile is my term for converting your shop to Agile development by making a series of small improvements in the development process as it exists. As you make these small improvements, you will build street cred with your fellow developers and with management. From these small steps you can eventually build up to Agile, and it will seem like a natural progression. You’ll look like a total genius during the process, which is good for your career. You also won’t run into nearly as much resistance, because you’re not promoting giant changes, just small, logical, useful steps.
Step #1: Nightly Build & Deploy to Test
Setup a nightly build that deploys to a test server. People have leaky brains. When a developer writes some code and checks it in all knowledge of that code immediately starts leaking out of his brain. Similarly, if a tester finds a problem the details of that problem starts leaking out of their brain once they send the bug report.
So if you’re not building and deploying nightly, then you have a large amount of brain leakage between the developer checking in some code, the tester finding a problem, the developer fixing the bug, the fix getting deployed, and the fix getting verified. Plus there’s a lot of friction between QA and Development about whether or not a fix is available to QA and so on. Lots and lots of unproductive waste.
To illustrate, say a developer checks in some code on Monday. Let’s say there’s a deployment on Friday afternoon at 4:30pm. So on Monday #2, the tester notices a problem, and posts a bug report. It’s now been a week. Maybe the developer knows, maybe he doesn’t where the problem is; its been a week. Let’s say he figures out the problem and fixes it though on Monday #2, a quick turnaround. The fix will now languish until Friday. Monday #3 is now the earliest the tester will be able to verify the fix, assuming they remember how to reproduce the problem. That means it takes 2 weeks to close a bug! That’s with a weekly build, your shop probably isn’t doing even that. If you’re building “when the developers feel like it”, it may be even longer.
If you deploy nightly, if a developer checks in some code on Monday, it deploys Tuesday. If a tester finds the problem immediately, the developer can almost always pinpoint the problem quickly, because the problem is in the code they just changed. The developer immediately fixes the problem, checks in the fix, and on Wednesday the fix will be verified and closed. You’ve taken the cycle time for a bug fix from 2 weeks to 2 days!
If you setup a nightly build, QA no longer has to bug development about when the fix is ready for testing, and they don’t have to go through a giant backlog after every deployment to test. Instead, every morning QA can simply look through the fixed bugs and verify and close them. Then when a fix doesn’t work, or its only a partial fix, the development team gets immediate feedback and no leaky brains! QA and development also get in the habit of working closely together.
So Step 1: Setup a nightly build. Then give a speech about leaky brains and how a nightly build means everyone has less to remember. I recommend Hudson.
Step #2: Continuous Integration
Once you’ve got a nightly build going for a while, (hopefully with Hudson), it’s a quick jump to Continuous Integration. At this point, the other developers may even ask for this; that every check-in triggers a build and emails the development team if it fails. You don’t want to do a deployment for each check-in, because you’ll drive your QA department batty if you’re constantly taking the server up and down. But you do want engineers to get immediate feedback if they’ve broken the build, especially if you’re building for multiple platforms. If you’re working on Mac and you break Windows, you might not know for a while otherwise. Again, it’s a leaky brain problem.
Step #3: Unit Tests
Ok, so you’re now on a roll. Just changing the build process has built you a lot of street cred with QA and your fellow developers. You can now add running tests as part of the build. The way to sell this to your fellow developers is simply to point out that the nice thing about tests is that it’s often easier to check the code they just wrote via a unit test then it is to set up all the other pieces of the system. Writing test code makes checking your work easier, and then having that integrated into the build is just gravy.
Don’t try to sell them on Test Driven Development. Instead, just convince them it will make their lives easier. In most modern IDEs, its trivial to run a test case. In every development environment I’ve worked with though, the code I’m working on at the moment is part of a larger whole, and setting up the whole is a Pain-In-The-Ass. It’s easier to write test code for a unit then it is to fire up 20 other units. Nearly all developers will buy into this once you show them how the IDE can run test cases for them. Plus, running the code you just wrote through a test fills your success well.
This is a good time for you to think about building some test infrastructure; like the ability to load YAML files for data, setup/teardown a database, etc. Integration testing is always a pain, but if you have some good foundations, you can eventually get your co-workers to write integration tests as well. For one thing, they now run every night and every check-in, so they’ll see the immediate value. You have to get them to write some tests first though, but once they do, they’ll realize it’s another leaky brain issue, and they’ll start to prefer code with tests.
Step #4: Respect QA
See my post here: QA
Step #5: Check In [CI] Emails
Start sending to everyone in your group and your boss an email titled “[CI] Your name YYYY-MM-DD” with bullet points for the following:
- What you did that day
- What you plan on doing tomorrow
- What you’re stuck on.
(Yes, it’s a 1-person scrum in email form. I don’t call them that, because I invented them in 1991 at my first job, which predated scrum, so there.)
It will take about a week for about half of the rest of the development team to follow because actually, your co-workers are interested in what you’re working on so they’ll start doing it too. By the end of the week your boss will mandate it across your development team, he’ll like it so much. Ask your boss to do his own [CI] email so you know what’s going on with the big picture. Thank him/her when they comply.
Step #6: Release Trains
Once you’ve got your code being built nightly and tested to some degree, you can introduce the concept of release trains. The idea here is that there will be regularly scheduled releases (whatever release means) as if there are trains pulling out of a station hourly. So it’s not critical to fix every bug and then to “freeze”. If what’s on the web site now has more bugs than what is in test, why not push out the new version? Why make users use the old crappy version?
Of course, to do this, you must have confidence with what’s in testing, which is why Steps 1-4 need to be done first. And some amount of “freezing” is important. But modern version control systems support branching well enough this shouldn’t be an issue. So you branch for the release, and then development can continue on the head branch. The branch is the release train that is pulling out of the station, the head is understood to be the next train. If you’ve got continuous integration running, you should be able to set up a second test server for the branch that is built by the same build system.
The benefits here are more intangible. It gets people focused on the big picture of the development process of release, release, release. It keeps both QA and development continuously productive and gets them in the habit of continuous testing and development. It lowers the stress of both QA and development when they know that a bug fix can catch “the next” train. This is also the beginning of transitioning to Agile, but don’t tell anyone that!
Step #7: Project Planning Step Zero – A To Do list
Ok, so now development is humming along, but are they doing the right things? The next few steps relate to introducing good project planning to the organization. So now you can push for building a list of everything that has to be done. This involves doing a certain amount of design up front. I call this “Step Zero” for project planning; making a to-do list of what you have to do.
You may think you’re doing this already. Are you sure? It’s very hard to know everything you have to do without doing at least some engineering design work. Joel On Software has some good stuff on this.
This step will probably meet the most resistance in shops with no project management, where engineers just want to “start coding”.
Step #8: Prioritize the To-Do List
Ok, so you’ve got a to-do list. There’s probably too much stuff on it! Start organizing the list so that the stuff that’s most important (technically hard , important feature, or risky) gets on the earliest release trains. This will seem stupidly obvious to everyone once you point it out. Suggest people work on the to-do list in order.
Step #9: To-Do Poker & Story Points
Now you have to actually sell. Convince people that relative estimates are more accurate than regular estimates (Joel On Software is wrong in this case). Introduce the concept of to-do poker for estimating relative complexity. Add complexity estimates to your to-do list.
Step #10: Start doing Iterations & Burn Down
It’s probably at least a year later now, and everyone thinks step 1-9 were your own brilliant ideas; you’ve been promoted to manager of the group. You’re doing release trains already, so breaking those up into Sprints, and tracking your Story Point Velocity and doing a Burn Down chart is going to seem natural. So are Scrum meetings.
Whether you tell anyone your development team is now “Agile” is up to you.
Here are some books to get you started: