Cross post from LinkedIn
When we last left our story, we were taking a ten thousand foot overview of some cognitive biases that can reach out and wreck a project. Both programmers and clients are victims of these states of mind, and it’s important to remember that we are swimming in them all the time. The particular cognitive bias that we were looking at is called “Illusion of Transparency”. For our discussion, this is when the client thinks that the programmer knows more about the client’s project than the programmer actually does, AND the programmer in turn thinks that they know more about the client’s project than they actually do. So we’re getting it from both ends. This problem is exacerbated by the lack of definition in a project, and reliance on the mistaken belief that that knowledge is there that isn’t. The obvious solution to this problem is to define a project to the Nth degree, so that there is no room for misunderstanding. Unfortunately, that may turn out to be a bad idea also. Those sorts of projects are called “Waterfall projects”.
Waterfall development is fine for those sorts of projects that have processes that are ACTUALLY well understood by all, and unlikely to change… at all. Unfortunately, the majority of projects are not like that and by the time that you reach the end, you may have confusion or just plain fail. Check out the paper by the Standish group here. OK, just to go down the rabbit hole here for a bit, why is the failure rate of Waterfall development so much higher than Agile development? My theory, unsupported by any papers at this point, is that when groups spend the time to completely define a software project before writing any code, they run afoul of the requirements (or reality) changing before the end of the project. Some software projects that are defined and managed by the waterfall method can go on for years. After that period of time, the need for the solution may go away, or the requirements for it may be so different that the project just doesn't fit reality any more.
Welcome to the other end of the bell curve. We’ve discussed too hot and touched on too cold. How about Just Right? Right now, Just Right is embodied by the Agile Methodology. Specifically, for Phoenix Web Group, scrum. When we're talking to clients, we emphasize the “ping pong-ing” of the project. This is the rapid (one or two weeks) turn around of each iteration. At the end of each of these iterations we assess where we are, and determine whether we are on the right track. Since we do this with the client, we can be pretty sure that we're doing it right. If we find out that we've strayed or reality has changed, it's only for a week or so, and we can get back on track easily. The trick here is to make sure to strike a balance between too much information, and not enough. If you can do that, you're golden.