The product manager's lament

Life is not easy when you're working in an old-fashioned waterfall development process, no matter what role you play. But I have a special sympathy for the "product manager" in a startup that is bringing a new product to a new market, and doing their work in large batches. I met one recently that is working on a really innovative product, and the stories I heard from their development team made me want to cringe. The product manager was clearly struggling to get results from the rest of the team. These are smart people trying hard to all row in the same direction. So why are they having so much difficulty?

Let's start with what the product manager does. He's supposed to be the person who specifies what the product will do. He writes detailed specs which lay out exactly what features the team should build in its next iteration. These specs are handed to a designer, who builds layouts and mockups of all the salient points. Then the designs are handed to a team of programmers with various specialties. Each specialist takes up his part of the spec (UI, middleware, backend) and cranks out code. Last, the QA team builds a test plan based on the spec, and then tests the features to make sure they conform to the plan.

This system naturally lends itself to a pipeline approach, which the product manager organizes. While the programmers are off building the next major feature, he is busy writing specs so that, when they finish, there won't be any idle time before they can start the next iteration. If the programmers go idle, it's bad news for the product manager, because he's supposed to keep them busy building product. Otherwise, the company is wasting serious money.

When I met this team, some acrimony had built up. The last few features came out pretty different from what was origianlly spec'd, and took far too long, to boot. The programmers keep asking for more say in the designs and direction that they work on. So the team has been spending more and more time in the spec and design phases, trying to get team buy-in on what they are going to build. For some reason, though, despite the increased buy-in, the final product often doesn't look anything like the original spec. The VP Engineering spends all of his time trying to make sure the programmers understand and implement the spec. Each iteration takes longer than the previous one. Frustration is mounting.

It doesn't take long to discover that the product manager is being forced to write every spec five times. First, he writes it nice and clear. Next he works with the designer to build out the design spec, and with QA to build out the test plan. When the programmers get it, they often start negotiating with him about what's going to be built. They exchange countless emails, and he's being constantly interrupted and being asked to clarify exactly what the spec means. The fourth spec exists only in these emails, which are changing the design in an ad-hoc fashion. By the time QA gets the feature, their test plan is badly out of date. So the product manager winds up actually having to use the software, by hand, updating the spec and helping create a new test plan. Naturally, the deviations from the spec are so severe, that he has to rewrite the spec he's currently working on (for the next major feature) to take them into account.

Ironically, this system was designed to keep each functional group 100% utilized, so nobody goes idle, including the product manager. But as the iterations get longer, he's spending more and more of his time answering questions. The interruptions are so bad, he has to write his new specs at 3AM, just to keep the pipeline full.

What's wrong with this picture? Everyone is working at 110%, with full dedication. But the team is falling further and further behind.

Here are the changes I'm working with this team to implement

• Work in cross-functional teams. Each team has a representative of each function. To start, we'll try a product manager, designer, programmer or two, and QA. The team owns a complete feature end-to-end.
  
• Focus on speed of iteration rather than utilization of every function. Let people go idle, if they can't help the current iteration succeed. I'm contiuously amazed how many people have untapped creativity and resourcefulness. They don't want to be idle. By letting them focus on the success of their team exclusively, you empower them to do whatever it takes to make the team successful. Will that mean someone in design will jump in to help QA get the release certified? We'll see.
• Switch the spec process from push to pull. Start with a one-page spec, no more. Then, let the team ask questions of the product manager whenever they need clarification. In exchange, the team agrees to show each piece of working code to the product manager for his approval. They'll find points of disagreement much faster, and resolve them in realtime. Plus, the team will get better and better at interpreting the concise specs that only have to be written once. (Eventually, they may abolish specs altogether)

There's much more this team can do to eliminate waste in the way that they work and thereby iterate faster. Eventually, I hope to get them on a full agile diet, with TDD, scrums, sprints, pair programming, and more. But first I think we need to save the product manager from that special form of torture only a waterfall product development team can create. Once the different parts of the team can trust each other again, we'll have the basis we need to start a true continuous improvement feedback loop.