How much work should you do on a new product before involving customers? If you subscribe to the theory of the minimum viable product, the answer is: only enough to get meaningful feedback from early adopters. Sometimes the best way to do this is to put up a public beta and drive a limited amount of traffic to it. But other times, the right way to learn is actually to show a product prototype to customers one-on-one. This is especially useful in situations, like most B2B businesses, where the total number of customers is likely to be small.

This case study illustrates one company’s attempt to do customer development by testing their vision with customers before writing a single line of code. In the process, they learned a lot by asking initial prospects to sign a non-binding letter of intent to buy the software. As you’ll see, this quickly separated the serious early adopters from everyone else. Mainstream customers don’t have enough motivation to buy an early product, and so building in response to their feedback is futile.

Along the way, this case study raises interesting ethical issues. The lean startup methodology is based on enlisting customers as allies, which requires honesty and integrity. If you deceive customers by showing them screenshots of a product that is “in-development” but for which you have written no code, are you lying to them? And, if so, will that deception come back to haunt you later? Read on and judge for yourself.

The following was written an actual lean startup practitioner. It was originally posted anonymously to the Lean Startup Circle mailing list, and then further developed on the Lean Startup Wiki’s Case Studies section. If you’re interested in writing a future case study, or commenting/contributing to one, please join the mailing list or head on over to the wiki. What follows is a brief introduction by me, the case study itself, and then some Q&A led by LSC creator Rich Collins. Disclaimer: claims and opinions expressed by the authors of case studies are theirs alone; I can’t take credit or responsibility. – Eric Ries

In April of 2009 my partner and I had an idea for a web app, a B2C platform that we are selling as SaaS [software-as-a-service]. We decided from the get-go that, while we clearly saw the benefits and necessity of our concept, we would remain fiercely skeptical of our own ideas and implement the customer development process to vet the idea, market, customers etc, before writing a single line of code.

My partner was especially adamant about this as he had spent the last 6 months in a cave writing a monster, feature-rich web app for the financial sector that a potential client had promised to buy, but backed out at the last second.  They then tried to shop the app around, and found no takers.  Thousands of lines of code, all for naught -- as is usually the case without a customer development process. (See Throwing away working code  for more on this unfortunate phenomenon. -Eric)

We made a few pencil drawings of what the app would look like which we then gave to a graphic designer.  With that, the graphic designer created a Photoshop image. We had him create what we called our "screenshots" (which suggests that an app actually existed at the time) and had him wrap them in one of these freely available PS Browser Templates. Now armed, with 4 "screenshots" and a story, we approached our target market, some of which was through warm introductions, and some, very literally, was through simple cold-calling.

Once we secured a meeting, we told our potential customers that we were actively developing our web app (implying that code was being written) and wanted to get potential user input into the development process early on.  Looking at paper print-outs of our "screenshots", no one could tell that this was simply a printout of a PSD, and not a live app sitting on a server somewhere. We walked them through what we thought would be the major application of our product.  Most people were quite receptive and encouraging.  What proved to be very interesting was that we quickly observed a bimodal distribution with regards to understanding the problem and our proposed solution:

• people either became very excited and started telling us what we should do, what features it needed and how to run with this, or
  
• they didn't think there was a real problem here, much less a needed solution.

We ruminated on this for a while. The vehemence of those that didn't get it surprised us.  Perhaps we had a super-duper-hyper-ultra-cool idea  --- but not enough customers existed to make it worth the effort. We visited each potential customer a minimum of twice, if not three times.  Each time we would come back with a few more "screenshots" and tell them that development was progressing nicely and ask them for more input. We also solicited information as to how they were currently solving the problem and how much they paid for their solution.

On the third visit, we pressed those who saw merit in the idea to sign a legally non-binding Letter of Intent.  Namely, that they agree to use it free of charge if we deliver it to them and it is capable of X, Y and Z.  And not only do they agree to use it, but that they intend to purchase if by Y date at X price if it meets their needs.

By the way, this LOI was not written in legalese.  Three quarters of it was simple everyday English.  In fact, we customer dev-ed the LOI itself.  The first time, we asked a client to sign it before we had even written it.  When they agreed to sign it, we quickly whipped it up while sitting in a coffee shop and emailed it off to them.  This would help us separate the wheat from the chaff when it came to determining interest and commercial viability.  Once we had two LOIs signed and in-hand, we actually began to write code.

We also implicitly used the LOIs for price structure and price discovery - which we are still working on.  We backed into prices from all sorts of angles, estimating the time-cost of equivalent functionality, competitive offerings, other tools we were potentially displacing -- but in the end, we lobbed a few numbers at them and waited to see if they flinched.

Customer A got X price, Customer B got X + Y price, and so on.  So far, our customers have never mentioned price as an objection, which suggests to me that at this point we are very much underpriced. The LOI was also useful as we leveraged it by approaching the competitor of one of those who signed by simply letting them know that their competitor will be using our app.  They returned our cold intro email within 8 mins.

We have two customers that have balked at signing LOIs, but want to use our product.  This has been somewhat of a quandary for us.  When we decided to go the LOI route, we thought that we would not bend and that we would only service those customers who would sign the LOI.  In the end, we decided that these two customers were large enough to help us with exposure, provide good usage data and worth the risk of them wasting our time.  Time will tell if this theory proves correct.

Right now, the app itself is pretty ugly, a bit buggy and slow -- and doesn't even do a lot.  It is borderline embarrassing.  Don't get me wrong, it does the few necessary things.  BUT it definitely does NOT have the super-duper-hyper-ultra-cool Web 2.0 spit and polish about it. Interestingly enough, our ratio of positive comments to negative comments from actual users is about 10 to 1.  One of our first customers had a disastrous launch with it, yet, has signed on to try it again (granted, they did get it for free and we did offer it for free for this next time). But they didn't hesitate to try it again.  I thought we would have to plead, beg and beseech.  But for them, it was a no-brainer.  So, we have to be doing something right.

Our feature set is very limited and being developed almost strictly from user input.  While I personally have all sorts of super-duper-hyper-ultra-cool Web 2.0 ideas --- we are holding ourselves back, and forcing ourselves to wait for multiple, explicit and overlapping user requests.  We have seen our competitors whose feature sets are very rich, to say the least, but we think in some cases, are as over-engineered as they are feature-rich.

Only time and the market will tell if they are innovative and we are slow, lazy pigs or they have gotten ahead of themselves/the market and our minimalist solution will be better received.

Rich Collins, founder of the Lean Startup Circle, responded to the poster with some Q&A.
LSC: What is your response to some of the people on Hacker News that questioned the ethics of taking this approach?

Some of the commenters have some good points.  It definitely explores ethical boundaries.  However, I don't think we indulged in any zero-sum game type deception.  By that, I mean our intentional fuzziness about the state of development did not cause harm in any manner to our prospective clients.  In fact, just us showing up at their offices and talking about our screenshots benefited our prospective clients tremendously as:

1. Those clients who had never even entertained the functionality we were proposing gained significant knowledge.
2. With that knowledge, they could (and did) Google our competition and start exploring the space and current offerings. 

We did, in fact, tell one of our prospects in the beginning that our screenshots were simply mock-ups.  However, that makes the prospect feel as if you are wasting their time and they then are unlikely to provide input.

"Oh, this is just a Photoshop file?  Well, come back to us when you are further along." which defeats the whole purpose of getting face time for Customer Development!

When you tell them, the app is in development (and it was, even before coding, we were spending a lot of time on what we wanted and didn't want, how it would look, use cases ‚ etc) the prospects are interested in providing input and shaping the product.  They need to feel and see some momentum.

LSC: Your use of a non-binding letter of intent was another interesting tactic.  Did the customers that signed it end up paying for your product?

Yes and no.  We had a dispute with one signee and couldn't convert them.  However, we successfully converted others.  I should also mention that there was one client who refused to sign an LOI, but we are in the process of converting them.

The LOI was designed to give us hard, non-bullshit-able feedback instantly.  Too often people will affirm your idea so that you (or they) can save face, which BTW is a form of well-intentioned and socially acceptable deception.  This is why, IMHO, friends, wives, and significant others are probably not good people to talk to about your idea.  At the end of the day, no one knows if the idea is any good.  The market will tell you.

LSC: Would you respond to a few selected Hacker News comments?
"If I were one of your prospects, I would never sign a letter of intent based on drawings only. I'd make you come back later with something, anything I could play with ... Come back when you have something real to show. Until then you're no different from any other poser."

I myself probably would never sign an LOI on screenshots only.  However, our customers did a lot of stuff that I would never do.  Lesson learned:  I am not my customer.  We think differently.  We solve our problems differently.  We have different needs and wants.  Repeat after me:  You are not your customer.

LSC: And one more: "Except the LOIs in this case are utterly meaningless. I've been on the customer side of LOIs that were signed on request, knowing that it obligated us to nothing."

Wrong.  We got instantaneous feedback on the validity of the idea and started our sales process concurrently.  While legally non-binding, customers who have signed an LOI are a lot less likely to disappear or make themselves hard to get a hold of.  LOIs, while clearly not as good as signed sales contract, do have meaning and are valuable.  I encourage B2B startups to keep them in their customer development arsenal.

Special thanks to Rich Collins, the Lean Startup Circle practitioners, and to everyone who has contributed to the Case Studies on the wiki. And thanks to these entrepreneurs for sharing their story. Have a case study you’d like to share? Head on over to the Lean Startup Wiki.
Read More »

Inc Magazine has a great new piece up about the increasing use of the Minimum Viable Product by businesses (and not just startups). Here's an excerpt; some of my comments are below:

One of the most gut-wrenching moments for a company is the rollout of a new product. A significant swing and miss can break a company's momentum -- and maybe its bank account. Unfortunately, after months or even years of development, many companies discover that customers aren't willing to buy their new wares. That's why some entrepreneurs are trying another approach to product launches: marketing a product online before spending much on research and development or inventory.

Consider the method used by TPGTEX Label Solutions, a Houston-based software company that specializes in bar codes and labels for manufacturers and chemical companies. Like many companies, TPGTEX rolls out new products several times a year. But instead of spending the time and money to develop products on spec, TPGTEX creates mocked-up webpages that list the features of a potential new product -- such as a system for making radio-frequency identification, or RFID, labels -- along with its price. Then, the company spends no more than a few hundred dollars marketing the product through search engines and to the contacts in its sales database and LinkedIn. It isn't until a customer actually clicks or calls to place an order that TPGTEX's developers will build the software. "We do not develop a product until we get a paying customer," says Orit Pennington, who co-founded the six-employee company with her husband in 2002. Development time is typically no more than two to three weeks, and it generally takes just a few orders to cover development costs.

TPGTEX's approach is an example of a trend in business that has been dubbed minimum viable product or microtesting. The idea is to develop something with the minimum amount of features or information needed to gauge the marketability of a product online. That might mean mocking up a website with potential features and seeing how many visitors click on the item. It might also involve buying pay-per-click ads to see how easy it is to gain potential customers. Or it might mean selling a few products on a site like eBay to see how well they perform before ordering in bulk from a wholesaler.

What sets this approach apart from practices like using focus groups is that companies base product development decisions not just on what customers say they want but on how they vote with their wallets.

Read the rest...

This article is part of a trend that has taken me a bit by surprise: the adoption of lean startup techniques outside the traditional domain of high-tech startups. The theory predicts this, of course, because the definition of a startup as “a human institution creating a new product or service under conditions of extreme uncertainty” says nothing about sector, size of company, or industry. Still, it’s always a relief to see practice and theory converge.

Of course, as more people attempt to use the Minimum Viable Product as a tactic, there are a lot of misconceptions possible. The biggest is the confusion over why this tactic is useful. The Inc story, and many others, does a good job emphasizing its lean-ness. By allowing customers to “pull” value from the company in small batches, you reduce the risk of building a product that nobody wants. Like all lean transformations, this is powerful – it increases the value of every dollar invested in new product creation.

But MVP is most powerful when it is used as part of an overall strategy of learning and discovery. And this is the most confusing, because MVP does not pay off under this strategy if we are attempting to build a minimal product. For that, release early, release often will suffice. But if our aspiration is to change the world, we need something more.

The key ideas are customer development, the pivot, MVP, and root cause analysis. Each is described in separate essays on this blog, but let me say a few words about how they work together – especially for companies with big ambitions. Big visions take a long time to develop, and require an exceptionally high degree of product/market fit. That’s just a fancy way of saying: customers have to really, really like your product. Being specific, it means that their behavior powers one of the three fundamental drivers of growth with a large coefficient. But if big products and big visions take a long time to develop, it’s exceptionally risky to build it based on vision alone. That’s because for a big product to take off, it needs to be right in many key respects. Miss just one, and you can find yourself just a few degrees off – and moving with too much momentum to change course. Think Friendster, the “achieving a failure” startup I’ve written about, Apple’s Newton, Webvan, etc. In each of these, the failure of the initial idea led to the failure of the company (or division).

Building an MVP can help mitigate that risk. But it’s not enough. What if customers hate the MVP? Does that mean your product vision is fundamentally flawed, or just that your initial product sucks? There is no way to know for sure. That’s why entrepreneurship in a lean startup is really a series of MVP’s, each designed to answer a specific question (hypothesis). Being systematic about these hypotheses is what customer development is all about. By testing each failed hypothesis leads to a new pivot, where we change just one element of the business plan (customer segment, feature set, positioning) – but don’t abandon everything we’ve learned. In order to work, these pivots have to be heading in a coherent direction, which is why vision is still such a critical part of entrepreneurship, even in a data-based decision making environment. (See “It’s a startup, not a spreadsheet” for more.)

And yet, even that is not enough. The more visionary the entrepreneur, the more difficult it is to really pivot, really seek out what’s in customers’ heads, and really create a minimum viable product. And so startups – great and terrible alike - are prone to give these ideas lip service, but fail to really take maximum advantage. That’s why a process of rigorous root cause analysis is so critical. After every major milestone, the company has to ask: what did we learn? Why didn’t we learn more? And, most importantly, make incremental investments to do better next time. This is the ultimate startup discipline, the hardest to master, and the one that pays biggest dividends. If you can embrace continuous improvement from day one, you can actually speed up as you scale. It’s an awesome thing to watch.

Read More »

One of the most important lean startup techniques is called the minimum viable product. Its power is matched only by the amount of confusion that it causes, because it's actually quite hard to do. It certainly took me many years to make sense of it.

I was delighted to be asked to give a brief talk about the MVP at the inaugural meetup of the lean startup circle here in San Francisco. Below you'll find the video of my remarks as well as the full slides embedded below. But I wanted to say a few words first.

First, a definition: the minimum viable product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.

Some caveats right off the bat. MVP, despite the name, is not about creating minimal products. If your goal is simply to scratch a clear itch or build something for a quick flip, you really don't need the MVP. In fact, MVP is quite annoying, because it imposes extra overhead. We have to manage to learn something from our first product iteration. In a lot of cases, this requires a lot of energy invested in talking to customers or metrics and analytics.

Second, the definition's use of the words maximum and minimum means it is decidedly not formulaic. It requires judgment to figure out, for any given context, what MVP makes sense. As I talked about in a previous interview, IMVU's original MVP took us six months to bring to market. That was a pretty big improvement over a previous company, where we spent almost five years before launching. Yet in another situations we spent two weeks building a particular feature that absolutely nobody wanted. In retrospect, two weeks was way too long. We could have found out that nobody wanted the product a lot sooner. At a minimum, a simple AdWords smoke test would have revealed how utterly bad the concept was.

Without further ado, the video:


Slides are below:

Minimum Viable Product

View more presentations from Eric Ries.

Read More »

I recently say down with Venture Hacks for an interview. Part one is up on their site today, in text, audio and slide format. Here are some topics and excerpts of what we covered, edited lightly for how I wish I'd said it at the time. To hear full audio and a complete transcript, click through to Venture Hacks.

Is "release early, release often" enough?

The issue there is, if you just follow the release early, release often mantra, you find yourself running around in circles, because you ship code, you get some feedback from people, you do a focus group.

Customers say,”Give me feature X,” “Give me feature Y,” and sometimes you do what they want, maybe sometimes you’re going to do what you want, and then they get mad at you. Pretty soon you’re chasing your own tail a little bit because you’re not operating against a clear, long-term vision of what you’re trying to accomplish.

The idea of minimum viable product is useful because you can basically say: our vision is to build a product that solves this core problem for customers and we think that for the people who are early adopters for this kind of solution, they will be the most forgiving. And they will fill in their minds the features that aren’t quite there if we give them the core, tent-pole features that point the direction of where we’re trying to go.

So, the minimum viable product is that product which has just those features (and no more) that allows you to ship a product that resonates with early adopters; some of whom will pay you money or give you feedback.

Solving the chicken-and-egg platform problem

Developers don’t want to develop unless there are customers who are there to buy their products, and customers don’t want to come on the platform unless developers are there selling them something useful.

What we did is we took early adopter developers and we told them a story about how IMVU was going to take over the world and be this really powerful product for mainstream customers ... and we gave them an economic incentive that said, the earlier you get on board with the platform, the bigger your take is going to be for derivative products that get created down the road.

We shipped a product that had almost no customers — certainly no mainstream customers — but, because we had told that story effectively and we really understood those early adopter developers, we got a ton of them on the platform developing. They felt like they were in the middle of a gold rush, despite the fact that there was really no evidence to support that belief yet.

Starting with just a landing page

What we should have done, and what we did for a lot of features thereafter, is started with a landing page that promised people that product. Then we should have taken out the AdWords we were planning to take out, drive traffic to that landing page, and offer people to buy the experience that we are talking about.

What we would have found out if we were doing that experiment is 0% of people would have clicked through, which means it doesn’t matter what is on the second page.

The first page is so bad, not because it is badly designed, but because the features are wrong that you don’t need to go through the effort of building out the product. So we wished we had done that, and we did make that mistake really

Overcoming the fear of the false negative

As long as you’re not afraid of the false negative, that is, if you don’t get discouraged because you’ve built your first paper prototype of it and shown it to people and nobody wanted it. That can’t mean that like you give up because,”Oh, forget it, we’ll never make it.” You’ve got to say, "OK, well then let’s iterate some more.”

If you keep iterating at it, you keep making it a little bit more sophisticated, at a certain point after you’ve been through 10 iterations, that you still got no uptake whatsoever, and the feedback you’re getting from customers is still a yawn, you might say to yourself,”You know what? We’re not moving in the right direction. In fact, we’re past the point of minimum viable product. This just isn’t a viable product.”

Read the rest of the interview at Venture Hacks.

Read More »