Case Study: Rapid iteration with hardware
(I am often asked to explain how to apply Lean Startup approaches to domains beyond software. In order to answer, I have taken to drawing a two-axis diagram.
On one axis we have the degree of market uncertainty for a given industry. For "cure for cancer" type businesses, there is no question about who the customer is and what the customer wants, and therefore there is no market uncertainty. On the other extreme, modern web-based applications face almost no technical risk, and are governed by high market uncertainty.
On the other axis we have the underlying cycle time of the industry in question. Slow-moving cycles, like drug discovery or new automobile models, govern the slow part of the axis. On the extreme opposite end are rapid iteration businesses like software or fashion.
The key to understanding Lean Startup is to recognize two things:
The following case study looks at one such industry, consumer electronics, where the pace of iteration has taken a marked turn towards high speed. It is written by Ronald Mannak, who is currently the CEO of a startup named Yobble. What follows are solely his opinions. -Eric)
In a bar in Amsterdam in 2005, my two cofounders and I came to the sad conclusion that startup we tried to built for two years was doomed. In 2003 we started developing a martial arts motion sensing toy, a full three years before the Nintendo Wii changed the world of motion sensing. The toy (we called it Ninja Master) consisted of two hardware units, attached to both wrists. When a child would perform a perfect karate move (or better yet: a combo of several karate moves in a row), Bruce Lee-like karate sounds would emerge from a small speaker in the device. We loved the product. Test users loved the product. It was way ahead of its time. We thought we were visionaries and believed the future was motion control. Yet, we failed to sell the toy. We talked to every toy company imaginable, but none wanted to license our toy. " Kids nowadays don't want to move, they play Playstation" was the most often heard reply, even though our user tests suggested otherwise. To make matters worse, we lived in a country (Holland) without a proper functioning startup, VC and angel ecosystem. The company was doomed. My co-founders decided startup life wasn't for them.
However, one new idea emerged at that meeting. What if we could make an air drum? Drum sticks with sensors in them. Now that was an idea. Music is much easier to sell (to toy companies) than the abstract martial arts Ninja Master toy. Besides, we could easily expand the line with with an air guitar and a device to link the air instruments to a PC. How cool. I loved the idea so much that I decided to pursue the idea.
I envisioned the product would be popular with 8 to 12 year old boys. I thought the price couldn't be higher than $40. I already knew how the product would be used. Boy, was I wrong.
Waterfall
I previously worked on a couple of IT projects that used the 'waterfall model' where specifications were written down by one team, thrown over an imaginary wall and implemented by another team. Every single waterfall project I encountered turned out to be a disaster in every way. Specifications turned out to be open to multiple interpretations, usability was the last priority (if a priority at all). It Just Did Not Work. As a beta tester of the first Borland Delphi, I learned the wonders of rapid prototyping and fast iterations. I wondered if we could do the same for hardware development. It turned out we indeed could.
The first hire
The first hire was critical. I wanted somebody who was creative first and technical secondly. I found the perfect person at the department of Industrial Design Engineering of the Delft University of Technology. Joris. Joris was creative and eager to learn. Better yet, he plays drums. Even better than that: he likes to tinker with electronics. Hiring him was a no brainer, and he didn't disappoint.
The internship only lasted six months. That's not much time, considering the scope of the project. I convinced the university that Joris should not be writing specifications and other nonsense first, but start right away building prototypes. And he did.
Joris suggested that before he started working on electronics, we should invite children, give them wooden drum sticks and let them pretend they were playing air drums. It turned out to be an excellent idea. Children are perfect test subjects. To our surprise, every single child did something we didn't anticipate. Without any exception, they all whacked the wooden drum sticks *sideways* and made 'crashing sounds'. I certainly didn't think of sideways movements when I created the first ideas, but apparently it was a good idea to implement.
The prototypes
The next day we started building the first prototype to see if the sensors actually behaved like they were supposed to, and to see if we could measure the sideway movements. The prototype was crude. Joris taped sensors on his arms with duct tape and started drumming in the air with wooden drum sticks (that did not contain any electronics). We connected the sensors to a seven year old pc with an Arduino-like interface that ran a simple drum program we developed. The results were amazing. It actually worked. (A video of the first prototype can be found here.)
We now knew what kids liked and we knew the product was technically feasible. Yet, I still felt we didn't know all we needed to know and wanted to test more. And I'm glad we did.
For the next prototypes we placed the sensors in PVC pipes to optimize sensor angles and added features to the pc software.
We made another discovery we did not anticipate. We found out that parents (who came along with their children for user testing) often liked the prototype as much as their kids. We decided to interview the parents and quickly found out that the parents who like our product were video games players. Of course we liked our product, but we never would have guessed other grown ups would like our product too. Knowing this, we invited test users from 12 to 30. They also loved the prototypes. Our target audience just exploded in size. We decided to make a few changes that would make the product less 'toy' and more 'gadget'.
Over a period of six months, we made eight generations of prototypes, each version adding more features and making the product more reliable. By testing each generation, we learned that a lot of hypotheses were correct, but a large number of hypotheses were incorrect. By testing early and often, we were able to adjust the product. I believe that we demonstrated that it is indeed possible to iterate fast and often with hardware development.
Product Launch
After some financing-related delays, the products went on sale in Europe and Asia in the summer of 2008. The retail selling price was $40, exactly what we targeted. In less than six months, we sold over 90,000 units. All shops sold out our products two months before christmas, all without spending one penny on marketing. The products were voted 'best music gadget' on television program The Gadget Show, became the best selling music toys on Amazon.co.uk and the best selling products on Firebox.com. Best of all, users love the products. On Firebox.com, the average user rating (740 users) is 4.5 out of 5 stars (link). We couldn't have been happier.
Post mortem
We demonstrated it is possible to iterate often and fast. I believe a lot of the product's success can be attributed to the iterative development process. We didn't find every issue though. We didn't test the price and we didn't see the Nintendo Wii or Guitar Hero coming. We chose to enter the market though the low margin toy market, where (in hindsight) we should have positioned the products as video games with higher video games margins.
Another thing we missed: after we launched we received many requests to add double bass drum, as often used in metal. The drums include two drum pedals and a double bass drum could have been added by a simple and minor change to the embedded software. However, updating the embedded software in sold devices isn't possible with the microcontroller we used. We could have included the feature in a 1.1 version of the product, but the toy manufacturer we licensed the toy to, wasn't interested in a new version, as the original version continues to sell well to this day.
Tools to develop hardware get better and cheaper. Open source projects like Arduino and SuperCollider make iterative hardware development cheaper and faster than ever. We learned that connecting the prototypes to PCs and user the PC to run the program is a very good way to test hardware (developing on a PC is still much faster than embedded developing on a standalone hardware device).
In the summer of this year, I moved to San Francisco and founded a new startup that makes music related games and hardware controllers that connect to the iPhone. There are a lot of new opportunities. New cheap flashable micro controllers make firmware updates possible for low cost hardware. With hardware connected to the internet (in our case through the iPhone) it should be possible to use continuous deployment: small and very frequent updates of the firmware instead of less frequent large updates. Bugs in firmware could be fixed within minutes or hours instead of weeks or months.
(Continuous deployment of hardware is an exciting new capability. In addition to continuous deployment of firmware via the Internet, it is also possible to do continuous deployment by taking advantage of a small-batch production process. When the complete cycle time of assembly is low and the design can be specified mostly through software, it's conceivable that each batch rolling off the line could have a different design. -Eric)
As a final thought, I am convinced iterative design depends mostly on the mindset of the team and the company culture and less on the tools. I was lucky to have a great team of A-players that were willing to take responsibility and risk. If the company culture is such that mistakes are punished, I am pretty sure iterative development won't work.
On one axis we have the degree of market uncertainty for a given industry. For "cure for cancer" type businesses, there is no question about who the customer is and what the customer wants, and therefore there is no market uncertainty. On the other extreme, modern web-based applications face almost no technical risk, and are governed by high market uncertainty.
On the other axis we have the underlying cycle time of the industry in question. Slow-moving cycles, like drug discovery or new automobile models, govern the slow part of the axis. On the extreme opposite end are rapid iteration businesses like software or fashion.
The key to understanding Lean Startup is to recognize two things:
- Lean Startup techniques confer maximum benefit in the upper-right quadrant, namely high market uncertainty coupled with fast cycle time.
- Every industry on Earth is currently undergoing a disruption that is causing it to move along both axes: more uncertainty and faster cycle times.
The following case study looks at one such industry, consumer electronics, where the pace of iteration has taken a marked turn towards high speed. It is written by Ronald Mannak, who is currently the CEO of a startup named Yobble. What follows are solely his opinions. -Eric)
In a bar in Amsterdam in 2005, my two cofounders and I came to the sad conclusion that startup we tried to built for two years was doomed. In 2003 we started developing a martial arts motion sensing toy, a full three years before the Nintendo Wii changed the world of motion sensing. The toy (we called it Ninja Master) consisted of two hardware units, attached to both wrists. When a child would perform a perfect karate move (or better yet: a combo of several karate moves in a row), Bruce Lee-like karate sounds would emerge from a small speaker in the device. We loved the product. Test users loved the product. It was way ahead of its time. We thought we were visionaries and believed the future was motion control. Yet, we failed to sell the toy. We talked to every toy company imaginable, but none wanted to license our toy. " Kids nowadays don't want to move, they play Playstation" was the most often heard reply, even though our user tests suggested otherwise. To make matters worse, we lived in a country (Holland) without a proper functioning startup, VC and angel ecosystem. The company was doomed. My co-founders decided startup life wasn't for them.
However, one new idea emerged at that meeting. What if we could make an air drum? Drum sticks with sensors in them. Now that was an idea. Music is much easier to sell (to toy companies) than the abstract martial arts Ninja Master toy. Besides, we could easily expand the line with with an air guitar and a device to link the air instruments to a PC. How cool. I loved the idea so much that I decided to pursue the idea.
I envisioned the product would be popular with 8 to 12 year old boys. I thought the price couldn't be higher than $40. I already knew how the product would be used. Boy, was I wrong.
Waterfall
I previously worked on a couple of IT projects that used the 'waterfall model' where specifications were written down by one team, thrown over an imaginary wall and implemented by another team. Every single waterfall project I encountered turned out to be a disaster in every way. Specifications turned out to be open to multiple interpretations, usability was the last priority (if a priority at all). It Just Did Not Work. As a beta tester of the first Borland Delphi, I learned the wonders of rapid prototyping and fast iterations. I wondered if we could do the same for hardware development. It turned out we indeed could.
The first hire
The first hire was critical. I wanted somebody who was creative first and technical secondly. I found the perfect person at the department of Industrial Design Engineering of the Delft University of Technology. Joris. Joris was creative and eager to learn. Better yet, he plays drums. Even better than that: he likes to tinker with electronics. Hiring him was a no brainer, and he didn't disappoint.
The internship only lasted six months. That's not much time, considering the scope of the project. I convinced the university that Joris should not be writing specifications and other nonsense first, but start right away building prototypes. And he did.
Joris suggested that before he started working on electronics, we should invite children, give them wooden drum sticks and let them pretend they were playing air drums. It turned out to be an excellent idea. Children are perfect test subjects. To our surprise, every single child did something we didn't anticipate. Without any exception, they all whacked the wooden drum sticks *sideways* and made 'crashing sounds'. I certainly didn't think of sideways movements when I created the first ideas, but apparently it was a good idea to implement.
The prototypes
The next day we started building the first prototype to see if the sensors actually behaved like they were supposed to, and to see if we could measure the sideway movements. The prototype was crude. Joris taped sensors on his arms with duct tape and started drumming in the air with wooden drum sticks (that did not contain any electronics). We connected the sensors to a seven year old pc with an Arduino-like interface that ran a simple drum program we developed. The results were amazing. It actually worked. (A video of the first prototype can be found here.)
We now knew what kids liked and we knew the product was technically feasible. Yet, I still felt we didn't know all we needed to know and wanted to test more. And I'm glad we did.
For the next prototypes we placed the sensors in PVC pipes to optimize sensor angles and added features to the pc software.
We made another discovery we did not anticipate. We found out that parents (who came along with their children for user testing) often liked the prototype as much as their kids. We decided to interview the parents and quickly found out that the parents who like our product were video games players. Of course we liked our product, but we never would have guessed other grown ups would like our product too. Knowing this, we invited test users from 12 to 30. They also loved the prototypes. Our target audience just exploded in size. We decided to make a few changes that would make the product less 'toy' and more 'gadget'.
Over a period of six months, we made eight generations of prototypes, each version adding more features and making the product more reliable. By testing each generation, we learned that a lot of hypotheses were correct, but a large number of hypotheses were incorrect. By testing early and often, we were able to adjust the product. I believe that we demonstrated that it is indeed possible to iterate fast and often with hardware development.
Product Launch
After some financing-related delays, the products went on sale in Europe and Asia in the summer of 2008. The retail selling price was $40, exactly what we targeted. In less than six months, we sold over 90,000 units. All shops sold out our products two months before christmas, all without spending one penny on marketing. The products were voted 'best music gadget' on television program The Gadget Show, became the best selling music toys on Amazon.co.uk and the best selling products on Firebox.com. Best of all, users love the products. On Firebox.com, the average user rating (740 users) is 4.5 out of 5 stars (link). We couldn't have been happier.
Post mortem
We demonstrated it is possible to iterate often and fast. I believe a lot of the product's success can be attributed to the iterative development process. We didn't find every issue though. We didn't test the price and we didn't see the Nintendo Wii or Guitar Hero coming. We chose to enter the market though the low margin toy market, where (in hindsight) we should have positioned the products as video games with higher video games margins.
Another thing we missed: after we launched we received many requests to add double bass drum, as often used in metal. The drums include two drum pedals and a double bass drum could have been added by a simple and minor change to the embedded software. However, updating the embedded software in sold devices isn't possible with the microcontroller we used. We could have included the feature in a 1.1 version of the product, but the toy manufacturer we licensed the toy to, wasn't interested in a new version, as the original version continues to sell well to this day.
Tools to develop hardware get better and cheaper. Open source projects like Arduino and SuperCollider make iterative hardware development cheaper and faster than ever. We learned that connecting the prototypes to PCs and user the PC to run the program is a very good way to test hardware (developing on a PC is still much faster than embedded developing on a standalone hardware device).
In the summer of this year, I moved to San Francisco and founded a new startup that makes music related games and hardware controllers that connect to the iPhone. There are a lot of new opportunities. New cheap flashable micro controllers make firmware updates possible for low cost hardware. With hardware connected to the internet (in our case through the iPhone) it should be possible to use continuous deployment: small and very frequent updates of the firmware instead of less frequent large updates. Bugs in firmware could be fixed within minutes or hours instead of weeks or months.
(Continuous deployment of hardware is an exciting new capability. In addition to continuous deployment of firmware via the Internet, it is also possible to do continuous deployment by taking advantage of a small-batch production process. When the complete cycle time of assembly is low and the design can be specified mostly through software, it's conceivable that each batch rolling off the line could have a different design. -Eric)
As a final thought, I am convinced iterative design depends mostly on the mindset of the team and the company culture and less on the tools. I was lucky to have a great team of A-players that were willing to take responsibility and risk. If the company culture is such that mistakes are punished, I am pretty sure iterative development won't work.
0 comments:
welcome to my blog. please write some comment about this article ^_^