Below the Cut Line
At a company where I worked, an impactful prototype developed by a researcher sat on the vine, constantly ending “below the cut line”. Frustrated, he took matters into his own hands and implemented it within the product under a feature branch, even coding in a framework he had little familiarity with, ultimately succeeding at making it work in record time (note: before our era of AI-assisted coding!). Unfortunately that still proved insufficient as the feature never made it into the official product.
Many established companies employ PhDs that conduct research, perhaps file patents and even publish. Much of that work however does not ultimately end up in a working product. Startups can find themselves iterating on a clever idea yet unable to achieve a good product-market fit. VCs may back teams with world-class technology but no clear path to a product or adoption. Public-sector groups can fund impressive technology through startups, yet hand it off to a contractor for “integration” where the technology stagnates or dies.
The core challenge across tech-focused organizations of all sizes is the same: How to nurture innovations and bring them to life in the real world.
Many a Slip Between Prototype and Product
I have seen one or more of the following failure modes play out at almost every company I have worked at. I’ve been on both sides - a researcher inventing new things, seeking to productize them, and a user of someone else’s invention, also wanting to productize it. It’s not been easy on either side. And I am not alone in this. Please see a few articles I’ve linked below for more. Now for the kinds of tech transfer failures:
Type 1: A Joint Project
In companies with dedicated research teams working on longer term research, product teams encountering challenging problems might try to collaborate with the research teams. When incentives are not fully aligned, both sides may not be equally excited. Even if that were to happen and some good technology is developed, it still takes significant time, energy, creativity, and several iterations to make it work in the real world. Many scientists have a natural inclination to look for the next cool project. Meanwhile, the product team, faced with daily pressures of development and support, is unable to prioritize the productization of the technology. Often, they also do not feel a strong sense of ownership or desire to actually see the innovation through to customers, because the idea originated elsewhere. This is not a fault but human nature to be excited about our own ideas and not be seen as an implementer of somebody else’s ideas.
Type 2: Diffusion
One might argue that tying research too close to product development constrains innovation to what the customer wants, rather than being truly disruptive. One might feel that researchers should be left to their own devices and think freely about revolutionary ideas. Left to themselves however, researchers tend to spend time working on problems of interest to them and not necessarily those that serve the company the best. At one of my previous companies, highly skilled and brilliant researchers were left to define their own agenda and pursue it. At yearly meet ups, I was always amazed and thrilled at the possibilities their research entailed. When I would ask who they’re collaborating with to transition their work into a product, I would often get a blank stare or an attitude that that was someone else’s job. On the flip side, sometimes the researcher is eager to productize something but the product team does not want to adopt it (e.g. the case I opened this article with).
Type 3: Research Team Rolls Up Its Sleeves
There is a particular breed of scientist-engineers who are not only able to do ground breaking research, but are also willing and able to do production grade software development. Teams with such scientist-engineers owning and shipping a product-line often run into political battles with established product teams. In a glaring instance, a product team went out of their way to stop the running of a key processing pipeline from a research team. I’ve seen instances where projects are seized from research teams, sending the research team back to its silo. The latter might still be okay if the team receiving custody of the project continues evolving it. Instead, the project often withers away because the team doesn’t fully grasp the longer term vision, gets bogged down with its own daily grind and pressures, and does not feel a sense of ownership for reasons similar to the cases outlined above.
Having recognized the above failure modes, several years ago, I wrestled with the question: Can we design an organizational structure and incentives so that innovations emerge organically, and more of them get translated into positive impact on a company’s bottom line?
Beating this Unique “Valley of Death”
A few key lessons
- Innovation needs time and space to develop. An engineer or scientist cannot churn out deeply innovative work in their spare time on a consistent basis without burning out.
- Productization requires deep focus, commitment, and sufficient time. This is often the major bane for innovation and requires the proverbial 99% perspiration. It can be long, boring, and tedious.
- Technology developed by an innovation team and transferred to a different team often suffers from a downright failure to productize, or a lack of focus and continued evolution.
Part of the solution is thoughtful hiring into both research and development teams, a topic for a future article. I also assume that the important work of establishing the market need for a given innovation has been done - this is another major topic outside the scope of this article.
Once a clear signal of a prototype’s impactfulness is determined, it is time to create an MVP. The innovator is provided the resources for productization and continued refinement of their prototype, and importantly, held accountable for it. The resources provided are in the form of a skunkworks full-stack team including frontend and backend engineers, DevOps, product management, design, and delivery staff. Ideally, this could be an existing team that is able to take the work on as part of its roadmap. Otherwise, it will be a separate “tiger team” that is assembled.
A technical lead for the MVP creation is required. In some cases it could be the innovator herself. In others, the innovator partners with a tech lead. In either case, there needs to be a single person responsible for the delivery of the MVP. The technical lead should be excited for the product line, have the technical chops to drive productization and to debug issues at some depth. They also need some business acumen to understand the product’s value (ability to do demos is a plus) and help shape a clear product vision. This can be substituted by periodic consults with sales/delivery/mission leads.
The team is enabled to deliver their MVP to the customer, receive feedback, and iteratively improve their MVP. It is crucial for the original innovator to be deeply involved in this phase. There will be edge cases and challenges that will undoubtedly emerge. There will be new insights and opportunities that will unfold. The early phases could be rocky but things will settle down as the product gets adopted and customers develop familiarity. The team (including the tech lead/innovator) is evaluated on customer satisfaction and growth. Key metrics include how well the team is able to ship, balancing quality, speed, innovativeness, and responsiveness to customer concerns and requests. They are free to follow or adapt the company’s established development processes where it makes sense (especially need for meetings and their cadence).
Over time, as ownership of the innovation and its continued evolution emerges naturally in the team, the original innovator is free to peel away and tackle the next big problem. The timeline for this happening is variable.
Seeding the Next Big Idea
Innovation needs a mindset, and emerges from some curiosity and enabling work habits. Curiosity not just for technologies but also for the company’s business. Innovation can span the range from those that can have near-term impact on customers to those that can completely disrupt current approaches to solving problems.
The innovator needs to understand customer problems, so that s/he can make connections between relevant technology and product. This means talking to sales staff, delivery engineers, product managers, and other colleague groups who spend significant time with customers, and ideally customers when feasible.
Keeping up with emerging technology today is hard, with insane levels of AI hype. While it takes more work today to pick out signals from daily chatter than even three years ago, there are many more tools available for prototyping. Developing unique insights and connecting the dots requires periodic reflection and thinking from first-principles.
Innovation starts by the building of a prototype. For their prototype, the creator might require some AI skills, definitely engineering, and even some front-end development. No need to be a master of all these areas, but at least a few of them, and filling gaps with collaborators. Importantly, the company needs to develop a conducive work culture that supports these activities.
Perils in the path from working prototype to a successful product form a unique “valley of death” in tech. It can be solved with the right team structure, appropriate skills, and aligned incentives. When prototypes are given a chance to survive this “valley of death” they will live and die by what should actually count: success at customer missions.
Related Articles
- Dina Bass and Jack Clark: How Microsoft Plans to Beat Google and Facebook to the Next Tech Breakthrough (2016)
- Viswanath, Kaushik: Why Corporate America Gave Up on R&D (2020)
- Arora et al: The changing structure of American innovation: Some cautionary remarks for economic growth (2020)
- Scott D. Anthony: Three Invisible Hurdles to Innovation (2025)