At the end of projects, it is common to conduct a retrospective meeting to learn what could be done better next time. While this is certainly good practice, it does not help those who are beginning a new innovation project for the first time. What lessons learned will hindsight reveal with painful accuracy only at the end of the project?

In the spirit of trying to avoid learning things the hard way, I queried the members of the LinkedIn group Innovation Management: How do innovation projects go wrong? Four members shared their insights from many years as innovation consultants. This informal survey yielded some familiar responses that I have organized below.

Internal alignment

A lack of internal alignment is often talked about as groups look back over failed projects. While this term is often used, it’s not a single issue. A lack of alignment, or disagreements, latent or expressed, can occur at a number of different places within an organization that will harm project outcomes.

Cross-functional teams might never gel or overcome organizational silos. A skunk works team might be so separate from the internal politics of an organization that they fail to get the management support that they need when they attempt a full scale pilot within the larger organization. Still worst, an outside consultant might encourage wild ideas to foment disruptive change projects only to discover that the ideas are far from the firm’s corporate strategy. These examples illustrate just a few of the ways that a lack of alignment can harm a project.

Crossed Incentives

skunk worksSometimes poor alignment is not simply caused by poor communication. Individual and organizational incentives play a role in subtly undermining internal alignment on a project. When I was working on a new product development project within a large software vendor, I learned that the team was very concerned about the appeal of the product for both prospective customers and a group within the company that would do the promotions and trainings of sales teams on new products. They needed the latter in order to get out of the building with a new product.

Unfortunately, even with strong customer interest for the new product (and executive support of the underlining technology), they failed to get the intermediary group aligned. This group saw the product as a threat because it involved tying together two separate divisions within the organization—two divisions that were not accustomed to collaborating and were accountable to separate VPs. It was unclear how individuals would be compensated for a product that was not wholly from within their division. Thus, the project failed to find traction and support to exit the R&D phase.

Need for Market Information

Another strong tendency on the part of organizations that are developing new product or technologies is to want to keep them secret for as long as possible. Organizations are motivated by the understandable desire to prevent their competition from commercializing their new ideas before you can. While there are some differing opinions on the frequency with which trade secrets are actually successfully exploited by outsiders, the practice of maintaining confidentiality is very common within enterprise R&D departments.

The challenge to innovation execution arises from the need to understand the customer segment, test prototypes, and clear any other market or regulatory barriers before spending a lot of time and money on an innovation project. I imagine the new product development process like building a bridge where work must done from both sides at the same time. One side is the product or technology side and the other is the market and regulatory context.

It’s no wonder that many products fail. New product development and innovation projects in general are hard work that for medium- to large-sized companies involve the active collaboration of many people and the strong support of the executives or upper management.