What to Collect: An Overview
So you’re convinced that collecting data about program outcomes is important. What then, should you collect? There are three main guidelines to keep in mind when developing a data collection and reporting plan:
- Include graduates in your plan
- Understand the importance of baseline data
- Collect basic data points that will help the industry demonstrate its value
Graduate Firms: Demonstrating Persistency
One of the things you hear about most when incubators issue press releases
is how many companies have graduated from the program. This data point
is easy to collect and certainly is important – but what are those
companies contributing to the local economy right now?
Far more effective than tallying graduates is providing quantifiable evidence of company success postgraduation.
“What the incubation industry should demonstrate is that the success rate for incubated firms is a polar opposite compared to the standard quoted failure rate for new business start-ups,” says Skip Farrar, manager of business development at Southern California Edison, a public utility in California.
People quote statistics about business failure rates (often incorrectly) all the time. The U.S. Small Business Administration has found that after four years, only 44 percent of firms remain in business.* Contrast that with what NBIA historically has found about incubator graduates: among incubation programs whose average age was nearly ten years, 87 percent of graduate firms were still in business.**
Clearly, there is a huge difference between the survival rate of incubator graduates and the survival rate of small businesses in general. “If we could demonstrate that twice as many firms are still in business four years after graduating from an incubation program, that’s effectively doubling the success rate and bringing credibility to the incubation industry,” Farrar says. “But you can’t figure that out until you know how long a graduate remains in business – its persistency in the market.”
It is critical to note that these separate data sets – NBIA’s data on incubator graduates and the SBA’s data on small businesses – are not directly comparable. Looking at them side by side, however, is striking and suggests that incubation plays a significant role in helping companies succeed. “Those that have gone through the entire incubation process show a high survival rate,” says Dinah Adkins, NBIA president & CEO. “Incubated companies show great persistence. But making any direct comparisons perpetuates the error.” For a detailed discussion about why these data sets are not comparable, and for tips on discussing the success rates of small businesses in general and graduates of incubation programs, see Comparing Stats on Firm Survival.
Some incubator managers don’t track graduate data because they believe they can’t take credit for accomplishments (or mistakes) of firms after they’re out on their own. How is an incubator manager to know if a series of decisions that led to a company’s ultimate success or failure is due to its former association with the incubator? How can an incubator say it had a hand in creating jobs that come into existence after a company leaves the program?
Skip Farrar, business development manager at Southern California Edison, a public utility in California, says it’s important for incubators to continue to track and report this data. “It’s a but/for argument,” he says. “But for the incubator, this business might never have existed, and the persistency wouldn’t be there." In Farrar’s opinion, the role the incubator plays in getting a business off the ground is extremely important. “Therefore, the incubator gets to claim results over time,” he says.
But what about those who say the business might have started on its own? Or that its success might have nothing to do with services provided by the incubator long ago? “Think about it this way,” Farrar says. “But for the contributions of Steve Wozniak, Apple Computer never would have existed. So does he get to keep claiming responsibility for Apple Computer’s success over time? Absolutely.” Farrar notes that this line of thinking is prevalent in the academic realm as well, with universities taking credit for the wage-earning capabilities of graduates.
* U.S. Small Business Administration (SBA), “Frequently Asked Questions,” www.sba.gov/advo. Accessed on March 9, 2007.
** Molnar, Lawrence A., Donald R. Grimes, Jack Edelstein, Rocco De Pietro, Hugh Sherman, Dinah Adkins and Lou Tornatzky, Business Incubation Works. Athens, Ohio: NBIA Publications, 1997.