Program Evaluation and Metrics for SBA Regional Innovation Clusters

Presented by: Berna Demiralp, Ph.D.

Optimal Solutions Group, LLCM Squared Research Park | University of Maryland

5825 University Research Court, Suite 2800College Park, MD 20740

301-306-1170 | info@OptimalSolutionsGroup.com

Introduction

• SBA’s Regional Innovation Clusters Initiative provides funding to ten clusters with the goal ofstrengthening clusters and enhancing their

ability to assist small businesses;increasing opportunities for small business

participation and growth within the clusters; stimulating regional economic development

and growth.

What are clusters?

• “Clusters are geographic concentrations of interconnected companies, specialized suppliers, service providers, firms in related industries, and associated institutions (e.g., universities, standard agencies, and trade associations) in particular fields that compete but also co-operate” (Porter, 1998).

SBA Regional Innovation Clusters

• Clusters selected for the SBA’s Regional Innovation Clusters Initiative represent a broad spectrum of geographic areas and industries.

Agricultural Innovation Cluster (Monterey – Santa Cruz – San Benito, CA): agricultural innovation and leading-edge agriculture research.

Carolina Nuclear Cluster (NC, SC): improving nuclear energy supply chain.

Northeast Electrochemical Energy Storage Cluster (CT, NY, MA, ME, VT, NH, RI): developing, manufacturing, and deploying fuel cell and hydrogen technologies

Advanced Power and Energy Cluster (MN, ND, SD, WI): power generation, storage, and alternative

Illinois Smart Grid Regional Innovation Cluster (IL): Smart Grid innovation improving electric transmission infrastructure.

Enterprise for Innovative Geospatial Solutions (MS, LA): geospatial businesses

Upper Michigan Green Aviation Coalition (MI): expanding green aviation industry.

NorTech FlexMatters (OH): advanced energy and electronic devices printed on flexible substrates.

Huntsville Advanced Defense Technology Cluster Initiative (Huntsville, AL): aerospace technology

San Diego Advanced Defense Technologies Cluster (San Diego, CA): autonomous systems, cyber security, and other technologies

SBA’s Regional Innovation Clusters• Small businesses-oriented activities/events:

Business counseling; Workshops, training, and mentoring; Matchmaking and networking activities;Assistance in obtaining financing;Commercialization and technology transfer;Export services;Intellectual property and patent services;Marketing services; and,Contract counseling and other

legal services.

Evaluation Components

• Implementation evaluation Are clusters reaching the targeted small

businesses in their regions?Are the cluster activities / events meeting the

needs of the targeted small businesses?

• Impact evaluation Are the small businesses participating in cluster

activities / events attaining their desired goals?

Program Parameters

1. Variety of cluster configurations and business models;

2. Varied small business recruitment strategies;

3. Varied inclusion and exclusion criteria for selecting small firm; and,

4. Clusters were free to choose the types and frequency of services/activities.

Data Collection• Both quantitative and qualitative data:

Cluster Administrator SurveySmall Business SurveyLarge Organization SurveyInterviews with Cluster Administrators Interviews with selected small businessesCluster Quarterly ReportsQuality Indicators Survey (to be completed by

both the cluster administrator and the evaluator)

Data Collection

• Secondary data sources:Dun & Bradstreet databaseQuarterly Census of Employment and Wages

(BLS)ADP Small Business ReportLongitudinal Business database

Program Metrics and Outcomes

• Proximal: Access to capital; New alliances formed; Exports; Development of new products and services; Patents; Access to cleared secure facilities; Acquiring a contract or subcontract; Participation in industry chain; and Commercialization of new technology.

Program Metrics and Outcomes

• Distal:Number of businesses created;Revenue;Number of employees; and,Total compensation.

Metrics and Methodology

• How do the performance outcomes of small businesses participating in cluster activities/events change during the first year?Non-experimental:

Performance metrics (levels)Growth models

Metrics and Methodology

• How do the changes in outcomes experienced by small businesses participating in cluster activities/events compare to the well-known benchmarks?Compare the changes in participating firms’

performance metrics (revenue, number of employees and total compensation) to similar small businesses in the same industries and geographic areas using secondary data sources.

Metrics and Methodology

• What impact did cluster participation have on small businesses’ performance metrics (revenue, number of employees, total compensation) during the first year?Quasi-experimental:

Difference-in-Difference modelsMatched pair models

Metrics and Methodology• What impact does cluster participation have

on regional economies?Utilize input-output models for each metropolitan

area.Input-output models take into account inter-

industry interactions and transactions between producers, intermediaries, and consumers.

Alternative input-output models: RIMS II (Census Bureau)IMPLANREMI

Metrics and Methodology

• Input-output models estimate the impact on the regional economy:direct effect (e.g. 10 new jobs created in

firm A);indirect effect (i.e. increase in output and

employment in industries that supply firm A); and

induced effect (i.e. the effect of increased household income and spending).

Metrics and Methodology

• What is the direct effect that will be analyzed in the input-output model?The change in participating firms’ performance

outcomes (revenue, number of employees, total compensation) over and above the change in the outcomes experienced by the comparison group in the quasi-experimental models.

Conclusion• Producing more cluster activities/events do not

necessarily yield innovation and key outcomes;• Investing in clusters do not necessarily yield key

outcomes in the short-run (“not shovel ready”);• Literature suggests that clusters can produce

sustainable improvements in innovation and key outcomes; and,

• However, there are few rigorous evaluations of innovative clusters that provide generalizable findings.