Entrepreneurship & Innovation Case Studies
7(j) and 8(a) Program Evaluations
U.S. Small Business Administration
Optimal evaluated the effectiveness of 7(j) online training for participants’ ability to obtain Federal Government contracts. Optimal developed a quasi-experimental research design evaluation to test the effectiveness of 7(j) instructor-led online training provided to 8(a) firms. Optimal also implemented a mixed methods data collection approach that integrated administrative and qualitative data collected from business counselors and small businesses.
Procurement Scorecard Evaluation
U.S. Small Business Administration
Optimal conducted a rigorous, quasi-experimental program evaluation to assess the effectiveness of the small business scorecard methodology change and the scorecard overall in increasing small business opportunities and improving the ability of federal agencies to meet their small business procurement goals.
Evaluation of Microloan Lending Program
U.S. Small Business Administration
Optimal conducted a quasi-experimental design evaluation that examined the job creation/retention, revenue growth, and business tenure outcomes of Microloan program borrowers. Optimal designed survey instruments that created stratified statistically representative samples and collected data for the evaluation. Optimal’s methodological approach to instrument development and validation integrated qualitative and quantitative methodologies and obtained OMB PRA clearance.
SBA Program Economic Impact Study
U.S. Small Business Administration
Using administrative data, Optimal data scientists and economists used an input-output model (Regional Economic Models, Inc. [REMI]) to determine the flow of economic activity across all SBA set-aside programs. For this ongoing work, Optimal has (will) facilitate after-action reviews with multiple programs executives and staff.
Evaluation of 8(a) Certified Firms
U.S. Small Business Administration
Optimal conducted a rigorous quasi-experimental evaluation using sensitive administrative data from multiple systems (System for Award Management [SAM] and the Federal Procurement Data System – Next Generation [FPDS-NG]) to assess the short- and long-term impacts of the 8(a)-certification program. Researchers also conducted interviews with stakeholders (agency executives, industry/business organizations). Optimal also facilitated an after-action review with agency executives and program managers.
Logic Models for SBA Economic Development Programs
U.S. Small Business Administration
Optimal conducted literature and administrative documentation review, developed Z types of the logic model, conducted focus groups and interviews with stakeholders (i.e., SBA executives and program staff) to build consensus on program(s) theories of change, refined and augmented the logic models, and developed the research questions based on the logic models for future program evaluations.
For background see Optimal evaluated four Small Business Administration programs using logic models to determine outcomes.
Women-Owned Small Business (WOSB) NAICS Analysis
U.S. Small Business Administration
Optimal conducted a Congressionally mandated rigorous disparity study to assess substantial under-representation of WOSBs in the federal marketplace. Data scientists leveraged administrative data (SAM, FPDS-NG, Dun & Bradstreet) and secondary data sources (Census) and created Human-Centered Design data visualizations to engage stakeholders (agency executives and program officials, and industry and advocacy organizations) and facilitated mid-course stocktaking and after-action reviews.
HUBZone Economic Impact Evaluation
U.S. Small Business Administration
Optimal researchers and data scientists leveraged agency administrative data (SAM and FPDS-NG) and an input-output model (Regional Input-Output Modeling System [RIMS]) to determine the flow of economic activity (i.e., employment, earnings, value added, total economic output) in geographic areas (selected counties and all States) resulting from prime contract awards to HUBZone-certified firms. Optimal facilitated action-action reviews as well as pause-and-reflect sessions with agency executives that resulted in a follow-on study to assess the economic impact across all SBA set-aside programs.
Regional Innovation Clusters Evaluation
U.S. Small Business Administration
Optimal has conducted evaluations for the Regional Innovation Clusters since 2010. In consultation with the Technical Working Group, Optimal developed an overall evaluation framework for SBA’s Regional Innovation Clusters Initiative and has developed and suggested new and clarified existing output and outcome measures and metrics. Optimal researchers also developed nationwide virtual baseline and follow-up surveys and interviews with cluster administrators, and successfully developed an OMB Information Collection Request package. Optimal facilitated after-action reviews with agency executives and program staff and provided capacity building and technical assistance support to cluster administrators.
Read the previous RIC final report
Watch Lessons Learned from Evaluating SBA’s Regional Innovation Cluster Initiative
Consolidating Previous MEP Studies Leveraging Census Data
National Institute of Standards and Technology
Optimal has designed and is currently conducting a quasi-experimental study to assess how learning activities and network ties impact small and mid-sized manufacturing establishments’ performance. This study uses secondary, survey, administrative datasets, including micro-level Census data, Longitudinal Business Database (LBD). Optimal’s researchers link the establishment level data using fuzzy matching techniques and employ difference-in-difference techniques in estimating the impact of learning activities and networks has on business outcomes in comparison to otherwise similar manufacturing establishments that were not part of these same types of learning activities and networks. This analysis is taking place at the Maryland Federal Statistical Research Data Center.
Optimal’s NIST MEP team will be presenting at the Eastern Economic Association conference in February 2023. For more information, see: https://www.ramapo.edu/eea/49th-annual-conference/
Assessment of the 602 Non-Profit Disposition Program
U.S. Department of Housing and Urban Development
Optimal prepared three case study reports that describe the operations and socio-economic conditions in each of the sites, as well as a cross-site report that describes the evaluation strategy and the prospective statistical analysis developed for future evaluations. These reports also discuss near-term progress of the 602 Program in the three sites and provide readers with a general understanding of the operations and potential benefits of the 602 program.
Scale-Up America Program
U.S. Small Business Administration
Scale-Up American was designed to help small firms with high potential “scale up” and grow their businesses so that they will provide more jobs and have a greater economic impact, both locally and nationally. Optimal wrote the quasi-experimental design study and methodological approach and collected primary data for the Scale-Up America Initiative evaluation. Optimal then transferred the design study and data to the Center for Economic Studies (CES) at the Census Bureau to conduct the study using restricted-use micro-data (i.e., Business Register).
Emerging Leaders Initiative Evaluation (Follow on)
U.S. Small Business Administration
Optimal is tasked with conducting an evaluation to examine the implementation and follow-up results of the Emerging Leaders Initiative/T.H.R.I.V.E. Emerging Leaders Reimagined training for the current year and for up to three past cohorts of participants. Optimal is also responsible for primary data collection from program participants. Optimal’s data collection solution has provided the SBA district office staff (application reviewers) and SBA HQ staff with easy-to-use interfaces and near-real-time reports. Optimal collaborated with the training contractor to develop and disseminate training videos to the district offices actively reviewing applications.
Evaluation Services Community Navigation Pilot Program
U.S. Small Business Administration
Optimal is conducting a formative evaluation of an innovative pilot program to engage and provide technical assistance to underserved populations. Optimal has designed a mixed methods evaluation using creative performance measures derived from the marketing literature and is using linked administrative and secondary data sources to assess grantee performance at multiple hierarchies (Hubs and Spokes). The evaluation design also includes a quasi-experimental design for assessing the ongoing and long-term impact of the community navigators’ approach to engaging underserved populations.