Upload
vuongque
View
227
Download
2
Embed Size (px)
Citation preview
Volume 2, Number 2, Special Issue 2011
*Associate Professor, Technology, Leadership, and Innovation
Director, Certificate in Entrepreneurship and Innovation Program
Purdue University
35
The Engineering Entrepreneurship Survey:
An Assessment Instrument to Examine Engineering Student
Involvement in Entrepreneurship Education
Nathalie Duval-Couetil*, Teri Reed-Rhoads, and Shiva Haghighi
Abstract - This paper describes the Engineering Entrepreneurship Survey
(EES), an assessment instrument designed to examine engineering student
involvement in entrepreneurship education and related outcomes. It was
developed as part of an NSF-funded study “Entrepreneurship Education and its
Impact on Engineering Student Outcomes: The Role of Program Characteristics
and Faculty Beliefs.” It has been used successfully over the past two years to
gather data from senior-level students at multiple institutions that have
entrepreneurship courses or programs that are available to engineers. The 135-
item survey examines: 1) engineering student attitudes towards entrepreneurship,
2) engagement in entrepreneurial activities and behaviors, 3) knowledge of
entrepreneurship-related terms and concepts, 4) entrepreneurial self-efficacy, 5)
student perceptions of engineering faculty attitudes toward entrepreneurship.
Item categories and subscales show high reliability. The validated survey
collects baseline data that can be valuable in program development and
evaluation.
1. Introduction
Entrepreneurship courses and programs may be one of the fastest growing curricular areas within
engineering schools. Economic realities such as global competition, downsizing, decentralization,
re-engineering, mergers, and new technologies have made career paths more complex and
uncertain for graduates in all sectors (Gibb, 1996). In this new environment, it is widely accepted
that entrepreneurial ventures are key to innovation and economic growth. Therefore, students
who are able to identify opportunities, understand market forces, commercialize new products,
communicate, and lead teams, in addition to having strong science and technical skills, are likely
to have more value in the marketplace for jobs. To meet this need, colleges of engineering are
increasingly developing entrepreneurship courses and/or are making those offered through
business schools or multidisciplinary programs accessible to their students. This movement has
garnered support from influential publications and professional organizations such as the National
Academy of Engineering (NAE) and the American Society for Engineering Education (ASEE)
(Dabbagh and Menascé, 2006; Rover, 2005).
The characteristics of entrepreneurship programs offered to engineers can vary widely as do the
beliefs and practices of the faculty who teach them. A multi-institution, NSF-funded study,
Entrepreneurship Education and its Impact on Engineering Student Outcomes: The Role of
Program Characteristics and Faculty Beliefs, was undertaken to address how these factors
influence student outcomes. The purpose of the study was to explore the relationship between
N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi
The Journal of Engineering Entrepreneurship 36
program models, faculty beliefs, and student outcomes to help inform program development,
improve educational experiences, and evaluate their success (Figure 1). Consequently, a primary
component of this study was to investigate the engineering student involvement in
entrepreneurship education within their engineering programs and its impact on them. This
required the development of a comprehensive assessment instrument capable of collecting
baseline data on a broad range of attitudes toward and outcomes of entrepreneurship education on
engineering students across multiple institutions.
Figure 1. Components and objectives of the NSF-funded study Entrepreneurship
Education and its Impact on Engineering Student Outcomes: The Role of
Program Characteristics and Faculty Beliefs
Somewhat surprisingly, although an increasing number of students in all disciplines are being
exposed to entrepreneurship education (Streeter et a., 2002), minimal published research has
examined related learning outcomes or their influence on student attitudes, behaviors, career
goals, or professional competence. Pittaway et al. (2009) observed that most of the research on
entrepreneurship pedagogy focused on program design and implementation, and that assessment
practice was a major gap in the field. One of the major reasons contributing to the lack of
assessment in entrepreneurship education pointed to within the management literature is a lack of
consensus on what entrepreneurship or being entrepreneurial is. Another reason is the wide range
of learning objectives that exist across courses and programs. This led Henry et al. (2005) to state
that the "content of syllabi of courses developed by entrepreneurship scholars differs to such an
extent that it is difficult to determine if they even have a common purpose" (p. 103).
A comprehensive review of the entrepreneurship education assessment literature uncovered few
validated instruments available to faculty and administrators to assess the impact of
entrepreneurship education at the course or program level (Duval-Couetil et al., 2010a). The
review identified three primary types and categories of assessment: 1) course level evaluations
which measure student reactions to a particular class or specific activities; 2) focused instruments
which measure very specific aspects of, or constructs related to, entrepreneurship; and 3) program
evaluations which assess a wider range of outcomes. Most published work appeared to be related
to developing entrepreneurship-related constructs that are most appropriate for management or
Entrepreneurship Program models
Engineering School-based
Business School-based
Multidisciplinary
Faculty beliefs and practices
Engineering faculty
Business School Faculty
Lecturers or Part-time faculty
Practitioners
Student outcomes
Behaviors
Attitudes
Knowledge
Self-efficacy
Explore relationships to help inform program development,
improve educational experiences, and evaluate their success
The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education
Volume 2, Number 2 Special Issue2011 37
professional contexts. Of the instruments reviewed, only those by Shartrand et al. (2008)and
Lucas et al. (2009) were developed to be used primarily with engineering students.
The purpose of this paper is to describe the development, use, and preliminary validation of the
Engineering Entrepreneurship Survey (EES) (Appendix A). This survey instrument has been
used successfully to collect data among engineering students enrolled at universities that have
entrepreneurship programs accessible to engineering students. The data gathered from the
administration of the survey over the past two years has served as the basis for several conference
presentations and papers (Duval-Couetil et al., 2010b, 2011a, b & c). Given the breadth of data
collected, it can serve as a valuable tool for engineering programs or faculty interested in
gathering student data as part of their program development and/or evaluation processes.
2. Development of the Engineering Entrepreneurship Survey
2.1. Research Questions
The EES instrument was developed to capture the data necessary to investigate how
entrepreneurship education impacts a broad range of attitudes toward and outcomes of
entrepreneurship education for engineering students. The primary focus of the research study was
to examine the extent and nature of changes in behaviors, knowledge, attitudes, and self-efficacy
that result from exposure to entrepreneurship education. It was important to provide a context for
the research by providing background data related to the extent to which engineering students
were involved in or being exposed to entrepreneurial activities within their academic programs.
Therefore, the instrument addressed the following research questions:
To what extent do engineering students participate in entrepreneurship education and
related activities?
To what extent is entrepreneurship addressed in their engineering programs?
What are engineering student attitudes toward entrepreneurship as a career?
Why are students interested or not interested in entrepreneurship?
How familiar are engineering students with entrepreneurship terms and concepts?
What are engineering student perceptions of their entrepreneurship-related abilities?
What are the characteristics of engineering students participating in entrepreneurship
education?
Student demographic and program data were collected to investigate differences within and
across the following groups:
Engineering students who had and who had not participated in entrepreneurship courses
or programs
Demographic characteristics (e.g., sex, culture, ethnicity, familial entrepreneurial
background)
Engineering disciplines
Individual programs and universities
2.2. Selection of the Appropriate Assessment Method and Administration Period
Given the varying program models, the first step in creating the assessment instrument was to
review the requirements of the entrepreneurship programs across the institutions. This was
necessary in order to determine the type of assessment that would be most appropriate to capture
N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi
The Journal of Engineering Entrepreneurship 38
involvement in entrepreneurship education, and the period within students’ academic programs
when it would be most appropriate to administer the survey. The review found that the programs
involved in the study could vary significantly in their length and when they began and ended.
Consideration was given to administering pre- and post-tests to engineering students involved in
entrepreneurship education given their value in establishing benchmarks against which changes in
attitudes, behaviors, knowledge, or self-efficacy can be measured. It was determined, however,
that they could not be used in this study given the varying program models and lengths. This was
based on the principle that to be valid, pre- and post-assessments should be used with highly
structured curriculum where learning objectives are addressed to all subjects in a systematic
manner at equivalent times within and across academic programs. Another factor that precluded
the use of pre- and post-tests was the three-year grant period, which limited our ability to collect
data for more than three semesters. Given the time necessary to develop the assessment
instrument and analyze results, three semesters was not sufficient to capture exposure to
entrepreneurship education in an adequate sample of students across the three institutions.
As a result, it was decided that a self-report instrument administered to students involved in
senior-level capstone design courses would provide the necessary sample homogeneity across
institutions. It would capture involvement in entrepreneurship and innovation-related activities,
such as formulating an idea for a product/business, developing prototypes, presenting, researching
markets, and preparing business plans, since they typically occur at the capstone level (Dabbagh
and Menascé, 2006). It would also allow for the investigation of differences across the desired
demographic characteristics and groups. To minimize logistics and challenges related to survey
administration across multiple institutions, it was decided that a web-based version would be used
(Qualtrix).
2.3. Review of Existing Assessment Instruments
As stated in the introduction to this paper, a comprehensive literature review of entrepreneurship
education assessment was conducted both within and outside the domain of engineering. The
results of this review were published in a conference paper (Duval-Coutil et al., 2010a) and
disseminated in a workshop on entrepreneurship assessment (Duval-Coutil et al., 2010b). The
literature review identified two existing scales developed that were integrated into the EES
instrument with permission from their authors and with minor modifications:
Venturing and Technology Self-efficacy (Lucas et al., 2009): This scale is
comprised of 15 items to measure engineering students’ confidence in their
venturing and technology application skills based on authentic tasks they might
encounter in the workplace. It emphasizes the leadership of innovation rather
than a narrow focus on the formation of companies. Items consist of short task
descriptions that involve self-confidence in one’s ability to recognize
opportunities, write business plans, manage projects and human resources,
estimate the costs of, and market a new product, as well as items that examine
self-efficacy related to moving from science concepts to applications. The scale
was validated with a sample of approximately 400 third and fourth year
engineering students. The category of items uses a response scale reflecting 0-
100% in intervals of 10, which the authors state is easily understood by
engineers.
Motivations and Barriers to Starting a Business (Shinner et al., 2009): Two
scales were selected from a larger survey instrument designed to investigate
The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education
Volume 2, Number 2 Special Issue2011 39
attitudes toward entrepreneurship and its education in order to assess demand for
a multidisciplinary entrepreneurship course. The survey was administered to 317
students and 87 faculty members in a variety of disciplines. Original survey
items were modified slightly and the original 5-point Likert scale based on
relative importance was changed to level of agreement (1=strongly disagree,
5=strongly agree).
Consideration was given to using the NCIIA’s Entrepreneurship Inventory as a source for a
subset of survey items related to familiarity with entrepreneurial terms and concepts (Shartrand,
et al., 2008). The instrument was developed to measure engineering students’ familiarity with
105 terms and concepts, and it provides information on validated scales related to five categories;
becoming an entrepreneur, finance and accounting, people and human resources, sales and
marketing, and product ideation and development. Its integration was not feasible, however, due
to its length. The need to minimize the number of response scales in the EES was another factor
in the decision.
In addition to the literature review, course- and program-level instruments used at each institution
were reviewed. This was done to determine the extent to which existing assessments used at each
institution could be leveraged and to evaluate the degree to which there might be overlap or
interference with historical assessments still taking place at each university. Each program had
used assessments internally for assessment and/or program development purposes, and they were
in various stages of development and validation. Of the internal instruments reviewed, the
research team drew most heavily on pre- and post-survey instruments used at their own
institution, Purdue University, as part of its multidisciplinary Certificate in Entrepreneurship and
Innovation Program. The entry survey has 74 items and the exit survey 61 items focused on work
experience, post-graduation plans, motivations for taking an entrepreneurship course, familiarity
with entrepreneurial topics, entrepreneurial self-efficacy, family/parental history in
entrepreneurship, and demographics. It has been used with hundreds of students primarily for
program development and reporting purposes. A number of items and scales from these
instruments were modified and/or expanded upon to better address the needs of engineering
students.
Where no assessments were found to address research topics, the research team created new
survey items. This occurred primarily in categories having to do with entrepreneurial activity or
behaviors occurring within engineering programs:
Engineering student participation in entrepreneurial activities: This category of
ten items was developed to examine the extent to which students had been
exposed to entrepreneurship education or related experiential learning activities.
It was generated from the literature review and knowledge of entrepreneurial
activities available to engineering students either through or external to their
engineering programs. Students were asked to respond yes or no to whether they
participated in activities such as taking a course, interning with a startup,
conducting market research for a new product, developing a product for a real
client, writing a business plan, delivering an elevator pitch, protecting intellectual
club, participating in an entrepreneurship-related competition, joining an
entrepreneurship student organization, or participating in non-credit workshops.
Extent to which entrepreneurship is addressed in engineering programs: This
category of nine items examined student perceptions of the extent to which
entrepreneurship was addressed within their engineering programs. Items
N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi
The Journal of Engineering Entrepreneurship 40
investigated the degree to which students agreed that they were taught or
encouraged to develop entrepreneurial skills, take entrepreneurship courses,
participate in experiential learning opportunities related to entrepreneurship, and
interact with entrepreneurs. It also addressed the degree to which engineering
faculty discussed entrepreneurship.
2.4. Format and Structure
Given the number of sources of existing and newly created items, it was necessary to structure the
survey in a way that was user-friendly for the subjects. Effort was made to minimize the number
and types of response scales used. Most had 5-point Likert scales with endpoints either strongly
disagree (1) strongly agree (5) or poor (1) to excellent (5) with the exception of Lucas and
Cooper’s Venturing and Technology Self-Efficacy Scale. So that it would not interrupt the flow
of the survey, the scale was placed near the end. Survey codes were developed for each of the
subscales or categories of questions, and these are presented in Table 1. Attention was paid to the
length of the survey; it had to be comprehensive enough that faculty would recognize its value yet
short enough so students would complete it. It was estimated that it would take students
approximately 15 minutes to complete. Feedback from the faculty who participated was positive.
Over 90 percent of the students who started the survey completed it.
3. Sample
Over a period of three semesters, the EES was administered to engineering students enrolled at
senior-level engineering capstone design courses at three large public universities with
established entrepreneurship programs available to engineering students. Two of the
entrepreneurship programs were predominantly engineering-based. The other was a
multidisciplinary program, whereby engineering students took “core” entrepreneurship courses
with students in a variety of majors and complemented them with approved engineering courses
to complete program requirements. Students received the survey via the faculty members
teaching these courses. Contact with faculty members was made either through personal contacts
at each institution, or through Internet searches to identify department heads and others able to
approve and/or facilitate survey administration. Faculty members were sent an email describing
the intent of the study and what would be required of them. If faculty agreed to participate, they
were sent an email to be forwarded to students, which included a brief explanation of the survey
and its URL. Over the course of the month that followed initial distribution of the survey to
students, faculty members were asked to remind students to complete it. Over three semesters,
501 engineering students completed the survey. It was distributed to approximately 30 courses
across the three institutions. Response rates per course ranged from 3 to 58 percent, with a mean
of 21 percent. Preliminary results from an initial sample of 345 subjects and final sample of 501
have been reported (Duval-Couetil, 2011a &b).
The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education
Volume 2, Number 2 Special Issue2011 41
Table 1. EES Survey Item Categories and Scales, Topics Addressed, and Item Sources
Category, Code, No.
of Items
Specific Topic Addressed by EES Item
Categories and Scales Source of Items
BEHAVIORS
ACTIVITIES (10) Extent to which engineering students participate
in entrepreneurship education and related
activities
New items
POSTGRAD (7) Students’ post-graduation career plans Purdue surveys
BUSINESS (1) Number of students who had, have, or intend to
have a business
New items
VENTURE (1) Type of businesses students are interested in
starting
Purdue surveys
ATTITUDES
PROGRAM (9) Extent to which entrepreneurship addressed in
engineering programs
New items
INTEREST (7) Nature of engineering student interest in
entrepreneurship
Purdue surveys
STARTBUS (12) Reasons students would be interested in
entrepreneurship
Shinnar et al.
NOTSTART (14) Reasons students would not be interested in
entrepreneurship
Shinnar et al.
KNOWLEDGE
FAMILIAR (37) Student familiarity with entrepreneurship terms
and concepts
New items
Purdue surveys
SELF-EFFICACY
EFFICACY (15) Student perceptions of their technology
venturing and entrepreneurship-related abilities
Lucas et al.
SKILLS (6) Student perceptions of their skills in areas
related to entrepreneurship
New items
Purdue surveys
ABILITY (1) Student perceptions of their entrepreneurship
ability overall
Purdue surveys
BUSABILITY (1) Student perceptions of their ability to start a
business immediately
Purdue surveys
DEMOGRAPHICS
Multiple codes (14) Characteristics of engineering students
participating in entrepreneurship education
New items
Purdue surveys
N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi
The Journal of Engineering Entrepreneurship 42
4. Establishing Validity and Reliability
To be effective, a survey instrument should be both reliable and valid. Validity refers to the
ability of an instrument to measure the real concept under consideration. Reliability refers to
whether a particular technique applied repeatedly to the same object would yield the same result
each time (Babbie, 1990). Since reliability is an indication of measurement error, an instrument
must be reliable in order for validity to have meaning. Therefore, both are necessary and
individually are not sufficient.
4.1. Validity
The first type of validity to be considered in survey instrument development is content validity,
which refers to the degree to which a survey covers the range of meanings and content within a
specific domain (Babbie, 1990). EES content validity was evaluated through a comprehensive
literature review, as well as evaluation by a panel of approximately 20 experts. This panel was
comprised of engineering and entrepreneurship faculty at the institutions involved in the study,
members of the external advisory board for the grant and educational assessment specialists.
Feedback was integrated into revisions of the survey.
Face validity is a form of content validity and “may be of importance in determining its
acceptability and reasonableness to those who will be tested” (Messick, 1989). If a survey is
considered to be meaningless or irrelevant by subjects, they may be less willing to participate. To
determine whether the EES would be considered valid from the subject’s point of view,
engineering students who had and who had not been exposed to entrepreneurship education,
participated in a think-aloud protocol. This protocol allowed members of the research team to
observe subjects completing the online survey to evaluate comprehension, the instrument’s
relevance, and its user-friendliness. Subjects were asked to identify any ambiguities or
difficulties they encountered as they completed the survey. The think-aloud protocol resulted in
feedback that led to the revision of items, scales, survey instructions and length.
Construct validity refers to evidence of the extent at which a measure relates to other variables
within a system of theoretical relationships or constructs (Babbie, 1990). Often, experts in the
field are used to verify construct validity as was the case with the EES. Preliminary evaluation of
construct validity also consisted of data analyses to examine differences in the results of students
who had and who had not been exposed to entrepreneurship education. These were detected in
larger populations as data collection progressed.
Future analyses will examine predictive validity, which refers to the extent to which a given
instrument can be used to predict desired future performance, such as a student’s future success
within a given field. In the short term, this could be achieved by examining performance in
entrepreneurial activities while they are on campus (e.g., competitions). Longer term,
longitudinal research could examine engineering students’ post-graduation career choices,
income, and net worth as reported. A survey of entrepreneurship alumni of a business school
conducted by Charney and Libecap (2003) explored the impact of entrepreneurship education on
these factors.
4.2. Reliability
In this study, reliability was assessed through internal consistency to determine the extent to
which similar survey items give consistent responses. Cronbach’s coefficient alpha was used to
measure the overall internal reliability within the categories of survey items that addressed
The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education
Volume 2, Number 2 Special Issue2011 43
attitudes, beliefs, or perceptions that were most appropriate for such analysis (Table 2).
Categories of items that measured behaviors and past or future participation in activities were
excluded from the analyses.
Table 2. Cronbach’s alpha for EES survey item categories and scales
(see Table 1 for details of items and scales)
Category of Items/Scale Number of
items
Cronbach’s
Coefficient
Alpha
PROGRAM (9) 9 .89
INTEREST (7) 7 .92
STARTBUS (12) 12 .83
NOTSTART (14) 14 .86
FAMILIAR (37) 37 .96
EFFICACY (15) 15 .96
SKILLS (6) 6 .74
Notable results are the following:
All but one category, SKILLS, met the standard of 0.8, which is considered acceptable
for group comparisons (Nunnally and Bernstein, 1991). Items in this category addressed a
variety of skills including analytical, communication, and presentation skills, as well as
the ability to evaluate business ideas, level of risk tolerance and ability to deal with
uncertainty. These items were not written in a way that tied them to an entrepreneurial
context, and this will be addressed in future revisions.
The category FAMILIAR is comprised of 37 terms and concepts that are relevant to
entrepreneurship. Students were asked to rate their familiarity with each. Cronbach’s
alpha of 0.96 was high for the group, which is to be expected with such a large number of
items. Given the large number of items, this category was analyzed independently for a
conference presentation (Duval-Couetil, 2011)). This consisted of dividing 35 of the 37
terms into six categories based on content and face validity and calculating the internal
consistency for each. They included the following: engineering (α = .88); general
entrepreneurship (α = .86); 3); general business (α = .84); marketing (α = .93); finance (α
= .93); and professional skills (α = .75).
Future work will address the refinement of these scales.
5. Discussion
The Engineering Entrepreneurship Survey (EES) was developed based on a need to be able to
measure student attitudes toward entrepreneurship, their level of involvement and interest in
entrepreneurship education, and its impact on their entrepreneurial knowledge and self-efficacy.
The instrument was designed to collect a wide range of baseline data that would be useful to
make comparisons based on student characteristics and program characteristics both within and
across institutions. It was developed after conducting a comprehensive review of the
entrepreneurship education assessment literature both within and outside the field of engineering.
This review identified validated scales that were used in the development of the EES and
identified assessment gaps that were filled with new items and scales developed by the research
team. Attention was given to ensure that the survey would be administered at a period in a
N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi
The Journal of Engineering Entrepreneurship 44
student’s academic program when exposure to entrepreneurship education would be detected and
that the administration would not interfere or overlap with assessments used at institutions
involved in the study.
Using conventional analytical approaches, we have demonstrated that the EES is acceptable as a
self-report instrument to be used with engineering students. It was used over a period of two
years to collect data that, thus far, has been primarily used to measure levels of student interest in
and involvement in entrepreneurship education and to investigate differences between groups of
students who have and have not had exposure to entrepreneurship education. Future analyses will
explore differences in attitudes based on demographic characteristics such as sex, ethnicity, and
parent involvement in entrepreneurship. They will also explore differences in institutions to
examine the extent to which program characteristics and faculty beliefs and practices impact
student attitudes and outcomes, which is the overarching objective of the NSF-funded study.
As it stands, the EES can be used as an effective assessment tool for needs assessment, program
development, and research purposes. It can be used alone to measure differences in groups, as has
been described in this manuscript. There are, however, opportunities to refine the instrument as a
whole and to further explore the validation of subscales focused on the following areas:
Demand for entrepreneurship education: Items addressing student interest in taking
entrepreneurship courses, as well as their post-graduation career plans can be used to
estimate potential demand for entrepreneurship courses or related activities.
Entrepreneurial climate: Items related to student perceptions to the extent that
entrepreneurship is addressed within engineering programs or by engineering faculty can
assess whether a particular environment is conducive to and supportive of entrepreneurial
education or activities.
Entrepreneurial mindset: Items pertaining to student involvement in entrepreneurial
education, activities, courses, and whether students have ideas for ventures; the nature of
student interest in entrepreneurship can be used to measure the degree to which students
have an entrepreneurial mindset.
Gains in entrepreneurial knowledge and self-efficacy: Used as a pre- and post-test in
contexts with homogeneous conditions, it can measure gains related to exposure to
entrepreneurship education or related activities. It can also be used to compare
knowledge or gains across courses, engineering departments or institutions.
Body of knowledge for engineering entrepreneurs: Items pertaining to familiarity with
terms and entrepreneurial self-efficacy can identify the emphases, strengths and
weaknesses of programs being offered to engineering students. Items related to barriers
to entrepreneurship can identify areas that must be addressed in an educational program
designed to make entrepreneurship a desirable and/or accessible choice.
As with any instrument, it is important to examine the sample on which it is tested. To date, the
external validity of the survey, or degree to which the results can be generalized to populations
beyond the sample, is limited. The instrument has been used primarily within large, public
universities with established engineering and entrepreneurship programs where interest and
participation in entrepreneurship may be higher than in the general population of engineering
schools. Purposive sampling has been used to ensure adequate representation of students who
had exposure to entrepreneurship education in order to make comparisons between groups. The
representation of underserved populations and certain engineering disciplines in the sample was
too low to draw conclusions about these groups. Improving the external validity will require
administration of the survey to a more diverse group of institutions, with varying program models
and student populations.
The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education
Volume 2, Number 2 Special Issue2011 45
Internal validity refers to the recognition of bias that could be introduced by the researchers
themselves through their beliefs or actions. This can be manifested in the research design or the
way the sample was obtained. For this particular instrument, and the way it has been administered
to date, bias could arise from self-selection of faculty and students who chose to participate.
Senior design faculty members from three institutions were asked to participate in the study;
however, those who chose to do so may have a positive bias toward entrepreneurship education.
Since the survey was voluntary for students, it is possible that those who chose to complete it had
a positive bias toward entrepreneurship as well. Furthermore, its administration as an online
survey may have resulted in lower response rates since students completed them outside of class
instead of in-class. Future work must address these sampling limitations.
6. Conclusion
The Engineering Entrepreneurship Survey (EES) can be an effective tool as universities and
departments develop entrepreneurship courses and programs for engineering students. The
literature review on which it is based can serve as a resource for researchers developing
assessment instruments in the field of entrepreneurship education. Thus far, it has been used
effectively to detect differences across groups of students who have and who have not been
exposed to entrepreneurship education. Future work will examine differences by program and
demographic characteristics. The EES is certain to undergo refinement, and future work will
explore developing individual instruments that focus on specific constructs or content. However,
in its current form, it can provide valuable baseline data for faculty and administrators who are
involved in teaching, developing, or evaluating the impact of entrepreneurship courses delivered
to engineering students.
References
Babbie, E. (1990). Survey Research Methods. Belmont, CA: Wadsworth.
Charney, A. and Libecap, G. (2003). The contribution of entrepreneurship education: An analysis
of the Berger program. International Journal of Entrepreneurship Education, 1(3):276-288.
Dabbagh, N. and Menascé, D. (2006). Student perceptions of engineering entrepreneurship: An
exploratory study. Journal of Engineering Education, 95(2): 153-163.
Duval-Couetil, N., Reed-Rhoads, T., and Haghighi, S. (2010a). Development of an assessment
instrument to examine outcomes of entrepreneurship education on engineering students.
Proceedings of the 40th ASEE/IEEE Frontiers in Education Conference. Washington, DC.
Duval-Couetil, N., Reed-Rhoads, T., and Haghighi, S. (2010b). Assessing the outcomes of
entrepreneurship education on engineering students. Workshop of the NCIIA Annual Meeting.
San Francisco, CA.
Duval-Couetil, N., Reed-Rhoads, T., and Haghighi, S. (2011a). Investigating the impact of
entrepreneurship education on engineering students. Proceedings of the NCIIA Annual
Meeting. Washington, DC.
Duval-Couetil, N., Reed-Rhoads, T., and Haghighi, S. (2011b). Engineering students and
entrepreneurship education: Involvement, attitudes and outcomes. International Journal of
Engineering Education, in press.
Duval-Couetil, N., Reed-Rhoads, T., and Haghighi, S. (2011c). Entrepreneurship literacy for
engineering students. Proceedings of the 41st ASEE/IEEE Frontiers in Education
Conference. Rapid City, South Dakota.
N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi
The Journal of Engineering Entrepreneurship 46
Gibb, A. (1996). Entrepreneurship and small business management: Can we afford to neglect
them in the twenty-first century business school? British Journal of Management, 7(4): 309-
321.
Henry, C., Hill, F., and Leitch, C. (2005). Entrepreneurship education and training: Can
entrepreneurship be taught? Part 1. Education & Training, 47(2): 98-111.
Lucas, W., Cooper, S., Ward, T., and Cave, F. (2009). Industry placement, authentic experience
and the development of venturing and technology self-efficacy. Technovation, 29(11): 738-
752.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational Measurement. New York, NY,
England: Macmillan Publishing. 60-68.
Nunnally, J. and Bernstein, I. (1991). Psychometric Theory. New York: McGraw-Hill.
Pittaway, L., Hannon, P., Gibb, A., and Thompson, J. (2009). Assessment practice in enterprise
education. International Journal of Entrepreneurial Behaviour & Research, 15(1): 71-93.
Rover, D. (2005). New economy, new engineer. Journal of Engineering Education, 94(4): 427-
428.
Shartrand, A., Weilerstein, P., Besterfield-Sacre, M., and Olds, B. (2008). Assessing student
learning in technology entrepreneurship. Proceedings of the 2008 ASEE/IEEE Frontiers in
Education Conference. Saratoga Springs, New York.
Shinnar, R. Pruett, M., and Toney B. (2009). Entrepreneurship education: Attitudes across
campus. Journal of Education for Business. 84(3):151-159.
Streeter, D., Jaquette, P. & Hovis, K. (2002). University-wide entrepreneurship education:
Alternative models and current trends. WP 2002-02 March 2002 Working Paper -
Department of Applied Economics and Management. Ithaca, NY: Cornell University.
The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education
Volume 2, Number 2 Special Issue2011 47
Appendix A
Engineering Entrepreneurship Survey (EES)
This survey is to gather information about your experiences and involvement in
entrepreneurship activities and education. Entrepreneurship refers to the
practice of starting business ventures based on the development of new products
and/or services.
This survey is voluntary and you may skip any questions. If necessary, you can
stop the survey and complete it at a later time. Use the same link used to access
the survey to pick up where you left off. You must be 18 years or older to
participate. Personal information, such as your name and email, is requested so
that we may conduct follow-up interviews with a small sample of students. At
the end of the survey, you will be given the option to opt-out of being contacted
for a follow-up interview. Identifying information will not be released in any
way.
Please select "yes" indicating that you have read the above statement and agree
to be included in this survey. If you do not wish to continue, you may close the
Web browser now.
I agree to participate in this survey. (AGREE)
Yes (1)
No
INSTRUCTIONS:
Please take your time to answer each question as honestly and as
accurately as possible.
You will need to click on the arrow button at the end of each page to
save it and move on to the next.
Please be sure to click the 'Submit' button on the last page to complete
the survey.
The survey should take approximately 15 minutes. If you need assistance or
have questions, please contact:
Survey Administrator
University Name
Email Address
Phone
Thank you for your time.
N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi
The Journal of Engineering Entrepreneurship 48
What university do you attend? (UNIV)
University A
University B
University C
What is your major? (MAJOR1)
Aeronautics and Astronautics Engineering
Agricultural and Biological Engineering
Architectural Engineering
Biomedical Engineering
Biomolecular Engineering
Chemical Engineering
Civil Engineering
Computer Engineering
Computer Science
Construction Engineering
Electrical Engineering
Environmental/Ecological Engineering
Industrial/Systems Engineering
Interdisciplinary Engineering
Manufacturing Engineering
Materials/Materials Science Engineering
Mechanical Engineering
Multidisciplinary Engineering
Nuclear Engineering
Textile Engineering
Other (please list)
If you have a double major, please list it (if not, leave blank). (MAJOR2)
If you have one or more minors, please list it/them (if not, leave blank). Minor 1
Minor 2
Minor 3
What is your sex? (SEX) Male
Female
Please enter your age (in years). (AGE)
Please indicate your residence status. (RESIDENCE) In-state student
Out-of-state student
International student
If you are an international student, what is your home country?
(HOMECOUNTRY)
The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education
Volume 2, Number 2 Special Issue2011 49
What is your ethnic/racial background? Select all that apply. (RACE)
American Indian or Alaska Native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian or Other Pacific Islander
White
Multiracial
Other (please specify)
Decline
Are either of your parents entrepreneurs? (PARENTS)
Yes
No
I’m not sure
Is anyone in your family (not including your parents) an entrepreneur? (FAMILY)
Yes
No
I’m not sure
How many entrepreneurship courses have you taken in your college career, outside of
engineering? (COURSES)
None
1
2
3 or more
If you have taken any entrepreneurship courses outside of engineering (see previous
question), please list the course name(s) and number(s). (COURSE NAME)
Have you participated in any of the following types of academic entrepreneurship
programs? (ESHPROGRAM)
Certificate in Entrepreneurship
Major in Entrepreneurship
Minor in Entrepreneurship
Other
N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi
The Journal of Engineering Entrepreneurship 50
While in college, have you done any of the following: (ACTIVITIES)
No Yes
Taken an entrepreneurship course within the College
of Engineering at your university
Interned or worked for an entrepreneurial or start-up
company
Conducted market research and analysis for a new
product or technology
Developed a product or technology for a real
client/customer
Given an “elevator pitch” or presentation to a panel of
judges about a product or business idea
Been involved in patenting a technology or protecting
intellectual property
Been involved in entrepreneurship- or business-
related student organizations
Written a business plan
Participated in an entrepreneurship-related
competition (e.g., product development, business
plan)
Participated in entrepreneurship-related workshops
(extra-curricular, non-credit)
Rate your level of agreement with the following: In general, in my engineering courses…
(PROGRAM)
Strongly
disagree Disagree Undecided Agree
Strongly
agree
Faculty discuss
entrepreneurship
Students are taught
entrepreneurial skills
Students are encouraged to
develop entrepreneurial skills
Students are encouraged to
take entrepreneurship courses
Students are encouraged or
required to participate in
entrepreneurship-related
activities
Students are encouraged to
consider starting their own
companies
Entrepreneurship is presented
as a worthwhile career option
There are opportunities to
interact with entrepreneurs
Students should learn more
about entrepreneurship
The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education
Volume 2, Number 2 Special Issue2011 51
Consider your post-graduation options and please rate your level of agreement with the
following: I plan to… (POSTGRAD)
Strongly
disagree
Disagree Undecided Agree
Strongly
agree
Start my own business or
be self-employed
Work for a small business
or start-up company
Work for a medium- or
large-size business
Work for the government
Serve in the military
Work for a non-profit
organization
Attend
graduate/professional
school
Please rate your level of agreement with the following: (INTEREST)
Strongly
disagree Disagree Undecided Agree
Strongly
agree
I have a general interest in
the subject of
entrepreneurship
I want to become an
entrepreneur
I have an idea for a
business product or
technology
I would like to know if I
have what it takes to be an
entrepreneur
I am interested in taking
entrepreneurship classes
Entrepreneurship
education can broaden my
career prospects and
choices
I would like to learn about
entrepreneurship in my
engineering courses
Please check the answer that best fits your current situation. (BUSINESS)
I had my own business
I have my own business now
I would like to start a business in the next year
I would like to start a business in the next 5 years
I would like to start a business in the next 10 years
I don’t have any plans to start a business at this time
N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi
The Journal of Engineering Entrepreneurship 52
If you are interested in being an entrepreneur, what type of business are you interested in
starting? Please describe the product or service and industry or market. (VENTURE)
Please rate your level of agreement with the following: I would start a business in order
to… (STARTBUS)
Strongly
disagree Disagree Undecided Agree
Strongly
agree
Focus on a technology
that interests me
Satisfy a need in a market
Solve a social problem
Create something of my
own
Have more flexibility and
independence
Have more free time
Make more money
Be at the head of an
organization
Manage people
Create jobs
Follow a family tradition
Gain high social status
If there are any reasons unlisted above for why you would start a business, please list them
here. (STARTBUS2)
The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education
Volume 2, Number 2 Special Issue2011 53
Please rate your level of agreement with the following: I would NOT start a business in
order to… (NOTSTART)
Strongly
disagree Disagree Undecided Agree
Strongly
agree
Lack of ideas regarding
what business to start
Lack of assistance
available to assess
business viability
Excessively risky
Lack of initial capital for
start-up
Lack of legal assistance
or counseling
Lack of knowledge of the
business world and the
market
Lack of experience in
management and finance
Current economic
situation
Irregular income
Lack of support from
people around me
(family, friends, etc)
Fear of failure
Doubts about personal
abilities
Having to work too many
hours
Problems with employees
and colleagues
If there are any reasons unlisted above for why you would start a business, please list them
here. (NOTSTART2)
N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi
The Journal of Engineering Entrepreneurship 54
Please rate your level of knowledge or skill in the following areas related to
entrepreneurship. (FAMILIAR)
Poor Below
average Average
Above
average Excellent
Characteristics of entrepreneurs
Role of entrepreneurship in the
world economy
Business ethics
Risk management
Legal structures for ventures
Intrapreneurship
Social entrepreneurship
Leadership
Managing teams
Project management
Negotiation
Product development
Product life cycle
Economies of scale
Feasibility study
Prototype
Intellectual property
Technology commercialization
Patents
Finance and accounting
Venture capital
Equity
Company valuation
Balance sheet
Income statement
Break even
Market research
Competitive analysis
Target market
Product positioning
Product distribution
Advertising and promotion
Sales and selling
Executive summary
Business plan
Business models
Business incubator
The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education
Volume 2, Number 2 Special Issue2011 55
For each statement select a number from 0 (0% Not at all confident) to 10 (100%
Completely confident) to indicate how confident you are that you could perform that skill
or ability now. (EFFICACY)
0 1 2 3 4 5 6 7 8 9 10
Know the steps needed to place a
financial value on a new business
venture
Pick the right marketing approach
for the introduction of a new service
Work with a supplier to get better
prices to help a venture become
successful
Estimate accurately the costs of
running a new project
Recognize when an idea is good
enough to support a major business
venture
Recruit the right employees for a
new project or venture
Convince a customer or client to try
a new product for the first time
Write a clear and complete business
plan
Convert a useful scientific advance
into a practical application
Develop your own original
hypothesis and a research plan to
test it
Grasp the concept and limits of a
technology well enough to see the
best ways to use it
Design and build something new
that performs very close to your
design specifications
Lead a technical team developing a
new product to a successful result
Understand exactly what is new and
important in a groundbreaking
theoretical article
Translate user needs into
requirements for a design so well
that users will like the outcome
Overall, how would you rate your entrepreneurial ability? (ABILITY)
Poor
Below average
Average
Above average
Excellent
N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi
The Journal of Engineering Entrepreneurship 56
How would you rate your ability to start a business now? (BUSABILITY)
Poor
Below average
Average
Above average
Excellent
Rate your skill levels in the following areas: (SKILLS)
Poor Below
average Average
Above
average Excellent
Communication skills
Presentation skills
Analytical skills
Ability to evaluate business
ideas
Level of risk tolerance
Ability to deal with
uncertainty
Please enter your first name. (FIRSTNAME)
Please enter your last name. (LASTNAME)
Please enter your university assigned email address. (EMAIL1)
Please list an alternate email address, if applicable (optional). (EMAIL2)
Are you willing to be contacted for a follow-up interview? (FOLLOWUP)
Yes
No