Background
The NC State Alumni Survey of baccalaureate alumni who graduated
from December 1990 through August 1993 sought among other things
to identify the extent to which bachelor's alumni were satisfied
that they had received quality training in professional skills
and abilities at NC State. This report, the third in a series
of reports on data gleaned from the survey, focuses on how all
responding alumni (n = 3,179) rated items concerned with the importance
of various skills and abilities, the extent to which alumni believed
that their undergraduate education at NC State had provided them
with those skills and abilities, and the differences between those
two ratings. The differences between preparation and importance
ratings are called gap scores; they are used to measure the extent
of any disjuncture between perceptions of alumni preparation and
workplace requirements. Mathematically, this may be expressed
by the formula:
Communication Skills
Preparation in all communication areas was judged to be between
average and good by respondents, a positive finding. Fairly wide
negative gap scores are evident on communication skills overall
and on listening skills. This finding may be interpreted
as an indication that alumni judge the level of importance of
overall communication skills and listening skills in the workplace
they experience to be greater than the level of preparation they
received at NC State in these areas.
Technical Skills
Judging from the minimal gap scores, the importance of technical
skills and the degree to which responding alumni perceived they
had been prepared at NC State were quite similar. This can be
interpreted to mean that, overall, NC State graduates perceived
they are adequately prepared in technical skills relative to the
importance of these skills in the workplace. When disaggregated
to the college level, substantial variation by college became
apparent on overall technical knowledge. The gap score
on this item for the School of Design was negative and fairly
large.
Computer Skills
Overall, respondents indicated they had received average preparation
in computer skills at NC State. When disaggregated by college,
substantial variation is evident. Concerning overall knowledge
of computer applications and basic computer skills, the
gap scores were negative and fairly large for three colleges (Management,
Agriculture and Life Sciences, and Humanities and Social Sciences.
For the School of Design, the gap score was negative and quite
large on both items. These findings indicate an uneven level
of preparation among NC State alumni by college. The sizable
gap scores found on these two items for some schools or colleges
on overall knowledge of computer applications and basic
computer skills reinforces the notion that student access
to and understanding of computers and computing technology will
continue to require substantial attention if NC State is to retain
its competitive edge in attracting, retaining, and graduating
highly qualified students.
Workplace Skills
All items in this area were rated between important and very important
by responding alumni; preparation was judged as average to good.
For leadership and management skills, a fairly large negative
gap score was evident. This may be taken as an indication that
alumni judge the importance of leadership and management skills
in the workplace to be substantially greater than the level of
preparation they received at NC State.
Professional Traits, Skills, and Attitudes
Items in this area were rated as important by responding alumni, although ability to work with persons from diverse cultural and ethnic backgrounds was rated the lowest in this group. Preparation in this area was rated between average and good. Judging from the magnitude and negative direction of the gap scores, it is evident that alumni perceive the level of importance of three areas in particular to be greater than the level of preparation they received at NC State: professionalism, confidence in your ability to perform well and ability to grow on the job. This last item, in particular, is cause for concern since it is a component skill of lifelong learning.
Return to Table of Contents
Return to OIRP Survey Page
Introduction
Background
The NC State Alumni Survey of baccalaureate alumni who graduated
from December 1990 through August 1993 sought among other things
to identify the extent to which bachelor's alumni were satisfied
that they had received quality training in professional skills
and abilities at NC State. This report, the third in a series
of reports on data gleaned from the survey, focuses on how alumni
responded to questions concerned with the importance of various
skills and abilities, the extent to which alumni believed that
their undergraduate education at NC State had provided them with
those skills and abilities, and the differences between those
two ratings.
Sample
Data obtained from this survey represent the largest and most
comprehensive effort in the history of NC State to gain feedback
from alumni. The survey sample included 7,491 alumni. Three full
mailings and a telephone follow-up resulted in a 51.2% response
rate, yielding 3,179 usable questionnaires.
Quality assurance
Data quality assurance steps taken yielded the following findings:
Rating scales used
All items were rated by respondents on five-point scales. For
items assessing the importance of various areas, the scales
ranged from 5 = very important to 1 = not important. For items
assessing the level of preparation in each area, the scales
ranged from 5 = excellent preparation to 1 = poor preparation.
Responses in the "not applicable" category were omitted
in calculation of the mean score for both importance and preparation
items. In order to facilitate interpretation of the results of
this survey, similar items are grouped together in the discussion
below.
Interpretation of rating scales
In this report, three distinct pieces of information are presented
on each survey item concerned with professional preparation.
First, mean ratings of the relative importance of each area to
graduates' jobs are presented. Next, mean ratings of the quality
of preparation received in each area are shown. For items disaggregated
to the college/school level, tests of significance were performed
on these means; significant departures from the university-level
means are noted. Finally, the means of observed differences between
levels of importance and levels of preparation are presented.
These means of observed differences are labeled "gap scores"
and are included to provide an indicator of the size and direction
of areas for improvement (indicated by a negative gap score) or
areas in which level of preparation exceeds perceived workplace
requirements (indicated by a positive score). T-tests were conducted
to determine whether or not these gap scores were significantly
different from zero; significant gap scores are noted in the tables.
When viewing the results of this survey, it is important to remember
that all three elements should be considered - importance,
preparation, and gap scores - before arriving at a judgment as
to the correct interpretation of the findings.
As a guide to interpreting the relevance of the gap score, in
any instances where the gap score is larger than -1.0, the item
most probably merits further investigation. In other cases, it
may be useful to examine the standard deviations for both importance
and preparation. In situations where the gap score is negative
and larger than the standard deviation of either the importance
or preparation rating, further investigation may be warranted.
In situations where the gap score for a particular item is negative
and larger than the standard deviation for both importance
and preparation ratings, a real difference between importance
and preparation is implied and further investigation is indeed
warranted. Note: due to rounding and to the differing number
of respondents to items on importance and preparation, mean gap
scores may not exactly total the difference between the mean importance
and mean preparation ratings.
Communication skills were judged by survey respondents to be of
high importance to their current positions. The one exception
was foreign language skills, which was judged to be of
much lower importance. Among specific dimensions of the communication
process, listening skills received a notably high importance
rating. Preparation in all communication areas was judged to
be between average and good by respondents. Preparation in reading
skills received the highest rating (3.65), while preparation
in foreign language skills (2.98) was rated lower relative
to other communication dimensions. These results are given in
table 1.
| |||||
Communication skills overall | |||||
Written communication skills | |||||
Public speaking and presentation skills | |||||
Reading skills | |||||
Listening skills | |||||
Foreign language skills |
To permit a visual comparison of these findings, ratings of importance
and preparation are displayed in chart 1.
The large positive gap score for foreign language skills
is a result of low ratings on importance and much higher ratings
on preparation. Fairly wide negative gaps are evident on communication
skills overall and on listening skills. This finding
may be interpreted as an indication that alumni judge the level
of importance of overall communication skills and listening skills
in the workplace they experience to be greater than the level
of preparation they received at NC State in these areas. Listening
skills may be thought of as the ability to receive and understand
information transmitted orally, whether from clients, co-workers,
or supervisors.
To a land-grant research university, the importance of undergraduates
acquiring a thorough grounding in technical skills cannot be overemphasized.
Three items designed to capture alumni estimates of importance
and preparation in these skills were included on the survey instrument:
overall technical knowledge, ability to apply scientific
principles, and ability to apply mathematical skills.
Of these, overall technical knowledge received the highest
rating in terms of importance (4.04) and ability to apply math
skills received the highest rating in terms of preparation
(3.83). These results are displayed in table 2.
Item |
|||||
Overall technical knowledge | |||||
Ability to apply scientific principles | |||||
Ability to apply mathematical skills |
To permit a visual comparison of these findings, ratings of importance
and preparation in technical skills are displayed in chart 2.
Judging from the minimal gap scores, the importance of technical skills and the degree to which responding alumni perceived they had been prepared at NC State were quite similar. However, the large standard deviations on these items should also be taken into account when interpreting the ratings. Not only is there a wide variance in how important survey respondents perceived technical skills to be, but there is a similarly wide variance in the perception among respondents as to how well they were prepared at NC State. To facilitate an understanding of these ratings, a further analysis by college was performed on all three items; this is set forth in tables 2a - 2c.
Overall Technical Knowledge | |||||
Agriculture and Life Sciences | |||||
Design | |||||
Education and Psychology | |||||
Engineering | |||||
Forest Resources | |||||
Humanities and Social Sciences | |||||
Physical and Mathematical Sciences | |||||
Textiles | |||||
Management | |||||
University Level |
Results displayed in table 2a support the notion that substantial
variation exists by college with regard to importance and preparation
in overall technical knowledge. Negative gap scores are largest
on this question for the School of Design and smallest for the
Colleges of Textiles and Engineering. A large negative gap score
on this item indicates that responding alumni reported a discrepancy
between the overall technical preparation they received at NC
State and the importance of technical knowledge overall in the
workplace they experience.
Ability to Apply Scientific Principles | |||||
Agriculture and Life Sciences | |||||
Design | |||||
Education and Psychology | |||||
Engineering | |||||
Forest Resources | |||||
Humanities and Social Sciences | |||||
Physical and Mathematical Sciences | |||||
Textiles | |||||
Management | |||||
University Level |
Table 2b clearly shows that respondents' judgments on the importance
and preparation in ability to apply scientific principles varies
somewhat by college. On this item, most colleges display positive
gap scores; no significant negative gap scores were found. This
indicates that respondents felt rather better prepared in their
ability to apply scientific principles than was demanded of them
in the workplace.
Ability to Apply Mathematical Skills | |||||
Agriculture and Life Sciences | |||||
Design | |||||
Education and Psychology | |||||
Engineering | |||||
Forest Resources | |||||
Humanities and Social Sciences | |||||
Physical and Mathematical Sciences | |||||
Textiles | |||||
Management | |||||
University Level |
Alumni survey results displayed in table 2c confirm that little
disparity exists by college on respondents' ratings of the importance
and preparation in ability to apply mathematics skills. Gap scores
were either positive or not significantly different from zero
on this item by college. Again, this indicates that respondents
felt somewhat better prepared in their ability to apply mathematics
skills than was demanded of them in the workplace.
Three items on the alumni survey were concerned with computer
skills: overall knowledge of computer applications, basic
computer skills (word processing, etc.), and technical
computer skills (programming, etc.). Basic computer skills
were rated the most important of these three items, and respondents
judged their preparation about equal in overall knowledge of
computer applications and in basic computer skills.
These findings are given in table 3.
Item |
|||||
Overall knowledge of computer applications |
| ||||
Basic computer skills | |||||
Technical computer skills |
To permit a visual comparison of these findings, ratings of importance
and preparation in computer skills are displayed in chart 3.
To facilitate an understanding of these ratings, a further analysis by college was performed on all three of these items; this is set forth in tables 3a - 3c.
Agriculture and Life Sciences | |||||
Design | |||||
Education and Psychology | |||||
Engineering | |||||
Forest Resources | |||||
Humanities and Social Sciences | |||||
Physical and Mathematical Sciences | |||||
Textiles | |||||
Management | |||||
University Level |
Results displayed in table 3a clearly demonstrate a differential
level of preparation by college. For three colleges, Management,
Agriculture and Life Sciences, and Humanities and Social Sciences,
the gap scores exceed -1.00 and demonstrate an area of concern.
For the School of Design, the gap score of -2.20 is especially
noteworthy and is cause for in-depth investigation.
Basic Computer Skills | |||||
Agriculture and Life Sciences | |||||
Design | |||||
Education and Psychology | |||||
Engineering | |||||
Forest Resources | |||||
Humanities and Social Sciences | |||||
Physical and Mathematical Sciences | |||||
Textiles | |||||
Management | |||||
University Level |
Table 3b clearly shows that, as in table 3a, substantial variation
exists by college with regard to basic computer skills (such as
word processing, spreadsheets, etc.). Gap scores exceed -1.00
for four schools and colleges: Design, Management, Agriculture
and Life Sciences, and Humanities and Social Sciences. Again,
this finding is cause for concern about the level of preparedness
of NC State graduates for dealing with technology in the workplace.
Technical Computer Skills | |||||
Agriculture and Life Sciences | |||||
Design | |||||
Education and Psychology | |||||
Engineering | |||||
Forest Resources | |||||
Humanities and Social Sciences | |||||
Physical and Mathematical Sciences | |||||
Textiles | |||||
Management | |||||
University Level |
Table 3c demonstrates little variation among colleges with regard
to importance and preparation in technical computer skills, and
little discrepancy for most colleges and schools between importance
and preparation ratings. One important exception is the School
of Design, where a substantial gap score exists for this item.
Given the highly technical graphics and design packages in current
workplace use within Design disciplines, this is a special cause
for concern.
A recent authoritative national poll of students conducted by
the Art and Science Group, Inc. (1997) found that "a majority
of the [new student] market rates the quality of a college's computer
facilities as very important in college choice - equal in importance
to a college's science and research laboratories" (p. 10),
and that "access to e-mail, the Internet and the software
required for academic work in college, among other computer resources,
is very important to a high percentage of students" (p.15).
The sizable gap scores found for some schools or colleges on
overall computer skills and basic computer skills
reinforces the notion that student access to and understanding
of computers and computing technology will continue to require
substantial attention if NC State is to retain its competitive
edge in attracting, retaining, and graduating highly qualified
students.
Exploring those skills and abilities that connect the academy
to the world of work was a major theme of the NC State Alumni
Survey, and a number of survey items were designed with that in
mind. Among these dimensions were the ability to work in teams,
leadership and management skills, and various ways in which
alumni might be expected to apply their learned skills in project
planning and problem solving activities. All items in this area
were rated between important and very important by survey respondents;
items concerned with the importance of problem solving abilities
were rated especially high. Preparation in these areas was judged
overall between average and good. Preparation in areas concerned
with solving problems achieved the highest ratings, whereas preparation
in leadership and management skills was rated the lowest of the
skill sets included in these items. These results are presented
in Table 4.
| |||||
Leadership and management skills | |||||
Ability to work in teams | |||||
Using knowledge to solve problems overall | |||||
Planning projects | |||||
Defining problems | |||||
Solving problems | |||||
Thinking creatively | |||||
Bringing information/ideas together from different areas |
|
To permit a visual comparison of these findings, ratings of importance
and preparation in workplace skills are displayed in chart 4.
Most items in this group display gap scores that, while significant,
are smaller than the standard deviations of importance and preparation
ratings and as such are not particular areas for concern. However,
a large negative gap score is evident for leadership and management
skills. This may be interpreted as an indication that alumni judge
the importance of leadership and management skills in the workplace
to be substantially greater than the level of preparation they
received at NC State in this area, and argues for more specific
exposure to leadership and management concepts and practices in
undergraduate curricula.
Professional traits, skills and attitudes
Continuing the theme of exploring commonalities between NC State
and the world of work, another broad group of sixteen items was
designed to elicit information on professional traits, work attitudes
and skills, and professional development. All importance items
were rated between important and very important by respondents,
although ability to work with persons from diverse ethnic and
cultural backgrounds received the lowest rating (4.02) among
this group. Items concerned with the quality of preparation at
NC State were rated between average and good with the exception
of the ability to work independently, which was rated as
good (4.06).
Traits overall | |||||
Professionalism | |||||
Conducting work activities in an ethical manner |
| ||||
Resourcefulness | |||||
Confidence in your ability to perform well | |||||
Work attitudes and skills overall | |||||
Ability to adjust to new job demands | |||||
Working under pressure | |||||
Making decisions under pressure | |||||
Ability to work independently | |||||
Ability to work with persons from diverse ethnic and cultural backgrounds |
| ||||
Being dependable and punctual | |||||
Professional development overall | |||||
Ability to learn independently | |||||
Ability to grow on the job | |||||
Willingness to accept new responsibilities |
To permit a visual comparison of these findings, ratings of importance
and preparation in professional skills, traits and attitudes are
displayed in chart 5.
Judging from the magnitude of the gap score, it is evident that
alumni perceive the level of importance of three areas in particular
to be greater than the level of preparation they received at NC
State: professionalism (gap score = -1.00), ability
to grow on the job (gap score = -0.96), and confidence
in your ability to perform well. Professionalism may
be thought of as the ability to meet or exceed professional standards
of conduct and practice in the workplace, and confidence in
your ability to perform well serves as a measure of the assurance
an alum perceives relative to the quality of their overall preparation.
Ability to grow on the job is one of the component abilities
of being a lifelong learner, and as such is an important outcome
indicator for NC State.
Return to Table of Contents
Return to OIRP Survey Page
Return to OIRP Home Page