Results of the National Survey of Student Engagement

A Brief Report Submitted to the Faculty and Staff of St. Ambrose University
by the St. Ambrose University Assessment Committee 

Regular and ad hoc Assessment Committee members: Greg Conderman, Brenda DuBois, Ann Freeberg, Martin Hansen, Paul Koch, Rich Legg, Brenda Peters, Ann Preston, Lew Sanborne, and Jim Van Speybroeck 

Method: During the spring 2001 semester, the National Survey of Student Engagement (NSSE) was administered to both first-year and senior students at St. Ambrose University (see the home of NSSE for more complete information).  Using data provided by St. Ambrose University, a random sample of first-year and senior students was selected for inclusion in the sample.  Students had the opportunity to respond to the survey either by mail or by filling out the survey on the web.  The response rate from St. Ambrose was approximately 47% (n = 220), which was well above the national average for response rates to this survey. 

 Results: Results were presented to St. Ambrose in three ways: 1) a means report which compared our students’ response frequencies to those of other students at Master’s institutions (we are classified as a Master’s I institution using the Carnegie classification system), 2) an institutional benchmark report examining 5 benchmarks developed by collapsing individual questions into new variables, and 3) an institutional engagement index, which attempts to correct benchmark scores based on institutional characteristics. 

 You are strongly encouraged to visit the following St. Ambrose web sites to examine more complete results from the NSSE for St. Ambrose.

• general information on the NSSE and the benchmark scores for St. Ambrose, and

• means report for individual questions for St. Ambrose.

• Summaries of the results will also be placed on reserve at the library.

It must be cautioned that these results represent a snapshot of our first-year and graduating students at a particular point in time.  Interpretation of these data (both those items that we see as positive and those that pose a concern for us) must occur within this context.  Never the less, these results deserve serious discussion among the members of the campus community.  Initial plans are to discuss these results with department chairs and program directors this spring.  Additional analyses will also be conducted on the data in an attempt to increase explanatory power. 

 The Assessment Committee is currently discussing whether the NSSE should be part of a regular assessment cycle so that we can determine the usefulness of responses over time and where changes have or have not occurred.

Means report

             The means report provides information on individual questions, and you are encouraged to visit the SAU web site mentioned above.  The following information summarizes only where there were significant differences between SAU and other Master’s institutions for either first-year or senior students.  There were many more items where significant differences did not exist between SAU and other Master’s institutions; these items can viewed on the web site mentioned above.  The existence of differences between SAU and other Master’s institutions, or a lack of such differences, must be evaluated within the local context to determine whether action is necessary.  Unless otherwise indicated, higher scores are preferable.  

Academic, Intellectual, and Social Experiences (1 = never, 2 = sometimes, 3 = often, 4 = very often)

-Came to class without completing readings or assignments: (1^st-year students) SAU = 1.85, Master’s Institutions = 2.06 (lower scores are preferable on this item), no difference for seniors

-Worked with other students on projects outside of class to prepare class assignments: (1^st-year students) SAU = 2.07, Master’s Institutions = 2.29, no differences for seniors

-Discussed ideas from your readings with others outside of class: (1^st-year students) SAU = 2.39, Master’s Institutions = 2.68, no differences for seniors

-Had serious conversations with students of a different race or ethnicity than your: (1^st-year students) SAU = 2.19, Master’s Institutions = 2.54, (Seniors) SAU = 2.26, Master’s Institutions = 2.56

-Had serious conversations with students who differ from you in terms of their religious believes, …, or personal values: (1^st-year students) SAU = 2.46, Master’s Institutions = 2.80, no differences for seniors

 Mental Activities (1 = very little, 2 = some, 3 = quite a bit, 4 = very much)

-Synthesizing and organizing ideas, information, or experiences into new, more complex interpretations and relationships: (1^st-year students) SAU = 2.50, Master’s Institutions = 2.76, no differences for seniors

 Quality of Advising (larger numbers are “better”)

-Overall how would you evaluate the quality of academic advising you have received at your institution?  (1^st-year students) SAU = 3.21, Master’s Institutions = 2.91, (Seniors) SAU = 3.13, Master’s Institutions = 2.82

 Time Usage (1 = 0 hours/week, …, 8 = more than 30 hours/week)

-Prepared for class (studying, reading, …, and other activities related to your academic program): (1^st-year students) SAU = 3.57, Master’s Institutions = 3.82, no differences for seniors

-Working for pay off campus: (1^st-year students) SAU = 3.65, Master’s Institutions = 2.76, (Seniors) SAU = 4.56, Master’s Institutions = 4.10

 Institutional Emphasis (1 – very little, 2 = some, 3 = quite a bit, 4 = very much)

-Provide the support you need to help you succeed academically: (Seniors) SAU = 3.16, Master’s Institutions = 2.85, no differences for first-year students

Quality of Relationships (1 = unfriendly, unsupportive, sense of alienation to 7 = friendly, supportive, sense of belonging)

-Relationships with faculty members: (1^st-year students) SAU = 5.81, Master’s Institutions = 5.39, (Seniors) SAU = 6.02, Master’s Institutions = 5.55

-Relationship with administrative personnel and offices: (Seniors) SAU = 5.03, Master’s Institutions = 4.61, no differences for first-year students

 Satisfaction (1 = poor, 2 = fair, 3 = good, 4 = excellent)

-How would you evaluate your entire educational experience at this institution? (Seniors) SAU = 3.44, Master’s Institutions = 3.21, no differences for first-year students

Benchmark Statistics

The PowerPoint presentation on the SAU web site mentioned above provides more detailed information on the benchmark statistics and how they are defined, and you are strongly encouraged to visit this site (generally, higher scores should be seen as more positive outcomes).

             The Institutional Engagement Index (IEI) provides an alternative view to institutional performance.  The IEI predicts what our benchmark statistics should be while controlling for institutional characteristics such as the types of students that attend SAU as well as other institutional characteristics.  St. Ambrose’s actual scores were generally lower (except for Supportive Campus Environment) than those predicted based on the IEI model.  When an analysis of the differences between the actual and predicted scores (residuals) was conducted, St. Ambrose tended to fall in the lower percentiles (the percent of schools at or below a particular standardized residual score).  The analysis of these standardized residuals gave an estimate of the degree to which St. Ambrose exceeded or fell short of its predicted score on each benchmark relative to all other NSSE institutions.  If our score fell below the predicted score it would lead to a percentile less than 50%, and if our score fell above the predicted score it would lead to a percentile above 50%.  It should be emphasized that the percentiles are relative to a predicted score based on the model, and not relative to some absolute standard or criterion.  The percentiles are in parentheses after the actual scores.