Student self-assessment, diagnosis and referral with the Self-Assessment for Success Screening Inventory

The SASSI Instructor''''s Manual

Raymond A. Launier

Santa Barbara City College

interpersonal strengths and work habits. The AMP is a psychometrically sound instrument with good reliability and validity. However, both the AMP and the CSFI focus on engrained personality characteristics of students. This focus on traits makes them less relevant to identifying changeable behaviors related to student success. A review of 16 other assessment instruments revealed similar shortcomings: either focused too much on personality characteristics, or not sufficiently focused on behaviors perceived by faculty as especially relevant to student success.

For the purposes of promoting student success through assessment and feedback, a practical shortcoming of these instruments lies with the fact that the instruments are lengthy and time-consuming to administer in class, require extensive and ongoing staff support and financial resources to purchase, administer, score and interpret the results to the students. These instruments are limited in the extent to which they address the assessment criteria of key academic, motivational and situational factors related to student success identified by faculty. Nor do the existing instruments provide self-diagnosis and self-referral to relevant student services or campus resources. Accordingly, this report summarizes the progress made and results obtained in developing a more useful instrument for self-assessment, self-diagnosis and self-referral in the promotion of student success.

Method

Initial Scale Construction and Pilot Testing

Procedures

Items in the initial Self-Assessment for Success (SAS) scale were derived from the 1996 SBCC Report on student preparedness in which faculty reported wide ranging deficiencies shared by a large percentage of students. Fifteen items were written for the SAS scale and were grouped into three areas: basic literacy skills, key college success skills, and attitude and commitment to education. In addition, information was gathered about gender, course units carried, hours worked, and grade point average.

Psychometric scale development seeks to satisfy four criteria: ease and clarity in administration, reliability, validity and utility. The SAS is self-administered and takes about ten minutes to complete. Feedback from students about ease and clarity of use was largely positive with some suggestions for refinement. The test-retest reliability (r_s = .76, p < .001, N = 112) of the SAS scale was calculated by re-administering the scale to students in a general psychology class with a two-week interval between administrations.

The predictive validity of the SAS scale scores was determined through their correlation with measures of academic performance and grades. Higher scores on the SAS scale indicate greater amounts of self-reported academic difficulties. Data combined from several psychology classes generated a negative correlation coefficient (r = -.32, p <.001, N = 152) between student scores on the midterm exam and total SAS scores. For those students who reported their G.P.A. (about 65%) in general psychology, philosophy and political science classes, a negative correlation coefficient (r = -.54, p < .001, N = 216) was obtained between G.P.A. and total SAS scores.

To the extent that college students can assess their own level of preparedness, compare their individual results with college-wide norms, and reflect upon the relationships between performance, motivation and success, then students have a basis and guidelines for seeking assistance in the particular areas indicated. These preliminary findings showed that the SAS scale could be easily self-administered and self-scored, and that it provides reliable, valid and useful results.

The SAS scale was revised based on results and recommendations from students and colleagues in the pilot testing stage. Instructions for using the scale were improved. The items were rewritten to improve clarity and specificity. A 5-point scale was substituted for the 4-point scale for the rating of each item, to improve the interpretive and psychometric properties of the scale. An open-ended question was included: "What else or what interferes most with your success at SBCC?" Demographic information was solicited at the end instead of at the beginning of the scale. Finally, self-diagnostic guidelines were included, together with a detailed listing of college resources for self-referral purposes.

Participants

The revised scale was administered to a random sample of students from GE (general education) courses in the Social Sciences. To draw a representative sample of Social Science Division students, sixteen classes (two from each Department) were randomly selected from all the GE classes offered in the Social Sciences Division. The students who participated in the survey were drawn from classes in anthropology, communications, ethnic studies, history, philosophy, political science, psychology and sociology. The sample is representative of beginning general education students who are often at higher risk of failing than the more focused and mature students. The sample represents the target population of particular concern to faculty and administration.

Initial Results

Overall, the total SAS scale score bore a significant negative correlation (r = -.46, p < .001, N = 569) with GPA. Students who reported having more problems also reported lower GPAs. All items in the SAS scale were significantly correlated with self-reported GPA, but more so for male than for female students. Time management was a significant problem for 50.6% of the men, and for 44.6% of the women. In relation to part time jobs, 53.6% of the men were working 20 or more hours per week; 44% of the women were as well. Attending classes was a problem for 22.2% of the men, and or 11.1% of the women. For 18% of the men and for 6.6% of the women, commitment and motivation to succeed in college was a third order or an even less significant priority. This is further evident in the finding that 18.8% of the men and 8.7% of the women report watching on average 4 or more hours of television per day. Moreover, 36.8% of the men, 19.4% of the women, report that for every hour in class they only put in an additional 1/2 hour or less of time for study and assignments. Although work and time management problems appear to be key contributors to impaired academic success, other factors are also involved. Personal problems were significant for 32.8% of the men and for 20.5% of the women. However, both men (54.2%) and women (44.8%) reported that it was difficult and problematic to seek out and get help when they needed it. Close to 30% of both men and women reported having problems in the more traditional areas of academic performance: reading class assignments; participating actively in class; memorizing course-related material. When averaged across the 15 areas assessed by the SAS scale, 31.1% of the men and 24.3% of the women report having problems in areas directly related to college success.

Final Scale Revision: The Self-Assessment for Success Screening Inventory

Procedures

Prior to conducting a survey with a more representative sample of SBCC students, a final revision of the SAS instrument was made. Based on additional suggestions from SBCC faculty, and with a desire to increase the predictive validity of the instrument, four additional items were added. These included items related to high school grades, degree of participation in cooperative learning, frequency of intoxication, and overall satisfaction with the college experience. Items were also weighted to reflect their relative strength of relationship with GPA in the calculation of the total score. Questions were included to assess demographic characteristics, such as age, gender, amount of part-time work, current course load, major, semester at college, educational goals and language of childhood. The revised 19-item instrument was renamed: the Self-Assessment for Success Screening Inventory – SASSI.

Participants

To conduct a college-wide survey, a random sample of courses offered at SBCC during the following Fall semester was selected. The 51 introductory 100 or 101 level courses offered were listed in alphabetical order and numbered. A random numbers table was used to select 20 courses for administering the SASSI instrument. To represent more advanced students, the 20 advanced 200 series courses were also listed and from which 10 courses were randomly selected. Solicitation for instructor assistance, guidelines for the administration of the instrument, and a packet of survey forms were forwarded to the instructors whose courses had been selected. Instructions included a request to administer the survey within two weeks following midterms. Seventy percent of the instructors administered and returned the surveys.

In the sample of 784 students surveyed, 55.5% were female. Ages ranged from 16 to 61 with an average age of 22.8, median of 20 and mode of 19. Seventy-four percent of the students worked part-time: 36% less and 38% more than 20 hours a week. English was a second language for 16.5% of the students; 41.5% were in the first or second semester; 65% had goals of pursuing a four year degree or higher. Student majors were represented as follows: 13.5% were in the Health and Human Services Division, 14.7% were in the English/Communications Division, 5.4% were in the Fine Arts Division, 5.7% were in the Technologies Division, 4.1% were in the Business Education Division, 51.4% were in the Social Sciences Division, and 5.2% were in the Sciences Division.

Final Results

The results from this survey are presented in greater detail in Tables 2 and 3. Table 2 presents a summary of the percentage of responses in each scale level of the 19 SASSI items. For instance, with regard to item 14 focused on the amount of time spent studying outside of class, 40.1% put in two to three hours for each hour of class time; 26.9% put in a half hour or less. Managing time is problematic or worse for 42.5% of the students. Information provided in Table 2 serves as a base rate on the norms of student behavior and experience. This base rate information can be useful to students when comparing their own self-assessment to the norms of student behavior.

When the SASSI items are divided into problematic and non-problematic categories and then averaged across the 19 items, then one can conclude that on average 75% of our students are able to manage the demands and challenges of SBCC courses relatively free of serious problems and/or difficulties. Alternatively, 25% are in another boat, floundering and in need of attention. Moreover, 9.2% are having a very difficult time and in danger of failing.

Table 3 provides a summary of significant gender differences in the amount and kind of problems students do experience. With the exception of math and computational assignments in which women report significantly more difficulties, men report doing significantly worse in 11 of the 19 areas assessed. On average, men watch more television, get intoxicated more often, have more difficulty managing time, participate less in class, attend classes more poorly, study less and are less likely to seek help.

Table 3 also summarizes the correlations between GPA, total SASSI score and the individual scale items. For the sample as a whole, all 19 items were significantly related to GPA, four of which were significant for men but not for women. The correlation (r = -.53, p < .001,

N = 784) between the total SASSI score and GPA is considerably stronger than the correlation of .35 reported with the College Factors Success Index. For males, the multiple R of -.67 indicates that 45% of the variance in GPA is associated with variations in SASSI assessed student behavior. Of the 19 items assessed, four were particularly strong predictors of GPA: math difficulties, high school grades, time management and attendance.

Demographic and situational factors were associated with some of the variation in student SASSI scores and GPA. The following differences were statistically significant, based on t-test comparison of independent means with alpha set at .01 unless otherwise indicated. The average SASSI score for men was 56.3, and 51.7 for women, which is also associated with a slightly higher average GPA of 3.13 for women compared to 3.05 for men (p. <.05). Men also reported significantly lower levels of satisfaction with the college experience. Age was significantly correlated with SASSI and GPA. The average SASSI score was higher for students under twenty-one (55) compared to (52.4) for those twenty-one or older. Older students also had a slightly higher GPA of 3.17 compared to 3.00 for those under twenty-one. Younger students were also less satisfied.

Students for whom English is a second language had lower SASSI scores but otherwise were the same in average GPA and satisfaction with the college experience. The amount of student part-time work did not correlate with GPA or with SASSI scores. The only significant difference between those working part-time and those who don''t was a lower level of satisfaction with college for those who work 20 or more hours per week. Similarly, levels of parental education did not correlate nor distinguish students with regard to SASSI scores nor GPA. The educational goals of the students did make a difference. Those with higher educational goals scored lower in SASSI and higher in GPA.

Discussion: Use of the SASSI

Self-assessment, self-diagnosis and self-referral were design criteria in developing the SASSI instrument. The normative and correlational results from this survey show that particular student behaviors itemized in the SASSI instrument are significantly and strongly associated with student success. The predictive validity of the 19-item SASSI is considerably stronger than the 80-item College Factors Success Index, associated respectively with 45% versus 12% of the variance in GPA.

Instructors and counselors can easily use the SASSI instrument in classes and groups. This scale can be administered and completed in class in approximately 10 minutes. The SASSI scale items are presented in a single page. The self-diagnostic interpretive and self-referral guides are provided on the backside to the instrument. For those students whose scores are indicative of difficulties, resources and services available to students on campus specific to areas in which attention is warranted are described and listed. A specimen sample of the SASSI is presented at the end of this report. This referral and student service information can be modified to fit the specific situation and resources of other college campuses.

There are several limitations to the SASSI instrument. One, the norms are derived from one community college. SBCC may or may not be representative of other community colleges or of four-year colleges and universities. Re-norming the instrument on other campuses may well produce self-diagnostic guidelines that are more valid for students at each campus. Similarly, the self-referral guidelines to student services and resources need to be campus specific. Finally, as a screening inventory, the SASSI relies upon self-reported problems. It does not provide detailed feedback to the student on best practice behaviors, attitudes and habits to do well in college. To address this last point, the author has developed a more comprehensive self-assessment instrument: The Wellness Inventory of Student Effectiveness-Revised (Launier, 2001).

On a more general note, I believe that it is more empowering to students to provide them with instruments and tools for their use and potential benefit than to provide instruments to those already professionally trained. The developmental tasks for students is to grow into greater self-reliance and independence; to learn to do for themselves; to engage in metacognition and self-awareness; to monitor, correct and direct the course of their own dreams, aspirations and action plans. In this context, the SASSI instrument, based on the contributions of many faculty and students, and as a self-help tool, can contribute to the success of college students.

References

Bracey, G. W. (1997, November). What happened to America''s public schools? American

Heritage.

California League of Community Colleges, (1996). Preparing to Serve the Student of the Future.

Gose, B. (1998, January 16). More freshmen than ever appear disengaged from their studies,

survey finds. The Chronicle of Higher Education.

Hallberg, Hallberg & Sauer, (1993). The College Success Factors Index. Sierra Madre, Ca:

Ombudsman Press.

Launier, R. (1997). Student Success from a S.A.S.S.I. perspective. SBCC Fall 97 survey findings

with the Self-Assessment for Success Screening Inventory. Unpublished report: Santa

Barbara City College.

Launier, R. (1998). An evaluation and comparison of the College Success Factors Index and the

Self-Assessment for Success Inventory to assess and promote student success in college.

Unpublished report: Santa Barbara City College.

Launier, R. (2002). W.I.S.E. Students Succeed. Dubuque, Iowa: Kendall/Hunt Publishing.

Mandel, H. P., Friedland, J. G. & Marcus, S. I. (1996). Achievement Motivation Profile. Los

Angeles: Western Psychological Association.

Santa Barbara City College, (1996). Strategies for Promoting Student Success in Their Courses.