Annual student assessment report

              Oklahoma State System
of
Higher Education
Annual
Student
Assessment
Report
June 23, 2011
OKLAHOMA STATE REGENTS
FOR HIGHER EDUCATION
Joseph L. Parker
Chairman
Tulsa
William Stuart Price Marlin “Ike” Glass, Jr.
Tulsa Secretary
Newkirk
Michael C. Turpin James D. “Jimmy” Harrel
Oklahoma City Assistant Secretary
Leedy
Julie Carson Don Davis
Vice Chair Lawton
Claremore
Ronald H. White John Massey
Oklahoma City Durant
Glen D. Johnson
Chancellor
The Oklahoma State Regents for Higher Education in compliance with Titles VI and VII of the Civil Rights Act of 1964, Executive Order 11236 as amended, Title IX of the Education Amendments of 1972, Americans with Disabilities Act of 1990 and other federal laws do not discriminate on the basis of race, color, national origin, sex, age, religion, handicap, or status as a veteran in any of its policies, practices, or procedures. This includes but is not limited to admissions, employment, financial aid, and educational services.
This publication is issued by the Oklahoma State Regents for Higher Education, as authorized by 70 O.S. 2001, Section 3206. Copies have not been printed but are available through the agency website at www.okhighered.org. Two printout copies have been deposited with the Publications Clearinghouse of the Oklahoma Department of Libraries. ANNUAL STUDENT ASSESSMENT REPORT
Table of Contents
Executive Summary ..................................................................................................................................... 1
Background .................................................................................................................................................. 5
Analysis ....................................................................................................................................................... 8
Entry-Level Assessment .............................................................................................................................. 9
General Education (Mid-Level) Assessment .............................................................................................. 27
Program Outcomes (Exit-Level) Assessment ............................................................................................ .38
Student Satisfaction Assessment ................................................................................................................. 41
Graduate Student Assessment ..................................................................................................................... 45
Licensure and Certification ......................................................................................................................... 46
Assessment Budgets ................................................................................................................................... 51
Tables
Number of Students Enrolled in Remediation by Institution ...................................................................
Secondary Test Cut-Scores by Subject and Institution ............................................................................
APPENDIX
Policy on Assessment ............................................................................................................................... INTENTIONALLY LEFT BLANK
1
Oklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher EducationOklahoma State Regents for Higher Education
ANNUAL STUDENT ASSESSMENT REPORT
2002002008-09
Executive Summary
The fifteenth annual report on student assessment in the Oklahoma State System of Higher Education is presented as required by the State Regents’ policy on “Assessment.” Reports submitted by each institution are provided as an overview of the 2008-09 academic year assessment activities. Background
Oklahoma legislation paved the way for development of a statewide assessment plan in 1991 by allowing institutions to charge students up to one dollar per credit hour to support the student assessment effort. The State Regents’ Assessment Policy was adopted in October 1991 with the purpose of maximizing student success.
The purpose of assessment is to maximize student success. The institutional assessment plan requires the systematic collection, interpretation, and use of information about student learning and achievement to improve instruction. The assessment policy also addresses the need to demonstrate public accountability by providing evidence of institutional effectiveness.
Each institution must evaluate students at four levels (graduate student assessment is optional): Entry-Level Assessment and Course Placement - to determine academic preparation and course placement. General Education (Mid-Level) Assessment - to determine general education competencies in reading, writing, mathematics, and critical thinking. Program Outcomes (Exit-Level) Assessment - to evaluate outcomes in the student's major. Assessment of Student Satisfaction - to ascertain students' perceptions of their educational experiences including support services, academic curriculum, faculty, etc. Graduate Student Assessment - to assess student learning beyond standard admission and graduation requirements and to evaluate student satisfaction.
Institutions submit an annual assessment report to the State Regents, which describes assessment efforts at each of these levels. Information on number of students assessed, results of the assessment, and detailed plans for any institutional and instructional changes due to assessment results are to be provided in the report. Entry-Level Assessment and Placement
The purpose of entry-level assessment is to assist institutional faculty and advisors in making course placement decisions that will give students the best possible chance of academic success. Beginning in fall 1994, the State Regents implemented a required score of 19 on the ACT in the subject areas of English, mathematics, science, and reading as the "first-cut" for entry-level assessment. Students may also demonstrate curricular proficiency by means of an approved secondary assessment process.
Students are enrolled in developmental courses after being unable to demonstrate proficiency in one or more subject areas. These courses are below college-level and are not applied toward degree requirements. A supplementary per credit hour fee is assessed to the student for these courses. 2
As required by policy, institutional assessment plans not only assess the basic academic skills of incoming students for course placement purposes, but also track students to measure their success rate. In addition to measuring basic academic skill competencies, institutions are collecting data on student attitudes and perceptions of college life. Institutions are offering orientation courses, computer-assisted instruction, tutoring, and learning resource centers, all of which are intended to make the initial college experiences both positive and successful. General Education (Mid-Level) Assessment
General education assessment is designed to assess the competencies gained by students in the college general education program. Institutions are required to assess students in the areas of reading, writing, mathematics, and critical thinking. Mid-level assessment normally occurs after completion of 45 semester hours and prior to completion of 70 semester hours. For associate degree programs, mid-level assessment may occur halfway through the program or at the end of the program. More typically, this assessment occurs at the end of the program after students have had sufficient time to develop basic skills.
Assessments at mid-level and in the major academic program provide important information to institutions about the degree to which their general education programs facilitate student achievement of desired knowledge and competencies. Results of this process have led some institutions to redesign their general education programs. The types of courses and delivery methods have been closely examined. Program Outcomes (Exit-Level) Assessment
Program outcomes assessment, or major field of study assessment, is designed to measure how well students are meeting institutionally stated program goals and objectives. As with other levels of assessment, selection of assessment instruments and other parameters (such as target groups, when assessment occurs, etc.) is the responsibility of the institution. Institutions are encouraged to give preference to nationally standardized instruments that supply normative data. The instrument selected should measure skills and abilities specific to the program and to higher level thinking skills. Results are used to revise curricula. Assessment of Student Satisfaction
Student and alumni perceptions are important in the evaluation and enhancement of academic and campus programs and services because they provide an indication of the students' subjective view of events and services, which collectively constitute their undergraduate experiences. Student satisfaction assessment can be accomplished in several ways including, but not limited to, surveys, interviews, and focus groups. The results are used to provide feedback to improve programs and services.
Assessment survey results indicate student satisfaction with the availability and interest of faculty and staff, academic preparation for future occupations, classroom facilities, campus buildings and grounds, class size, libraries, cost, and other services. Common areas of dissatisfaction were food services, course availability, veteran’s services, availability of student housing, job placement assistance, financial aid services, student activity fee uses, and parking.
Changes have been implemented as a result of student feedback. Common changes include upgrades and addition of technology resources to improve academic and administrative services, student access to computers and the Internet, expanded orientation programs, enhanced tutoring services, student activities, food services, and career counseling and placement. New facilities have been constructed and older facilities have been renovated to meet students’ needs. 3
Graduate Student Assessment
Beginning fall 1996, higher education institutions that charge graduate students the student assessment fee must perform assessment beyond the standard requirements for admission to and graduation from a graduate program. All ten universities offering graduate programs (OU, OSU, UCO, ECU, NSU, NWOSU, SEOSU, SWOSU, CU, and LU) reported graduate student assessment activities that include licensure, certification, and comprehensive exams; portfolios; capstone courses; practica; theses; interviews; and surveys. Licensure/Certification Assessment
An important measure of both student achievement and program effectiveness and appropriateness is the professional examination for licensure or certification. This is the second year institutions were asked to provide the number of students taking such examinations and the number of them passing. Assessment Budgets
This is the second year that assessment budgets figures were requested. In compliance with State Regents’ policy regarding the use of fees, it is important to monitor how assessment fees are being allocated for the support of assessment activities. An analysis of assessment budgets are planned for future reports. Analysis
As evidenced by the institutional reports, Oklahoma’s colleges and universities are achieving the two major objectives of student assessment: to improve programs and to provide public accountability. As institutional implementation of student assessment has evolved, continued enhancements and improvements have been documented.
Institutions have improved the process of gathering and using assessment data. Specific days or class times for assessment have been designated to encourage and facilitate student participation in general education and program outcomes testing. Strategies for increasing the response rates to surveys have been evaluated. Assessment results have been integrated into other institutional review processes and shared widely with faculty and students.
Areas of concern include the variance in secondary institutional placement cut-scores for a given instrument. Secondary testing for science is not practiced at all institutions; however, some institutions use a combination of reading and math scores and others use science tests. Also, institutions are using one or more of seven different assessment instruments; this variation diminishes the ability to compare practices across the state or with institutions in other states.
Administration of general education assessment varies in methodology among the state’s higher education institutions with several using locally developed tests. Using nationally-normed exams could provide more consistency and comparison to national benchmarks.
Persistence and graduation rates depend on the ability of a student to succeed not only in higher-level courses, but also globally in the business and industry. Implementation of state-wide outcomes assessments in writing and mathematics could insure that students have the requisite skills to be successful in further education and in the work place. Pass rates of outcomes assessments could be included in the annual student assessment report as a means of monitoring progress and increasing public transparency and accountability. Such assessments also could assist in accreditation. 4
Additional information on related institutional polices and student performance are available in annual reports from the Oklahoma State Regents for Higher Education, including the Annual Student Remediation Report and the High School Indicators Project Reports: Mean ACT Composite Scores; High School to College-Going Rates; Headcount, Semester Hours and GPA; and Remediation Rates.
5
OKLAHOMA STATE REGENTS FOR HIGHER EDUCATION
ANNUAL STUDENT ASSESSMENT REPORT
2002002008-09
The fifteenth annual report on student assessment in the Oklahoma State System of Higher Education is presented as required by the State Regents’ policy on “Assessment.” Reports submitted by each institution are provided as an overview of the 2008-09 academic year assessment activities. Background
Oklahoma legislation paved the way for development of a statewide assessment plan in 1991 by allowing institutions to charge students up to one dollar per credit hour to support the student assessment effort. The State Regents’ Assessment Policy was adopted in October 1991 with the purpose of maximizing student success.
The purpose of assessment is to maximize student success. The institutional assessment plan requires the systematic collection, interpretation, and use of information about student learning and achievement to improve instruction. The assessment policy also addresses the need to demonstrate public accountability by providing evidence of institutional effectiveness.
The policy is a proactive, comprehensive assessment program, which addresses institutional quality and curricular cohesiveness. It is designed so that the results of the assessment efforts will contribute to the institution's strategic planning, budgetary decision-making, institutional marketing, and improving the quality of student services.
Each institution must evaluate students at four levels (graduate student assessment is optional): Entry-Level Assessment and Course Placement - to determine academic preparation and course placement. General Education (Mid-Level) Assessment - to determine general education competencies in reading, writing, mathematics, and critical thinking. Program Outcomes (Exit-Level) Assessment - to evaluate outcomes in the student's major. Assessment of Student Satisfaction - to ascertain students' perceptions of their educational experiences including support services, academic curriculum, faculty, etc. Graduate Student Assessment - to assess student learning beyond standard admission and graduation requirements and to evaluate student satisfaction.
Institutions submit an annual assessment report to the State Regents, which describes assessment efforts at each of these levels. Information on number of students assessed, results of the assessment, and detailed plans for any institutional and instructional changes due to assessment results are to be provided in the report.
Although all institutions currently use the ACT as the first entry-level assessment, testing instruments used for secondary evaluation vary. Commonly selected commercial instruments include the ACT Assessment of Skills for Successful Entry and Transfer (ASSET), the Accuplacer Computerized Placement Test (CPT), ACT Computer-Adaptive Placement and Support System (COMPASS), and the Nelson-Denny Reading Test. Institutionally-developed writing and mathematics tests, as well as a predictive statistical model, are also used. Each institution is responsible for establishing secondary testing cut-scores.
6
As required by policy, institutional assessment plans not only assess the basic academic skills of incoming students for course placement purposes, but also track students to measure their success rate. In addition to measuring basic academic skill competencies, institutions are collecting data on student attitudes and perceptions of college life. Institutions are offering orientation courses, computer-assisted instruction, tutoring, and learning resource centers, all of which are intended to make the initial college experiences both positive and successful. General Education (Mid-Level) Assessment
General education assessment is designed to assess the competencies gained by students in the college general education program. Institutions are required to assess students in the areas of reading, writing, mathematics, and critical thinking. Mid-level assessment normally occurs after completion of 45 semester hours and prior to completion of 70 semester hours. For associate degree programs, mid-level assessment may occur halfway through the program or at the end of the program. More typically, this assessment occurs at the end of the program after students have had sufficient time to develop basic skills.
Mid-level assessment is accomplished with a combination of locally developed and standardized testing instruments such as the ACT Collegiate Assessment of Academic Proficiency (CAAP), the Riverside College Base Academic Subjects Examination (BASE), and the Test of Adult Basic Education (TABE). These nationally validated instruments are useful, because they provide regional or national benchmark data from other participating institutions. Several institutions have developed local instruments for mid-level assessment in some subject areas. More qualitative assessments, such as portfolio assessments and course-embedded techniques, are also being used.
Assessments at mid-level and in the major academic program provide important information to institutions about the degree to which their general education programs facilitate student achievement of desired knowledge and competencies. Results of this process have led some institutions to redesign their general education programs. The types of courses and delivery methods have been closely examined. Program Outcomes (Exit-Level) Assessment
Program outcomes assessment, or major field of study assessment, is designed to measure how well students are meeting institutionally stated program goals and objectives. As with other levels of assessment, selection of assessment instruments and other parameters (such as target groups, when assessment occurs, etc.) is the responsibility of the institution. Institutions are encouraged to give preference to nationally standardized instruments that supply normative data. The instrument selected should measure skills and abilities specific to the program and to higher level thinking skills. Results are used to revise curricula.
Program outcomes assessment methods used by State System institutions are diverse. Faculty members in each academic program or major field of study are responsible for developing their own methods of assessing to what degree students meet stated program goals and objectives. Assessments include structured exit interviews, surveys of graduating seniors and employers, Educational Testing Service’s (ETS) Major Field Assessment Tests (MFAT), national graduate school admission exams (GRE, MCAT, GMAT), the ACT College Outcome Measured Program (COMP), senior projects, portfolios, recitals, national and state licensing exams, internships, capstone courses, theses, transfer GPAs, admission to professional schools, retention rates, and job placement. Assessment of Student Satisfaction
Student and alumni perceptions are important in the evaluation and enhancement of academic and campus 7
programs and services because they provide an indication of the students' subjective view of events and services, which collectively constitute their undergraduate experiences. Student satisfaction assessment can be accomplished in several ways including, but not limited to, surveys, interviews, and focus groups. The results are used to provide feedback to improve programs and services.
Assessment survey results indicate student satisfaction with the availability and interest of faculty and staff, academic preparation for future occupations, classroom facilities, campus buildings and grounds, class size, libraries, cost, and other services. Common areas of dissatisfaction were food services, course availability, veteran’s services, availability of student housing, job placement assistance, financial aid services, student activity fee uses, and parking.
Changes have been implemented as a result of student feedback. Common changes include upgrades and addition of technology resources to improve academic and administrative services, student access to computers and the Internet, expanded orientation programs, enhanced tutoring services, student activities, food services, and career counseling and placement. New facilities have been constructed and older facilities have been renovated to meet students’ needs.
Nationally standardized surveys are used most often, but locally developed surveys are administered at some colleges and universities. Students are often surveyed at entry, during their college experience, and after they graduate. Many institutions also survey withdrawing students. The ACT Student Opinion Survey (SOS) is the most commonly used instrument. Others include the Noel-Levitz Student Satisfaction Inventory (SSI), the ACT Alumni Survey, the ACT Withdrawing or Non-returning Student Survey, and the ACT College Outcomes Survey (COS). Graduate Student Assessment
Beginning fall 1996, higher education institutions that charge graduate students the student assessment fee must perform assessment beyond the standard requirements for admission to and graduation from a graduate program. All ten universities offering graduate programs (OU, OSU, UCO, ECU, NSU, NWOSU, SEOSU, SWOSU, CU, and LU) reported graduate student assessment activities that include licensure, certification, and comprehensive exams; portfolios; capstone courses; practica; theses; interviews; and surveys. Licensure/Certification Assessment
An important measure of both student achievement and program effectiveness and appropriateness is the professional examination for licensure or certification. This is the second year institutions were asked to provide the number of students taking such examinations and the number of them passing. Assessment Budgets
This is the second year that assessment budgets figures were requested. In compliance with State Regents’ policy regarding the use of fees, it is important to monitor how assessment fees are being allocated for the support of assessment activities. An analysis of assessment budgets are planned for future reports. 8
Analysis
Student assessment in the Oklahoma State System of Higher Education is defined as:
“A multi-dimensional evaluative process that measures the overall educational impact of the college/university experience on students and provides information for making program improvements.”
As evidenced by the institutional reports, Oklahoma’s colleges and universities are achieving the two major objectives of student assessment: to improve programs and to provide public accountability. As institutional implementation of student assessment has evolved, continued enhancements and improvements have been documented.
Institutions have improved the process of gathering and using assessment data. Specific days or class times for assessment have been designated to encourage and facilitate student participation in general education and program outcomes testing. Strategies for increasing the response rates to surveys have been evaluated. Assessment results have been integrated into other institutional review processes and shared widely with faculty and students.
Areas of concern include the variance in secondary institutional placement cut-scores for a given instrument. Secondary testing for science is not practiced at all institutions; however, some institutions use a combination of reading and math scores and others use science tests. Also, institutions are using one or more of seven different assessment instruments; this variation diminishes the ability to compare practices across the state or with institutions in other states.
Administration of general education assessment varies in methodology among the state’s higher education institutions with several using locally developed tests. Using nationally-normed exams could provide more consistency and comparison to national benchmarks.
Persistence and graduation rates depend on the ability of a student to succeed not only in higher-level courses, but also globally in the business and industry. Implementation of state-wide outcomes assessments in writing and mathematics could insure that students have the requisite skills to be successful in further education and in the work place. Pass rates of outcomes assessments could be included in the annual student assessment report as a means of monitoring progress and increasing public transparency and accountability. Such assessments also could assist in regional and departmental accreditation.
Additional information on related institutional polices and student performance are available in annual reports from the Oklahoma State Regents for Higher Education, including the Annual Student Remediation Report and the High School Indicators Project Reports: Mean ACT Composite Scores; High School to College-Going Rates; Headcount, Semester Hours and GPA; and Remediation Rates. 9
Entry-Level Assessment
Entry-Level Assessment and Placement is defined in State Regents’ policy as an “evaluation conducted prior to enrollment which assists institutional faculty and counselors in making decisions that give students the best possible chance of success in attaining academic goals”.
Each institution uses ACT subscores to provide a standard for measuring student readiness. Students scoring below the minimum level established by the State Regents in the four subject areas of science reasoning, mathematics, reading, and English are required to undergo additional testing to determine the level of readiness for college level work consistent with the institution’s approved assessment plan, or successfully complete remedial/developmental course work in the subject area.
Institutions are required to report to the State Regents the methods, instruments, and cut-scores used for entry-level course placement, as well as the student success in both remedial and college-level courses. Instructional changes resulting from an analysis of entry-level assessment is also to be reported.
Several institutions use a combination of high school grade point averages, ACT subject scores, and secondary test scores to determine course level placement. Minimum scores required for college level work are listed in tables with each institution. Some institutions adjust math cut-scores upward if the student’s anticipated major field of study requires a higher level of mathematics skills.
The following listing by institution includes the testing instruments used for determining course placement, the subject area scores necessary for enrollment in college-level courses, and actions taken as a result of tracking student performance in their first college-level course. While a few of the tests were developed locally, the majority were obtained from testing companies. The COMPASS and ASSET instruments are produced by ACT; Accuplacer, CPT, and Writeplacer are products of The College Board. ASSET is a pencil-and-paper version of COMPASS, a computer-based format. Accuplacer and CPT are the same.
University of Oklahoma (OU)
Placement instruments: COMPASS, standardized writing sample, and/or Calculus COMPAS for higher level math placement (standardized writing sample / Calculus COMPAS were developed locally)
Subtest
Cut-Score
Course
Notes
Reading
81+
College Reading
English
85+
College English
Algebra
60+
General Education Math
College Algebra
50+*
College Algebra
*If HSGPA < 3.5 or Transfer GPA < 2.8 College Algebra
45+*
College Algebra
*If HSGPA > 3.5 or Transfer GPA > 2.8
Annual analysis evaluates the effectiveness of programs designed to increase academic success. Cut scores, GPA levels, and other assessment criteria are modified to assure that students are being placed appropriately. Individual entry-level math course success rates were evaluated, and findings indicate that students continue to struggle most with math courses. Analysis also indicates students may struggle with study skills and knowledge of material. As a result, a comprehensive walk-in evening tutoring program (UC Action) was started in Fall 2007. Usage of UC Action has increased from 1,367 visits since Fall 2007 to 3,144 visits in Spring 2009, with the return rates suggesting students find the program beneficial. 10
A locally developed New Student Survey has been used since 1975 to assess new freshman student backgrounds and attitudes. Each year, changes are made in the survey to address such things as technology as well as other issues. The data generated from the New Student Survey have been useful in conducting retention and academic studies to determine the type of student who drops out of the University as well as to identify those students not succeeding academically. In 2008, items were added to the New Student Survey measuring student use of e-mail and Facebook.
Based on findings of these evaluations, the Assessment and Learning Center is in the process of making program additions to provide directed career assessment and exploration leading to effective major selection for deciding students. Integration of COMPASS data with Banner is also in progress.
Oklahoma State University (OSU)
Placement instruments: COMPASS and Entry-Level Placement Analysis (“ELPA” - developed by OSU)
Subtest
Cut-Score
Course
Reading
81+
(no restrictions)
English
56+
College English Mathematics
55-71
Intermediate Algebra Mathematics
72+
College Algebra Science
(no restrictions) Reading or
71+
Algebra
55+
Each enrolled new student (freshmen and transfer students with fewer than 24 credit hours) receives a Student Assessment Report that summarizes information used for entry-level assessment. This report includes the student’s academic information (ACT scores, high school GPA and class rank), the results of ELPA, areas of curricular and performance deficiencies requiring remediation, and recommendations and requirements for course placements as per OSU guidelines that have been approved by the State Regents. The Student Assessment Reports are produced by the Office of Institutional Research and Information Management and are distributed to students by the New Student Orientation Office. Entry-level assessment also includes evaluation of educational readiness, educational goals, study skills, values, self-concept, and motivation. These evaluations are included in the assessment process when each student meets with his/her advisor prior to enrollment.
Many resources are available to OSU students for academic support. University Academic Services (UAS) offers free tutoring services to all OSU students. The Math Learning Resources Center provides individual tutoring in mathematics. The Writing Center provides tutors, writing coaches, a grammar hotline, and assistance with word processing. University Counseling provides services to help students improve their study habits, deal with test anxiety, develop better time management skills, and explore careers.
The CIRP Freshman Survey is conducted in alternate years at OSU as part of a nationwide study conducted jointly by the American Council on Education and the University of California at Los Angeles’ Higher Education Research Institute. The study provides information about the expectations, attitudes, and experiences of OSU freshmen and college freshmen nationwide. The survey results help identify areas that may become problems for students during their first year, and these areas can then be addressed 11
in orientation classes and by academic advisors. NSSE and CIRP data are being shared with colleges and departments this year and may lead to instructional changes.
University of Central Oklahoma (UCO)
Placement instruments: Accuplacer CPT
Subtest
Cut-Score
Course
Reading
75+
Freshmen Level Reading
Sentence Skills
77+
English Composition I Pre-Algebra
75+
General Education Math Pre-Algebra
98+
College Algebra
The Admission Officer determines which students require secondary placement testing based on the placement policy. The Academic Support Center offers computerized tutorials in a wide range of subjects and one-on-one tutoring in mathematics and English. Other departments on campus offer free tutoring by subject. Rose State College offers the developmental courses on the UCO campus and reports completion rates each year. The University has formed a student retention committee composed of members from Student Affairs and Academic Affairs. In Fall 2009, the CIRP was administered to students enrolled in English 1113. CIRP data was used to develop the “First Day” experience in the College of Liberal Arts, and in Student Affairs programming. Enrollment Management uses CIRP data for prospective student services also.
East Central University (ECU)
Placement instruments: COMPASS; Integrated Process Skills Test II (IPST II) for science
Subtest
Cut-Score
Course
Reading
77+
(no restrictions)
Writing Skills
43+
English Composition I Algebra
0-51
Intermediate Algebra Algebra
52+
College Algebra Science
28+
(no restrictions)
The 2008-09 placement distributions for English show improvement compared to the average placements for the 2003-04 through 2007-08 freshman classes (2003-08), with a significantly higher percentage of students passing the COMPASS writing secondary placement. The placement distribution for reading also indicates overall improvement with more students passing the COMPASS reading comprehension placement test over the previous five-year average with fewer students required to take the developmental courses in these areas. However, the placement distribution for mathematics and science indicates fewer students passing the COMPASS algebra and science secondary placement tests over the previous five-year average with more students required to take the developmental courses in these areas. This is primarily due to an increase in the required passing scores on the COMPASS modules for these subject areas. A subject ACT score of 19 or higher does not appear sufficient to guarantee consistent success in College Algebra (MATH 1513), the Reading Courses Group, or in the Science Courses Group. Furthermore, the data provide further evidence why entry-level assessment and placement at ECU has not significantly improved the retention rates of the freshman classes. Given the poor academic backgrounds 12
of many entering freshmen, the remediation offered at ECU is not sufficient in many cases to provide some students with the skills necessary to succeed at the college level.
Northeastern State University (NSU)
Placement instruments: Accuplacer
Subtest
Cut-Score
Course
Notes
Reading
75+
(no restrictions)
English
80+
English Composition I
Accuplacer English
5+
Intermediate Algebra
WritePlacer Mathematics
44-74
Intermediate Algebra
Mathematics
75+
College Algebra
Students not meeting the required ACT score are assessed by the First Year Experience/Enrollment Services department. This assessment is done with the Accuplacer and includes English, mathematics and reading. This office conducts testing daily by appointment with most activity during the spring and summer semesters. Test results are generated and proper enrollment is done at the same time in the First Year Experience counselor’s office. Tutoring is provided for the students who have difficulty in the zero level course work. Progress of first time full-time students is now monitored at mid-semester and grades are posted electronically on Blackboard by the tenth week. Students are allowed to re-test one time after 30 days have elapsed.
The analysis of zero level math and English remains fairly consistent from year to year. Spring percentage pass rates are usually lower than the preceding fall. Pass rates in mathematics in the fall are usually between 60 and 65 percent and between 45 and 65 percent in the spring. English pass rates are usually between 65 and 75 percent in any given fall and lower in any given spring. Overall, the pass rates have remained the same over the past two years. NSU considers the method and effectiveness of placement decisions to be effective. Cut scores have changed minimally over the past several years.
The department of mathematics revised the two developmental courses and is now using different textbook and material as a result of recent data and student performance. The same textbook is being used for both Mathematics (MATH) 0123 and MATH 0133. There have been additional sections of MATH 0123 added to keep class size at a reasonable number. There has been an attempt by several mathematics instructors to pilot a zero level algebra course that is somewhat self-paced and where students are allowed to proceed at a benchmark (criteria driven) level. Public school teachers with appropriate experience are being hired as adjunct faculty. Administrative withdrawals are being issued for nonattendance to students in all zero level courses. Both English and mathematics faculty teaching zero level courses have made adjustments and are using common syllabi. The department of mathematics is rethinking the use of College Algebra as a General Education course offering.
The English faculty members have changed textbooks and continue to utilize a multi-station writing laboratory for those in all zero level and beginning English coursework. A writing laboratory director is now in place at the NSU and Broken Arrow campuses and the computers in the writing lab have been upgraded in number and quality. The office of Assessment and Institutional Research is cooperating with the Writing Laboratory to determine the effect of laboratory time on student writing abilities.
13
Northwestern Oklahoma State University (NWOSU)
Placement instruments: Accuplacer
Subtest
Cut-Score
Course
Reading
76+
(no restrictions)
English
88+
English Composition I Algebra
45-75
Intermediate Algebra Algebra
75+
College Algebra Arithmetic
55+
(no science restrictions)
Northwestern has taken steps to ensure success for academically underprepared students, including assuring the availability of developmental courses for incoming freshmen during the fall semester, standardizing its developmental education placement policy across all developmental disciplines (math, English, reading and science); and providing training for faculty members who teach developmental education courses. Accuplacer cut-scores for mathematics were re-evaluated and adjusted approximately four years ago. Beginning in Fall 2010, supplemental instruction was added in all sections of the MATH 0013 Pre-Intermediate Algebra course. The objective is to not only improve success in developmental mathematics, but also improve retention and success in credit-bearing mathematics as well.
Southeastern Oklahoma State University (SEOSU)
Placement instruments: Accuplacer (CPT) and CPT Companion Test for English, math, and reading; Stanford Test of Academic Skills for science
Subtest
Cut-Score
Course
Notes
Reading
74+
(no restrictions)
Sentence Skills
85+
English Composition I
Algebra
42-54
Intermediate Algebra
Algebra
75+
College Algebra
Science*
20+
(no science restrictions)
*Stanford Science Test
Student progress was measured by course pre-post test scores, course GPA, and overall GPA. The pre-post test scores show gains after completing one semester of instruction. A comparison of course GPAs and overall GPAs for students who matriculated into regular college courses portrays a positive image of student success as students who completed at least one semester of remediation compared favorably with those students who were not required to remediate. At this time, no adjustments to cut scores are recommended. An additional study completed in August 2009 of students who made passing scores on secondary placement tests administered between the years 2004-2009 also indicates that current cut scores required to pass the secondary tests are effectively placing students at the proper course level.
Southwestern Oklahoma State University (SWOSU)
Placement instruments: CPT Accuplacer
Subtest
Cut-Score
Course
Reading
75+
(no restrictions)
Sentence Skills
75+
English Composition I Algebra
85-94
Intermediate Algebra 14
Algebra
95+
College Algebra
Students are advised of academic support through notification in various handbooks, bulletins, and university websites as well as by staff and faculty during clinics, orientation, registration, and advisement. Academic departments also provide advisement as well as tutoring assistance in special labs by student tutors and faculty. Faculty members review the structure of developmental English, mathematics, and reading courses for ways to improve student achievement. A tracking study of a cohort of Fall 2001, Fall 2002 and Fall 2003 entering freshmen reveals more specific success in following academic years. Final data shows Fall 2001 first time freshmen with a 50 percent success rate, including withdrawals, in developmental courses; and a 63 percent success rate, excluding withdrawals, in developmental courses.
Cameron University (CU)
Placement instruments: Computerized Placement Tests (CPT)
Subtest
Cut-Score
Course
Reading
77+
College Reading and Study Strategies
English
95+
English Composition I Mathematics
65-74
Intermediate Algebra Mathematics
74-97
Survey of Mathematics Mathematics
98+
College Algebra
The “Early Alert” system allows faculty members to work through the Office of Enrollment Management and notify at risk students of potential problems in their entry-level courses. This procedure is improving retention efforts with these students. Additionally, students who completed developmental courses are tracked through successive courses with results indicating improved retention and pass rates.
Cameron University established a developmental English laboratory designed to target specific Basic Composition and Developmental Writing deficiencies. Special one-on-one tutoring is available for students in these classes. This assessments requires students to keep a portfolio of their work in Basic Composition and Developmental Writing courses to include copies of each essay and its revision, all tests, quizzes, and daily work.
Cameron’s “Early Alert” system was improved to provide more effective communication with students in all entry-level courses. The new entry-level mathematics course is improving student skills and providing better preparation for the affected students taking Introductory Algebra.
Langston University (LU)
Placement instruments: CPT Accuplacer
Subtest
Cut-Score
Course
Reading Comprehension
75+
(no restrictions)
Sentence Skills
75+
English Composition I Elementary Algebra
<75
Intermediate Algebra Elementary Algebra
75+
College Algebra
Data gleaned from the entry-level assessment database for 2008-2009 cannot be compared to previous years. LU transitioned from a paper and pencil test format to an electronic mode of assessment that is 15
scaled differently. Given the past years trend line, LU assumes Fall 2008 results under the new format to yield similar results.
LU indicates course placement decisions are effective and meet current student needs. The Office of Academic Affairs makes necessary adjustments when errors of judgment surface. Additionally, LU’s tracking suggests a happy, well-informed student is an academically productive student.
The cut scores are evaluated periodically against both internal and external benchmarks. These benchmarks have been a relatively good barometer for student success in a higher education environment. Collectively, cut-score evaluations and analyses of entry-level basic skills scores have resulted in relatively few changes to the entry-level assessment process. The Vice President for Academic Affairs critiques each assessment cycle against our predetermined goals and objectives to ensure continuous qualitative and quantitative improvement. During 2008-2009, the secondary entry-level assessment instruments were administered in one (1) session of one hundred (100) students twice daily during the assessment period. The result will be compared and contrasted to Fall 2009 results to measure the impact of such change.
Computer aided instructions were continued in the mathematics, reading and writing laboratories during the 2008 – 2009 fiscal year. Adding technology to enhance student learning remains a priority given funding challenges in Oklahoma. Research suggests this is an appropriate strategy for the benefit of both the student and the university.
University of Science and Arts of Oklahoma (USAO)
Placement instruments: COMPASS for math and writing; locally developed science test for science
Subtest
Cut-Score
Course
Notes
Writing
75+
English Composition I
Pre-Algebra
0-55
Basic Math Skills
Pre-Algebra
56+
College Algebra
Science Placement*
50+
(no science restrictions)
*In-House Science Test
A review of the cut scores was conducted and the results provided to appropriate departments. No action was indicated based upon the data provided. The entry-level assessments indicate student placement is appropriate. Those students who have not done well either in developmental or college level courses did not do well due to reasons aside from not being able to accomplish the work. Another review of the cut scores will be conducted in the next academic year.
No instructional changes have occurred or are planned. Advisors, however, have more closely tracked their advisees entering with lower scores. USAO reports additional faculty may result in the ability to better address areas of concern.
Oklahoma Panhandle State University (OPSU)
Placement instruments: Accuplacer (CPT)
Subtest
Cut-Score
Course
Reading
70+
(no restrictions)
English
87+
English Composition I 16
Algebra
0-73
Intermediate Algebra
Algebra
74+
College Algebra
During the 2008-2009 academic year, a Freshman Expectations Survey was administered to all students enrolled in the Student Success Seminar during the fall of 2008 The survey asks a series of questions regarding their demographic background, what decisions affected their choice in attending OPSU, and what they expected in their first year of college. There were 176 students who completed the survey during the fall of 2008.
When looking at the trend over the last five years, there has been a dramatic increase in the number of students requiring developmental coursework. The college will continue and expand its services in the areas of special tutoring, counseling, and personal attention to students. These offices will work closely with all students in developmental courses to assist in various matters the student may encounter while attending OPSU.
Rogers State University (RSU)
Placement instruments: COMPASS for English, reading, and mathematics; Stanford Test of Academic Skills in Science for science
Subtest
Cut-Score
Course
Notes
Reading
82+
(no restrictions)
English
82+
English Composition I
Algebra
36-45
Intermediate Algebra
Algebra
46+
College Algebra
Science
82+
(no restrictions)
Stanford Science Test
The success of RSU’s Entry-Level Assessment and Placement Program is measured by a number of factors, including validation of cut scores, retention levels, and success in both developmental and college-level coursework. The effectiveness of placement decisions and appropriateness of cut scores are evaluated on the basis of retention of students in each developmental course; achievement in developmental courses; and performance in subsequent college-level coursework. No changes to existing cut scores were made during the 2008-2009 academic year.
During the 2009-2010 year, the University Assessment Committee examined the student success rates in developmental and college-level courses based on placement; collaborated with the new Developmental Studies Coordinator and departmental faculty; and made recommendations to the Academic Council as appropriate.
Connors State College (CSC)
Placement instruments: COMPASS and ASSET; Accuplacer (CPT) as a back-up placement exam
Subtest
Cut-Score
Course
COMPASS
Reading
76+
(no restrictions) Writing
75+
English Composition I Algebra
51-65
Intermediate Algebra Algebra
65+
College Algebra 17
Subtest
Cut-Score
Course
ASSET
Reading
41+
(no restrictions) Writing
45+
English Composition I Algebra
44-48
Intermediate Algebra Algebra
49+
College Algebra 18
Subtest
Cut-Score
Course
ACCUPLACER
Reading
79+
(no restrictions) Writing
79+
English Composition I Algebra
53-72
Intermediate Algebra Algebra
73+
College Algebra
The College Board Accuplacer is used as a back-up placement examination when computer network problems prevent the administration of the COMPASS. The ASSET is used for off campus populations that are not allowed computer access to the Internet, such as the two correctional sites served by CSC. ASSET is also utilized by the financial aid office as a back-up test for students who do not obtain the minimum “Ability to Benefit” score on the COMPASS.
In the area of developmental reading, the instructor has instituted several instructional modifications. A major change in the structure of the program was to move away from mostly independent work for students based on individual study plans to direct instruction in comprehension, fluency, and vocabulary strategies in harmony with recent research.
Eastern Oklahoma State College (EOSC)
Placement instruments: COMPASS
Subtest
Cut-Score
Course
Reading
72+
(no restrictions)
Writing
62+
English Composition I Pre-Algebra
0-44
Developmental Math Pre-Algebra
45+
College Algebra
Murray State College (MSC)
Placement instruments: COMPASS and ASSET
Subtest
Cut-Score
Course
Subtest
Cut-Score
Course
ASSET
COMPASS
Reading
39+
(no restrictions)
Reading
71+
(no restrictions) Writing Skills
36+
English Composition I
Writing Skills
70+
English Composition I Intermediate Algebra
34-38
Intermediate Algebra
Algebra
26-39
Intermediate Algebra Intermediate Algebra
39+
College Algebra
Algebra
40+
College Algebra
Once students were enrolled in the courses appropriate to their testing level, both peer and professional tutors were available for assistance in that course. Tutorial services on the Tishomingo campus were offered at one central location in the “Help Center” located in the Library. On the Ardmore campus, tutorial services are available in the Ardmore Higher Education Center lobby. Scheduled hours were 19
published for tutorial assistance in a variety of subject areas, including writing, math, and science. Microcomputers with tutorial software were also available for student use.
On the Tishomingo campus, student progress was tracked in particular by the individual student’s academic advisor and in general by the Academic Advisement Center. On the Ardmore campus, student progress was tracked by the academic advisors at the MSC office. At the end of the semester, each academic advisor received a grade report for his/her advisees that indicated student success or lack of success for both developmental and college-level courses. The academic advisor and the student then made any necessary changes to the student’s class schedule for the subsequent semesters.
On a semiannual basis, the Director of Academic Advisement reviews and discusses the effectiveness of student placement with instructors of the developmental. Reports of any recommended changes from those semiannual reviews are submitted to the MSC Academic Council consisting of administrators and faculty. There is ongoing refinement of the curriculum based on communication between instructors of developmental courses and instructors of college-level courses. The institution is currently evaluating all developmental programs.
As a result of this review, Basis English II courses have been added to the curriculum. The Standard Writing Scores on the COMPASS test have been revised. As of June 2009, students scoring from 0-37 are required to take Basic English I and students scoring from 38-69 are required to take Basic English II.
Northeastern Oklahoma A&M College (NEO A&M)
Placement instruments: CPT
Subtest
Cut-Score
Course
Sentence Skills
78+
English Composition I
Reading
77+
(no restrictions) Elementary Algebra
53-72
Intermediate Algebra Elementary Algebra
73+
College Algebra Science
Elementary Algebra and
53+
Reading
77+
Students who do not meet the CPT cut scores in individual subjects are placed in developmental courses. The developmental reading and basic composition courses include classroom instruction and supplemental computerized laboratory assignments. Paraprofessional personnel are available to assist students with their laboratory assignments. The developmental mathematics courses provide classroom instruction. NEO A&M provides a mathematics lab staffed by support personnel with a at least a baccalaureate degree in mathematics to provide tutoring services for mathematics classes up to calculus. Tutoring is available for eligible students in Student Support Services TRIO program. Students also may seek assistance in preparing for the assessment test through the Testing Center.
The Testing Center personnel monitor student progress to ensure the students are enrolling in the appropriate developmental and college-level courses. Each semester, the Testing Center coordinator receives a computer-generated report identifying students who have not enrolled properly in the developmental courses and notifies the students' advisors. The Enrollment Management staff verifies that students enroll in the appropriate developmental courses.
The NEO A&M has created a Center for Academic Success and Advisement, which opened in the fall of 20
2009. One of the goals of the center is to identify students at risk for not completing the semester and to intervene early in order to help these students be successful and complete the semester. The center is staffed with full-time academic advisors and a part-time retention advisor to assist academically at risk students.
Northern Oklahoma College (NOC)
Placement instruments: COMPASS
Subtest
Cut-Score
Course
Reading
81+
English Composition I
E-Write
8+
(no restrictions) Algebra
42-72
Intermediate Algebra Algebra
73+
College Algebra Science
Algebra and
80+
Math
25+
It is the intent of NOC to provide webstreams of NOC faculty addressing various topics that students may wish to review prior to re-testing. It is the intent of NOC to explore the possibility of having specific modules of self-paced learning for students to review prior to re-testing.
In prior years a significant number of students have placed into Pre-Algebra or Concepts of Algebra. At the recommendation of NOC mathematics faculty, the COMPASS mathematics placement test was modified to utilize only questions concerning pre-algebra, algebra, and college algebra domains, excluding the domains of geometry and trigonometry. The recommendation was issued after determining NOC used COMPASS scores for college algebra placement purposes and not the higher level mathematics courses. As a result, students placed into Intermediate Algebra, which happened rarely under previous assessment instruments. Student satisfaction with their math placement was improved and faculty was very pleased with the placement.
In previous years the COMPASS writing skills examination was used as the challenge examination for English composition. NOC implemented the COMPASS E-write for English Composition placement purposes and faculty was very pleased with the transition.
NOC is now in the process of evaluating the pre-test/post-test COMPASS results to evaluate the effectiveness of the foundational/developmental program as a whole. The COMPASS results are being linked to the CAAP results for overall program effectiveness. The faculty has been pleased with the results.
Tulsa Community College (TCC)
Placement instruments: Accuplacer (CPT)
Subtest
Cut-Score
Course
Reading
80+
English Composition I
Sentence Skills
80+
(no restrictions) College Algebra
0-40
Intermediate Algebra College Algebra
41+
College Algebra 21
Beginning with the Fall 2009 semester, TCC began administering the ACT Compass for entry-level placement and diagnosis. Student success rates (earning a C or better) in developmental courses was reported, as was student success (earning a C or better) in subsequent college coursework. Because TCC is an Achieving the Dream (AtD) college, all five AtD goals were measured: [1] successfully complete developmental courses; [2] successfully complete gateway course; [3] complete coursework with a C or better; [4] persist from one semester to the next; and [5] increase degree attainment and completions.
TCC discontinued use of the Accuplacer CPT in Summer 2009 and subsequently implemented the use of the ACT Compass for Fall 2009 placement, cut score analysis is underway during for the 2009-2010 academic year. Results will be reported in the 2009-2010 Annual Student Assessment Report.
Although research through the American Association of Community Colleges (AACC) and the Achieving the Dream initiative indicate success rates achieved at TCC are commonly experienced at community colleges, TCC is not satisfied with these results and wishes to increase student success. Consequently, developmental reading was selected for analysis and intervention during the 2008-2009 academic year with developmental mathematics to be highlighted in 2010-2011, followed by developmental writing in 2011-2012.
Oklahoma State University – Oklahoma City (OSU-OKC)
Placement instruments: COMPASS
Subtest
Cut-Score
Course
Reading
80+
(no restrictions)
Writing Skills
82+
English Composition I Algebra
43-75
Intermediate Algebra Algebra
76+
College Algebra
Entering students are tracked, especially those in developmental studies courses. A master report is generated on an annual basis which tracks such items as successful outcomes (grade of C or better) and persistence to the next course in the sequence. These students are also assessed via pre- and port-test methods in the developmental course sequence. A more robust master report, incorporating cohort tracking, was developed and implemented during the 2009-2010 academic year. Data gathering items include: Success rates for each developmental studies course; Success rates in college level courses for students at different entry points in developmental studies (longitudinal cohort tracking); and Grade distributions for developmental courses by age, gender, and ethnicity with term-to-term trending.
Assessment of developmental programs remains a center point of the Developmental Studies Department. There is a renewed focus on all students at OSU-OKC, especially those just beginning. OSU-OKC is committed to delivering educational programs that “prepare individuals to live and to work in an increasingly technological and global community.” By continuously assessing these programs and their effectiveness, the institution actively works towards fulfilling this commitment. In the Fall 2009 semester OSU-OKC installed a department head at a faculty rank, a new developmental studies instructor, and additional mathematics faculty.
22
Oklahoma State University Institute of Technology (OSUIT)
Placement instruments: COMPASS
Subtest
Cut-Score
Course
Reading
81+
(no restrictions)
Writing Skills
74+
English Composition I Algebra
45-67
Intermediate Algebra Algebra
68+
College Algebra Science
Reading and Algebra
149+
All secondary assessment of basic skills (ACT COMPASS) was available for administration online at the OSUIT campus and at remote sites approved by the college. Student Success camps sponsored by the Arts & Sciences division and the College Readiness Center (CRC) allowed students to work at their own pace where they could complete remediation in as little as one day. The camp was free; however, if students desired to stay on campus, they were responsible for lodging and food.
OSUIT continues to implement the Early Alert System, an electronic intervention system used by faculty to alert the institution when a student is in danger of failing or when a student is not attending classes regularly. When the Early Alert System is activated, Arts & Sciences faculty distribute electronic notices to the student’s advisor in his or her technical program of study. Subsequently, the advisor schedules an appointment with the student to discuss possible solutions and makes appropriate recommendations for the student to seek academic support services available on the campus. In this way, students in college-level coursework are enabled to stay on track and receive academic or social interventions as needed.
The Assessment Committee and faculty in the College Readiness Center (CRC) reviewed the cut scores for entry-level assessment that were revised prior to the 2005-2006 academic year; these cut scores were retained through 2008-2009.
Western Oklahoma State College (WOSC)
Placement instruments: COMPASS
Subtest
Cut-Score
Course
Reading
80+
(no restrictions)
Writing
70+
English Composition I Algebra
28-49
Intermediate Algebra Algebra
50+
College Algebra
The Assessment Committee has a two-pronged plan. Developmental assessment consists essentially of a competency-based assessment of each individual course, similar to the plans implemented for program assessment, as well as longer-term studies of student success by tracking students proceeding from developmental courses through specific college level courses.
Tracking encompasses many factors including success rates, grade point averages, grade distribution, and most importantly, comparison of developmental students verses non-developmental students. Ultimately, tracking will provide WOSC information pertaining to the effectiveness of placement testing and provide 23
a clearer picture of the entire collegiate process from entrance to graduation.
WOSC provides free academic peer tutoring services to all students in the Tutoring Center for courses such as English, computer sciences, computer applications, basic mathematics, all algebra content areas (e.g., beginning, intermediate, and college), economics, and financial and managerial accounting. Science areas covered generally include chemistry and biology. The peer tutors have accommodating scheduled hours throughout the day and evening during the week. Attendance varies with each semester and subject, but approximately 150 students are helped throughout the year.
The PASSKEY software program is used for students who place in English Fundamentals and Developmental Reading III. A key feature is the software allows the developmental course instructors to administer diagnostic tests to better determine each student’s strengths and weaknesses. In addition, all scores can be linked to the COMPASS scoring. This process bridges the gap between weaknesses and instruction by preparing an individual prescription for the student by assigning particular lessons from the software. No lessons are assigned from areas where the student has the acquired knowledge. The student then progresses through the developmental courses quicker. The PASSKEY software does not allow a student to progress to the next lesson until they have achieved a grade of 80% or better; therefore, the student does not bypass a problem area.
ACADEMIC SYSTEMS software is being used for developmental students in Basic Math and Beginning Algebra. A key feature of this software is that it will allow students to work at their own pace to complete the course. This may enable the student to progress through the developmental mathematics courses at a pace consistent with their abilities. In addition to the computer based mathematics courses, traditional classroom lecture courses are available for those students preferring this method of instruction.
Redlands Community College (RCC)
Placement instruments: COMPASS or ASSET
Subtest
Cut-Score
Course
Reading
80+
(no restrictions)
Writing Skills
68+
English Composition I Algebra
36-66
Intermediate Algebra Algebra
67-69
"Decision Zone" Algebra
70+
College Algebra
The COMPASS placement test is primarily used for those students testing on RCC’s main campus, while ASSET is used for testing students at RCC’s outreach sites. Evaluation of cut scores occurs periodically at RCC. COMPASS cut scores were revised in 2007 to include more “decision zones.” Since retention is a major concern, RCC employs a retention specialist. This individual works with both students and faculty members to improve students’ academic experiences during the entire academic year.
Carl Albert State College (CASC)
Placement instruments: COMPASS or CPT
Subtest
Cut-Score
Course
Subtest
Cut-Score
Course
COMPASS
CPT
Pre-Algebra
<45
Developmental Math
Algebra
45-65
Intermediate Algebra 24
Algebra
66+
College Algebra
Writing Skills
74+
English Composition I
First-time entering freshmen levels of past academic experience are evaluated in order to assess educational readiness. Results from entry-level assessment are utilized during advisement and enrollment so students have the highest probability of success during their collegiate experience. Finally, results from entry-level assessment are used to evaluate and recommend any changes to the orientation class, the developmental education curriculum, and the registration and advisement process.
Rose State College (RSC)
Placement instruments: COMPASS or Accuplacer (for distance learning/transfer students)
Subtest
Cut-Score
Course
Reading
81+
(no restrictions)
Writing Skills
74+
English Composition I Algebra
51-75
Intermediate Algebra Algebra
76+
College Algebra
Students receive academic support for assessment testing through a variety of sources. COMPASS diagnostic testing is offered in the Engineering and Science Division Mathematics Laboratories. Reference materials are provided in the Learning Resources Center (LRC) in mathematics, reading, and English. Study guides for the COMPASS are available online with an additional link to ACT’s website where additional practice items can be found. Paper copies of the study guide are available in the Testing Center. Library reference materials outlined in the study guide are held on reserve in the LRC. In addition to the COMPASS Study Guide, a literary reference specific to preparation for COMPASS assessment, Chart Your Success on the COMPASS, by Callahan, Commander, and Cotter is available in the LRC and RSC mathematics laboratory.
Additionally, the Student Success Center was established January 2009 to help students’ personal growth, professional development, and academic progress from enrollment through graduation. An instructor may refer a student if they have any concerns about the student, whether academics or personal. Through this early alert system, referrals for established services (tutoring, laboratories, personal counseling, career counseling, academic advisement, etc.) and mentoring programs can assist a student before their problems become insurmountable.
The Placement and Testing Committee, reflecting a cross-section of faculty, continues to review the cut scores for validity when trends of unsuccessful performance warrant evaluation. However, for the last several years the committee has focused on methodology related to mathematics placement. The branching methods within the COMPASS assessment tool were modified based on mathematics faculty recommendation; however, outcome placement ranges were not modified. The changes to mathematics/pre-algebra routing have yielded significant course placement adjustments in developmental mathematics. Success rates for students in the areas of pre-algebra, elementary algebra, and intermediate algebra indicate they were on par or in many cases more successful than students who enrolled through another means. RSC Placement and Testing Committee’s consensus has been that the mathematics changes are resulting in positive improvements in student outcomes.
RSC continues to use The Entering Student Descriptive Report as a research tool, which provides useful information related to student placement in initial courses and the number of students placing in those 25
courses. This information is utilized by academic divisions as a tool for student course scheduling.
26
Oklahoma City Community College (OCCC)
Placement instruments: COMPASS; ASSET; Accuplacer; Riverside Biology and Chemistry tests for science
Subtest
Cut-Score
Course
Subtest
Cut-Score
Course
COMPASS
CPT
Reading
80+
(no restrictions)
Reading
71+
(no restrictions) Writing Skills
82+
English Composition I
Writing
83+
English Composition I College Math
0-49
Intermediate Algebra
Algebra
39-75
Intermediate Algebra College Math
50+
College Algebra
Algebra
76+
College Algebra
Subtest
Cut-Score
Course
ASSET
Reading
41+
(no restrictions) Writing
45+
English Composition I Numerical Skills
35-55
Elementary Algebra
OCCC regularly reviews the placement of students. Information for the review is obtained from faculty surveys and student completion rates in specific classes. Periodically, surveys are administered requesting information on whether faculty believes each student in their class was placed appropriately. The information from this survey is reviewed for patterns or trends. If the grouped data reveals more than five percent of the students are placed at the wrong level, the cut scores are reviewed for possible adjustment. This survey is carried out once every three years, upon request, or a year after a new test is implemented.
Course completion rates are also reviewed. A review is initiated to identify possible reasons for fluctuation if more than a ten percent completion rates is experienced. If placement is determined to be a part of the problem, then a recommendation to change placement scores may be made. 27
General Education Assessment
Mid-level assessment is designed to assess the competencies gained by students in the college general education program. Institutions are required to assess students in the areas of reading, writing, mathematics, and critical thinking. Mid-level assessment normally occurs after completion of 45 semester hours and prior to completion of 70 semester hours. For associate degree programs, mid-level assessment may occur halfway through the program or at the end of the program. More typically, this assessment occurs at the end of the program after students have had sufficient time to develop basic skills.
Mid-level assessment is accomplished with a combination of locally developed and standardized testing instruments such as the ACT Collegiate Assessment of Academic Proficiency (CAAP), the Riverside College Base Academic Subjects Examination (BASE), and the Test of Adult Basic Education (TABE). These nationally validated instruments are useful, because they provide regional or national benchmark data from other participating institutions. Several institutions have developed local instruments for mid-level assessment in some subject areas. More qualitative assessments, such as portfolio assessments and course-embedded techniques, are also being used.
Assessments at mid-level and in the major academic program provide important information to institutions about the degree to which their general education programs facilitate student achievement of desired knowledge and competencies. Results of this process have led some institutions to redesign their general education programs. The types of course and delivery method have been closely examined.
University of Oklahoma
Over the 2009 calendar year, several ongoing projects provided data and findings:
PAC-GEO (Provost’s Advisory Committee for General Education Oversight) meets monthly to review course proposals and determine transfer credits for general education. The committee of almost 30 members approves proposals based on the criteria outlined in the Policies, Procedures, and Guidelines. Over 50 course proposals were reviewed as potential general education courses. Approximately 10 percent were returned for revision primarily for purposes of clarifying writing assignments, elaborating on reading expectations or explaining grading or examination criteria.
PAC-WRITE (Provost’s Committee on Writing) convenes specifically with a charge from the Provost to review issues related to writing. In past cases, the committee has addressed enrollment, placement, and capstone courses. Since the entry-level writing courses (delivered from English and from Expository Writing) are general education, and because all capstone courses are also general education, the committee plans to coordinate with PAC-GEO to review the 50 capstone courses. All capstones, by virtue of the general education designation, and as culminating experiences for majors, should contain an intensive writing component; whether and how that writing is taking place is the next investigation.
Writing Across Campus and Writing Fellows Program initiatives are supported by the Office of the Provost and by the Vice Provost for Instruction. The director of WAC efforts is involved with general education assessment and functions as an advisor to PAC-GEO and a leader of PAC-WRITE. The model of Writing Fellows places advanced graduate students with instructors as curricular revision partners.
Oklahoma State University
Information about OSU’s general education learner goals is available on the OSU website (http://osu.okstate.edu/acadaffr/aa/gened-CriteriaGoals.htm). Three approaches are used to evaluate the general education program: Institutional Portfolios, Review of General Education Course Database, and 28
college-, department-, and program-level approaches.
Institutional portfolios provide direct evidence of student achievement of the overall goals of general education. Institutional portfolios have been developed in five areas representing the overall goals of the general education program: written communication, critical thinking, mathematics problem solving, science problem solving, and diversity. Since 2001 OSU has collected samples of student work that represent student achievement of the general education goals from courses across campus. These student work samples are then assessed by a panel of faculty members using rubrics. The results from this process provide direct evidence of student achievement of the general education goals. To make the best use of limited resources institutional portfolios are collected in every content area on an alternating schedule. Four areas were assessed in 2009: written communication, critical thinking, science problem solving, and diversity. In 2008-2009 460 samples of student work were collected and evaluated by a panel of faculty members using rubrics developed and approved by OSU faculty members. The percent of samples scored as a ‘3’ or higher (on a 5-point scale) was 77 percent for critical thinking, 60 percent for written communication, 52 percent for science problem solving, and 59 percent for diversity.
The General Education Advisory Council (GEAC) periodically evaluates every general education course to ensure alignment with the goals of the general education program. As part of this certification process instructors identify which general education goals are associated with courses, describe the course activities that provide students the opportunity to achieve the goals, and explain how student achievement of the goals is assessed within the course. Each course with a general education designation is reviewed every three years.
Many colleges, departments, and programs include elements from the general education goals in their own assessment efforts. These assessment activities are included in the program outcomes assessment section.
In response to these findings, the institution has decided to continue to fund the Provost’s Faculty Development Initiative: Focus on General Education in 2009-2010. In addition, a group of faculty and staff members is being formed to further study the critical thinking findings and to identify possible approaches the institution may use to improve the results. OSU is also engaged in a number of initiatives to improve students’ diversity scores (http://diversity.okstate.edu/). Assessment data are also used to monitor recent changes to the general education program.
All results will be shared broadly with faculty members and relevant councils and committees at OSU and publicly on the OSU general education assessment website (http://uat.okstate.edu/assessment/assessment_at_osu/gened/index.html). Additional discussions about how to respond to results and take steps to improve will be held during the sharing of results.
University of Central Oklahoma
Course embedded assessment focus on the following general education goals:
1. To provide students with an understanding of the universality of the human experience and the common goals and needs that drive that experience through a multicultural and global perspective; (Student focus group; Student Symposium survey);
2. To instill communication and information management skills necessary for participation within society; (English 1113 pre/post essay; Communications 1113 anxiety inventory; Student Symposium survey; Student Symposium presentation evaluations, NSSE Survey Results);
3. To instill skills of analytical thinking, information processing, reasoning, and research necessary for personal and professional development; (Math 1113 and 2013 embedded test questions; Biology 29
1214 lab experiment; English 1213 research paper artifact; Student Symposium presentation evaluations, NSSE Survey Results);
4. To develop an understanding of the cumulative human experience from historical, cultural, and scientific perspectives; (Humanities course pre/post test);
5. To appreciate humanity’s creative talents and to understand the effect of these endeavors on social, economic, philosophical, and political thought; (NSSE Survey Results);
6. To understand humanity’s place in and responsibility to the natural world; (Biology 1114 pre/post test; Student Symposium survey); and
7. To guide students in the exploration and appreciation of moral and ethical concerns common to all. (Philosophy pre/post survey).
Assessment practices include student focus groups, Student Symposium survey, NSSE survey results, presentation evaluations, research papers, pre/post tests, and embedded test questions.
The Cooperative Institutional Research Program (CIRP) survey is administered every fall semester. One section of the survey focuses on expectations of general education curriculum. The College of Liberal Arts conducts syllabi reviews regarding writing requirements. As a result, there has been an increase in the number of writing assignments required in liberal arts courses.
East Central University
ECU assessed nine student outcomes for six academic skill areas during 2008-09. These outcomes covered critical thinking, library skills, oral or expressive communication, reading, mathematical skills, and written communication. During 2008-09, 418 students took one of the Collegiate Assessment of Academic Proficiency (CAAP) multiple-choice sections on Reading (133), Critical Thinking (99), or the CAAP Writing Essay (186) as part of the course requirements for UNIV 3001, the general education capstone course. The test was administered to all UNIV 3001 students. This is the third academic year ECU has used CAAP, so longitudinal data is limited. The 2008-09 cohort’s scaled score for Critical Thinking was 61.1 while the cohort’s scaled score for Writing was 63.0. Both scores were about equal to the 43rd percentile nationally. On the other hand, ECU students scored 3.2 on the Writing Essay section of the CAAP, about equal to the 59th percentile nationally.
Northeastern State University
NSU has determined Riverside’s College Base matches well with the goals of general education as described by the university. College Base does not assess every facet of the general education curriculum. NSU is not satisfied with using College Base as the tool to assess its general education program. NSU’s results on the College Base have both been above and below the national average. As a result, institutionally-developed instruments were utilized to assess humanities, speech, and health/nutrition. During 2008-2009, NSU did not use the College Base because the NSU Assessment Committee was looking for more effective ways to determine general education effectiveness. The Vice President for Academic Affairs formed a General Education Committee to revisit the total general education program to include evaluation. A General Education Capstone course has been proposed and accepted by the Vice President for Academic Affairs. Education continues to administer the OGET as a prerequisite to program admission
Northwestern Oklahoma State University
The general education assessment strategy is a two-pronged approach. Both assessments measure general education competencies as developed by the General Education Committee. One assessment is standardized testing for mid-level evaluation of the general education program. As NWOSU became part 30
of Voluntary System of Accountability (VSA) the MAPP test was chosen for the standardized tests for mid-level assessment of general education in order to complement the on-going VSA testing requirements for freshmen and senior. The second assessment is a program accompanying the new general education curriculum which calls for significant expansion of assessment to include additional measures. All students in general education courses with designated competencies are assessed in the course level assessments. Assessments are administered by each course professor. The general education course-level data is a web-based database and after data has been collected, it is analyzed by the Assessment Office.
Southeastern Oklahoma State University
SEOSU used a two-tiered system to complete mid-level assessment; one at the university level and the other at the departmental level. Approximately 1,290 students participated in the two university–wide aspects of mid-level assessment this academic year. At the university level, SEOSU used ACT CAAP Tests (Critical Thinking, Mathematics, Reading, Science Reasoning, Writing Skills, and Writing Essay) and the ACT College Outcomes Survey to assess student progress and perceptions regarding the goals and learning outcomes of the general education program. Average scores by SEOSU students were within one standard deviation of the national average for all six ACT CAAP Tests. Graduating seniors ranked “taking responsibility for my own behavior” and “acquiring a well-rounded general education” as the top two areas of personal growth on the ACT College Outcomes Survey; “acquiring a well-rounded general education” also was rated the highest for the college contribution to that growth.
Two techniques, Course-Embedded Assessment and Levels of Implementation Survey, were used at the departmental level for mid-level assessment. For Course-Embedded Assessment, assessment instruments, protocols, and benchmarks were developed to evaluate student progress in meeting the learning outcomes for all the goals that are addressed by each course. Students met, or exceeded, more than 60 percent of the more than 250 course-embedded benchmarks. The Levels of Implementation Survey was comprised of eight statements regarding various aspects of the General Education Program and the degree to which each one is put into practice at the departmental level. Department chairs completed the survey and indicated that progress was made in the general education program and its assessment during the last four years. Average score (1 = lowest; 5 = highest) has increased from 3.9 in 2006 to 4.4 in 2009.
Southwestern Oklahoma State University
Curriculum-embedded methods that are used in all general education courses on the Weatherford and Sayre campuses assess four main goals: (1) competency in written and oral communications; (2) mastery of core mathematics concepts and understanding of mathematics principles, symbols, and logic; (3) skills in problem solving and critical and creative thinking; and (4) understanding and competency in use of technology, computer literacy, and information systems.
Faculty members rely on feedback from formative methods to improve instruction and modify activities. Sharing the information with students and making the assessment part of the course requirements create an environment for meaningful participation of students.
Data indicate that benchmarks for student achievement are being met in the general education courses. Faculty reported revisions in methods of assessment and instruction as well as refinements of course objectives. The flexibility of curriculum-embedded assessment allows changes to be made and efficacy of changes to be assessed more efficiently.
While our current reporting of mid-level assessment is based on a two-year course rotation of curriculum-embedded assessments, ACT's CAAP has been utilized annually since Fall 2007. Out of 403 students 31
invited to participate in the CAAP this year, 40 (10 percent) actually contributed to this important project. The impressive results of the efforts from our examinees were slightly better than the national average. On a scale of 40-80, SWOSU juniors received an average score of 64.3 on the critical thinking objective test compared to the national average of 62.3. In reading, our students earned an average score of 64.0; the national 2 average was 62.5. SWOSU juniors did slightly better in the reading content area of social sciences than in the arts and literature portion of the reading test. SWOSU students received an average score of 66.2 on the Writing Skills objective test compared to the national average of 64.1. On a scale of 1-6, SWOSU juniors received an average score of 3.3 on the Writing Essay test; the national average was 3.2.
Cameron University
ACT’s CAAP examinations are used to measure General Education outcomes in Mathematics and English. Communications department faculty members measure the outcome of speaking effectively, using a rating system adopted for their program. This same process is used by some of the academic departments for program specific measures because it has excellent inter-rater reliability.
The General Education Committee is coordinating assessment activities with the Institutional Assessment Committee. Academic departments are reporting the results of assessment of student learning in their courses through the institution’s Program Quality Improvement Report process.
Langston University
The instruments used to assess college readiness as a secondary measure were also used to assess mid-level accomplishments. These instruments make comparisons easy and provide a predictive value for academic attainment in the established general education competencies. Results from the mid-level assessment are made available to all academic units, the responsibility managers, and executives who supervise and provide direction to responsibility managers. Additionally, the general education committee reviews the data and makes recommendations to the Academic Policy Committee and Faculty Senate for action.
University of Science and Arts of Oklahoma
USAO identified 170 juniors for mid-level assessment. The assessment tool is the ACT CAAP test. Students were informed that he/she needed to complete the test on the date established. The students who did not complete the test had an enrollment hold placed in their student file. Make up testing was available. Prior to enrolling for the next trimester, the student completed the test. Two testing thresholds exist, one beginning in March and ending in August.
CAAP tests in critical thinking, math, science, reading and writing are given at random. Each student is only required to complete one exam. Randomization of test distribution resulted in 35 juniors completing the writing exam, 35 completed the math exam, 35 completed the reading exam, 36 completed the critical thinking exam, and 29 completed the science exam. The mean scores for USAO ranged from 1.3 points above to 3.9 points below the national mean. Improvements were seen in the writing and math scores. Slight declines were seen in science, reading and critical thinking. This information has been provided to the departments for their action.
Oklahoma Panhandle State University
OPSU uses the Oklahoma General Education Test (OGET) to assess mid-level performance. The OGET 32
exam covers General Education content—English, mathematics, science, social studies, humanities, and writing—and can be taken by any student at any time via a computer. This assessment activity was linked to the general education program competencies by comparing student scores on these exams to the cut scores and state averages. All students applying to the Teacher’s Education Program were required to take the OGET exam. There were a total of 34 students who took the OGET during the 2008-2009 academic year. The students were motivated to do well on the OGET because they would not be allowed to proceed in their chosen academic areas unless they passed the exams. The results revealed that 29 students passed the OGET, and 5 failed providing a pass rate of 85 percent.
In the spring of 2009, OPSU implemented a new general education assessment plan. OPSU established three main goals (oral and written communication, analytical and quantitative reasoning, global understanding and cultural awareness) and with a total of ten student learning outcomes: read critically and express ideas clearly, logically, and persuasively in standard written English; express ideas clearly, logically and persuasively in standard spoken English; apply mathematics as a language; apply biological and physical science principles to the natural world; utilize principles of computer systems throughout the curriculum; interpret relationships of the creative processes, aesthetic principles and historic traditions of one or more of the humanities; identify the principles of history and culture of the United States; identify the principles of government, politics, and political organizations; recognize the ideas and principles that influence human thought and behavior; and identify the economic principles that effect macroeconomics. These outcomes were assessed using rubrics and/or pre-post testing in the required general education coursework at OPSU.
Rogers State University
Mid-level assessment relies upon course-embedded assessment of student performance by faculty. This strategy has its foundation in nine original General Education outcomes identified by RSU faculty. Faculty members also specify the core knowledge areas of each general education course and establish appropriate performance criteria and assessment procedures to measure student mastery of course content.
Reading, writing, mathematics, critical thinking, and other institutionally recognized general education competencies are addressed by the General Education Program outcomes described in the electronic portion of this report. The responsibility for the related data collection rests with the faculty who teach the general education courses, as well as the departments whose courses are part of the General Education Program. The University Assessment Committee is responsible for oversight of the mid-level assessment process and any curricular modifications that occur as a result of the assessment process.
The faculty and administration at RSU recognize that the mid-level assessment model has, in the past, tended to treat General Education as sub-components of the programs of the various academic departments rather than as an inter-disciplinary program that does not reside within individual departments, but instead crosses the entire curriculum.
The departmental general education assessment report/plan form has been revised in order to clarify, verify and amplify departmental assessment efforts. The University Assessment Committee, as peer reviewers, will examine the 2008-2009 general education reports and provide feedback for improvement to the departments.
Connors State College
The ACT CAAP tests were administered to all students planning to graduate in the 2008-2009 academic year. Advisors enroll students in EDUC 2320 Outcomes Assessment and students attend the class on the 33
designated day and time to complete the assessment. Test-times for the CAAP were intentionally scheduled to maximize student participation. Reminder-letters were mailed a week prior to testing to encourage partnership. Students were informed that the CAAP results would be on their transcript. It was explained to students at the onset of testing that results would be used for assessment of student learning.
Student progress was tracked into future semesters utilizing transfer reports from NSU, OSU, and OU; most CSC students transfer to NSU. Students transferring to NSU (n = 327) dropped an average of 0.88 grade points at NSU and students transferring to OU (n =6, too small to assume a representative sample) also decreased. There were 68 students who transferred to OSU, with 17 making higher GPA’s and 51 lower. A post-transfer drop in grade point is typical.
Murray State College
MSC uses the ACT CAAP test to measure reading, writing, mathematics, and critical thinking. The CAAP is curriculum based, so results can be related to college courses. The CAAP items are drawn from the general education college materials in humanities, social and natural sciences, and mathematics.
Two hundred thirty two students participated in the CAAP test for the 2008-2009 academic year. The identified population included potential 2009 spring graduates who entered MSC as first-time freshmen. The students were notified they were required to select one of three scheduled dates to participate in the CAAP. Students were encouraged to do their best on the CAAP through two means: (1) a sense of student responsibility to MSC and future students in that scores could impact the curriculum taught and (2) a direct benefit in that the scores could be reported to the four-year institution to which the student is transferring.
Northeastern Oklahoma A&M College
General education is an integral part of the curriculum at NEO A&M. The college uses the Measure of Academic Proficiency and Progress (MAPP) to assess general education and critical thinking skills. The Testing and Assessment Center personnel administer the assessment during the students’ final semester at NEO A&M. A comparison of the assessment results enables the College to determine value-added particularly in the realm of "general education."
The mean scores for each category had gradually increased from the academic year 2004-2005 through 2007-2008. However, the mean scores for each of the subcategories and the total mean score for students enrolled in a transfer degree program decreased for the students graduating during the 2008-2009 academic year. In contrast, the mean scores for each category and the total mean score increased for the technical/occupational programs. The students in these programs scored higher in all categories than those students enrolled in transfer degree programs. Both cohorts scored highest in reading. The technical/occupational cohort scored three (3) points higher than for the transfer degree programs in reading.
Northern Oklahoma College
NOC is in the process of evaluating a 3 year linkage report of all ACT subsections, COMPASS placement/challenging exams and the CAAP exams. The faculty will be reviewing the linkage reports in conjunction with the Office of Academic Affairs. It is anticipated that the faculty will begin an intensive discussion of the General Education program.
This conversation was put on hold while NOC finished the 3 year self-study and the HLC visit. The 34
administration and faculty scheduled the Fall 2008, following the Spring 2008 HLC visit, as the target date to begin an earnest evaluation of the general education program. As a result of the HLC visit, NOC is doing additional work concerning the assessment of program outcomes, especially in the transfer programs. Each division has spent a great deal of time reviewing the suggested outcomes. The HLC team requested less dependence on the CAAP exams. As a result the faculty members are looking at other measures. The students who have taken the CAAP exams are staying close to national norms.
Tulsa Community College
TCC’s mid-level assessment process, used for the past several years, centered around evaluation of one of the institution’s general education goals college-wide each year on a rotating basis. During the 2008-2009 academic year, faculty assessed Engaged Learning. A total of 4,372 students participated in the assessment of this general education goal, TCC third general education goal, and 97 percent success rate was indicated.
Additionally, during the 2008-2009 academic year, each general education goal was assessed by one or more disciplines or initiatives. Reading, mathematics, College Strategies, and all program/discipline faculty in the Critical Thinking Initiative (CTI) at the west campus linked curriculum to specific general education program competencies and assessed them accordingly. To assess the transition from entry-level to college mid-level, student measures of success were identified and progress was evaluated for students enrolled in ENGL 1003, College Strategies, students who enrolled in ENGL 0963, College Survival, and first-time freshmen who enrolled in neither course during the 2008-2009 academic year. Of these 3,655 students, 1,712 enrolled in College Strategies, 92 enrolled in College Survival, and 1,851 enrolled in neither course. To evaluate student success in both developmental courses and gateway courses, assessments were conducted between course grades of students who enrolled in Strategies, students who enrolled in College Survival, and first-time freshmen who enrolled in neither course. Few students from the College Survival course took college level course work, and most significant results are between students who enrolled in Strategies and first-time freshmen who did not. Strategies students earned significantly higher grades than non-Strategies first-time freshmen in: Basic Math Writing II College Algebra Biology for Majors US History 1492 to Civil War Era Introduction to Psychology
These results suggest that during the 2008-2009 academic year, student success was positively affected in College Strategies, increasing in persistence from fall to spring and persistence from fall to fall, increasing success (“C” or better) in six developmental and gateway courses, and increasing the efficacy of student self-testing and information processing abilities.
Oklahoma State University – Oklahoma City
In the 2008-2009 academic year, the Assessment Committee used the ACT’s CAAP as a mid-level assessment instrument. The Assessment Committee reviewed the different methods of assessing Mid-Level General Education and decided to have a sample of students complete the ACT CAAP starting in the fall of 2007 and continuing in the spring of 2008. The fall 2007 administration included the Writing and Critical Thinking assessment. The administration of the mathematics and reading was during the spring 2008 term. OSU-OKC administered the CAAP assessment to program courses whose instructors were approach by their respective division heads. The deciding factor was to assess students in courses had the respective general education prerequisite, so the likelihood the student having these skills was 35
higher. Because of the time difference between the assessments, the same students taking all four assessments were unlikely. Therefore, one should look at each subject area individually as opposed to all four together, since the sample of students was different for each assessment. The approach to the CAAP assessment was to take a snapshot of the students who are mid way through the educational experience; therefore, the reporting is at the institution level and not an indicator of individual student progress. The administration and analysis of CAAP results were also used during the 2008-2009 term.
36
Oklahoma State University Institute of Technology
Mid-level assessment of general education competencies was conducted as described in each program’s academic assessment plan. These assessments were developed by faculty specifically for each program. Five Core Objectives common to all programs of study, based on reading, writing, mathematics, critical thinking, ethics, diversity, and technical competencies grew from this process. A sixth core objective, Service Learning, grew from the emphasis placed on service learning by the accrediting bodies. The objectives are: 1) Communication – effectively communicate electronically, verbally, and in writing; 2) Critical Thinking – demonstrate logical, systematic problem-solving techniques; 3) Ethics – develop and display a sense of personal, social and professional work ethics; 4) Culture, History, and Diversity – explain the cultural heritage and primary elements of the history and government of the U.S. and its people, especially as it impacts one’s industry or field of study; 5) Technology – access and use technology appropriate to one’s field of study; and 6) Service Learning – Provide opportunities for students to effectively utilized learned technologies and processes to aid various constituencies in the community.
Western Oklahoma State College
In the past, WOSC has used CAAP testing to determine program competencies. CAAP, a product of ACT, was used so scores could be linked to student’s COMPASS and ACT scores. However, only those students who have taken both COMPASS and ACT tests were linked since both scores are needed to make a valid comparison. The comparison would indicate whether students have made progress since entering and attending the institution.
The Assessment Committee proposed to change the process and way the outcome testing is to be done. Therefore, the CAAP testing was not given for the 2008-2009 year and will resume for the 2009-2010 academic year. Changes are being made to the process and the testing rules in order to have a true sampling and comparison to determine program competencies.
Redlands Community College
The mean scores on the CAAP exams were examined in the areas of reading, mathematics, and science. The Assessment through Writing pilot study was initially administered during the 2001-2002 academic year, and has been continued through 2008-2009. English Composition II students wrote an essay of their choice from a list of prepared topics. Topics were drawn from the following areas: problem solving, leadership, and social problems. An evaluation rubric was attached to the list of essay topics for students to review prior to writing their essays. A team of RCC faculty from across the curriculum evaluated the student essays. Using a holistic grading system the evaluation team assessed the student’s ability to demonstrate knowledge of Standard English, to demonstrate the ability to write in an acceptable essay form, and to demonstrate critical thinking skills. Students not meeting the established standards can receive additional assistance by accessing a tutor through the Redlands Peer Tutor Program, by accessing computer tutorials through the Assistance Center, or by auditing an ENGL 1113 class.
Carl Albert State College
The objectives of mid-level assessment are to assess all students who have attained 45 or more hours in order to determine students’ academic progress and learning competencies in the areas of reading, writing, mathematics, and critical thinking. The results from mid-level assessment will be used to evaluate, to improve, and to recommend any changes to the general education and academic program curricula. 37
During the fall and spring semesters of 2008-2009, all CASC students that had completed 45 or more hours were notified about the CAAP testing and asked to participate. A total of 153 students participated in Fall 2008 and 224 students participated in Spring 2008 for a total of 377 for the academic year. Test modules administered were reading, writing skills, mathematics, science reasoning, and critical thinking.
The results of the CAAP were compared to national norms and to the performance of 21 CASC students who had tested with the ACT as entry-level assessment. Based on those results, CASC students performed at or close to national norm levels in all four levels.
Rose State College
RSC has been assessing all classes for critical thinking, effective communication, technology proficiency, and quantitative literacy, respectively, since Fall 2003. During the Fall 2008 semester, the full-time faculty reported on their assessment of 636 classes for effective communication. A total of 10,109 students participated with 8,540 (or 84.48 percent) demonstrating successful effective communication skills based on the context-specific criteria of the individual professors. Spring 2009 adjunct faculty assessed 420 classes. A total of 6,259 students participated with 5,151 (82.30 percent) demonstrating successful effective communication skills based on the context-specific criteria of the individual professors.
The Academic Assessment Committee requested that full-time faculty complete a survey during the Spring 2009 semester related to any changes they had made to their assessment of technology proficiency or new methods they planned to implement for Fall 2009 as a result of the outcomes and/or their classroom assessment experience. Information Technology Services continues to make available, through PeopleSoft and the College’s Internal Website, the ability to allow faculty to submit their assessment reports online.
Oklahoma City Community College
General Education assessment at OCCC examines student’s academic progress and learning on the four general education student learning outcomes including: 1) Human Heritage, Culture, Values and Beliefs; 2) Communication and Symbols; 3) Social, Political and Economic Institutions; and 4) Science. In 2009 the faculty General Education Committee decided to initiate a review which would include using rubrics to evaluate student artifacts. The General Education Committee created interdisciplinary teams with members from multiple divisions. Each team consisted of five members with two members specifically teaching in one of the General Education Core Areas. Also, at least one team member was a representative of the General Education Committee. The goal of this process was to evaluate one hundred artifacts from students having attained at least 35 hours of General Education Courses from OCCC. Areas tested include social institutions; writing; mathematical methods; scientific methodology; human heritage, culture, and value; and public speaking. Proficiencies in these areas varied widely among those tested. 38
Program Outcomes Assessment
Program outcomes assessment, or major field of study assessment, is designed to measure how well students are meeting institutionally stated program goals and objectives. As with other levels of assessment, selection of assessment instruments and other parameters (such as target groups, when assessment occurs, etc.) is the responsibility of the institution. Institutions are encouraged to give preference to nationally standardized instruments that supply normative data. The instrument selected should measure skills and abilities specific to the program and to higher level thinking skills. Results are used to revise curricula.
Based on examination of the various types of outcome data described below, institutions have made changes to include more direct assessment of student learning and assessment processes resulting in program improvement. Examples of changes made include providing annual funding for program assessment, implementing more direct communication with students and faculty members to provide feedback on program outcomes and assessments, and inclusion of available technology both in program outcomes as well as assessment tools.
Listed below are the methods and tools used by each institutions to assess program outcomes.
University of Oklahoma
Capstone courses, standardized exams, course evaluations, exit interviews, student surveys, portfolio reviews, alumni surveys, employer surveys, advisory board surveys, local/national contests, transcript review, professional exams/certifications
Oklahoma State University
Capstone courses, licensure exams, exit interviews, portfolios, projects and presentations, surveys, evaluations, writing assessments, ETS major field exams, standardized exams
University of Central Oklahoma
Surveys, exit interviews, focus groups, portfolio reviews, writing assessments, presentations, capstone courses, essays, written evaluations, standardized exams, course embedded assessment, ETS Major Field Exam, pre-post tests
East Central University
Portfolios, surveys, licensing and certification exams, capstone courses, locally developed exams, presentations, ETS Major Field Exam, comprehensive exams
Northeastern State University
Capstone courses, certification tests, ETS major field exams, portfolios, exit surveys, writing assessments, standardized exams, pre-post tests, presentations, locally developed exams
Northwestern Oklahoma State University
Licensure exams, course embedded assessment, ETS major field exams, exit interviews, capstone courses, portfolio reviews, surveys, locally developed tests, standardized exams 39
Southeastern Oklahoma State University
Standardized exams, locally developed comprehensive exams, certification tests, surveys, interviews, senior seminars, portfolio reviews, pre-post tests, capstone courses, ETS major field exams, exit interviews, oral presentations
Southwestern Oklahoma State University
Portfolios, exit interviews, ETS major field exams, surveys, course embedded assessment, standardized tests, licensure and certification exams
Cameron University
Portfolio reviews, locally developed and standardized tests, capstone courses, exit interviews, surveys, benchmarking
Langston University
Standardized tests, ETS major field exams, portfolios, locally developed tests, presentations, comprehensive exams, licensure and certification exams
University of Science and Arts of Oklahoma
Portfolios, locally developed and standardized tests, licensure and certification exams, comprehensive exams, ETS major field exams
Oklahoma Panhandle State University
Employment data, graduate school acceptance, standardized tests, exit interviews, surveys, course evaluations, capstone courses, licensure and certification exams, portfolios
Rogers State University
Portfolios, capstone courses, licensure and certification exams, standardized exams, surveys, ETS Major Field Exam, presentations
Connors State College
Licensure and certification exams, capstone courses
Eastern Oklahoma State College
Pre- and post-tests, locally developed exams, surveys, course embedded assessments
Murray State College
Locally designed tests, licensure exams
Northeastern Oklahoma A&M College 40
Capstone courses, licensure and certification exams, surveys, projects
Northern Oklahoma College
Licensure and certification exams
Tulsa Community College
Course-embedded assessment, employer surveys, licensure and certification exams, self-studies
Oklahoma State University – Oklahoma City
Capstone courses, portfolios, employer surveys, student surveys, pre-post tests, standardized and locally developed exams, comprehensive exams, certification exams
Oklahoma State University Institute of Technology
Capstone courses, comprehensive exams, pre-post tests, licensure and certification exams
Western Oklahoma State College
Course-embedded assessments, evaluations, portfolios
Redlands Community College
Pre-post tests, portfolios, focus groups, internships, advisory committees, surveys
Carl Albert State College
Licensure exams, surveys, capstone courses, program reviews, transfer reports, locally developed exams
Seminole State College
Course-embedded assessment, surveys, transfer reports
Rose State College
Capstone courses, portfolios, surveys, licensure exams, transfer reports
Oklahoma City Community College
Capstone courses, surveys, licensure exams 41
Student Satisfaction Assessment
University of Oklahoma
Local student satisfaction survey; National Survey of Student Engagement (NSSE), Complete Withdrawal Information Survey
Oklahoma State University
Undergraduate Program Alumni Survey, Graduate Program Alumni Survey, Graduate Student Satisfaction Survey
University of Central Oklahoma
National Survey for Student Engagement (NSSE), Cooperative Institution Research Project (CIRP), Graduating Student Survey (GSS)
East Central University
ACT Survey of Student Opinions
Northeastern State University
College Student Experiences Questionnaire, ACT Student Opinion Survey, Senior Survey, student evaluation of classes, Freshmen Inventory, UCLA Freshman Survey
Northwestern Oklahoma State University
Student Opinion Survey
Southeastern Oklahoma State University
Academic Advising and Outreach Center, College Outcome Survey, Council for the Advancement of Standards for Student Services, Graduate Survey, Junior Survey, Library Survey, National Survey of Student Engagement (NSSE), Noel-Levitz Student Satisfaction Inventory, Student Opinion Survey
Southwestern Oklahoma State University
Course/Instructor evaluations, ACT Survey of Student Opinions, Alumni Survey, NSSE
Cameron University
National Survey of Student Engagement (NSSE)
Langston University
ACT Student Opinion Survey
University of Science and Arts of Oklahoma
42
Course evaluations, Senior survey, NSSE
Oklahoma Panhandle State University
Student Satisfaction survey, Student Needs survey, Graduation survey, Alumni survey
Rogers State University
Student Opinion Survey, Course evaluations, Graduate Survey, NSSE
Connors State College
ACT Faces of the Future, housing and student activities surveys, library survey
Eastern Oklahoma State College
ACT Student Opinion Survey for Two Year Colleges
Murray State College
Locally developed Student Satisfaction Questionnaire
Northeastern Oklahoma A&M College
Student Satisfaction Survey
Northern Oklahoma College
Community College Survey of Student Engagement (CCSSE)
Tulsa Community College
Student Support Services survey
Oklahoma State University – Oklahoma City
Student Satisfaction surveys, Graduating Student surveys, Post-Graduation surveys
Oklahoma State University Institute of Technology
Instructor/Course Surveys, Noel-Levitz Student Satisfaction Inventory
Western Oklahoma State College
Entering Student Survey, Continuing Student Opinion Survey, College Outcomes Survey, Alumni Survey
Redlands Community College
Community College Survey of Student Engagement (CCSSE) 43
44
Carl Albert State College
ACT Alumni Survey for Two-Year Colleges
Seminole State College
Student Feedback on Classroom Instruction Form, ACT Faces of the Future Survey, Graduate Opinion Survey
Rose State College
ACT Student Satisfaction Survey, Graduate Survey
Oklahoma City Community College
ACT Student Opinion Survey, Community College Survey of Student Engagement (CCSSE), Student Input on Instruction (SII), graduate survey
45
Graduate Student Assessment
University of Oklahoma
Thesis reviews, teacher licensure exams, course evaluations, internships, exit surveys, alumni surveys, comprehensive exams, presentations/publications, job placement, employer surveys
Oklahoma State University
Survey of Alumni of Graduate Programs, Graduate Student Satisfaction Survey, comprehensive exams, presentations/publications, portfolios, exit interviews, National Certification Exam, ETS MBA Major Field Exam, Curriculum Examination for Oklahoma Educators
University of Central Oklahoma
Theses, National Praxis II Exam, Oklahoma State Practicum I Test, practice exam for licensure, presentations/publications, Board of Certification Exam, comprehensive exams
East Central Oklahoma
Portfolios, Various Constituent Surveys (VCS), State Elementary Principal Certification Exam, Oklahoma State Subject Area Test (OSAT), comprehensive exams, Oklahoma State Teacher Certification Exam, employer surveys, graduate surveys, Oklahoma Teacher Certification Test (OTCT), Oklahoma Teacher Certification Test for School Counselors (OTCT), Certification Examinations for Oklahoma Educators (CEOE)
Northeastern State University
National examinations, exit interviews, portfolios, theses
Northwestern Oklahoma State University
Comprehensive exams
Southeastern Oklahoma State University
Teacher certification tests, Counselor Preparation Comprehensive Exam (CPCE), Oklahoma State Certification Exam, Oklahoma Subject Area Test (OSAT), presentations, exit surveys, Advanced Certificate Portfolio (ACP), teacher evaluations
Southwestern Oklahoma State University
Comprehensive exams, portfolios, Oklahoma Subject Area Test (OSAT), Internship Candidates’ Evaluation, employer surveys
Cameron University
Portfolio reviews, performance ratings, standardized examinations, exit interviews, employer perceptions, graduate surveys, capstone courses, benchmarking
Langston University
Comprehensive exams, portfolio reviews, graduate surveys, National Physical Therapy Examination 46
Licensure and Certification
Number of Number of
Students Students
Program and/or Exam Tested Passing
University of Oklahoma
No licensure or certification data were reported.
Oklahoma State University
Elementary Education (OSAT Subtest 1)
114
104
Elementary Education (OSAT Subtest 2)
110
103
Elementary Education (OGET)
98
91
Early Childhood Education (OGET)
42
40
Mechanical Engineering (Fundamentals of Engineering exam)
68
55
Civil Engineering (Fundamentals of Engineering exam)
38
31
Secondary Education Social Studies (OGET)
41
39
Elementary Education (OPTE P-8)
104
101
Early Childhood Education (OPTE P-8)
35
35
Agricultural Education (OSAT)
38
38
University of Central Oklahoma
Nutrition and Dietetics
9
6
Nursing
185
165
Speech Pathology
23
21
Teacher Education
189
187
East Central University
Elementary Education
71
59
Nursing
51
48
Early Childhood Education
24
22
Physical Education Teacher Certification
22
14
Criminal Justice
14
14
English
13
11
History Education
11
10
Mathematics Education
7
7
Music Education
6
6
Family & Consumer Science Education
3
2
Northeastern State University
Elementary Education - BS/ED
341
Early Childhood Education - BS/ED
75
Health and PE - BS/ED
30
28 47
Northeastern State University (cont’d)
Counseling Psychology - MS
29
Vision Science - BS
28
28
Nursing - BSN
26
26
Special Ed - Mild/Moderate Disorders - BS/ED
23
18
Spch. and Lng. Pathology - BS
19
19
Social Studies Education - BS/ED
17
Speech, Language, Pathology - MS
17
17
Northwestern Oklahoma State
Nursing
20
20
Math Education
2
2
Natural Science Education
2
0
Education
20
17
Elementary Education
100
68
Social Science Education
13
9
Special Education
11
7
Health & Sports Science Education
4
4
Southeastern Oklahoma State
Elementary Education
81
64
Physical Education
30
23
English
16
10
Principal Core
14
13
Principal Elementary
11
9
Mild/Moderate
7
7
Mathematics
6
6
Biological Sciences
5
5
Reading Specialist
5
5
United States History & Principal Secondary
5
3
Southwestern Oklahoma State
Pharm.D.
84
82
Master of Education in Educational Administration
183
140
Elementary Education
126
93
Nursing
33
32
Technology (Engineering & Industrial)
19
10
School Counselor
18
18
Occupational Therapy Assistant
10
10
Early Childhood
10
3
Music Education
12
12
Physical Therapist Assistant
10
10
48
Cameron University
OSAT - Elementary Education
73
73
Langston University
National Physical Therapy Examination
10
8
University of Science and Arts of Oklahoma
Cert. Exam for Oklahoma Educators Elem Ed
23
23
Cert. Exam for Oklahoma Educators Early Childhood
10
10
Cert. Exam for Oklahoma Educators Deaf Ed
4
4
Cert. Exam for Oklahoma Educators Mathematics
3
3
Cert. Exam for Oklahoma Educators Social Studies
3
3
Cert. Exam for Oklahoma Educators Science
1
1
Cert. Exam for Oklahoma Educators English
1
1
Cert. Exam for Oklahoma Educators Physical Ed.
1
1
Cert. Exam for Oklahoma Educators Art
1
1
Oklahoma Panhandle State University
OPTE exam #75, State HPER Professional Licensure Exam
6
6
OPTE exam #76, State HPER Professional Licensure Exam
5
5
OSAT exam #50, State Elementary Education Content Licensure
6
3
OSAT exam #51, State Elementary Education Content Licensure
6
5
OSAT exam #7, Content Licensure Exam English
2
2
OSAT exam #11, Content Licensure Exam Advanced Math
1
1
OSAT exam #17, Content Licensure Exam US/OKHist/G
1
1
OSAT exam #25, Content Licensure Exam Mid Level Math
1
1
OSAT exam #42, Content Licensure Exam Agricul Ed
1
1
Connors State College
RN-NCLEX for Nursing Program
70
95.65
CDA Credential
6
100
Eastern Oklahoma State College
Nursing: NCLEX
70
70
Murray State College
Nursing
62
58
Physical Therapy Assistant
13
12
Veterinary Technology Assistant
8
8
Northeastern Oklahoma A&M College
Associate Degree Nursing - Registered Nurse
59
45
Medical Laboratory Technician
8
7
Physical Therapist Assistant
11
9
49
Northern Oklahoma College
ADN-Tonkawa
28
23
ADN-Enid
34
26
ADN-Stillwater
29
24
Rogers State University
NURSING (AAS) NCLEX-RN
61
47
National Registry EMT Paramedic
15
8
Tulsa Community College
Dental Hygiene
12
12
Respiratory Care
25
23
Phlebotomy
7
7
Medical Laboratory Technology
9
7
Nursing
107
103
Oklahoma State University – OKC
Sign Language Interpretation
10
8
Oklahoma State Veterinary Technician Exam
14
14
Veterinary Technician National Exam
14
7
CLEET Certification Exam
18
18
Nursing Exam
126
113
Oklahoma State University Institute of Technology
Environmental Protection Agency (EPA) Certification
15
15
National Council License Examination (NCLEX)
14
10
Western Oklahoma State College
Radiologic Technology
13
10
Nursing (RN)
79
73
Redlands Community College
Nursing Program
27
26
Carl Albert State College
Nursing
22
22
Physical Therapy Assistant
13
13
Radiography
6
6
Rose State College
Nursing Science (AAS)
116
110
Dental Hygiene (AAS)
12
12
Clinical Laboratory Tech (AAS)
13
13
Radiologic Technology (AAS)
15
15 50
Respiratory Therapist (AAS)
23
23
Health Information Tech (AAS)
10
10
Accounting (AAS) (ACAT)
13
2
Oklahoma City Community College
EMS, Paramedic
16
15
Nursing
180
167
Occupational Therapy Assistant
19
16
Physical Therapist Assistant
16
10
51
Assessment Budgets
Regents’ policy states that academic service fees “shall not exceed the actual costs of the course of instruction or the academic services provided by the institution.” (Chapter 4 – Budget and Fiscal Affairs, 4.18.2 Definitions)
Institution
Assessment fees
Assessment salaries
Distributed to other departments
Operational costs
Total Expenditures
University of Oklahoma
$681,673
$50,000
$220,872
$100,000
$370,872
Oklahoma State University
$496,824
$289,842
$266,900
$49,150
$605,892 Total Research $1,178,497 $339,842 $487,772 $149,150 $976,764
UCO
$0
$143,550
$70,000
$70,000
$283,550
East Central University
*
*
$6,305
*
$6,305
Northeastern State University
$215,516
$198,648
$0
$39,035
$237,683
Northwestern Oklahoma State University
$0
$102,788
$7,960
$16,925
$127,673
Southeastern Oklahoma State University
$0
$83,752
$10,000
$44,248
$138,000
Southwestern Oklahoma State University
$0
$183,200
$5,755
$65,000
$253,955
Cameron University
$288,078
$304,479
$15,666
$80,275
$400,420
Langston University
$70,401
$121,773
$425
$19,487
$141,685
University of Science and Arts of Oklahoma
$43,352
$41,577
$0
$14,286
$55,863
Oklahoma Panhandle State University
$33,128
$31,078
*
$2,050
$33,128 Total Regional $650,475 $1,210,845 $118,161 $349,256 $1,678,262
Carl Albert State College
$2
$0
$0
$31,600
$31,600
Connors State College
$0
$16,500
$10,000
$9,000
$35,500
Murray State College
$0
$58,711
$11,388
$81,197
$151,296
Northeastern Oklahoma A&M College
$90,000
$65,000
$6,000
$28,180
$99,180
Northern Oklahoma College
$83,000
$110,000
$0
$21,000
$131,000
Tulsa Community College
$470,691
$88,868
$58,845
$322,978
$470,691
OCCC1
$225,442
$215,836
$25,000
$14,000
$254,836
OSU-OKC
$116,059
$179,363
$10,000
$14,000
$203,363
OSU-IT
$70,640
$104,685
$0
$30,200
$134,885
Rogers State University
$224,423
$192,838
$0
$13,000
$205,838
Redl