Judy Marwick - Prairie State College
Abstract: Mathematics faculty at Prairie State College have spent the last three years working on an assessment plan that will both serve as a means of improving mathematicws instruction and satisfy North Central Accreditation requirements for non-grade based assessment. The journey as well as the final product will be discussed.
Assessment has always been a component of education. However, this topic has received much greater emphasis in the last two decades beginning with a series of commission studies of higher education prescribing assessment as the remedy to cure higher education’s ills (St. Louis Community College (CC), 1997). Coincidently, during this same period, there has been a paradigm shift in community college education changing the focus from teaching to learning (St. Louis CC, 1997). Thus the movement of academic outcomes assessment was born. This movement has much in common with the birth of most community colleges several decades earlier since both had student needs at their core (Ewell, 1994). Who then, could be better suited to lead educators in the pursuit of student learning through the means of outcomes assessment than the community college? This paper will explore the topic of academic outcomes assessment, its history, and present practices in the community college. The focus will be on the movement’s attempts to provide a mechanism for improving student learning.
Community colleges are well situated to act as leaders in the area of academic outcomes assessment since the purpose of this movement, improving student learning, is one and the same with the mission of the college. The concept of assessment is not new to the community college. What is new, is the increased focus on finding more effective instruments to assess student learning and ensure systematic feedback from the data. "Community colleges will be asked to respond to calls for increased educational excellence while maintaining access to educational opportunity for students who are least prepared to succeed" (Hirsh, 1987, p. 15). Assessment will be the tool used to accomplish this.
While educators have practiced assessment in many forms, the academic outcomes assessment movement focuses on measuring student learning as a direct outcome of the teaching process. It is essential for educators to state what is being taught and show clear evidence that the students have accomplished the stated objectives. The assessment initiative’s purpose is to encourage excellence in teaching and thereby improvement in learning. This differs from measures of institutional effectiveness such as placement rates, transfer rates, enrollments, and job placements. These measures are still important but they are not sufficient to illustrate learning and not what is meant by academic outcomes assessment. In a North Central Association (NCA) document on accreditation Cecilia Lopez states that "an institution’s program for the assessment of student academic achievement should be an independent, clearly identifiable component of any more comprehensive program to evaluate overall institutional effectiveness" (1997, p. 4). It is important to distinguish between assessing student academic achievement and evaluating the institution as a whole.
In the past many faculty have used course grades as the primary means of academic outcomes assessment. Many feel that this is no longer sufficient, asserting that grades are not a relevant measure because they are more likely to compare students within a class or school rather than to measure what has actually been learned. The purpose of academic outcomes assessment is not only to provide greater accountability but more importantly to improve the quality of student learning (Bray & Belcher, 1987).
Programs for academic outcomes assessment need to be widely accepted and routinely updated. The program should be ongoing and related to other planning and budgeting processes. The assessment plan must provide explicit public statements regarding the expectations for student learning and the information gained must be used to improve student learning (NCA, 1997). The goal of assessment "is to develop, initiate and sustain a continuous quality improvement process across the campus" (Triton, 1997, p. 3).
The scope of assessment is broad. Bray and Belcher define the process of classroom assessment as providing individual college teachers with useful feedback on what, how much, and how well their students are learning. They list the three areas which should be assessed as course-related knowledge and skills, learner attitudes, and learner reactions to instruction (1987). Palmer points out the importance of assessing skills at entrance, as an ongoing part of the curriculum, and as a follow-up evaluation upon program completion (1987). Whatever form assessment takes, its function is to engage faculty in thinking about the purposes and mission of education, that is student learning.
From 1960 to 1983 there was a seven fold growth in the number of community colleges (Bray & Belcher, 1987). Following this period of expansion came a movement to take stock of the quality of education being provided. This movement was led by government officials concerned about whether or not education was providing the quality of service that was necessary to educate the citizens to keep the country economically competitive. They wanted proof that institutions were delivering a quality service for the taxpayer’s dollar. The academic assessment movement was born from this concern. It was then joined by educational organizations and foundations as well as by accreditation commissions. With this impetus, educators have joined the bandwagon and, with their input, true academic outcomes assessment is emerging.
In 1983 A Nation at Risk documented the "rising tide of mediocrity" in education in the United States (Bray & Belcher, 1987). While the report focused mainly on the K-12 system, it led to similar concerns being expressed about higher education, particularly the community college because of its open door policy. Assessment as a solution to this concern was originally proposed in two national reports, Involvement in Learning (National Institute of Education, 1984) and Integrity in the College Curriculum (Association of American Colleges, 1985) (as cited in Ewell, 1994). In October of 1984 Secretary of Education William Bennett called on leaders of higher education to find ways to show the public that their institutions were making a valuable difference in the education and growth of their students (Bray & Belcher, 1987).
Ewell points out that in 1991 the United States had an "education president" and 50 state governors, all raising our expectations for education in a nearly unanimous voice. The cost of higher education was rising more quickly than inflation, and legislators needed accountability from educational institutions to show that the money was being well spent (Bray, 1987). Government officials saw great value in education but needed to justify the increasing costs to their constituents. "If payoffs for institutions to achieve high rates of degree-completion are high, one result may be to relax academic standards of progress" (Ewell, 1991, p. 48). Academic accountability seemed to be the solution.
Once born at the national level, the assessment movement was quickly embraced by legislators at the state level. State policy makers seized assessment as a powerful "lever for change" for improving quality in undergraduate education (Ewell, 1990). Ewell states that the majority of assessment activities occurring at American colleges and universities are due to state initiatives (1990). A fundamental shift in the way legislators perceive the public role of higher education had occurred. No longer would money be provided with trust that the educators would spend it wisely. Government officials began demanding documentation of value for the dollar spent. By 1992 more than three-quarters of the states had implemented a requirement that all public institutions engage in academic assessment and report on the results (Ewell, 1994). The specific requirements varied from state to state. Florida implemented The College-Level Academic Skills Test (CLAST) which is administered at the end of the sophomore year. In Illinois a program review is required of all public programs in higher education with results reported to the State (Ewell, 1990). The Tennessee Higher Education Commission provides financial support for conducting value-added testing, surveys of graduates or other forms of assessment. The Virginia Council administers a grants program focusing on quality improvement in instruction. (Ewell, 1994). Regardless of the mechanism used, the focus was on communicating for accountability in order to help higher education’s political supporters make the case for increased funding (Ewell, 1990).
The community college has moved to the forefront of the assessment movement for two reasons. First, because the political clout of the community college is generally weaker than that of four-year institutions, they are less able to resist governmental demands. Second, they have student learning, rather than faculty research, as their primary focus. In addition, the community colleges’ policy of open access has brought into question the value of the education being offered. In 1984 the Commission for Educational Quality stated that mere access had lost much of its value and questioned the quality of a community college education (Hirsch, 1987). On the other hand, because the community college is an institution whose mission focuses on student learning, assessment is not only highly consistent with current practices, but also gives these colleges a means to address questions of quality. Community colleges have always embodied a spirit of innovation and flexibility, and are therefore able to respond more quickly to new initiatives than other institutions. At the same time, the broad mission of the community college presents challenges for developing assessment tools that can accommodate all the varied educational tasks being undertaken.
Other agencies joined legislators to promote mandatory assessment initiatives for higher education. The American Association for Higher Education (AAHE) began an assessment forum in 1986 which meets annually at their national conference (Gardiner, Anderson, & Cambridge, 1997). AAHE has also sponsored a series of assessment workshops at University of California at Berkeley (Bray & Belcher, 1987). Since 1988 classroom research projects have been funded by the Ford Foundation and Pew Charitable Trusts (Bray & Belcher, 1987). The Fund for the Improvement of Post-Secondary Education (FIPSE) has also made available grants to support institutional efforts to evaluate educational effectiveness (Simmons, 1988). But perhaps the most far reaching efforts to mandate academic outcomes assessment at the post-secondary level have been made by the accreditation agencies.
Since the 1980s, the regional accrediting bodies, led by the Southern Association, have been pushing self-assessment as a means of accountability which does not impose outside standards (St. Louis CC, 1997). In 1989 the NCA commission called on all affiliated institutions to develop programs that assess and document student academic achievement (Lopez, 1997). It claimed that assessing student academic achievement is an essential component for evaluating overall institutional effectiveness. By July of 1995, all NCA’s affiliated institutions were required either to file an assessment plan with the commission or share a plan with an on-site team (NCA, 1997). These plans had to show evidence of an ongoing program for assessing academic achievement which assured that students were learning what faculty were trying to teach. Beginning with the 1995-96 school year, every self-study was also required to pay special attention to the institution’s assessment program (NCA, 1997). To be acceptable to NCA, an assessment plan must flow from the college’s mission and be faculty owned and driven. Thus the Commission does not prescribe a methodology for assessment but instead requires it to be structured around the college’s mission and educational purposes. NCA evaluators also require a seamless feedback loop so that what is learned as a result of the assessment process can be immediately applied in the classroom to improve teaching and learning. They also require that there be a process for evaluating the plan itself. NCA believes that assessment is cost-effective and does not restrict access, equity or diversity (Gardiner, 1997). "The Commission is committed to the concept that assessment of student academic achievement is critical to the future health of its institutions and to the public perception of higher education nationally" (NCA in Briefing, p. 4, as cited in Lopez, 1997).
The assessment of student learning is essential to Criteria Three and Four in NCA’s Handbook of Accreditation. Criterion Three states that "The institution is accomplishing its educational and other purposes" (NCA, 1997, p. 3). Criterion Four says, "The institution can continue to accomplish its purposes and strengthen its educational effectiveness" (NCA, 1997, p. 12). The pattern of evidence for Criterion Three and Four both emphasize a commitment to the use of assessment in evaluating and improving teaching and learning.
Although the current assessment movement was initiated by government and accrediting bodies, it can only produce the desired results if it is embraced by faculty. By requiring college faculty to explain what they are doing and to demonstrate how well they are doing it, outside agencies have caused discussion and introspection among faculty. "It appears that once teachers begin to raise questions about their own teaching and to collect data about its impact on learning, there is a self generated pressure to raise questions and discuss finding with colleagues" (Angelo & Cross, 1987, p. 382). Without requirements, some faculty would not have willingly participated in this collaborative effort, but once involved most become engaged and agree that the process of critical self examination is a positive vehicle for improving student learning. Now educators need to focus on taking assessment beyond politically mandated stages and have it assume its rightful role in improving curriculum and the quality of teaching and learning within the institution. If faculty do not adequately sustain this initiative, they will miss opportunities for increased funding. They will also motivate agencies outside the academy to attempt to assess educational results. Educators would do well to avoid this possibility and embrace academic outcomes assessment as their own, since good assessment practices will both improve their student’s learning and enhance their profession.
When NCA first mandated plans for academic outcomes assessment from each of its affiliated schools, it gave little direction as to what it considered good practices. NCA simply stated that it was important for each institution to develop an assessment plan that was individual, unique, and relevant to its institution (Resnick, 1987). In addition, they asked that the plan flow directly from the mission of the college and that it be owned and driven by the faculty with administrative support. Many colleges, such as Prairie State College in Chicago Heights, Illinois, struggled with this requirement because they did not know exactly what was required. As a result they treated academic outcomes assessment like any other requirement and assigned an administrator to address it and write the plan. This resulted in their plan being rejected twice by NCA. Not until the assessment plan emerged from extensive collegial discussion and planning was it accepted. Because so many colleges were struggling with this seemingly unstructured and vague new requirement, NCA has recently produced documents illustrating some best practices that evaluators have found on their accreditation visits (Lopez, 1997 & NCA, 1997).
NCA Consultant-Evaluators, a voluntary group of approximately 900 academic administrators and faculty believe that a program to assess student learning should:
NCA further states that there should be a strong relationship between institutional mission and the specific educational objectives of the individual departments. Since it is important the institution motivate, recognize and reward faculty assessment efforts, they recommend that the Chief Academic Officer of the college have overall responsibility for the assessment program but that a strong steering committee composed of faculty and administrators oversee the plan. Feedback loops are essential. Students should be made aware of the learning objectives and the purposes of assessment (NCA, 1997). NCA recommends that all three learning domains, cognitive, behavioral, and affective, be assessed (1997). Institutions must also give a realistic and appropriate time line for assessment, and the plan must show a strong likelihood that the program will lead to institutional improvement.
AAHE has also published documents to illustrate best practices for academic outcomes assessment. In 1992 the AAHE published principles of good practices for assessing student learning. These principles include:
These are somewhat different from NCA’s guidelines and offer faculty another perspective on this complicated issue. Faculty need to use all this information to create a plan for academic outcomes assessment which fits the needs of their program, their institution, and their students.
One of the most common mistakes faculty make when devising an assessment plan is to try to accomplish too much in too short a time. NCA (1997) recommends allowing twice the estimated time to collect and analyze assessment results. The assessment plan should be widely distributed throughout the institution, and each department or program area should submit an annual report of their progress and results. These reports should be compiled by the assessment committee and also widely distributed. The development of a source book containing material to help the academic units develop appropriate methods for gathering the information needed for assessment is recommended. The lack of such a source book is one of the weakest links in many colleges’ development of an assessment plan. Without research and the collection of a wide variety of materials, busy faculty are left to independently develop original assessment tools. This takes an incredible amount of time and duplicates work already done by others. Research and collaboration is a necessary initial step to facilitate the development of academic outcomes assessment in a timely manner.
Several examples of different forms of academic outcomes assessment have recently been provided to help faculty determine the best method of assessment for their programs. Since assessment should be done in multiple ways, many of the following strategies might be used at one institution. Angelo and Cross (1993) have published a book entitled Classroom Assessment Techniques which includes not only the characteristics of good classroom assessment but gives specific examples of tools faculty can use within the classroom for ongoing assessment. Angelo and Cross state that "the central purpose of Classroom Assessment is to empower both teachers and their students to improve the quality of learning in the classroom" (p. 4). They added that faculty using Classroom Assessment have reported more active involvement and participation from their students as well as greater student satisfaction. Cross further distinguishes between the Classroom Assessment techniques in the book and classroom research. She defines classroom research as "the study by classroom teachers of the impact of their teaching on the students in their classrooms" (1988, p. 3). She suggests assessment can be the "zipper" to close the gap between teaching and learning.
Value-added assessment, emphasized by Astin in 1984 (as cited in Belcher, 1987), measures the amount of knowledge that students have attained in a given course or program. This is assessed through pre and post testing, with both tests measuring the same objectives. ACT and other companies have programs that can be purchased for this purpose but faculty must be sure that these generic programs measure their stated objectives. An example of value-added assessment is being done across the state of Florida with the CLAST (Losak, 1987). This test is required by state law at all public institutions after the sophomore year. It has given increased support to Florida’s community colleges because their students have done better than some critics expected. Value-added testing has also been implemented at Northeast Missouri State University, using an ACT test upon entrance and at the end of the sophomore year (Mingle, 1986). One problem with this type of assessment is that the learning being measured may have occurred outside of the classroom. This is especially true of community college students who are usually not residential and often not full-time students.
Another problem with CLAST is that it only examines competencies in basic skills. CLAST satisfies political needs, but it cannot account for the aggregate of what students have learned (Bray & Belcher, 1987). The Rhode Island Office of Education found that value-added learning was "expensive to start, difficult to maintain, and require[d] a significant lag-time between the initiation of the process and the arrival of usable results" (Mingle, 1986, p. 12). Cohen and Brawer argue that a better approach to value-added assessment is criterion-referenced tests taken periodically by cohorts as they progress through college (as cited in Bray & Belcher, 1987).
Alverno College in Milwaukee, Wisconsin uses portfolios as a means of assessment (Mingle, 1986). The institution’s catalogue and course syllabi are very specific in explaining behavioral and performance objectives for their students. Assessment at Alverno is centered on the individual learner and feedback is a continuous part of a student’s learning experience. At Alverno assessment is considered to be a major part of a faculty member’s responsibility. All faculty work on assessment of curriculum on a weekly basis.
Other forms of assessing academic outcomes include a common writing assignment or performance test, an attitude survey, the capstone experience, outside evaluation or liscensure exams. For assessment to be used as a tool to improve student learning it must be tied directly to objectives and multiple forms of assessment must be used. Human intelligence, according to Howard Gardner, comes in multiple forms (as cited in Mingle, 1987), and therefore we must use multiple measures to adequately assess learning. Assessment is far more complicated than it at first appears but doing it well is worth the effort.
Faculty also need examples of what is not considered good academic outcomes assessment. One such example is a questionnaire asking students if their educational goals have been met. Another is the reports associated with program evaluation and review. A third type of non-measure is data gathered for administration purposes such as enrollment trends and graduation rates. NCA also considers grades or GPAs to be poor assessment instruments because, as Astin states, grades "tell us little of what the student has actually learned in the course and very little about what a student actually knows" (Assessment for Excellence, 1991, p. 11 as cited in Lopez, 1997). Although some faculty would disagree, NCA argues that grades are influenced by many factors other than direct student learning. Thus NCA does not accept grades as evidence of academic outcomes assessment.
As a unique institution, the community college must form assessment plans that differ from those at other institutions. The assessment plan of a community college must be linked to the five key functions of the college: transfer, career preparation, developmental education, continuing education, and access (Losak, as cited in Owens CC, 1994). Each segment of the plan needs to be linked to the overall mission of the community college. Ewell identifies the following priorities for assessment as the most appropriate for the community college (1994):
Academic outcomes assessment can be used not only as a tool to improve student learning at the community college but also as a way to show critics proof of academic excellence.
Conclusion
Academic outcomes assessment in the community college was begun as an act of compliance in response to outside directives. It is slowly changing and at many institutions has become a critical tool in a process of continuous improvement. Because the mission of the community college centers on teaching and learning, it is perfectly situated to embrace and profit from the assessment movement.
While it is important that an institution respond fully and accurately to a variety of public demands for accountability ... assessment of student academic achievement is key to improving student learning. ... Therefore, assessment of student academic achievement is critical to the future health of its institutions, to the educational accomplishments of students now and in the future, and to the enhanced perception by the public of the value of higher education (NCA, 1997, p. 5).
Government may no longer give money to higher education with faith that they will use it wisely. As in other areas of our society, accountability is required. If educators respond positively to this requirement and initiate true measures of academic outcomes achievement, the results will be two-fold. First, legislators will be more willing to increase much needed funding. Second, students will benefit from the continuous improvement in teaching practices. "There is now a large and compelling body of evidence that regular assessment of student academic achievement and the use of the results to improve student learning is an effective means of assisting institutions and their students" (Lopez, 1997, p. 19). If educators do not respond by assessing themselves, outside agencies will provide their own criteria and further requirements. This would not benefit higher education or their students.
In 1835 Alexis de Tocqueville wrote concerning democracy in America: "They all consider society as a body in a state of improvement, humanity as a changing scene, in which nothing is, or ought to be, permanent; and they admit that what appears to be good may be superseded by something better tomorrow" (as cited in Mingle, 1986, p. 1). The community college was born as a result of a desire to democratize American education. Let these institutions now lead the way toward improvement in student learning by embracing academic outcomes assessment.