Evaluation, Assessment, Grading, Norming

Assessing Learning v. Assessing Teaching

The relationship between assessing learning and assessing teaching should not be as dichotomous as the heading suggests. Rather, the two concepts should be interconnected. What follows are suggestions and tips to help instructors keep both of these concepts on their radar screen THROUGHOUT the semester. Waiting until the end of the semester to assess either learning or teaching can influence the degree to which one’s course is the best it can be.

Some Helpful Definitions

Difference between Evaluation and Assessment:

Evaluation is typically a broader concept than assessment as it focuses on the overall, or summative experience. Assessment is typically used to describe processes to examine or measure student learning that results from academic programs. Assessment is an ongoing process aimed at improving student learning, more so than evaluation, which is usually a final (or summative) result. Improving learning from assessment sometimes occurs immediately in classrooms or later because of changes for future students. However, these terms are often used interchangeably and so it is important to be aware of the larger context of the discussion at hand (Ewell, 2002).

Formative Assessment:

Assessment that provides feedback to the teacher for the purpose of improving instruction and other aspects of the teaching process (Wakeford, 2003).

Summative Assessment:

A culminating assessment, which provides information about the students’ mastery of content, knowledge, or skills (Wakeford, 2003).

Assessment and Evaluation of Learning

9 Principles of Good Practice for Assessing Student Learning (Astin, Banta, Cross, El-Khawas, Ewell, Hutchings, et al., 2003):

The assessment of student learning begins with educational values.
Assessment is the most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time.
Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.
Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.
Assessment works best when it is ongoing not episodic.
Assessment fosters wider improvement when representatives from across the educational community are involved.
Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about.
Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.
Through assessment, educators meet responsibilities to students and to the public

Assessment methods:

Multiple choice, true/false, matching, fill in the blank (or completion)
Essay Questions
Short Answer
Assessment of practical or laboratory work
Portfolios
Oral examinations

Issues to consider when developing assessments:

Is the assessment for formative or summative purposes?
Do you want to emphasize students’ ability to write and read?
Do you want to emphasize students’ ability to think creatively and critically?
How large is the class and what will class size mean for creating and grading the assessment?
Is the assessment reusable?
Have you considered ways to prevent cheating?

Tips for written assessment (Office of Faculty and TA Development, 2001):

Compose test items throughout semester.
Mix question types (include multiple choice, short answer, etc.).
Test early to demonstrate testing style.
Test often to keep students on task.
Test what you really want students to learn.
Proofread exams carefully; consider having a colleague proofread as well.
Create a test bank.
Avoid items that rely on correctly answering other items in the exam.
Pilot the exam with colleagues or former students.
Make appropriate accommodations for all students.
Provide enough copies.
Minimize distractions while the test is being administered.

[BACK TO TOP]

Grading and Norming

Validity, or does the assessment measure what it is supposed to measure?

To determine whether the assessment is valid, consider the following:

Is the assessment content appropriate for the students in the class?
What will the impact of the assessment be on the students—i.e., what is the perception by the students of what the instructor rewards based on the kind of assessment used?
Are the elements of the assessment related to the objectives of the course?

Reliability, or is the assessment dependable and accurate?

Possible sources of unreliability include:

Inadequacy in the length of assessment
Inconsistency among and/or between those grading the assessment
Inadequacy of specific questions on the assessment
Unclear expectations of the assessment

Fairness

Grade on the basis of mastery of knowledge and skills, not on effort, attendance, attitude, or other unrelated variables.
Keep students informed of their progress throughout the semester.
State grading procedures clearly in your syllabus.
Set policies on work that is turned in late and stick to them.
Return the first graded work before the add/drop deadline.
Keep accurate records of all students’ grades.
Record results numerically rather than as letter grades to assure greater accuracy when calculating final grades ( Davis, 1993).

Norming

A simple curve can be used as a grading method. To use a curve, the instructor will determine before any grading that a certain percentage of students will receive “A”s, “B”s, etc.
A normalized curve can be used as a grading method. This curve involves determining the class average and calculating a standard score for what each student earns. Using standard tables, these scores are converted into percentages based upon a normal curve.
Instructors can also choose to grade on a system that is not norm-based. One possible way to do this is to preset criterion to determine the value of an “A,” “B,” and so on. For example, if an A = 90% or above, any student who earns at least a 90% will receive an A.
There is debate about the pedagogical value of norming. It is important to be aware that your department may have strong opinions about grading on a curve. In addition, there is a great deal of concern about grade inflation. Be savvy about departmental expectations with regard to grading and systems others use in your department before deciding what system is best to use.

Tips that can assist in grading

Use a “rubric” or analytic scale to clearly define grading criteria and the weight given to each criterion.
Encourage students to conduct peer review of their work before turning the work in.
Provide formative feedback that is specific so that students can understand what they did well and what they need to work on.

[BACK TO TOP]

Assessment and Evaluation of Teaching

Educational Technologies at Missouri (ET@MO)

MU has easily accessible support for course assessment. ET@MO details specific ways that instructors can seek mid-semester feedback, self-evaluation, peer evaluation. See http://etatmo.missouri.edu/mocat for links to assist instructors in assessing their own teaching.

Classroom Assessment Techniques (CATs)

Classroom Assessment Techniques (Angelo & Cross, 1993) are instructional strategies that can be used as less formal means of assessment to allow on-going and immediate feedback. These techniques can determine how much, how well, or how students learn. Examples of CATs include:

One Minute Papers

Muddiest Point

One Sentence Summary

Background Knowledge Probe

See http://www.schreyerinstitute.psu.edu/Tools/InClass/, http://www.ntlf.com/html/lib/bib/assess.htm, and http://www.siue.edu/~deder/assess/catmain.html for other examples and details on how to facilitate CATs.

Early evaluations

Use early evaluations. Tabulate comments to find recurring themes and suggestions. Focus on potentially constructive changes. Report several issues (3-5) back to the class and explain the rationale for any changes you plan to make or are unable to make, based upon student feedback. At MU, you can request assistance in facilitating mid-semester evaluations. Students can receive on-line access to assessment forms to give instructors feedback so they can adjust their course and enhance student learning for the rest of the semester. To get more information about this, see http://etatmo.missouri.edu/eventsservices/mocat.htm.

[BACK TO TOP]

Final course evaluations

The empirical research on course evaluations or student ratings of teachers is vast. While there are studies that suggest otherwise, these ratings overall are reliable and valid and often used by instructors, administrators, and students (Cashin, 2003). These evaluations are valued and valuable, but should only serve as one measure of teaching effectiveness.

At MU, there are 8 different forms instructors can choose from to select the most suitable assessment tool for their class. Forms are delineated based upon the predominant format of the course:

Small lecture and discussion courses.
Large lecture course
Seminar-discussion course
Problem-solving course
Skill-oriented, "hands-on" course
Discussion section
Lecture courses which strongly emphasize textbook and homework problems
Laboratory instruction

In order to get the maximum benefit from evaluations:

Allow time at the beginning of class for students to complete the forms. Leaving them until the end of a class period often results in minimal open-ended feedback and students rushing to leave the classroom.
Tell students you want candid and constructive responses. Remind them that their feedback will not influence their course grade as the instructor will not receive a synthesis of the data until after the semester is over.
Keep the focus on potentially constructive changes. Don’t get bogged down by the comments over which you have no control.

[BACK TO TOP]

Assessment of students as it relates to assessing teaching

Remember that assessment of students is also a method to assess the instructors’ teaching. Based upon how the students answer a formative assessment in the course,

Are students learning the material?
Are you asking the right questions to measure learning?
Do you need to reinforce certain concepts?
Do you need to change your teaching style?
Are the students motivated?

Answers to many of these questions are embedded in the overall performance of students on your assessments. You may use these findings to adjust your own instructional strategies throughout your course or in subsequent courses.

For More Information

For more information about evaluation, assessment, grading, and norming, see the following on-line resources:

References

Angelo, T., & Cross, K. P. (1993). Classroom assessment techniques. San Francisco: Jossey-Bass.

Astin, A. W., Banta, T. W., Cross, K. P., El-Khawas, E., Ewell, P. T., Hutchings, P., et al.
(2003). 9 principles of good practice for assessing student learning. Retrieved March 13, 2008, from
Assessment @ Buffalo State Web site: http://www.buffalostate.edu/offices/assessment/aahe.htm

Cashin, W. E. (2003). Evaluating college and university teaching: Reflections of a practitioner. In J. C. Smart (Ed.), Higher education: Handbook of theory and research, Vol. 18 (pp. 531-594). London: Kluwer.

Davis, B. G. (1993). Tools for teaching (2 nd ed.). San Francisco: Jossey-Bass.

Office of Faculty and TA Development. (2001). Teaching at The Ohio State University:

A handbook. Retrieved June 26, 2004, from The Ohio State University, Office of Faculty and TA Development Web site: http://ftad.osu.edu/Publications/TeachingHandbook/

Wakeford, R. (2003). Principles of student assessment. In H. Fry, S. Ketteridge, & S.

Marshall (Eds.), A handbook for teaching & learning in higher education: Enhancing academic practice (pp. 42-61). London: Kogan Page.

[BACK TO TOP]

Contributed by Dr. Jennifer Hart Assistant Professor, Department of Educational Leadership and Policy Analysis