Activity 1.5

Clapping Hands*

Purposes:

    To develop guidelines for high quality performance assessments
    To explore the importance of quality: the impact of our assessments on students

Uses: This activity is at an early elementary level of difficulty. It can be used with Chapters 1 or 3 to illustrate features of quality alternative assessment. It provides a gentle introduction to quality issues. It should be used prior to the more detailed quality activities in Chapter 3-Activities 3.1, 3.3, and 3.6. In Chapter 3 the activity can also be used to illustrate design options and the effects these can have on teachers and students.

Rationale: Assessment is a deeply affective thing. It is about success and failure, about deeply held feelings, and about our sense of self-esteem. This activity is designed to show the impact of assessment both on performance and on the feelings at the receiving and rating ends of the process. It provides a graphic demonstration of the unintended consequences of how performance assessments are designed. It also results in a set of guidelines--do's and don'ts--for performance assessment.

We have successfully used this activity with teachers in all grades levels (K-12), and with both community college instructors and college of education faculty.

Materials:

Overheads:A1.5,O1--Clapping Hands Purposes; A1.5,O2--Clapping Institute Criteria

Handouts: A1.5,H1--The Meaning of Quality Summary Sheet (1 for each person) A1.5,H2--Performance Assessment--The Meaning of Quality 1 for each person); A1.5,H3--Clapping Institute Award Certificate (1 per group)

Optional Overheads: A1.5,O3--Equity Discussion Questions and A1.5,O4--Special Education Discussion Questions

Time Required:

60-90 minutes

Facilitator's Notes:

A. Set-Up (10-15 minutes)

The activity is experiential--it asks participants to commit and invest themselves in the process. As such, it might make some participants uncomfortable. Therefore, it is extremely essential to manage the activity well. Keep it light-hearted.

Once this exercise has been done with a group, it can't be done again with the same group. The complete exercise requires at least 15 people. (You can get by with 11--instructions for "small group" are included where needed.)

The presenter's outline below is set up for ease of use (list style; questions to participants in bold, etc.). Typical comments made by participants at various steps of the process are included at the end.

1.   Facilitator needs: paper and pencil, 1 copy of the award certificate Handout A1.5,H3--Clapping Institute Award Certificate), and this outline.

2.   Arrange 11 chairs (8 if the group is small) in a semi-circle. (See diagram below.) 2.   Arrange 11 chairs (8 if the group is small) in a semi-circle. (See diagram below.)

Image Loading...

3.   Introduce the exercise using Overhead A1.5,O1--Clapping Hands Purposes, as needed. Then:

Five people (4 if the group is small) will be asked to do a simple performance task, and five other people (3 if the group is small) will assess the performances. At the end everyone will be asked to describe how it felt to be in the position they took. The audience will participate at the end by questioning the volunteers on their feelings and adding their own observations.

4.   Ask for 10 volunteers (7 if the group is small) to come up and sit in the 10 (or 7) chairs arranged in a semi-circle. (Facilitator sits in the center chair--see diagram.)

5.   Note names of volunteers on a seating chart

6.   Point to participants to your right as the "assessees"-they will do a simple performance task

7.   Point to participants to your left as the "assessors"-they will evaluate the quality of performance on the task.

8.    Tell the volunteers:

This activity was developed to explore the consequences of being treated in different ways, so please be alert to them as the exercise progresses.

9.   Optional ways to get more people actively involved:

    a. Ask the audience to "mirror" the panel-each person picks one volunteer and considers how they would react in the same situation.
    b. Select 1-2 math people to compute the average scores given by the assessors.
    c. After the assessors give their scores, have audience "vote" on scores with their fingers.

B.   Assessment Task Administration (10-15 minutes)

1.   Assessee #1 (fifth person to your right; fourth person if the group is small)

(Give no verbal or non-verbal feedback.)

2.   Assessee #2 (fourth person to your right; skip this step if the group is small)

(Designate a "hall monitor" to ask the person to come back in when asked and to make sure the person can't hear the discussion.)

Assessors please assess (name)'s clapping on a scale of 1-5, with 1 being low and 5 being high.

(Record each rater's score beginning with the person on your immediate left; compute average; report to panel. Ask the participant to return, but give NO feedback.)

3.   Assessee #3 (third person to your right)

Assessors please assess (name)'s clapping on a scale of 1-5, with 1 being low and 5 being high.

(Record each rater's score beginning with the 2nd person to your left; compute average; report to panel. Ask the assessee to return.)

OK, (name), you got a score of _______. I hope you find this information useful.

4.   Assessee #4 (second person to your right)

After the assessee leaves the room, turn to the assessors:

I have good news and bad news. The good news is that I have heard from the International Clapping Institute. As you know, that's the organization that is developing the international standards for clapping that all our students must meet by the year 2000. They have begun a handbook on how to conduct the assessment. From now on they want us to assess all clapping performances on three traits--volume, appropriateness, and creativity.(Overhead A1.5,O2)

The bad news is that they haven't yet finished the handbook, so they really can't give any more guidance than that. So we just have to do the best we can. So, from now on we'll assess performance on each of the three traits, where 1 is low and 5 is high.

(Record the score for each trait--volume, appropriateness and creativity. Begin with a different assessor each time; for example, start with the 3rd person on your left for volume and the 4th person for appropriateness. Average the scores for each trait and prepare the certificate--A1.5,H3--Clapping Institute Award Certificate. Ask the assessee to come back, give him/her the certificate, and shake his/her hand.)

(Name), here are your scores. I hope you find them useful.

5.   Assessee #5 (first person to your right): This participant gets special treatment. Ask these questions:

To Assessors:

You're the experts on clapping; that's why you are here. Is there anything you want (name) to know that you'll be looking for when he/she claps?

Do you want to discuss the criteria given to us by the Clapping Institute?

Is there anything you want (name) to know is important to be effective? Anything in particular you'll be looking for in the clap?

To Assessee #5:

Assessors provide feedback as requested.

C.   Debriefing (15 minutes)

1.    Each person will have the chance to describe how he or she felt during this process and the thoughts and realizations he or she had. After each individual has had a chance to talk, the floor will be open to the audience to ask questions or provide observations.

Optional ways to get more people involved:

    Have 2-3 recorders note key comments on chart paper during the discussion (e.g., "lacked clarity," "wanted to try again," etc.).

    Have people "buzz" (discuss) their thoughts/feelings as mirrors before the panel reports.

2.   Ask Assessee #1 (5th to your right; 4th if the group is small):
     What thoughts or feelings did you have?

     How did you feel when we came down the line and other people got other kinds of preparation or feedback?

     How did you feel about getting no preparation or feedback?

3.   Ask Assessee #2 (4th to your right; skip this step if the group is small):

     What thoughts or feelings did you have?

     

How did you feel when you were asked to go out of the room?

     How did you feel about getting no feedback

4.   Ask Assessors (move these questions to step 6 if the group is small):

     How did you feel about rating the second person and not the first?

     How did you feel about having no criteria?

     Did anyone give any "protest" votes? (E.g., "I'm going to give everyone a '3' or a '5' because I was not given any guidance on what to do.")

     Did anyone have the thought, "I better give myself some wiggle room by not giving the first person higher than a '3'."

5.   Ask Assessee #3 (3rd to your right):

     What thoughts or feelings did you have?

     How did you feel when you were sent out of the room?

     When you got your score, was the feedback useful?

6.   Ask Assessors:

     Did it bother you that I asked you to give (name-#3) his/her scores but didn't ask you to give (name-#2) his/her scores?

     Did you compare (name-#3) with (name-#2) and (name #1)?

     (Implication: when we are given no criteria to make criterion-referenced comparisons, we start comparing students to each other—norm-referenced comparisons)

7.   Ask Assessee #4 (2nd to your right):

     What thoughts or feelings did you have?

     How did you feel when you came back in and got your certificate?

     Was the feedback useful?

     Could you hear people laughing while you waited outside? What did you think was going on?

     Did this method bother you? How did you feel about being assessed on traits you knew nothing about?

     Did you focus on any particular scores (like the lowest)?

8.   Ask Assessors:

     What were your reactions when I gave you criteria categories with no further directions?

     Were these criteria useful?

9.   Ask Assessee #5 (1st to your right):

     What thoughts or feelings did you have?

     Did you like the extra attention and information?

     Did you feel uncomfortable being treated differently?

10.   Ask Assessors:

     Was the extra discussion useful?

     In general, did hearing other people's scores affect your scores?

11.   Ask Assessees:

     Would any of you like to clap again? Why or why not?

12.   Ask the audience:

     Would anyone like to make an observation about what happened or ask the volunteers questions? (Don't let this stretch on too long.)

     Has anyone been in the position of any of the assessees or assessors? Please indicate the person and relate what happened. (Keep it to 3 or 4 people.)

D.   Activity "So-What." (15 minutes) (This part of the activity is essential. We want participants to be critical consumers of the performance assessments they see.)

1.   Ask participants to use Handout A1.5,H1—The Meaning of Quality Sheet to write what they learned from this activity about things to pay attention to when:

Participants will probably need "bias and distortion" defined. These are things that can mask achievement; things that can go wrong and result in drawing an incorrect conclusion about the quality of a student's work, skills, or knowledge. For example, (a) a high reading load on a math test, (b) student cultural background, or (c) student personality. See A1.5,H2—Performance Assessment—The Meaning of Quality for more examples. Note that these things don't always mess up the results. For example, if the purpose of the assessment is to assess reading comprehension, the length or difficulty of the passages is not a source of bias and distortion because that's what you're assessing. But if the amount of reading gets in the way of a student being able to demonstrate his/her math problem-solving ability, that's a source of bias and distortion.]

Give participants 5-10 minutes to make individual notes on Handout A1.5,H1.

2.   (5 minutes) Ask participants to note several points of agreement in table groups.

3.   Make a complete list in the whole group. This list comprises criteria by which to judge the quality of performance assessments. It is the beginning of a rubric for judging quality. (Listing quality statements on chart paper enables the facilitator to post the "criteria for quality alternative assessments" to refer or add to later.)

4.   Pass out Handout A1.5,H2—Performance Assessment—The Meaning of Quality to summarize what other groups have listed as important quality considerations.

Typical Responses (During Step C—Debriefing):

Assessee #1:   Apprehension because I was given no guidance; every time I asked for clarification I was told, "We'll get back to you." I didn't know what was expected. I was confused. I kept wanting to try again after the criteria came out because I was sure I could do better. But, toward the end I actually felt glad I went first because by then the pressure was on. I felt it was increasingly unfair. I got angry. I felt disengaged from the process and "off the hook." It didn't matter how I did because the task was so unclear I had a good excuse.

Assessee #2:   I had to guess at the criteria, so I fell back on general knowledge of what is expected in this type of situation. Therefore, personality, cultural background, and gender all might have an effect when things are unclear. I didn't know what was expected so I just tried to outdo the first person. I felt I was sent out of the room because I had done something wrong.

Assessee #3:   A low score reinforced my self-judgment that I am a klutz. I was angry that I was given no rationale for my score and no opportunity to improve it. A number with no rationale or scale is not useful. A gratuitous "5" from the judge rebelling because she didn't know the criteria was not useful; it felt condescending. I would have my "folks" come in and complain to the principal. I felt uncomfortable that the first person got no score at all and the second didn't find out what his score was; it wasn't fair.

Assessee #4:   I was embarrassed that I was being treated differently. I was embarrassed that some of my scores were low. I had a mental discussion with myself complaining about the exercise. I focused primarily on the low scores. I wanted to argue with the assessors. If I had known what the criteria were I could have done better. I heard people laughing and thought they were laughing at me. I was out of the room longer than anyone else so I figured I had done something wrong.

Assessee #5:   The pressure was on. How could I be better than the previous ones? Knowing I would be scrutinized by my peers made me nervous. They had no guidance and so could be excused for poor performance, I had no such excuse. I was embarrassed that I was treated differently. I was afraid that if I got better scores, the other "students" would be angry with me because they were treated differently. I wanted to not do a good job so that I wouldn't stand out. I appreciated the extra discussion; it made me feel more confident. Because of all the extra help, there was no excuse for not doing well.

Assessors:    I was unsure of the criteria so I was afraid of giving a high score. I was angry with not having any criteria so decided to give everyone a "5". I was uncomfortable that we were asked to rate the second person and not the first. I didn't have any criteria so I was only able to rate the second performance in relationship to the first. I felt sorry for the assessees and was glad I was not one of them. I wouldn't give the first person higher than a '3' because I didn't know what the other performances would be like. Hearing the others give their scores affected my scores.

The scoring areas (traits) helped some. I'm not sure what "appropriate" or "creative" means so I made up my own idea of what each meant. I wanted to rate other things besides these three. The three categories did not capture all the relevant dimensions of performance. Not all three categories would be appropriate to assess in all contexts.

Since we knew more about the person [person #5], we held her to a higher standard than the others. Even though I specified my criteria for success, I still felt she should go beyond them to get a high score. I didn't want the assessee in the room while we gave scores because she might challenge my judgment.

Someone:   One of the points invariably made is whether the clapping activity measures skills that are worth taking this much time to assess. The answer is probably "no." Clapping is just a familiar context in which to raise important points about assessment. If people have the reactions they do in this contrived situation, think about what happens in a real assessment. But, be sure to recognize that the point about "is this worth the time devoted to it" is a question we should ask of all assessments.

Variation 1: Teacher Education Faculty

A variation on the activity is to have teachers (or teacher education faculty) play the role of assessees (those being assessed) and student teachers (or students) play the role of assessors (those judging the performance). (Comment from Jamie Foster, Eisenhower Math/Science Consortium, Charleston, WV: "I did the activity immediately prior to lunch and participants discussed the activity and what they learned during lunch....The emotions that came from the teacher education faculty were insightful.")

Variation 2: Equity Enhancement

This activity brings to life with humor what is often in reality a major issue: are the assessments we use to gain portraits of student learning fair? Equitable? Do all students have the best opportunity we can provide to show their learning? Here are some additional ideas for increasing emphasis in this session on equity in assessment.

A.   Culture, Language, and Assessment. Present the task to one of the "assessees" in a language other than English. Question the "assessee" during the debrief about how this treatment affected his/her sense of competence (not just fairness). Elicit from the observers their reactions to this unexpected switch to instructions in a language which may be completely unfamiliar, or which they might be not quite fluent in. Or prompt reflection and discussion with the question, "How does a mismatch between the language of the student and the assessment affect the quality and accuracy of the assessment?" (See the first question on Overhead A1.5,O3—Equity Discussion Questions)

Depending on the audience, this extension might include opportunities for several participants to recall similar or related assessment problems from their own experience.

Continue the discussion with the following questions on Overhead A1.5,O3—Equity Discussion Questions:

B.   Special Education Students and Assessment. This is an assessment issue that raises strong feelings and conflicting solutions. Some people believe that students with physical, educational, or emotional challenges should be exempted from assessments—especially high-stakes assessments. Others argue that setting lower standards for some students reduces their opportunities to learn and succeed. Special educators often feel left out of professional development assessment experiences.

Select the questions below that best meet the assessment quality concerns that you wish to emphasize (Overhead A1.5,O4—Special Education Discussion Questions). (Remind the group that this activity is intended to activate and deepen their awareness of assessment quality issues that will be more fully addressed in other activities.)