Activity 2.1

Sorting Student Work

---

Purposes:

1. To become masters of the targets of instruction (Key 2: Clear Targets) through examining student work for characteristics of quality

2. To experience where criteria come from by "reconstructing" a scoring guide; to see that performance assessment is "do-able"

3. To experience the usefulness, both as staff development and as an instructional tool for students, of the process of developing generalized scoring criteria

4. To illustrate the integration of assessment and instruction—developing performance criteria also increases mastery of targets

5. To motivate teachers to want to pursue activities that increase their mastery of learning targets by showing that it will make instruction faster, easier and better

Uses:

This is an intermediate level activity that can be used in Chapter 2 to illustrate how development of criteria can help teachers clarify the targets they have for students. The activity can also be used with students for the same purpose. The activity can also be used in Chapter 3 to demonstrate how to develop generalized criteria, to demonstrate characteristics of quality criteria, and to demonstrate "consequential validity." Prerequisites might include a general stage setting activity (like Activity 1.1—Changing Assessment Practices, 1.4—Seeing Wholes, or 1.12—Assessment Principles), a defintional activity (like Activity 1.3—Post-it&tm; Notes), and a general introduction to the need for performance criteria (such as Activity 1.5—Clapping Hands, or 1.4—Assessment Standards).

Rationale:

Although overtly about developing scoring criteria, this activity is really about becoming a master of the targets of instruction. This activity develops participants' ability to define essential learning targets for students and identify work that exemplifies different levels of skill on the targets. Teachers nearly unanimously agree that such activities are extremely helpful to them to become better teachers.

Materials:

• Overhead projector, screen, blank transparencies, transparency pens

• Overhead A2.1,O1—Sorting Student Work Purposes

• Set of student work. Several sets are provided in Appendix B—Student Work Samples. These cover various skills, content areas, and grade levels. Pick the set of most interest to participants. Copy one set (single-sided) for each participant.

• Handouts A2.1,H1—Log Sheet; A2.1,H2—When Teachers Look at Student Work , A2.1,H3—The Horse Before the Cart: Assessing for Understanding ; A2.1,H4—Build Your Own Performance Criteria (one copy for each participant). Also pick performance criteria that correspond to the work samples being sorted. For example, if you are sorting math work, you might use the criteria from Kansas (Sample A.43) or Illinois (Sample A.8) in Appendix A—Sampler. (Other ideas are provided in the Activity 2.1 Index in Appendix A—Sampler.)

• Plenty of copies of the local content standards document for the content area being sorted

Time Required:

75-120 minutes

Facilitator's Notes:

1. (5 minutes) Explain the purposes of the activity using Overhead A2.1,O1—Sorting Student Work Purposes and the rationale presented above.

2. (10 minutes) Have participants work in groups of two or three. Pass out the set of student work. First, ask each group to do the performance task themselves—solve the problem, read the passage and answer the questions, sort the fish, etc. (depending on the work samples chosen from Appendix B.) If participants are sorting writing, don't ask them to write essays; it takes too long. If the task has a right answer (e.g., math), come to a group (all tables) consensus on what it is. Discuss questions or concerns about the task.

3. (15-30 minutes) Participants should put the student work into three piles based on its "quality." Describe this task in ways that cue what to do. For example, you can frame the task in terms of grading if it will help participants understand what to do: Put papers into stacks by what grade you would give to them. Or, if the use of grading wouldn't appeal to the group, the three stacks could represent levels of student sophistication.

   As participants sort the work, have them write down the comments they say to themselves as they place a work sample into one stack or another. (Use Handout A2.1,H2—Log Sheet.) Some examples of typical comments for math are:

   • High: I can tell exactly what the student did at every step. Concise and to the point. Correct answer. Logical and sequential. Good explanation. Precise. Correct labels. Explains thought process. Supports reasoning. Correct answer. Appropriate use of pictures, diagrams, and symbols. Used the right numbers in the right order. Sequencing was purposeful. Evidence that the answer was checked for reasonableness. I could tell exactly what the student did and why he or she did it.

   • Medium: Right idea, but computational errors. Forgot a step, but the rest is OK. Didn't go far enough. Didn't clarify. One has to make some inferences as to what the student did. Doesn't show all the steps. Used correct data, but incorrect process. Used correct process, but incorrect data.

   • Low: I couldn't follow the sequence. The answer doesn't look reasonable. Used wrong numbers. Used the wrong process. Mixed processes. Shooting in the dark. Illogical. Unclear. No explanation of answer. I had to make a lot of inferences as to what the student did. I was confused by the explanation.

   (Note 1: This level of detail is very helpful both to remind teachers what they meant, and to develop student-friendly versions of the criteria.

   Note 2: Sometimes this procedure leads to very negative phrases at the low end of the scale, such as "clueless," and "totally wrong." One way to avoid this is to suggest that participants think about the comments they would actually make to students when they handed back work—make "I" statements. "I was confused as to what you did," and "I would like to know more about why you used the numbers you did."

   Note 3: Asking participants to place work in stacks to represent "grades" has an added benefit—it clarifies the connection between grades and the features of student work that earns the grade, thereby removing some of the subjectivity from grading.)

4. Circulate and remind participants that the goal is not necessarily to place each paper into the correct pile; the goal is to come up with a good list of reasons for why each paper was sorted as it was. It's the specific comments such as "the student started out well, but seemed to get lost in the middle," rather than the general comments such as, "logical" that will be most helpful. Continually ask participants to "tell me more about that" or "what specific things done by the student were cues that the student used a logical process?" Continually challenge teachers to think beyond general words such as "voice," "perceptive," and "good communication." Students don't understand what is meant by those words. Since the criteria are supposed to increase student understanding of the features of quality, we need to be clear on the indicators in the work that cue our judgments of quality. Also, challenging teachers to "tell me what indicators in the student work made you judge it logical" causes them to refine their own understanding of what they mean by these terms.

5. (20-30 minutes) When participants are pretty much done, summarize comments on an overhead made from Handout A2.1,H2—Log Sheet. Write down comments exactly as stated, unless there seems to be a misleading choice of words. In this case write down better wording and ask, "Does this capture it for you?" As groups report out, again challenge participants to think past general terms like "logical" or "understands the problem" to the specific indicators in the work that cued the decision that the response was "logical" or that the student "understood the problem." Emphasize the importance of students knowing what teachers mean by the terms in the criteria.

6. The sorting process used thus far in this exercise is "holistic." Participants end up with a list of comments for high, medium and low performance; any single paper gets only one overall score. Usually during the listing of comments someone will say something to the effect that, "I had trouble placing this paper into one stack or another because it was strong on reasoning but weak on communication." This provides an excellent teachable moment to bring up the reasons for analytical trait scoring systems—scoring each student work sample along more than one dimension. The reasons are: (a) it better shows a profile of student strengths and weaknesses, (b) it assists students to be better able to discuss their own work, (c) novices often can't improve all parts of the performance at one time—this helps break the performance down into teachable segments.

   To develop "traits" (important dimensions of performance), you can ask participants to identify comments on the list that seem to go together, and indicate which comments relate to each trait at the high, middle and low levels. For example, people frequently see the traits of communication, problem solving/reasoning, mathematical understanding, and correct computation when sorting math work. Number these traits 1, 2, 3, 4. Then go through the comments they generated and ask them into which category each comment fits. Put the trait number next to each comment.

   For example, the trait of communication (#1) would include, at the high level: I can tell exactly what the student did at every step; good explanation; clearly explains thought process; I had to make very few inferences about what the student did and why he or she did it, etc. I usually color code the trait numbers so that participants can easily see which comments go under each trait.

7. (15-30 minutes) The "so what" discussion. Use this sequence:

   1. Ask participants to compare their list of valued features to local or state content standards that correspond to the type of work sorted. Ask them, "What similarities do you find?" (They always find lots.) This assists participants to see (1) how the content standards are represented in the daily work of students, and (2) that the content standards represent what they already value as outcomes for students; it's not new.

   2. Ask participants to look at similarities between their list of valued qualities and the statements in a set of criteria you previously selected from Appendix A—Sampler that correspond to the content being sorted. There are always many similarities, because, after all, how different can lists of quality criteria be? This enables participants to see that if done well, scoring criteria were not pulled out of a hat; they represent what we all already value in student work; someone has just taken the time to write it down. Point out the alignment between (1) what they value in student work, (2) the content standards, and (3) the way the content standards are being assessed.

   3. Ask participants whether they think this would be a useful activity for them to do with their colleagues. (They always answer, "Yes!" because: "It helps to be consistent in grading. It makes me more confident in teaching these skills. It helps us rethink what we really value. It makes us more consistent in instruction. It helps us understand what good work looks like. It helps me focus instruction on what is really important.") The paper Handout A2.1,H2—When Teachers Look... describes the value to teachers of the process we just completed. (In this paper, the focus was sorting portfolios.)

   4. Ask participants whether they think this would be a useful activity for them to do with their students. (They always answer, "Yes!" for the same reasons as above.) The paper Handout A2.1,H3—The Horse Before the Cart: Assessing For Understanding describes the instructional value of having students help develop criteria for quality.

   5. Discuss how the same process can be used for other skills listed in local or state standards such as critical thinking, writing, and reading fluency. Sample scripts:

      • Reading fluency. Ask teachers to tape students reading over the course of the year. At the end of the year, have teachers place these tapes into 4-5 levels of fluency. Ask them to write down what they are saying to themselves as they place the tapes into piles. They can compare their comments to content standards in reading and a sample reading development continuum—for suggestions see the Activity 2.1 Index in Appendix A—Sampler.

      • Critical thinking. Ask teachers to choose samples of student work that illustrate different levels of ability to think critically. For each content area, sort the samples into three stacks and write down reasons. Then compare the lists from the various disciplines to determine whether the same criteria might be used across disciplines. You can also try to find where in the content standards critical thinking comes in and compare their lists to sample criteria. Sample criteria are suggested in the Activity 2.1 Index in Appendix A—Sampler.

   6. Refer participants to Handout A2.1,H4—Build Your Own Performance Criteria to review the steps they went through when developing their scoring criteria.

This document's URL is:

Home | Up & Coming | Programs & Projects: Assessment | People | Products & Publications | Topics

© 2001 Northwest Regional Educational Laboratory

Email Webmaster
Tel. 503.275.9500