Toolkit98
The authors' premise is that innovative teaching techniques (e.g.,
cooperative learning, group discussions, hands-on experiments
and videos) do not guarantee a change in the "intellectual
quality" of what students are asked to do. For example,
"a portfolio that shows a variety of student work over a
semester might replace the final exam taken in one sitting, but
the portfolio itself could be filled with tasks" devoted
to remembering and listing isolated bits of information. Thus,
the merit of any teaching technique should be judged by its ability
to improve the "intellectual quality of student performance."
The document listed below is devoted to defining "intellectual
quality," developing criteria for judging the intellectual
quality of the school work students are asked to perform, assessment
tasks, and student work. The criteria were designed for use in
mathematics and social studies and many examples in these subjects
are included. But the authors encourage application to any content
area. The criteria can, therefore, serve to define delivery and
performance standards across the curriculum and across teaching
methods.
Attached are the criteria for assessing the intellectual quality
of student work in social studies. They appear to be appropriate
for grades 7-12.
A Guide to Authentic Instruction and Assessment: Vision, Standards
and Scoring, 1995
pp 94-101
Fred Newmann, Walter Secada, and Gary Wehlage
Document Service
Wisconsin Center for Educational Research
School of Education
University of Wisconsin
1025 West Johnson Street
Madison, Wisconsin 53706
(608) 263-4214
PART III: STUDENT PERFORMANCE
Parallel, but somewhat differently worded, standards are presented
for student performance in mathematics and social studies. The
three standards reflect two of the general standards for authentic
achievement as follows:
Construction of Knowledge: Analysis
Disciplined Inquiry: Disciplinary Concepts; Elaborated Written
Communication
A. Scores should be based only on evidence in the student's performance
relevant to the criteria. Matters such as whether the student
followed directions, neatness, correct spelling, etc. should not
be considered unless they are relevant to the criteria.
B. Scores may be limited by tasks which fail to demand analysis,
disciplinary conceptual understanding, or elaborated written communication,
but the scores must be based only upon the work shown.
C. Scores should take into account what students can reasonably
be expected to do at the grade level. However, scores should
still be assigned only according to "absolute" criteria
in the standards, not relative to other papers that have been
previously scored.
D. When it is difficult to decide between two scores (e.g., a
2 or a 3), give the higher score only when a persuasive case can
be made that the paper meets minimal criteria for the higher score.
If the specific wording of the criteria are not helpful in making
this judgment, base the score on the general intent or spirit
of the standard described in the introductory paragraphs of the
standard.
Mathematics Standard 1: Analysis
Student performance demonstrates thinking with mathematical content
by organizing, synthesizing, interpreting, hypothesizing, describing
patterns, making models or simulations, constructing mathematical
arguments, or inventing procedures.
This standard is intended to measure the extent to which the student
demonstrates higher order thinking, i.e., thinking that goes beyond
mechanically recording or reporting facts, rules, and definitions
or mechanically applying algorithms.
The term "mathematical" analysis calls attention to
the fact that the content or focus of the thinking should be mathematics.
If analysis occurs about other subject areas, without a connection
to mathematics, the performance would still score a 1 on this
standard.
There are two guiding questions here: First, has the student demonstrated
mathematical analysis? To answer this, consider whether the student
has organized, interpreted, synthesized, hypothesized, invented,
etc., or whether the student has only recorded, reported, or mechanically
applied rules, definitions, or algorithms. If work is not shown,
correct answers can be taken as an indication of analysis if it
is clear that the question would require analysis to be answered
correctly. Second, how often has the student demonstrated mathematical
analysis?
4 = Mathematical analysis was used throughout the student's work.
3 = Mathematical analysis was used in 50 percent or more of the
student's work.
2 = Mathematical analysis was used in less than 50 percent of
the student's work.
1 = Mathematical analysis constituted no part of the student's
work.
Example: If no work was shown and ¾ of the questions were
judged to require analysis to get the right answer, and the student
did all of those correctly, she or he would score a 3 on this
scale. On the other hand, if the student got them all wrong,
he or she would get a 1. In short, the scorer should estimate
the percent of the total task that the number of correctly answered
analytic questions comprised.
Mathematics Standard 2: Disciplinary Concepts
Student performance demonstrates an understanding of important
mathematical ideas that goes beyond application of algorithms
by elaborating definitions, making connections to other mathematical
concepts, or making connections to other disciplines.
This standard is intended to measure the extent to which the student
demonstrates use and understanding of mathematical concepts.
Prior to scoring the student's work, the rater should identify
what mathematical concepts, if any, a student must use and/or
understand to succeed in the task. Low scores may be due to tasks
which fail to call for understanding of mathematical concepts.
A guiding question for using this standard is, "Does the
student show understanding of the fundamental ideas relevant to
the mathematics used in the task?" Correct use of algorithms
does not necessarily indicate conceptual understanding of the
material. Such understanding can be demonstrated, for instance,
by elaborating upon the concept through definition, or by making
connections between the core concept and other related ones.
If work is not shown, correct answers can be taken as an indication
of conceptual understanding, if it is clear that the task or question
requires a conceptual understanding in order to be completed successfully.
Completion of the task is not necessary to score high.
4 = The student demonstrates an exemplary understanding of the
mathematical concepts that are central to the task. Their application
is appropriate, flawless, and elegant.
3 = There is substantial evidence that the student understands
the mathematical concepts that are central to the task. The student
applies these concepts to the task appropriately; however, there
may be minor flaws in their application, or details may be missing.
2 = There is some evidence that the student understands the mathematical
concepts that are central to the task. Where the student uses
appropriate mathematical concepts, the application of those concepts
is flawed or incomplete.
1 = There is little or no evidence that the student understands
the mathematical concepts that are central to the task, or the
mathematical concepts that are used are totally inappropriate
to the task, or they are applied in inappropriate ways.
Mathematics Standard 3: Elaborated Written Communication
Student performance demonstrates a concise, logical, and well
articulated explanation or argument that justifies the mathematical
work.
Performance that meets this standard could include, or consist
of, diagrams and drawings as well as prose. To score high on
this standard, the student must communicate, in writing, an accurate,
complete and convincing explanation or argument.
The score should not be based on the proportion of student work
that contains explanation/argument but on the quality of mathematical
communication, wherever it may occur in the work. We use the
prefix "mathematical" to draw attention to the fact
that the substantive content of what is written about must be
related to mathematics.
4 = Mathematical explanations or arguments are eloquent, clear,
complex, and complete. Mathematical communication is exemplary.
3 = Mathematical explanations or arguments are present. They
are largely concise, clear, and well articulated; however, they
may be slightly flawed or incomplete in minor ways.
2 = Partial mathematical explanations or arguments are present.
They are incomplete because they have not been finished, they
omit a major part of an argument, or they contain several mathematical
or logical errors. Note: Simply showing work on algebraic problems
(regardless of whether the answer is right or wrong) constitutes
a low level of explanation and should be scored a 2 if no other
part of the student's work on the task shows a higher level of
communication.
1 = Mathematical explanations or arguments are absent or, where
present, they are seriously incomplete, totally inappropriate,
and incorrect. This may be because the teacher's questions have
left no room for argument or explanation, e.g., fill-in-the-blank
and multiple-choice questions.
Social Studies Standard 1: Analysis
Student performance demonstrates higher order thinking with social
studies content by organizing, synthesizing, interpreting, evaluating,
and hypothesizing to produce comparisons/contrasts, arguments,
application of information to new contexts, and consideration
of different ideas or points of view.
This standard is intended to measure the extent to which students
demonstrate cognitive activity that goes beyond mechanically recording,
reporting or otherwise reproducing information. Analysis may
include proposing generalizations and supporting them with evidence;
articulating and testing different theories or points of view;
synthesizing and categorizing by applying abstractions to more
specific information (this could include comparing similarities
and differences); considering implications and application of
information in new contexts; raising broad questions that help
to interpret more specific information; or interpreting the meaning
of personal roles, ideas, or events. The essential question is
whether students demonstrate construction of knowledge through
thinking and the organization of information, versus reproduction
of knowledge by restating what has been previously given to them.
The rhetorical form of students' statements might qualify as analysis
(e.g., "The main reason for the American Revolution was taxation
without representation"), but to score high on analysis,
the student's work must appear to be reasonably original, not
merely a restatement of some analysis that was given previously
in a text or discussion. In assigning a 3 or 4, the rater should
be reasonably confident that no significant portion of the response
has been virtually copied from some other source (i.e., text or
oral statements of others).
4 = Substantial evidence of analysis. Most of the student's work
includes analysis. At least three statements indicate that the
student has successfully generalized, interpreted, tested, or
synthesized specific information.
3 = Moderate evidence of analysis. A central portion of the student's
work includes analysis. At least two statements indicate that
the student has successfully generalized, interpreted, tested,
or synthesized specific information.
2 = Some evidence of analysis. A small, but not central, portion
of the student's work includes analysis. At least one statement
shows that the student has successfully generalized, interpreted,
tested, or synthesized specific information.
1 = No evidence of analysis. Almost all statements consist of
recording, or reporting specific information, without evidence
of the student's organizing it or reflecting upon it; OR virtually
all analysis offered is unsuccessful or in error.
In scoring analysis, the proportion of work that illustrates analysis
is more important than the actual number of statements indicating
analysis.
Social Studies Standard 2: Disciplinary Concepts
Student performance demonstrates an understanding of ideas, concepts,
theories, and principles from the social disciplines and civic
life by using them to interpret and explain specific, concrete
information or events.
This standard is intended to assess the extent to which students
use important ideas of the social disciplines to make concrete
information and events more meaningful. Substantive concepts
(e.g., depression, social class, culture) are often grounded in
disciplinary knowledge from history and the social sciences, but
many important ideas in social studies (e.g., justice, freedom,
citizenship) may have no exclusive origins or associations within
a single discipline. The main issue is the extent to which the
student has used substantive social ideas to organize, explain,
interpret, summarize, and extend the meaning and significance
of otherwise discrete pieces of information.
Social studies concepts may be used even though they may not be
stated explicitly, and this may vary with student grade level.
For example, an elementary student's discussion of conditions
or oppressed groups might indicate use and understanding of the
concept of equality, without stating the specific word.
If the topic of the task is itself a substantive idea (e.g., revolution),
students should get full credit for successful use of it. Give
credit only for ideas that are used appropriately in the context
of the assignment. No credit should be given for serious errors
in application or interpretation. The phrase "social studies
concepts" means a minimum of one social studies concept.
4 = The student has used social studies concepts to organize,
explain, interpret, summarize, and extend the meaning and significance
of otherwise discrete pieces of information. The use of the ideas
illustrates exemplary understanding.
3 = The student has included social studies concepts to organize,
explain, interpret, summarize, and extend the meaning and significance
of otherwise discrete pieces of information. The use of the ideas
is somewhat limited and/or shows some flaws in understanding.
2 = Social studies concepts are included, but their use is significantly
limited and/or shows significant flaws in understanding.
1 = The work includes virtually no social studies concepts, or
the use of any that are included shows almost no understanding.
The score for social studies Disciplinary Concepts should be based
on the quality of use of social studies concepts, not on the proportion
of student work that reflects social studies concepts.
Social Studies Standard 3: Elaborated Written Communication
Student performance demonstrates an elaborated account that is
clear, coherent, and provides richness in details, qualifications
and argument. The standard could be met by elaborated consideration
of alternative points of view.
To use the criteria below, the scorer should identify specific
points in the student work that are elaborated, and should make
a judgment about the coherence of the overall framework in which
various points are communicated.
4 = Exceptional: The writer provides substantial and accurate
elaboration for two or more important statements. The details,
qualifications, and nuances are expressed within an overall coherent
framework intended for the reader, and relevant to the topic.
The response is so rich as to be worthy of display as an outstanding
example of writing in social studies.
3 = Elaborated: The writer provides some elaboration for two
or three important statements or provides substantial elaboration
for one important statement. In either case, the details, qualifications,
and nuances are expressed within a coherent overall framework
intended for the reader, relevant to the topic, and without major
inaccuracies.
2 = Minimal: The writer provides reasonably accurate elaboration
for at least one important statement.
1 = Unsatisfactory: The writer provides virtually no information
or provides only disjointed details OR the writer provides discrete
claims, broad generalizations, slogans, or conclusions, but none
are elaborated.
When a task includes several parts, the score for elaboration
should be based on the part(s) answered in prose.
This document's URL is:
Home | Up & Coming | Programs & Projects: Assessment | People | Products & Publications | Topics
© 2001 Northwest Regional Educational Laboratory
Email Webmaster
Tel. 503.275.9500
