ToolKit98

Sample A.20

On-Demand Speaking

Massachusetts Department of Education

In the early 1980's, Massachusetts conducted a series of studies on the feasibility of conducting large-scale performance assessments of speaking with eighth graders. The purposes were to monitor general student skill in speaking and promote more speaking instruction. Students spoke on four topics (description, emergency, sequence, and persuasion) in a one-on-one setting with the rater. It does not appear that advanced preparation time was given to students. Four forms of the test (four sets of topics) were prepared.

Attached are the performance criteria used to assess student oral presentations.

Development of the State Speaking Assessment Instrument: Reliability and Feasibility Study, 1983; and Reliability and Feasibility Study, 1982

State of Massachusetts

Department of Education

350 Main Street

Malden, Massachusetts 02148

Handout #3

Speaking Assessment Criteria*

CONTENT

The content rating focuses on the specific things which are said. It is concerned with the amount of content related to the task, the relevance of the content to the task and the adaptation of the content to the listener and the situation.

1 = The delivery is inadequate in meeting the requirements of the task.

The speaker says practically nothing.
The speaker focuses primarily on irrelevant content.
The speaker is highly egocentric. The speaker appears to ignore the listener and the situation.

2 = The content is minimal in meeting the requirements of the task.

The speaker does not provide enough content to meet the requirements of the task.
The speaker includes some irrelevant content. The speaker wanders off the topic.
The speaker adapts poorly to the listener and the situation. The speaker uses words and concepts which are inappropriate for the knowledge and experiences of the listener (e.g., slang, jargon, technical language). The speaker uses arguments which are self-centered rather than other-centered.

3 = The delivery is adequate in meeting the requirements of the task.

The speaker provides enough content to meet the requirements of the task.
The speaker focuses primarily on relevant content. The speaker sticks to the topic.
The speaker adapts the content in a general way to the listener and the situation. The speaker uses words and concepts which are appropriate for the knowledge and experience of a general audience. The speaker uses arguments which are adapted to a general audience.

4 = The delivery is superior in meeting the requirements of the task. Examples are:

The speaker provides a variety of types of content appropriate for the task, such as generalizations, details, examples and various forms of evidence.
The speaker adapts the content in a specific way to the listener and situation. The speaker takes into account the specific knowledge and experience of the listener, adds explanations as necessary and refers to the listener's experience. The speaker uses arguments which are adapted to the values and motivations of the specific listener.

NOTE: This rating is concerned with content in terms of quantity, relevance and adaptation. It is not concerned with content in terms of accuracy. Concerns with accuracy of content fall outside a speaking skills assessment. Also, make sure you are not unconsciously "filling in" content for a speaker because you happen to know something about the speaker's topic. If you add information, this fact should be reflected in your rating. REMEMBER, in this component you are rating the quantity, relevance and adaptation of what the student says, not the accuracy of what the student says.

*From the Massachusetts Department of Education Assessment of Basic Skills, Speaking Assessment Rating Guide

ORGANIZATION

The organization rating focuses on how the content of the message is structured. It is concerned with sequence and the relationships among the ideas in the message.

1 = The organization is inadequate in meeting the requirements of the task. Examples are:

The message is so disorganized that you cannot understand most of the message.

2 = The organization is minimal in meeting the requirements of the task. Examples are:

The organization of the message is mixed up; it jumps back and forth.
The organization of the message appears random or rambling.
You have difficulty understanding the sequence and relationships among the ideas in the message. You have to make some assumptions about the sequence and relationships of ideas.
You cannot put the ideas in the message into an outline.

3 = The organization is adequate in meeting the requirements of the task. Examples are:

The message is organized.
You do not have difficulty understanding the sequence and relationships among the ideas in the message. You do not have to make assumptions about the sequence and relationships of ideas.
You can put the ideas in the message into an outline.

4 = The organization is superior in meeting the requirements of the task. Examples are:

The message is overtly organized.
The speaker helps you understand the sequence and relationships of ideas by using organizational aids such as announcing the topic, previewing the organization, using transitions, and summarizing.

NOTE: Make sure you are not unconsciously "filling in" organization for a speaker, because you happen to know something about the speaker's topic. If you have to make assumptions about the organization, this fact should be reflected in your rating. REMEMBER, in this component you are rating how the student organizes the message, not what the student says.

DELIVERY

The delivery rating focuses on the transmission of the message. It is concerned with volume, rate, and articulation. Articulation refers to pronunciation and enunciation. Some examples of poor articulation include mumbling, slurring words, stammering, stuttering, and exhibiting disfluencies such as "ahs," "ohms," or "you knows."

1 = The delivery is inadequate in meeting the requirements of the task. Examples are:

The volume is so low that you cannot understand most of the message.
The rate is so fast that you cannot understand most of the message.
The pronunciation and enunciation are so unclear that you cannot understand most of the message.

2 = The delivery is minimal in meeting the requirements of the task. Examples are:

The volume is too low or too loud.
The rate is too fast or too slow. Pauses are too long or at inappropriate spots.
The pronunciation and enunciation are unclear. The speaker exhibits many disfluencies, such as "ahs," "umhs," or "you knows."
You are distracted by problems in the delivery of the message.
You have difficulty understanding the words in the message. You have to work to understand the words.

3 = The delivery is adequate in meeting the requirements of the task. Examples are:

The volume is not too low or too loud.
The rate is not too fast or too slow. Pauses are not too long or at inappropriate spots.
The pronunciation and enunciation are clear. The speaker exhibits few disfluencies, such as "ahs," "umhs," or "you knows."
You are not distracted by problems in the delivery of the message.
You do not have difficulty understanding the words in the message.

4 = The delivery is superior in meeting the requirements of the task. Examples are:

The speaker uses delivery to emphasize and enhance the meaning of the message. The speaker delivers the message in a lively, enthusiastic fashion.
The volume varies to add emphasis and interest.
Rate varies and pauses are used to add emphasis and interest.
Pronunciation and enunciation are very clear. The speaker exhibits very few disfluencies, such as "ahs," "umhs," or "you knows."

NOTE: In articulation you may be concerned with accent. However, articulation should be rated with respect to your ability to understand the message, not the social acceptability of the accent. One particular accent is not considered better than another. REMEMBER, in this component you are rating how the student speaks, not what the student says.

LANGUAGE

The language rating deals with the language which is used to convey the message. It is concerned with grammar and choice of words.

1 = The language is inadequate in meeting the requirements of the task.

The grammar and vocabulary are so poor that you cannot understand most of the message.

2 = The language is minimal in meeting the requirements of the task.

The speaker makes many grammatical mistakes.
The speaker uses very simplistic, bland language. The speaker uses a "restricted code," a style of communication characterized by simple grammatical structure and concrete vocabulary.

3 = The language is adequate in meeting the requirements of the task:

The speaker makes few grammatical mistakes.
The speaker uses language which is appropriate for the task, e.g., descriptive language when describing, clear and concise language when giving information and explaining, persuasive language when persuading. The speaker uses an "elaborate code," a style of communication characterized by complex grammatical structure and abstract vocabulary.

4 = The language is superior in meeting the requirements of the task.

The speaker makes very few grammatical mistakes.
The speaker uses language in highly effective ways to emphasize or enhance the meaning of the message. As appropriate to the task, the speaker uses a variety of language techniques such as vivid language, emotional language, humor, imagery, metaphor, simile.

NOTE: In language you may be concerned with students who come from backgrounds where a foreign language or a non-standard form of English is spoken. However, language should be rated with respect to your ability to understand the message, not the social acceptability of the communication style. If a speaker's use of incorrect or non-standard English grammar interferes with your ability to understand the message, this fact should be reflected in your rating. REMEMBER, in this component you are rating how the student conveys the message through langauge, not what the student says.