11
How can an examinee have dierent scale scores with the same number correct on equivalent test forms?
Two equivalent forms will not yield identical scale scores, except at the extremes. This is even more evident when
there are a limited number of reported levels (points possible). The dissimilarities in raw score statistics are
compensated for in the conversion to scale scores. The answer to Q34 explains why an equal number correct on
two forms does not mean equal ability levels.
What is meant by error associated with test scores?
D
RC reports the standard error for each estimate of ability across the full range of scores. Tests with
multiple-choice items allow an examinee to guess the correct answer. IRT models may include guessing as a
parameter and predict the extent to which an examinee may guess correctly. Raw scores falling at or below
the performance level predicted by chance receive the same lowest obtainable scale score (LOSS). The LOSS,
like the HOSS, has less precision (higher error) because of the difculty of establishing how low the ability is.
It should be noted that comparisons of groups (e.g., schools, students by gender) involve much smaller levels of
error (i.e., they are more stable) because variation at the level of the individual is averaged over the group.
How can an examinee be at an NRS ESL prociency level of 6, based on the average of scale scores
in two skill areas, yet the scores on the two skill areas are at NRS ESL prociency levels of 6 and 4?
NRS ESL levels are not averaged. It is the average (composite) scale score which determines the NRS ESL prociency
level. As noted in the answer to Q37, HOSS scores may be extreme enough to pull the average of the two scale
scores to an NRS ESL level that is at the same level as the test with the HOSS, even though the score on the
second test is much lower.
How can growth be measured?
There are a number of ways to assess ability (standardized tests, teacher-designed tests, project and homework
portfolios, teacher evaluations, etc.). However, standardized tests provide the best way to assess growth in ability.
Changes in scale scores on equivalent forms taken at the beginning and end of a course are the most direct
evidence, though growth may be dened in other ways.
Why do some students not show gain on a post-test?
Experienced teachers will often recognize when a student has not made sufcient progress in a course. On the
other hand, teachers may witness improvement that is not reected in test scores. This can happen for several
reasons, one of which includes measurement error. When groups are used for analysis, one often sees signicant
improvement in the group as a whole, even though not all students will show improvement.