Imagine this scenario.
Two students from the same state have identical Total
PSAT/NMSQT® scores from the October 2015 test. One is a National Merit Scholarship® Semifinalist,
and the other is not.
Because NMSC
cut-off scores have traditionally varied significantly from state to state,
it’s not unusual for students with the same PSAT scores in different states to have
different outcomes. For example, in Wyoming it only took a Selection Index of 202
to be named a NMS Semifinalist this year, but in New Jersey it took 225.
Despite these inequities across states, however, cut-off scores have been
consistently applied to all students living in a particular state for as long
as most people can remember.
But with the way in which the NMSC Selection Index has been
computed for the 2015
or ‘redesigned’ PSAT, it is certainly possible that students residing in
the same state with identical total scores could have very different outcomes
for the 2017 competition.
So how might this happen?
First, it’s important to break the habit of thinking of NMSC
Selection Index (SI) and Total PSAT
Score as one and the same. It’s no longer true.
The new Selection Index is the sum of three scores—math,
writing/language, and reading—each on a scale of 8 to 38 multiplied by
two. An alternative method of computing
is to double the overall ERW Score, add the Math, and then drop the zero. In this context, the zero is out of place and
adds nothing.
Both methods work. And both methods show how superior math students
might be finding their particular skill slightly suppressed by both the new
scale and the SI computation.
“Because the new Selection Index formula gives more weight
to the verbal score, we’ll see two students with the same PSAT score but
different SI,” explained Bruce Reed, of Compass
Education Group. “In this one sense,
the new scale ‘punishes’ the very high math scorer who is ‘capped’ at 760. That
student might have earned a score as high as 80 last year.”
Here is an example of how the formula also works against math
students. Two students have Total PSAT
Scores of 1460 out of a possible 1520.
Student A has 760 in Math and 700 in EWR, while Student B has 700 in
Math and 760 in EWR. Drilling down to
individual test scores, Student A has 38, 35, and 35. To arrive at the SI, the
scores are added together and multiplied by 2 for a total of 216. Student B has
scores of 35, 38 and 38, resulting in a Selection Index of (35 + 38 + 38) x 2 =
222.
With a span of six points, it’s easy to imagine a situation
in which Student A, the outstanding math student, would not receive an
invitation to compete as a National Merit Scholarship semifinalist, while
Student B would. And the difference could get even closer in many
circumstances.
There is a good deal of speculation about what the new test
and scale will do to the NMSC cut-off scores both nationally for Commended
Scholars and state-by-state for Semifinalists. The lower ceiling on scores—38
as opposed to 40—has caused a corresponding drop of the Selection Index ceiling
from 240 down to 228. This doesn’t
necessarily mean, however, that state cut-off scores will drop by 12 full
points.
Other forces come into play including the general upward
drift of scores.
“Scores for most students are higher on the 2015 PSAT scale
than they would have been on the 2014 PSAT scale,” said Reed. “But at the highest levels, scores are lower—a
240 in 2014 could be no higher than 228 in 2015. This conflicting set of forces
is what makes the National Merit scores particularly hard to predict this year.
It’s possible we’ll see cut-offs in some states rise and in other states fall.”
Out of this confusion, different approaches to forecasting
National Merit status have appeared. Many families are looking at percentiles
to make judgements about likelihood of earning “Commended Scholar” or
“Semifinalist” designation. And to arrive at these estimates, families are
using percentiles derived from the National Representative Sample, which is
prominently displayed on the front of the score report, when they should be
using the “Test User” sample—a more difficult to find number provided to
individual students online after they complete registration with the College
Board.
And then there are the concordance
tables generated by the College Board, which compare 2014 PSAT scores to
2015 PSAT scores. But without having a feel for where the Commended Scholar
cut-off—top 50,000 scorers—might come, it’s a little difficult to extrapolate
future commended status. Add to this the politics and vagaries of how the
National Merit Scholarship Corporations determines state cut-offs, and the
projections become even more difficult except at the very highest total score levels.
According to Bruce Reed, “Both percentiles and concordances
have weaknesses when making comparisons to last year’s cut scores. In general,
using percentiles to estimate the Commended cut-off gives a lower estimate than
concording between old scores and new. Statistically, they should give similar
results, so there appear to be inconsistencies in some of the College Board's preliminary
numbers.”
Once you get past all the numbers, percentiles, and
projections, the real decision of who will or will not be a National Merit Semifinalist
rests in the hands of NMSC corporate executives who largely operate behind
closed doors. Now may be an opportunity for them to reconsider how the program
works and rethink state cut scores.
But regardless of how they decide to proceed for next year’s
competition, unless something changes between now and next fall when 34,000
test-takers will be granted Commended Scholar status and an additional 16,000
students will earn Semifinalist status, two students living in the same state
and maybe even attending the same high school with the same Total PSAT scores
could easily have very different outcomes—one a merit scholar and one not.
Thank you again to Bruce Reed, of the Compass Education Group, for helping
sort this out.
And the bottom line:
the crystal ball method might be somewhat more enlightening for this question than the College Board’s glitzy marketing spin!
No comments:
Post a Comment