Jan 11, 2016

Will my child be a National Merit® Scholarship finalist?

Imagine this scenario.  Two students from the same state have identical Total PSAT/NMSQT® scores from the October 2015 test. One is a National Merit Scholarship® Semifinalist, and the other is not.

Because NMSC cut-off scores have traditionally varied significantly from state to state, it’s not unusual for students with the same PSAT scores in different states to have different outcomes. For example, in Wyoming it only took a Selection Index of 202 to be named a NMS Semifinalist this year, but in New Jersey it took 225. Despite these inequities across states, however, cut-off scores have been consistently applied to all students living in a particular state for as long as most people can remember.

But with the way in which the NMSC Selection Index has been computed for the 2015 or ‘redesigned’ PSAT, it is certainly possible that students residing in the same state with identical total scores could have very different outcomes for the 2017 competition.

So how might this happen?

First, it’s important to break the habit of thinking of NMSC Selection Index (SI) and Total PSAT Score as one and the same. It’s no longer true.

The new Selection Index is the sum of three scores—math, writing/language, and reading—each on a scale of 8 to 38 multiplied by two.  An alternative method of computing is to double the overall ERW Score, add the Math, and then drop the zero.  In this context, the zero is out of place and adds nothing. 

Both methods work. And both methods show how superior math students might be finding their particular skill slightly suppressed by both the new scale and the SI computation.

“Because the new Selection Index formula gives more weight to the verbal score, we’ll see two students with the same PSAT score but different SI,” explained Bruce Reed, of Compass Education Group.  “In this one sense, the new scale ‘punishes’ the very high math scorer who is ‘capped’ at 760. That student might have earned a score as high as 80 last year.”

Here is an example of how the formula also works against math students.  Two students have Total PSAT Scores of 1460 out of a possible 1520.  Student A has 760 in Math and 700 in EWR, while Student B has 700 in Math and 760 in EWR.  Drilling down to individual test scores, Student A has 38, 35, and 35. To arrive at the SI, the scores are added together and multiplied by 2 for a total of 216. Student B has scores of 35, 38 and 38, resulting in a Selection Index of (35 + 38 + 38) x 2 = 222.

With a span of six points, it’s easy to imagine a situation in which Student A, the outstanding math student, would not receive an invitation to compete as a National Merit Scholarship semifinalist, while Student B would. And the difference could get even closer in many circumstances.

There is a good deal of speculation about what the new test and scale will do to the NMSC cut-off scores both nationally for Commended Scholars and state-by-state for Semifinalists. The lower ceiling on scores—38 as opposed to 40—has caused a corresponding drop of the Selection Index ceiling from 240 down to 228.  This doesn’t necessarily mean, however, that state cut-off scores will drop by 12 full points.

Other forces come into play including the general upward drift of scores.

“Scores for most students are higher on the 2015 PSAT scale than they would have been on the 2014 PSAT scale,” said Reed.  “But at the highest levels, scores are lower—a 240 in 2014 could be no higher than 228 in 2015. This conflicting set of forces is what makes the National Merit scores particularly hard to predict this year. It’s possible we’ll see cut-offs in some states rise and in other states fall.”

Out of this confusion, different approaches to forecasting National Merit status have appeared. Many families are looking at percentiles to make judgements about likelihood of earning “Commended Scholar” or “Semifinalist” designation. And to arrive at these estimates, families are using percentiles derived from the National Representative Sample, which is prominently displayed on the front of the score report, when they should be using the “Test User” sample—a more difficult to find number provided to individual students online after they complete registration with the College Board.

And then there are the concordance tables generated by the College Board, which compare 2014 PSAT scores to 2015 PSAT scores. But without having a feel for where the Commended Scholar cut-off—top 50,000 scorers—might come, it’s a little difficult to extrapolate future commended status. Add to this the politics and vagaries of how the National Merit Scholarship Corporations determines state cut-offs, and the projections become even more difficult except at the very highest total score levels.

According to Bruce Reed, “Both percentiles and concordances have weaknesses when making comparisons to last year’s cut scores. In general, using percentiles to estimate the Commended cut-off gives a lower estimate than concording between old scores and new. Statistically, they should give similar results, so there appear to be inconsistencies in some of the College Board's preliminary numbers.”
Once you get past all the numbers, percentiles, and projections, the real decision of who will or will not be a National Merit Semifinalist rests in the hands of NMSC corporate executives who largely operate behind closed doors. Now may be an opportunity for them to reconsider how the program works and rethink state cut scores.  

But regardless of how they decide to proceed for next year’s competition, unless something changes between now and next fall when 34,000 test-takers will be granted Commended Scholar status and an additional 16,000 students will earn Semifinalist status, two students living in the same state and maybe even attending the same high school with the same Total PSAT scores could easily have very different outcomes—one a merit scholar and one not. 

Thank you again to Bruce Reed, of the Compass Education Group, for helping sort this out.  

And the bottom line:  the crystal ball method might be somewhat more enlightening for this question than the College Board’s glitzy marketing spin!

No comments:

Post a Comment