Skip to main content

Bucknell's Trouble With Math

Late this past Friday, at a time that is popular for dumping unpleasant news, Bucknell president John Bravman, well, dumped some unpleasant news on an unsuspecting public.

Like all the schools of the Patriot League, Bucknell prides itself as being one of the most selective schools in the country.  Not just anyone can get in; you need to be near the top of your class.

So it came as a shock to me, and plenty of others, when Mr. Bravman dropped the bombshell that Bucknell had reported false SAT and ACT averages for a seven year stretch from 2006 to 2012.

It was a shocking admission that makes one ask the question: Why?

As President, part of my responsibility is to protect the integrity of the University and keep the Board and other informed on issues that affect institutional integrity or risk negatively affecting our reputation in the high education and broader communities.

With that responsibility in mind, I am disappointed to report that when calculating the SAT scores (math and critical reading) of the classes entering the University from 2006 through 2012, the University omitted from the SAT scores of a number of students.  Some of these omitted scores were higher than the SAT scores that the University reported, but most were lower.  Meanwhile, for several of the years in which errant SAT data were reported, the University reported ACT scores for the entering classes that were actually one point lower than the correct figures.

The letter went on to say that over the seven years of misreported data, the SAT scores were "on average reported to be 16 points higher than they actually were," and - interestingly - ACT scores that were actually 1 point lower than reported.

Inside Higher Ed also reported on the misrepresented scores, too:

Bravman's letter said he was concerned that due to "national discussions about college admissions," some people "may reach the incorrect conclusion that the scores omitted were from some single cohort that people typically cite – such as student-athletes, students from underrepresented communities, children of substantial donors, legacies and so on. All such speculation would be in error. The students came from multiple cohorts, and of course the university will not disclose their identity."

The false data were discovered after Bill Conley, a new vice president for enrollment management, noted that the mean SAT score for incoming students this year was about 20 points below last year's reported average. He then investigated, and found the pattern of false reporting.

In an interview Saturday, Bravman said that he believed a single person had been responsible for the false data. SAT and ACT scores were reported to the institutional research office in aggregate form, he said. So the institutional research officials relied on that aggregate data and never had the raw data that might have raised questions.

Bravman said that he has had discussions -- which he described as unsatisfactory -- with the person who was responsible for the reporting, and whom Bravman declined to identify. Bravman said that this person denied trying to make the university's admissions process look better either for internal or external audiences, and never offered a real explanation for what had happened.
He went on to say that it showed "ignorance at best, incompetence at worst" in terms of the person responsible, though it also appeared to show a gap in their internal processes as well.  
Aggregate numbers, the letter stated, were prepared by "enrollment management leadership" which prepared the false numbers, distributed them to everyone and "posted them on the school's website", the letter explains.

That would seem to point to the person responsible as someone under the administration of Kurt Theide, the former Vice President for Enrollment Management under former Bucknell president Brian Mitchell.

Again, though, the giant question remains unanswered from this whole affair.  Why?  Why violate the integrity of their numbers, published numbers that are used basically everywhere?

With the vehement assertion from Mr. Bravman himself that the students came from "multiple cohorts", it's clear there is no cover-up to hide bad SAT numbers of, say, athletes or legacies or any other group, so that can't be it.


It can't be incompetence.  That might explain one year's worth of fudged numbers, or maybe even two or three - but seven consecutive years of numbers fudged upwards suggests a campaign to get those numbers up.

It could be that they wanted the numbers to look a little better for PR purposes - to move up a few spots in the U.S. News and World Report rankings, for example.

But Mr. Bravman himself described the variations were described as "small" and they do not expect "to have any impact" on those rankings, so it's hard to believe, too, that this was the goal.

While student-athletes weren't specifically targeted for omission, what should also be clear, though, is that this campaign, deliberate or not, probably affected football recruiting at Bucknell.

The Patriot League's Academic Index, or A.I. for short, consists of two measurements: 50% GPA, and 50% test scores, to determine student eligibility to compete on their intercollegiate sports teams.  It is similar, but not identical, to the AI used by the Ivy League.

Bucknell's average SAT, based on the old 1600 point scale, was 1309 in 2012 according to collegedata.com, and presumably hasn't been updated with the new numbers yet.  The GPA for the entire class, which should be unchanged, is 3.52.

In football, the average AI of the incoming class determines the breakdown of the academic qualifications for the football recruits, called "bands". 

Recruits can fit into one of four "bands", based on the academic profile of members of the entire incoming class and three AI "bands" below that to the "floor", which is 168.  For example, a student with perfect test scores and a 4.0 GPA would have a 200 AI.

As I mentioned, the AI for the Patriot League is different from the AI of the Ivy League.  However, using this Ivy League AI Calculator as a very rough guide, a shift of 10 SAT points on Math, Critical Reading, and Writing, using Bucknell's numbers, results in a two point shift in AI.



Two points doesn't seem that significant, you say?  Think again.

Why?  For that, you have to go back to statistics class.





The uppermost football band from the incoming class in the academic index is defined as the average AI of the incoming class up to the AI ceiling of 200.  As mandated by the league, Patriot League schools need to recruit no fewer than five athletes in this uppermost band.


Remember the bell-shaped curve from statistics class?  That's the concept of a data distribution where there's a standard deviation of 1, and 2/3rds of data points show up in the middle of the curve - the 34.1% blue bands in this picture to the right.

(General population SAT scores fall on a bell curve as well.  Scaled scores are in fact designed so that most scores fall in the 400-600 range.)

See that steep part of the curve that says 13.6%?  That's approximately where Bucknell's average SAT numbers fall on the bell curve - the top 10% of all students in the country. 

Move that needle over just two points - which, in effect, is what Bucknell's pumped-up numbers are doing - and the rate of qualified applicants you're talking about drops significantly, including athletes.  A 20 point swing is very significant in this steep part of the curve - it could mean the difference between, say, the top 11% of students in the country to the top 8%.

Furthermore, an inflated AI number could have changed the nature of the football bands at Bucknell.

The bands are determined by the overall standard deviation of the class, which I'm assuming should broadly have remained the same with the upwardly-fudged numbers.  (I'm assuming this to be the case, since even with about 5% of the incoming student body's SAT scores removed from an incoming class of 916 students, I don't believe the standard deviation would change enough to change this significantly.)

For those without a statistics textbook handy, the standard deviation is the square of the data points' difference from the average, which is then subsequently averaged and then square rooted.  But more broadly, the standard deviation computes the average variation between the average AI and the AI of each individual student.

Let's say that Bucknell's "inflated" AI were a 187, with a standard deviation of 6.  Their bands would be:

Band I: AI of 187 and above: 5 athletes
Band II: AI between 181 and 187: 10 athletes
Band III: AI between 174 and 180: 14 athletes
Band IV: AI between 168 and 174: 3 athletes

But let's say their actual AI were a 185, with the same standard deviation.  Their bands would then be:

Band I: AI of 185 and above: 5 athletes
Band II: AI between 179 and 185: 10 athletes
Band III: AI between 172 and 178: 14 athletes
Band IV: AI between 168 and 172: 3 athletes

Even this two point swing could have dramatic overall changes in football recruiting.  There are more and more students (and football recruits) available for each lower AI point by definition since SAT scores, remember, are by definition a standard curve.  A drop from 187 to 185 in the top band might not be all that significant, but a drop in the third band from 174 to 172 would be.

I do not know what the ultimate motivations are of the person or people that were responsible for Bucknell's "problems" with math.  What I do know is that their actions over the last seven years may have affected the academic index of their football team, making it just a little bit harder to admit football recruits. 

Comments

Anonymous said…
Your calculations are incorrect. The maximum AI is 240.

Popular posts from this blog

Remembering Lehigh's Battles With The Late Tubby Raymond

(Photo Credits: Delaware Online)

When I heard the news Tubby Raymond, legendary Delaware head football coach, died last week at the age of 92, two immediate memories came rushing back to me.

One occurred on October 16th, 1999, when Tubby had made a complaint to the local paper or radio in the run-up to Kevin Higgins' Mountain Hawks beating his Blue Hens on Delaware's homecoming, 42-35.

I have no idea if the quote even actually happened, but my recollection is that Tubby said that Lehigh had "St. Bartholomew's" on their schedule, and hadn't played anybody.  It was a verbal jab that many Delaware fans took with them to the stands to heckle the Mountain Hawk fans that made the short trip to Newark.

Up until that point, I had watched a bunch of Lehigh football games over the years.  I experienced their rise in the 1990s.  I enjoyed wins, and championships, and playoff victories.

But never had I felt a win so viscerally vindicating than the one over Tubby Raymond&…

Lehigh Wrestling Gets Superstar Treatment at PPL Center. Lehigh Football Needs The Same At Murray Goodman.

"We knew it would be nice," Lehigh wrestling head coach Pat Santoro said. "But it was even better than we expected."

Pat was talking about the reception his Lehigh wrestling squad experienced at Allentown's PPL Center this weekend, when a sellout crowd over nearly 10,000 people came to watch No. 1 Penn State grapple with No. 5 Lehigh in a collegiate wrestling event.

It was, by all accounts, something special to behold. 

"I thought it was really cool and an exciting place to wrestle," said Penn State wrestler Nick Nevills. "These fans were really into it, a rowdy bunch. It's a lot more fun as an athlete to wrestle in an environment like this. I'd say it was one of the most exciting times I have had in my career."

The sense of spectacle at the PPL center, though, puts a spotlight on what more can be done at Lehigh itself to make their athletic contests into spectacles.  It requires money to be spent and energy to be expended.  But the…

Troy Pelletier Hasn't Stopped Outworking His Rivals On His Journey To Professional Football

Many Patriot League football fans remember the 153rd meeting between Lehigh and Lafayette, one that ended happily for the Mountain Hawks.

They might remember the MVP performance of QB Brad Mayes, or perhaps the halftime speech by OL Zach Duffy that seemed to spur the Mountain Hawks to victory.

Or perhaps they might remember the spectacular single play of Mayes rolling right and finding WR Gatlin Casey in the end zone to give the Mountain Hawks a lead they didn't relinquish.  It was an incredible play by Mayes, who returns this upcoming year for his senior season, and Casey, who, having exhausted his eligibility at Lehigh, will be playing one more year at Middle Tennessee State.

As great as those individual moments are, though, they are not my biggest takeaway of that game.

Too many Lehigh people forget that Lehigh was down 31-21 at half, and that victory was no sure thing.  And they sometimes forget that so much of that victory came from the grinding of WR Troy Pelletier, deliver…