Placement Tests, HS GPA, and Multiple Measures … “Just the Facts”

We know that repeated statements are often treated as proven statements, even if the original version of the statement was not accurate.  In other words, if you want people to accept your point of view … don’t worry about whether it is accurate,  just make sure that your statement is repeated by lots of people over a period of time.  Like “HS GPA is a better predictor than placement tests”.

The original message seems to have been based upon a CCRC report (“High Stakes Placement … “, 2012 by Clayton). https://ccrc.tc.columbia.edu/publications/high-stakes-placement-exams-predict.html  The conclusion about tests versus HS GPA is this:

First, focusing on the first or second columns, which examine the predictive value of placement scores alone for slightly different samples, one can see that exam scores are much better predictors of math outcomes than English outcomes. The overall proportion of variation explained is 13 percent for a continuous measure of math grades, compared with only 2 percent for a continuous measure of English grades. This is consistent with the findings from previous research.

The date being referenced is this:

Not only is the ‘HS GPA is better’ not accurate for the original research (for mathematics), people never mention a fundamental issue:

The data for the 2012 study came from ONE “large urban community college system” (LUCCS)

Now, I don’t doubt the basic premise that including more variables can improve a decision (such as a test plus HS GPA).  The problem is that the message “HS GPA is better” has been repeated so often, by so many people, that decision makers accept it as truth.  The truthfullness depends a great deal on the decision being made — placement in English, or placement in Mathematics.  The situation looks pretty clear (in the LUCCS data) for English, where using the HS GPA only seems a better thing.  In Mathematics … not so much!

Researchers have developed models for placement in mathematics based on HS transcript data, though I’ve never seen a proven model using just HS GPA.  The variables connected to these research models involve:

• Specific mathematics courses completed in high school (especially grades 11 and 12)
• Specific grades received in those mathematics courses
• As a minor factor, the overall HS GPA

A good prototype of this scheme is the California “MMAP” work; see http://rpgroup.org/Portals/0/Documents/Projects/MultipleMeasures/DecisionRulesandAnalysisCode/Statewide-Decision-Rules-5_18_16_1.pdf  .  Rather than the ‘drive-off-the-cliff’ approach (North Carolina, Florida, etc), this is a scientific approach to a complicated problem.  Few of our colleges, and few states, are willing to invest the resources necessary for this truth-in-multiple measures approach.  [The fact that California can do this seems to have been a consequence of decisions about higher education in that state 50 and 60 years ago.  We probably won’t see that again.]

Some additional truth about HS GPA:

High School GPA transmits inequity

Here is some data from the US Department of Education transcript study (2009):

The issue is not that HS GPA transmits inequity while placement tests do not.  In the case of SAT Math (and ACT Math) the gaps are known to exist.  The issue is that HS GPA transmits the inequity without regard to the student’s abilities in a subject domain.

The race/ethnicity ‘gaps’ for HS GPA are just one way to establish that it transmits inequity.  Economic and geographical inequities are also apparent in the HS GPA data.  At least the test developers strive to minimize their inequity; items which show a significant differential impact are removed from the tests.

Placement tests are less harmful to students than HS GPA.

The truth about multiple measures is that it will only help students when implemented in a scientific manner in the location or region involved.  HS GPA by itself will harm students.

 Join Dev Math Revival on Facebook:

How to Impact Student Success

College leaders (presidents, trustees, chancellors, etc) have discovered “student success” as an issue, and they promptly implement systemic changes which impede student success.

In some ways, their errors are understandable.  We’ve got plenty of data which shows …

• Traditional remediation in mathematics most often functions as a barrier to students
• Students who complete college math in their first year are more likely to complete their program/degree
• Placement by single-measure tests tends to underplace 20% to 30% of the students

Leaders have also accepted the surface logic of “alignment” (At the Altar of Alignment  ), just like some folks accept the logic of ‘trickle-down-economics’.  Alignment takes many forms … from aligning K-12 and college expectations to selecting a math course for a student’s program.  Little data exists to show that alignment improves student success; like tax cuts, alignment is difficult to argue against — even though we should.

When I talk about student success, I am referring to the important measures of student success — learning, preparation, and a liberating education.  Passing my math course is not a measure of student success … being able to deal with mathematics in other situations IS.  Curiously, I asked by college president about measuring student learning as a component of student success; the response was that we should drop course grades and move to a portfolio.

So, here is the type of thing I mean by student success.

In a conversation with a small group of science faculty, they shared their frustration with student’s inability to apply math — algebra in particular — to scientific contexts.  A low level example was a simple temperature conversion:  T[sub C] = (5/9)(T[sub F] – 32), given temperature of 40 degrees C, convert to degrees F.

Many students treat this as a calculation problem (5/9)(40 – 32), instead of algebraic.  It seems to make no difference if subscripts are used or the letters C and F instead.

Student success is being able to reason (algebraically) in this case to get the job done.

In this case, we have ‘alignment’. The math course students took before the specific science course included replacements for both independent and dependent variables.  Alignment is a very (VERY) weak estimate of preparation for student success.

My goal of student success is not especially lofty.  In a nutshell, this is it:

Given a situation involving application of concepts and skills easily within the mathematical reach of the students, they will formulate a reasonable solution method and execute this solution with reasonable precision.

This goal is quite a bit above the useless definition of student success seen by college leaders: course completion one-at-a-time.  Student success means that my colleagues in other disciplines would be pleasantly surprised by how well our students apply mathematical concepts and relationships which arise in that discipline.  Those faculty would not need to dilute the scientific rigor of their course (in whatever discipline) just because the students we send to them lack quantitative understanding.

We live in an era of ‘completion obsession’.  It’s not that program completion is bad … completion is a great thing; the best day of my year is getting to see some of my students walk across the stage to get their degree.  The problem is that the obsession with completion devalues the education we are supposed to be providing to our students.  In the completion fixation, we watch students on the marathon course to make sure that they pass each critical point — without noticing that many students are running without understanding strategy or skill.  It’s like perseverance is the only trait we value.

Our job is to keep education in mathematics.  Student success means that we’ve made a difference in how our students are able to deal with quantitative situations; mathematics is an enabler of multiple career options for all students, not a subject to be gotten-done-with.

 
Join Dev Math Revival on Facebook:

At the Altar of Alignment

The answer to all questions is “42” (see Hitchhikers Guide).  The solution to all problems is “alignment”.  Academic leaders, government officers, and policy makers are using the word “alignment” in attempts to address many perceived failures in academia.  Alignment is not even a necessary property, and is certainly not sufficient, for an academic system to be successful.

At the micro-level, people tell us to align course outcomes.  If course A is a prerequisite to course B, then the outcomes should be “aligned”. In cases where our goals are strictly operational (just the doing, not the understanding nor the reasoning), we can align courses.  I’d suggest that this is a very weak methodology for a mathematics curriculum, since aligning outcomes directs our attention to the fine levels of granularity as opposed to the basic story line of a course.  A stronger design is to focus on mathematical abilities being developed over time … both within a course as well as across courses.  Alignment is often counter-productive in mathematics.

At the mid-level, we are told to align the mathematics required with the needs of the student’s program.  In other words, if the primary quantitative need of an occupation is the consumption of statistics, then the mathematics required for the program should be a statistics course.  As attractive as this alignment might be … the practice is based on two unfounded assumptions — (1) that a student KNOWS what they plan to become when they begin college, and (2) that this plan is relatively stable over time for each student.  Unless we plan to return society to pre-global, pre-fluid periods for occupations, alignment is a dis-service to many students.   Instead of alignment, we’d be better served by offering a good mixture of valuable mathematics, not specialized.

At the macro-level, we try to align K-12 mathematics with college mathematics (or, vice versa).  The unfounded presumption here is that K-12 mathematics exists primarily to prepare students for college mathematics. And, there is an assumption that this ‘alignment’ (whatever it means in this context) will make a significant difference.  Like aligning course outcomes, aligning levels of education tends to push our attention down to small details —  in other words, alignment is based on focusing on insignificant details while ignoring larger concerns.  For this level alignment, think about what would be more powerful:

• Students have mastered skills A1 to A5, B1 to B7, C1 to C4, and D1 to D8 which logically can be followed by A6 to A9, B8 to B12, C5 to C10, and D9 to D11.OR
• Students develop learning and academic skills (including mathematics) to develop reasonable proficiency as well as an ability to learn in a variety of situations using different tools.

We spend time at the altar of alignment, working on ‘solutions’ which have little chance of helping students.  Education is much more than the sum of a finite series of detailed objectives … education is much more than learning just the mathematics needed for an expected occupation … education is more than a series of steps which present a surface logic but lack power in a person’s life.

Our time would be better spent in seeking a vision and some wisdom on educating students, educating them for capacities and success.  The checklist success of alignment is worthless compared to the benefits of education done well.

 Join Dev Math Revival on Facebook:

“Envisioning our Future” launched … Mathematics in the First Two Years

I am developing a new page on this blog … ‘envisioning the future’, devoted to where we are (or should be) going with mathematics in the first two years.  See Envisioning Our Future

The rationale for putting a focus on this ‘envisioning’ is simple — too much of our effort is currently invested in either defending our traditional curriculum OR in responding to demands to change in specific ways.  We need to focus on where we want to go in the long term, instead of coping with demands resulting in short term changes.

Progress means that we are closer to our goals on this path called ‘college mathematics’.  Our strategies should place this progress at the center of our work whenever possible.

I hope that you will find some useful ideas, and perhaps even some inspiration in this content.

 Join Dev Math Revival on Facebook: