Zoom Logo

CAPR Multiple Measures Placement Using Data Analytics Webinar - Shared screen with speaker view
Jim Myers
13:41
I'm here from MVCC. My computer doesn't have a camera, so I'm on the phone, waiting patiently.
Kate Schiefen
36:41
HI
Kate Schiefen
36:44
I hear you
jody-lynn storm
36:53
i can hear you
Elisabeth Barnett
36:57
Hi. Please let Jennifer know if you can’t hear.
Michelle Nightingale
36:59
We can hear you
Elisabeth Barnett
37:11
Feel free to ask questions! Or comment!
Rafael Alicea-Maldonado
37:14
I can hear fine.
jennifer.miller@suny.edu
38:32
Thank you!
jennifer.miller@suny.edu
39:26
Thank you for joining us. Here is a link to the NYS Student Success Center website: https://nysssc.org/and on Twitter at @nysssc
jennifer.miller@suny.edu
40:34
Here is a link to the early results from CAPR: https://postsecondaryreadiness.org/multiple-measures-placement-using-data-analytics/
jennifer.miller@suny.edu
41:23
Please share any questions you might have in the chat box.
jennifer.miller@suny.edu
46:01
We are recording the webinar and will be sending the link out to all registered attendees.
Jenny Schanker
50:56
Do you have any research on how long the high school gpa might remain valid for placement?
Andrea Pulskamp
51:31
What is the minimum gpa that is predictive for placing students in credit-bearing classes?
Gayle Barrett
51:39
When can we expect to see data from the self-report GPA study you mentioned in WI and MN?
John Hetts
52:20
@JennySchanker HSGPA is as good as better predictor for as long as we have data - minimum 10 years. And even at 10 years, it looks like HSGPA is asymptoting to remain more predictive than standardized tests….
Jenny Schanker
53:00
Thanks, John!
Ryan Newell
53:02
Reflecting on GPA, has it been shown that there is any differences between cumulative GPA or core classes GPA (non-electives)?
Linda Pizzi
53:21
Has anyone looked at actual HS grades in Math or English?
Josh Willis
53:39
Hi - do you have any plans to assess HSE test scores as predictors for placement? HSE tests might include the TASC, Hi-Set, or GED 2014.
Nicole
53:57
Does the high school GPA use help to minimize possible ethnic. disparate impact?
Ryan Newell
54:26
Thank you for your feedback!
John Hetts
54:27
@LindaPizzi - grades in English and Math are predictive but overall HSGPA is a stronger predictor and typically washes out the utility of the specific courses.
Linda Pizzi
54:41
@john hetts THANKS!
John Hetts
54:46
@LindaPizza - what matters more is how far students got in the sequence.
John Hetts
56:46
@Nicole - Using HGSPA in CA does reduce disproportionate impact compared to test based placement but not all the way because there are also disparate distributions in HSGPA.
Peter Arvanites
56:48
Have there been any studies concerning the actual placement tests? I have taken sample placement tests and I have found that the tests are flawed. The questions asked do not necessarily indicate a student's knowledge in the subject.
Nicole
56:53
Thank you
Peter Arvanites
57:34
Thank you.
Mark Johnson
01:02:37
GEDTS has published Performance Level Descriptors that indicate what competencies test-takers have mastered in three different score ranges -- "High School Equivalent", "College Ready", and "College Ready Plus Credit". This third score range has specifice recommendations on not only bypassing Dev Ed, but also on introductory level postsecondary course credits for which students should be given advanced standing. https://ged.com/wp-content/uploads/Performance Level_Descriptors_Chart.pdf
John Hetts
01:02:39
(For more on better predictivity of HSGPA compared to SAT, ACT, and Accuplacer for both 2 and 4 year institutions, see also the work of REL- Northwest: http://bit.ly/Hodara2016
Gayle Barrett
01:06:30
Can you share the algorithm that SUNY used?
Peter Arvanites
01:06:33
....0.......
Nicole
01:08:15
What does MDRC stand for?
Josh Willis
01:08:44
@Mark, thank you for the link. Given the wide range High School Equivalency tests (i.e., TASC and Hi-Set, in addition to GED), I’m curious about any analysis that third parties such as CAPR have re: the overall function of these tests for postsecondary use. My guess is, as Elisabeth said, test scores mostly predict test taking abilities, not course success…but am curious, especially since GPA is unfortunately not available as a predictor for high school equivalency students.
jennifer.miller@suny.edu
01:09:25
MDRC: Founded in 1974 as the Manpower Demonstration Research Corporation, “MDRC” became the official name of our organization in 2003. https://www.mdrc.org/
jennifer.miller@suny.edu
01:11:10
Here is the link to MDRC's MM Assessment project: https://www.mdrc.org/project/multiple-measures-assessment-project#overview
Nicole
01:13:06
Thanks. I have looked at some of their studies but could not locate info about who they were.
Scott Putorti
01:13:54
Hi! We know for CC collecting student historical records can be difficult. For students with missing data points how did the algorithm take that into account? Thank you!
John Hetts
01:16:48
(If people wanted the reference to the UC use of self-reported transcripts for admissions, here’s the original Insider Higher Ed coverage of the shift of four-year colleges to accept student self-reported data for admission purposes - all of the UCs and CSUs here do (but telling students they will be verified once admitted): https://www.insidehighered.com/news/2008/10/08/do-it-yourself-transcripts)
John Hetts
01:22:07
If you didn’t catch it, for English, you should notice that _every_ subgroup has higher placement and better outcomes with multiple measures than the highest subgroup of the control group.
jennifer.miller@suny.edu
01:23:17
Just a reminder, here is the link to the early results from CAPR: https://postsecondaryreadiness.org/multiple-measures-placement-using-data-analytics/
Linda Pizzi
01:26:04
One of the difficulties we've seen with HSGPA is that students with disabilities likely had an adjusted curriculum, but that is difficult to discern from the HS transcript.
Michelle Nightingale
01:27:23
Do you know why only one college in the study utilized Regents exam scores in their placement? Why did others not use Regents?
Peter Arvanites
01:27:45
I have found that looking at the actual courses that students have taken and how they perofrmed in each course is more important than the cumualtive gpa, especially when gym courses may be used in the gpa calculation (not to put gym courses down, but they are not pertinent to math and English performance).
Scott Putorti
01:28:10
Was there any research done on students who received exemptions to be placed into college level courses directly? If so, how were their success outcomes?
Michelle Nightingale
01:28:14
Did any of the participating schools in the study use the Reading Comprehension portion of Accuplacer in their algorithm for English course placement?
Michael Rio
01:28:32
Is anyone in the study using DSP?
Julie Martin
01:28:39
when creating the algorithms using historical data, what was the outcome you were predicting? ALso, how do you handle missing some or most of the algorithm predictors?
Nicole
01:28:41
I'd like to hear more about the weight of the non-cognitive measures.
Josh Willis
01:28:43
Here’s a slightly out of the box question, so apologies in advance. Is one of the assumptions here that ‘more advanced placement’ = ‘more advanced coursework/rigor’? Or could ‘more advanced placement’ = ‘different pedagogical approach’ (which would be an important variable)? For ex., I think of the vastly different success rates with traditional developmental courses vs. supported development courses (such as Start)
Jen Higgins
01:28:59
Did any of your subgroup analyses look at students with disabilities?
Peter Arvanites
01:29:15
I have read in a research journal that performance in a college algebra course is the best indicator of college success. I have been finding this to be true as I advise students.
Anne Benedict
01:29:18
Are any colleges using Success Navigator as one of their measures? If so, how are they using it?
Kathy McCoskey
01:29:20
To echo Linda Pizzi's point, I would like to know more about how to approach disabilities transcripts (for the reason(s) she noted)
Nicole
01:30:01
Were the schools that use Accuplacer using the Sentence Skills portion or the autograded essay for writing placement?
Julie Martin
01:30:40
directed self placement
John Hetts
01:30:41
That’s directed self-placement
Greg Sevik
01:30:44
directed self placement = DSP
Mary Ertel
01:30:47
directed self placement--DSP
Mark Johnson
01:31:26
I think colleges here in Wisconsin have broken the Accuplacer sentence skills section out separately -- we'll know more in a year when the newer MN/WI study is done..
Josh Willis
01:33:13
Sorry - CUNY Start
Josh Willis
01:33:24
a supported dev. course/program
John Hetts
01:33:28
@Jen Higgins and @KathyMcCoskey - in CA, we did subgroup analyses of students with disabilities and both the algorithm like work (the Multiple Measures Assessment Placement placement recommendations) and our more recent work on who should be placed into developmental education at all are both highly similar for students with disabilities. There is a small subset of students with disabilities with severe cognitive impairment/developmental delay that are best handled by your local specialists in students with disabilities in working with your local K-12 districts…
Kathy McCoskey
01:34:18
What advice do you have for how to go about evaluating the results of mulitple measures in our own colleges? It seems like a very difficult task, especially with little IR support.
Jen Higgins
01:35:12
@John Hetts, thank you!
Sally Jackson
01:36:00
As an IR person, it is not a matter of difficulty, it is a matter of quality of data and time.
Peter Arvanites
01:36:30
Thank you. This was helpful.
Greg Sevik
01:36:46
Thanks!
Kathy McCoskey
01:36:47
Thank you!