The Mixed Methods Blog

Perspectives from our researchers, highlights from recent studies, and other news about CCRC

2018 Innovations Conference Showcases CCRC Research

college campus students working together

The League’s Innovations Conference in National Harbor, MD, which ran from March 18–21, gave our researchers a chance to update community college faculty and staff on several projects CCRC has been working on, and to explain interim findings that might give some answers to the common problems colleges face.

The researchers presented research on tools used to facilitate high-quality advising, the use of multiple measures for placing students into developmental education, and engaging adjunct faculty.

Session 1: Designing a Toolbox for Developmental Advising Leveraging Technology

Presenters: Andrea Lopez, CCRC; Stefanie Crouse, Montgomery County Community College; Melissa Boynton, MDRC

It’s becoming more and more clear that intensive advising is an important ingredient in any effort to transform a college to help more students graduate. This session described the design and application of an advising toolbox, a resource CCRC and MDRC developed in partnership with colleges that are reforming their advising practices as part of a research project on technology-mediated advising. The toolbox outlines strategies to help advisors ask open-ended questions and focus on the important interventions that will create a stronger, more effective advising relationship with students. Each participating college created its own toolbox while taking into consideration the college’s institutional context and culture.

The toolbox, which is flexible enough to account for students with different needs, builds the conversation around what is going on with the student, what the student wants to achieve, and how the student will accomplish his or her goals. As part of the advising redesign being undertaken by the colleges, advisors develop a set of learning outcomes to ground their interactions with students.

“We want the advisor to probe deeper and have engaging conversations with students,” said Andrea Lopez, a senior research assistant with CCRC.

Session 2: Student Assessment and Placement Systems: Initial Outcomes From An RCT

Presenters: Elisabeth Barnett, CCRC; Elizabeth Kopko, CCRC; Michelle Ragucci, Schenectady County Community College

The Center for the Analysis of Postsecondary Readiness (CAPR), a research center led by CCRC and MDRC funded by the Institute of Education Sciences, is studying the effectiveness of using multiple measures to place students into developmental education instead of using the more common placement tool, a single standardized test. Seven colleges in the State University of New York system developed placement algorithms based on their own historical data that weigh factors such as high school GPA, the number of years since a student has graduated high school, the type of high school diploma, and other factors depending on available data.

The research was spurred by earlier studies that found that placement tests misplace large numbers of students into remediation—29 percent in English and 18 percent in math—who could get a B or better in college-level math or English.

Using GPA alone is already more predictive than using a placement test, said CCRC Senior Research Scientist Elisabeth Barnett, but “using multiple measures gives you some extra boost.”

The study is tracing the outcomes of students randomly assigned to be placed the old way or using the multiple measures algorithm. In early results for nearly 5,000 students at five colleges, the study found that placement in, enrollment in, and completion of a college-level math course was several percentage points higher for students placed using multiple measures.

In English, the results were much more dramatic. All three metrics went up by double digits. Total college-level credits earned also went up slightly.

“We’re hoping we’ll see some larger gains as we track students for a longer period of time,” said Elizabeth Kopko, a postdoctoral research associate with CCRC. A report on the early findings will be released in the coming months.

Members of the audience questioned whether GPA is a reliable measure of students’ ability when the rigor of the programs in their feeder high schools varies so much. But Brad Bostian, the director of first year experience at Central Piedmont Community College in North Carolina, who was in the audience, said his research shows that GPAs measure comparable skills. In his studies of students from the feeder schools to North Carolina’s 58 colleges, he found that even though some schools are harder graders and the content of their courses may differ, the attributes students need to earn an A, such as good study habits, consistent attendance, and completion of homework, are similar.

“Those skills do carry over,” Bostian said.

Session 3: Achieving the Dream’s Engaging Adjunct Faculty Initiative: Report From the Field

Presenters: Jonathan Iuzzini, Achieving the Dream; Jennifer Roberts, Community College of Philadelphia; Dallas Dolan, Community College of Baltimore County; Jasmine Sanders, CCRC

With 67 percent of the community college teaching staff made up of adjunct faculty, Achieving the Dream (ATD) launched a project to help six colleges develop methods to support adjunct faculty to improve instruction and to become engaged in student success initiatives. CCRC is evaluating the work.

The project started with the question, “Who are adjunct faculty, and what are their experiences?” A survey, interviews, and focus groups at the six participating ATD leader colleges found that adjunct faculty have diverse backgrounds but generally have significant teaching experience. Thirteen percent of adjunct faculty members have a doctorate, and 73 percent have a master’s degree. Also, half have a degree or credential in teaching. The average number of years of teaching experience is 18.

Some are K-12 educators, some are retired faculty who didn’t want to leave teaching altogether, and others have primary careers in other fields. Some teach at many colleges, some only at one.

“I feel like it debunks a lot of myths about who adjunct faculty and part-time faculty are,” said Jasmine Sanders, a senior research assistant with CCRC, of the survey.

Along with their strengths, the research also identified challenges part-time faculty face, including a lack of access to pertinent information and the need to accommodate changing college schedules, which makes it hard to hold down other jobs, Sanders said. Because they impact the instructors’ preparation, these challenges can also impact the quality of education the students receive. 

Colleges are addressing these challenges and engaging adjunct faculty through efforts such as adjunct faculty institutes and workshops, teaching circles that bring together faculty teaching the same course, the creation of websites with information on colleges’ support services and teaching resources, adjunct faculty appreciation events, and the development of adjunct faculty work centers. Colleges are also bringing faculty into the planning and implementation of initiatives such as the introduction of high impact practices and guided pathways.

CCRC will publish the research findings in a brief this spring, and Achieving the Dream will publish a practitioner’s guide later this year, said Jonathan Iuzzini of ATD.

Looking Ahead at the Next Questions About Guided P...
Thomas Bailey, Founding Director of CCRC, Named Ne...