A Multi-Study Research Project to Inform the Bill & Melinda Gates Foundation’s Postsecondary Success Initiative

In May 2009, the Community College Research Center (CCRC), Teachers College, Columbia University received a $5 million grant from the Bill & Melinda Gates Foundation to build a rigorous base of research knowledge on strategies for accelerating progression and increasing success among low-income young adults attending community colleges. This three-year project will produce a set of concrete recommendations to help the foundation select effective investments for its Postsecondary Success (PS) initiative, with the goal of doubling the number of low-income adults who by age 26 earn a postsecondary degree or credential—an increase of about 250,000 graduates each year.

Objective: Testing Seven Strategies

CCRC will analyze existing research literature and conduct a new set of studies to assess the effectiveness of seven promising yet largely untested strategies for promoting student success among disadvantaged young adults. Each of these seven strategies will be addressed through one or more of eight specific studies plus a survey of assessment practices at a national sample of community colleges. CCRC will examine these strategies in terms of their impact on student success, cost-effectiveness, and feasibility. It will also identify program characteristics and organizational practices that support effective implementation of each strategy on a large scale. The following hypotheses explain the reasoning inherent in each broad strategy.

1. Actionable Assessment – Colleges will have more success with students, both in developmental and college-level education, if assessments identify what students need to be successful, in addition to identifying the level of skills and knowledge that they have at the time of the assessment.

2. Structure/Focus – Underprepared students are more likely to complete programs that are highly structured and focused, especially when they are aimed at preparation for credentials and job placement in career fields.

3. Online Learning – High-quality and engaging online learning opportunities increase access, improving progression through school for low-income and underprepared students.

4. Acceleration – The rate at which academically underprepared students complete remedial instruction and succeed in college-level courses can be increased by helping students to proceed through requirements more quickly or by encouraging them to enroll in higher level courses where additional academic support is provided.

5. Contextualization – Low-skill students can learn more effectively and can advance to college-level programs more readily when skills are taught in the context of instruction in a content area.

  

6. Non-Academic Preparation (“student success” skills and college/career planning) – Students who enroll in college underprepared for college-level work benefit from assistance with college and career planning and from instruction in “student success” skills, in addition to academic remediation.

  

7. Systemic Institutional Alignment/Improvement – Institutions that are most effective in serving disadvantaged students are those that align and manage their programs and services strategically to increase the success of such students. Even successful individual programs in isolation cannot benefit students on a substantial scale without a supportive institutional environment—colleges must shift their focus from program effectiveness to organizational effectiveness.

Research Approach: A Three-Stage Process

For each hypothesis, CCRC will carry out research activities in three stages:

1. Literature review – CCRC will conduct a systematic analysis of current research to define existing knowledge on each hypothesis.

2. New studies – CCRC will build on that foundation with strategically selected new studies that can yield high-quality results in under three years. CCRC will conduct some studies of strategies that are administered on a small number of campuses, but most of the strategies to be studied are implemented across the community college system of an entire state (or city in the case of a study involving CUNY’s six community colleges).

3. Integrated report – CCRC will integrate the initial literature reviews, findings from the new studies, new evidence from other ongoing research projects, and input from the project’s advisory board members to produce a hypothesis-driven report that focuses on implications for policy and practice.

Eight New Studies in Four States

CCRC will conduct eight new studies. Six of these will use community college system data (from Washington State, Virginia, and CUNY). The remaining two studies will use institutional data from community colleges in Colorado. All of the studies will involve multivariate quantitative analysis. Each study will also include a qualitative component to investigate the program-level mechanisms and the institutional policies and supports that underlie effective implementation.

Study 1. Evaluation of I-BEST (Washington State). Using data on two-year college students from the Washington State Board for Community and Technical Colleges (SBCTC), CCRC will conduct an evaluation of I-BEST, a contextualized instruction model in which adult basic skills and career-technical faculty jointly design and teach college-level occupational courses for adult basic skills students. CCRC will examine both educational and labor market outcomes of I-BEST students and conduct fieldwork at I-BEST programs that are found through quantitative analysis to be particularly effective in serving low-income young adults.

Study 2. Structure of effective occupational programs (Washington State). This study will examine the structure and supports provided by community and technical college occupational programs that are effective in enabling low-income young adults to complete credentials and obtain employment. This study will test the hypothesis that more structured and focused programs that provide students with a more directed curriculum and proactive student supports lead to better outcomes for students who are not well-prepared for college-level work. CCRC will develop and apply a protocol for measuring key dimensions of program “structure” and then use unit record data from the SBCTC to compare student outcomes between more- and less-structured programs.

Studies 3, 4, and 5. Studies in partnership with the Virginia Community College System (VCCS). Through its strategic planning process, VCCS has set ambitious goals for improving student retention, with a special focus on disadvantaged students. CCRC has begun several quantitative studies using Virginia data (supported by Lumina Foundation as part of the Achieving the Dream initiative) examining Online Learning (Study 3), Student Success Courses (Study 4), and developmental education policies and practices that contribute to Gatekeeper Success for Remedial Students (Study 5). The new research in Virginia will build upon these studies through qualitative investigation of the practices of programs and colleges found through quantitative analysis to be effective in serving low-skill young adults.

  

Study 6. Developmental education assessment, placement, programming, and outcomes (CUNY). The CUNY system includes six community colleges, which enroll large numbers of minority, low-income, and underprepared students. CUNY provides an excellent setting to examine the effectiveness of remedial placement policies because, unlike many other college systems, it tracks students’ initial placement exam scores, remedial placement recommendations, developmental course enrollments, and exit exam scores. Individual CUNY community colleges have also implemented a wide variety of innovations to improve outcomes for developmental students, including acceleration, non-academic preparation, and contextualization. CCRC will use student unit record data from all six colleges to examine remedial assessment and placement policies, programmatic interventions, and the impact of each on students, particularly low-income young adults.

 

Study 7. Study of accelerated developmental education model FastStart (Community College of Denver). This program, which provides intensive accelerated and contextualized instruction with extensive student supports, has received a great deal of attention nationally through its association with the Ford Bridges to Opportunity and Breaking Through initiatives. The program was first implemented in 2005, and now enrolls approximately 150 students each semester. Participants are largely age 22 or younger and non-white; about half are Pell grant recipients. Both the Community College of Denver (CCD) and the Colorado Community College System (CCCS) will share unit record data with CCRC that will allow a comparison of outcomes of FastStart students with non-participating CCD students who have similar characteristics.

Study 8. Study of accelerated learning communities (Front Range Community College, Colorado). Front Range’s program is designed to accelerate underprepared students’ progress through college-level courses through team-taught “blended” courses. These courses combine developmental content in reading and writing with college-level content in introductory degree-credit courses, enabling remedial students to move directly into college-level work. The program, which began around 2005, is expected to grow to 160 students per semester by the fall of 2010. Administrators are planning to institutionalize the learning community model, with the goal of linking 85 percent of their highest-level remedial English courses with transfer-level general education courses. Front Range Community College has agreed to share unit record data with CCRC to investigate the model’s effectiveness.

In evaluating Front Range’s learning communities as well as the Community College of Denver’s FastStart and the Washington State I-BEST programs, CCRC will collect cost data and estimate the cost-benefit of each model. This analysis will aid in the assessment of the potential productivity gains that could be achieved by implementing these approaches on a large scale, and of the feasibility of doing so.

Nationwide Assessment Survey

An examination of system-wide and institution-specific assessment practices is included in the Virginia, CUNY, Denver Community College, and Front Range studies. These studies will provide an in-depth perspective on the effectiveness of current developmental education assessment and placement within those particular contexts. A wider question remains, however, about the range of current assessment and placement practices at community colleges across the country and the interaction between state policy and institutional practice. CCRC will conduct a five-state, 50-institution survey to chart the current landscape of assessment and placement practices in community colleges.

Research Advisory Board

This project will benefit from the breadth of experience and the judgment of a nine-member research advisory board. About half its members will be researchers, including at least two institutional researchers working in community colleges; the other half will be college presidents with strong research backgrounds or state officials or researchers at the state level. Specific plans for the studies will be discussed with advisory board members. The board will also review the initial results of the literature reviews and discuss whether those findings imply any changes in direction for the overall project. The board will also meet to discuss the results from specific studies and to deliberate on draft recommendations for the PS initiative.

Project Products

This project will result in the following deliverables:

 

1. Individual study reports and briefs. For each of the eight new studies, these reports will include estimated program effect sizes, including effects on low-income and underprepared young adults, and assessment of the cost-effectiveness and feasibility of implementing effective strategies at scale within community colleges.

2. Hypothesis-driven reports. The reports on each hypothesis will integrate findings from multiple studies (e.g., the institutional alignment report will draw findings from all eight studies). The reports will present project findings in the context of the larger literature on the given program strategy.

3. Comprehensive final report. CCRC will work with the research advisory board to develop recommendations based on the research findings for practice and policy. A comprehensive final report will pull together results across the seven hypothesis areas and make recommendations in terms of the most cost-effective investments for achieving the PS goals for increasing student success.