Skip to content
  • Research
  • About Us
  • News
  • Community College FAQs
  • Blog
  • Pandemic Recovery

Focus Areas

  • Dual Enrollment
  • Developmental Education
  • Guided Pathways
  • Advising & Student Supports
  • Teaching & Learning
  • Transfer
  • College to Career
Menu
  • Dual Enrollment
  • Developmental Education
  • Guided Pathways
  • Advising & Student Supports
  • Teaching & Learning
  • Transfer
  • College to Career

Publications Library

CCRC’s complete collection of publications

Presentations

Webinars and conference presentations with CCRC researchers

Guided Pathways Workshops

Materials from our do-it-yourself workshop series

Policy Resources

Our collection of federal policy briefs and fact sheets

  • About CCRC
  • CCRC Staff
  • Research Affiliates
  • Advisory Board
  • Biennial Report
  • Employment
  • Contact
Menu
  • About CCRC
  • CCRC Staff
  • Research Affiliates
  • Advisory Board
  • Biennial Report
  • Employment
  • Contact
  • CCRC in the News
  • Opinion
  • Press Releases
Menu
  • CCRC in the News
  • Opinion
  • Press Releases
  • Overview
  • Important Dates
  • FAQs
  • Overview
  • Important Dates
  • FAQs
  • Overview
  • Important Dates
  • FAQs

From Course to Skill: Evaluating Large Language Model Performance in Curricular Analytics

By Zhen Xu, Xinjin Li, Yingqi Huan, Veronica Minaya & Renzhe Yu

Curricular analytics (CA)—the systematic analysis of curricula data to inform program and course refinement—is an increasingly valuable tool to help institutions align academic offerings with evolving societal and economic demands. Large language models (LLMs) are promising for handling large-scale, unstructured curriculum data, but it remains uncertain how reliably LLMs can perform CA tasks.

In this AIED conference paper, the authors evaluate four text alignment strategies based on LLMs for skill extraction, a core task in CA. Using a stratified sample of 400 curriculum documents of different types and a human-LLM collaborative evaluation framework, they find that retrieval-augmented generation is the top-performing strategy across all types of curriculum documents. Their findings highlight the promise of LLMs in analyzing brief and abstract curriculum documents, but also reveal that their performance can vary significantly depending on model selection and prompting strategies. This underscores the importance of carefully evaluating the performance of LLM-based strategies before large-scale deployment.

Download Links

View conference paper (subscription may be required)
July 2025

Additional Resources

For more policy briefs and fact sheets, visit CCRC’s Policy Resources page.

  • Our Research
  • About Us
  • Publications
  • News
  • Community College FAQs
  • Blog
  • Pandemic Recovery
  • Our Research
  • About Us
  • Publications
  • News
  • Community College FAQs
  • Blog
  • Pandemic Recovery

Community College Research Center, Teachers College, Columbia University
Box 174 | 525 West 120th Street, New York, NY 10027

  • 212.678.3091
  • ccrc@columbia.edu

© 2025. All rights reserved.

Facebook-f Twitter Linkedin Youtube Instagram
Join our mailing list
  • Our Research
    • Focus Areas
    • Publications Library
    • Presentations
    • Guided Pathways Workshops
    • Policy Resources
  • About Us
    • CCRC Staff
    • Research Affiliates
    • Advisory Board
    • Biennial Report
    • Employment
    • Contact
  • News
  • Community College FAQs
  • Blog
  • Pandemic Recovery
  • Our Research
    • Focus Areas
    • Publications Library
    • Presentations
    • Guided Pathways Workshops
    • Policy Resources
  • About Us
    • CCRC Staff
    • Research Affiliates
    • Advisory Board
    • Biennial Report
    • Employment
    • Contact
  • News
  • Community College FAQs
  • Blog
  • Pandemic Recovery