EMERGING DIALOGUES IN ASSESSMENT

• Andy Browne, Ph.D.
  Assistant Professor
  University of Tennessee Chattanooga

Abstract

Student success outcomes can improve through the inclusion of Prior Learning Assessment (PLA) or Credit for Prior Learning (CPL) in their college experience. Integrating artificial intelligence into the PLA process can assist instructors with assessing and identifying clear, articulable experiences that might reach the level of college credit. Following the integration of artificial intelligence (AI), additional opportunities were found to help maximize the amount of PLA credits earned.  With clear benefits for students, including artificial intelligence in the assessment process is an important consideration for future discussions. 

Introduction

Credit for prior learning (CPL), also known as Prior Learning Assessment (PLA), is a tool that allows students to receive college credit for their prior learning experiences. These experiences take place outside of the traditional academic classroom but can still be valuable and rise to the level of college credit (Colvin, 2012). As more contemporary learners make their way into the college landscape it is important to consider how they can be best served. Beyond credit recognition, CPL has been shown to improve levels of student persistence in their studies and graduation rates among other benefits (Rust & Ikard, 2016). The award of credit for prior learning also provides the student with a win, demonstrating the importance of their work and life experiences.

Beyond the recognition for students’ previous experiences, the awarding of CPL speeds the progression of degree achievement in many cases. Students who were awarded 12 or more credit hours through PLA are estimated to save 9 to 14 months of time as they complete their degree (Klein-Collins et al., 2020). Additionally, those same students saved up to $10,200 in tuition and fees (Klein-Collins et al., 2020). Through better outcomes in student persistence, faster progression, and reduced costs, students can be more successful in their degree pursuits, making CPL an important consideration for higher education.

In the current CPL course at the University of Tennessee Chattanooga, continuous improvement and development of assessment processes have led to the exploration of integrating artificial intelligence into the CPL Learning Portfolio workflow. As part of the CPL Learning Portfolio, students complete an autobiography with information related to work history and activities spanning from the time immediately following high school graduation to the present. Historically, faculty assessed these autobiographies to help identify areas of competence the student may be able to demonstrate at a college credit level. While this proved to be a successful practice, there were questions about the possibility of potential credit opportunities being missed during the process, so the faculty decided to identify improvements to the assessment practice. One of the most obvious options for assistance in the assessment of autobiographies was the integration of emerging technology.

Following discussion, faculty decided to begin working toward integrating Artificial Intelligence (AI) into the assessment of student learning autobiographies. There were concerns that would have to be addressed such as student consent to use AI in the process, as well as anonymizing content they would enter into a Large Language Model (LLM). Once these issues were addressed, some early testing started. Initial attempts to use AI and LLMs to assist with the identification of student competency areas provided roughly the same information that had previously been identified by faculty alone. However, as adjustments were made to the LLM prompting, new competencies started to emerge that had not been identified earlier. Some of the basic prompts were used to identify general areas of competency that might be included in the learning autobiography. This often provided a broad group of topics based on the student’s work experience. Additionally, more specific prompts started to uncover areas that were rich with student experience and that had greater potential for rising to the level of potential college credit.

While refining prompts helped bring student experiences into focus, it also helped to make connections between various experiences that were combined into a single topic supported by robust demonstrations of competency and theoretical backgrounds. The previous assessment process fell short when identifying experiences that were able to be combined. Many times, faculty viewed these experiences based on an industry or job role. For example, one student had experience in hotel management, as well as multiple positions in local government. The initial human assessment of the learning autobiography led to competencies that were very specific to the industries the student had worked in. A subsequent assessment of the autobiography using a LLM identified areas across multiple employment assignments where the student had demonstrated competency in Leadership and Customer Service, a competency that could be used for elective credit in an Applied Leadership program. Another student had experience in the electronic sporting industry as well as experience in nonprofit management. While these two seem like unlikely companions, the LLM was able to identify connections between the two areas based on responsibilities such as launching programs from inception through completion, identifying stakeholder needs, and implementing leadership principles across diverse organizations.

In addition to assisting with the assessment of student Learning Autobiographies for competency areas, AI and LLM inclusion was helpful in identifying and developing demonstrable learning outcomes for the competencies. Through careful prompting, the LLM established connections between the student’s autobiography and the learning outcomes, providing specific information relating to how the student demonstrated their achievement of the learning outcomes. An example of a prompt that has been used to identify and establish these connections might be “Identify three areas of competence related to leadership that are supported by the information in this text.”  Following this prompt with another asking “Are there any areas that overlap to form stronger areas of competence?” has helped find areas of competency that are not always noticed by reading the autobiography. This has allowed students to have a clear understanding of how their backgrounds have been converted to college level learning experiences. The students then take the information and make connections to relevant leadership literature to complete the development of each competency area. Anecdotally, all students who completed the CPL program so far and were awarded credit have gone on to earn their degree.

The inclusion of AI and LLM input in evaluating student learning autobiographies supports authentic assessment of the student’s autobiographies, particularly when dealing with lived experiences. Wiggins (1990) explained that “Authentic tasks involve "ill-structured" challenges and roles that help students rehearse for the complex ambiguities of the "game" of adult and professional life” (p. 1). These students are seeking college credit for their adult and professional experiences, with a requirement of demonstrating clear, articulatable, theory-based competence. Adding the use of AI and LLM in the assessment process has helped identify areas of competency that were previously overlooked in the prior practice of evaluating autobiographies.

As artificial intelligence rapidly becomes ingrained in many everyday tasks, there is a need to discuss ways in which we can employ it in a way that is beneficial for the assessment of student work. Hao et al. (2024) explained that advances in AI and LLMs have provided new ways to approach the assessment and evaluation of student work. While the application of artificial intelligence in the learning autobiography example above is not directly related to applying a score to an assignment, it represents the deployment of the technology in assessing and identifying areas of student performance that can lead to improved student success.

References

Colvin, J. (2012). Earn college credit for what you know. Council for Adult & Experiential Learning; Kendall/Hunt Pub. Co.

Hao, J., von Davier, A. A., Yaneva, V., Lottridge, S., von Davier, M., & Harris, D. J. (2024). Transforming assessment: The impacts and implications of large language models and generative AI. Educational Measurement: Issues and Practice, 43, 16–29. https://doi.org/10.1111/emip.12602

Klein-Collins, R., Taylor, J., Bishop, C., Bransberger, P., Lane, P., & Leibrandt, S. (2020). The PLA boost: Results from a 72-institution targeted study of prior learning assessment and adult student outcomes (Rev. ed.). Council for Adult and Experiential Learning & Western Interstate Commission for Higher Education. https://www.wiche.edu/resources/pla-boost-report-updated-12-2020/

Rust, D. Z., & Ikard, W. L. (2016). Prior learning assessment portfolio completion: Improved outcomes at a public institution. The Journal of Continuing Higher Education, 64(2), p. 94-100. https://doi.org/10.1080/07377363.2016.1177871

Wiggins, G. (1990). The case for authentic assessment. Practical Assessment, Research, and Evaluation, 2(1), 2. https://doi.org/10.7275/ffb1-mm19