EMERGING DIALOGUES IN ASSESSMENT

Promoting Equity and Empowering Faculty with Data Coaching

September 28, 2023

Reem Jaafar, PhD, and Milena Cuellar, PhD, LaGuardia Community College

In recent years, the recognition of data disaggregation as a crucial step in addressing equity gaps in higher education has grown significantly. Numerous studies have focused on this approach alongside culturally responsive assessment methods (Henning et al., 2022a; Henning et al., 2022b; Montenegro & Jankowski, 2020; Witham et al., 2015). However, faculty often face challenges in accessing and analyzing their student data due to a multitude of responsibilities and resource constraints.

To bridge this gap, the authors, two faculty members have embarked on a data coaching initiative supported by a grant. The primary goal of this initiative is to promote a cycle of continuous improvement and support colleagues in designing equity-minded actions at the classroom and, potentially, program levels. This paper will discuss the measures we have undertaken, the alignment of our efforts with best practices, and explore strategies for scaling and sustaining this data coaching initiative (Carlson, 2022).

Why Data Coaching?

LaGuardia Community College, an open-access Hispanic Serving Community College, is recognized for its general education assessment (Boehman et al., 2021; Lehman & Rogers-Cooper, 2021). With a diverse student body representing over 150 countries, the institution has doubled its 3-year graduation rate over the past decade. However, in pursuit of continuous improvement, we started discussing with our colleagues the need for data disaggregation and its implications for student success. Data disaggregation refers to the process of breaking down collected data into smaller subgroups, often based on demographic factors, to reveal patterns or disparities that might be masked in aggregated data. Through these conversations, it became evident that navigating the complexities of data disaggregation requires support. Thus, data coaching emerged as a practical approach to assist our colleagues in interpreting data through an equity lens. We believe it plays a vital role in our collective efforts to promote equity in student success. The goals of our data coaching include: 

1. Enhancing Faculty Engagement with Data
1. Faculty members across various disciplines possess varying levels of experience with data. Data coaching serves as a tool for professional development, enabling faculty (and staff) to better understand student success data pertinent to their specific courses and programs.

2. Unveiling Hidden Equity Gaps
1. As diversity, equity, and inclusion become prioritized objectives across higher education institutions, data coaching provides faculty the opportunity to examine and interpret equity gaps. This could potentially catalyze the implementation of "equity-minded actions" (McNair et al., 2020).
3. Promoting Student Success
1. The insights gleaned from the data can assist in identifying students who may require additional support, informing effective teaching strategies, and evaluating the success of specific "equity-minded" actions. These insights can also enrich the outcomes of periodic program reviews and curriculum revisions.
4. Aligning with Accreditation Requirements
1. Data coaching can aid institutions in enhancing their assessment procedures, thereby demonstrating their alignment with certain accrediting bodies' Diversity, Equity, and Inclusion (DEI) requirements (Kelderman, 2023).

How We Started    

We initiated this project with a grant¹, with additional support from the Office of the Provost.  

Faculty Cohort

We recruited three faculty and a data coach from STEM fields. All faculty previously knew each other. Our reasons were two-fold. First, we wanted to build a cohort that could become coaches to the second cohort. Second, our institution has quantified equity gaps in outcomes in STEM courses. Faculty committed to participating in a year-long seminar that included coaching. The two co-PIs liaised between faculty and Institutional Research (IR) and guided the seminar with appropriate resources and materials.

Data Coach

The data coach, a key player in the success of our program, embodied several important characteristics (Carlson, 2022; Glover, 2017; Hasbrouck, 2017).

These sources converge on certain ideal qualities in studies analyzing coach characteristics for faculty development. Coaches should:

• Recognize faculty's professional development needs
• Align strategies with the institution’s mission
• Collaborate with senior leadership for a supportive coaching environment
• Possess strong listening and dialogue-based questioning skills
• Prioritize relationship-building
• Have a student-centered approach
• Have significant teaching experience and deep content knowledge
• Maintain a collaborative spirit with colleagues
• Be authentic, patient, and approachable, especially when handling challenging content

Our coach, a colleague with a strong teaching background and comprehensive institutional knowledge, had prior collaborations with the authors and faculty in the cohort. This insider's view enabled the coach to customize guidance in alignment with our institution's values and needs. Faculty collaboratively work with the coach to analyze and interpret data from Institutional Research related to course and program outcomes, identifying any additional data needs. Together, they concentrate on examining disaggregated data to identify underlying patterns. In our case, the coach was our colleague with a successful teaching background and institutional knowledge and had prior collaborations with the authors and faculty in the cohort. This internal perspective allows the coach to tailor the guidance to align best with our institution's needs and values.

Crucial to the model is the approach of one-on-one coaching. This enables meaningful connections, deep discussions, and personalized guidance. Periodic follow-ups are conducted to maintain momentum and ensure continuity in learning about and questioning the data. This consistent interaction forms the backbone of our approach, supporting faculty in understanding and addressing equity gaps in student learning outcomes.

Highlights and Timeline

The cohort participated in a year-long professional development seminar, meeting several times per semester as a group and having regular check-ins with the data coach in between sessions.

Fall Semester

First, we introduced the concepts of embedding equity in assessment, culturally responsive pedagogies, and equity index (see, for example, Bensimon et al., 2003; Montenegro & Jankowski, 2020). Next, we devoted time to a thorough analysis of our institutional contexts. This involved understanding our College's unique demographics, strengths, and challenges. We explored the student data dashboard that IR provided. 

One of our major achievements during this period was the collaborative effort to frame a research question. Each faculty member worked hand-in-hand with the data coach to develop a pertinent question based on their experience. They also provided each other feedback. 

Lastly, we approached our IR with carefully framed research questions to request the necessary data. Upon receiving the data, we revisited and supported the faculty in reframing the questions as needed. This iterative process ensured that our research was grounded in available and appropriate data given our institutional contexts.

Outlined below are the themes of the faculty projects:

• Faculty #1: "Are We Failing Disabled Algebra Students? Outcomes for Students with Registered Disabilities in College Algebra."
• Faculty #2: "BIPOC Student Achievement Gaps in Nursing."
• Faculty #3: "Exploring Equity Gaps in Two Key Introductory Science Courses."

Details about each project will be the subject of a future publication.

Winter Semester

During the winter session, a reflection session was conducted, where each faculty undertook a syllabus revision for their Spring courses as an equity-minded action. As part of this process, inclusive language was incorporated into the syllabi emphasizing how students could be encouraged to utilize available resources and attend office hours.

Additionally, faculty members dedicated time to analyzing the data received from IR and requested any necessary supplementary data. They also developed other equity-minded actions to implement throughout the Spring semester.

Spring Semester

Faculty implemented and assessed the impact of their equity-minded actions. They also created a data story based on their work. The goal was to present a comprehensive narrative that uses both quantitative and qualitative data to investigate the performance of students in a certain program and with certain characteristics. Occasionally, the analysis entails comparison to students in other programs. For example, the data story of faculty #1 examines the performance of students with disabilities in College Algebra by providing context on why this group and subject are essential, then presents data comparing pass rates between students registered with disabilities and those not. Outcome gaps emerged from the analysis. The story acknowledges potential data limitations and shares personal observations and anecdotes about student engagement and help-seeking behaviors. To disseminate the findings, we organized a college-wide showcase, allowing our cohort to present their data story and emphasize the positive outcomes of these equity-minded actions to the entire College.

Sustainability

Data coaching leveraging course-level, program-level, and institutional data presents a promising strategy for promoting faculty engagement with assessment and equity-focused action in higher education. By empowering faculty to seek and interpret data related to students in the course, major and in the College, this model fosters a culture of inquiry and continuous improvement, driving efforts to ensure that higher education is a space where all students can thrive.

To replicate the success of the data coaching initiative, institutions should follow key steps while embracing the right mindset. Fostering institutional buy-in with faculty in leadership roles and IR's support is crucial. Identifying an effective data coach for the first cohort will ensure success. Through mentoring, the coach can empower faculty from the initial cohort to become coaches for subsequent cohorts.

Institutions can integrate data coaching into ongoing professional development programs, benefiting faculty and staff. This could help institutions establish a culture of data-informed decision-making. Data coaching may prompt actions at different levels, not just in the classroom. Leveraging faculty development opportunities can be an effective means for institutions to scale and sustain these practices systematically.

Acknowledgments

The authors extend their heartfelt gratitude for the funding provided by the PSC-CUNY Enhanced Award Cycle 53 and the Academic Affairs Division at LaGuardia. Special thanks are also due to the Office of Institutional Research and Assessment, especially Mr. Jeff Weintraub and Dean Nava Lerer.