- About AALHE
- Member Portal
- Support AALHE
|EMERGING DIALOGUES IN ASSESSMENT|
Transformative Assessment: A Proposed Process
Friday, June 9, 2017
As faculty members work to make student learning transformational, assessment practitioners should work to make assessment transformational for faculty. Assessment can transform the experience of teaching faculty and bring them closer to their students and student learning. If the heart of transformational learning is a change in perspective or frame of reference of students (King, 2002), transformational assessment would change the perspective of faculty members. First, faculty would change their perspective of assessment; rather than seeing assessment solely as an external and unnecessary demand for data, faculty members would see assessment as a transformative experience that brings them closer to their understanding of their students and student learning. Faculty would also transform their experience of students and student learning as they are empowered to change the curriculum and their teaching to improve student performance.
While many assessment practitioners may envision assessment as transformative, not all faculty are equally convinced. Assessment practitioners can learn specific techniques and ways of being to inform their interactions with faculty. Adapting Kegan and Lahey’s work (2001, 2009) can give assessment practitioners some examples for understanding faculty and creating ways of working with faculty that can provide opportunities for altering faculty perspectives about assessment and about teaching and learning.
If assessment is to be transformative, it will place faculty members at the center of the assessment process. Transformative assessment would be assessment that faculty members design, choosing what works best for the organization’s context, which includes the discipline type. Assessment makes most sense to faculty when it mirrors the kind of research faculty do. For example, faculty from a bench science discipline may find that data from a quantitative assessment of a multiple-choice exam resonates more with their understanding of what students should be learning, whereas faculty members who practice qualitative research may find that results from interviews or open-ended questions make more sense to the faculty. The disciplinary context, as well as the political context of the department, budget constraints, the number of teaching faculty members and other factors, so that the assessment is part of the environment, rather than an additional requirement that does not fit with the reality of the organization.
Of course, there are requirements from accreditors, states, institutions and other entities to which we must attend. These requirements may have no other solution but to take an additional exam, survey or other measures that do not speak to the organization’s context. In such cases, there may be no other solution but to simply comply. Even in these cases, however, faculty can introduce assessment efforts that do hold value for them, even creating methods to be intentionally low-effort in acknowledgement of the other requirements thrust upon them. Accreditor requirements that do not match the faculty members’ context is the hard work of being an assessment practitioner that requires creativity, perseverance, and acceptance of what is possible and what is not. In other situations, however, keeping an open mind to interweave requirements with transformational assessment practices can produce the happiest of circumstances: assessment procedures that speak to the faculty members and the accreditors at the same time.
Transformative assessment would also encourage faculty to develop genuine relationships with students and with the data that they collected from the students through an analysis of authentic information about student performance with both direct and indirect measures, taking the faculty members’ observations and knowledge of their students into consideration. Assessment would become important to the faculty because it would speak directly to their experience. Transformative assessment takes faculty perspectives as legitimate and necessary means to create effective assessment strategies that accurately measure and analyze student learning.
All of that may sound good, but as many assessment practitioners know, faculty members do not always embrace assessment to make such an approach possible. Faculty members often have resentments toward assessment that may be well earned: frustrations with administration, budget fears, past destructive use of assessment results, and previously ineffective assessment approaches have left some faculty with a skeptical view of assessment. Faculty may even greet a new assessment approach with open hostility, even when they are deeply invested in teaching and learning, their students, and their research.
When assessment practitioners approach faculty, they can keep in mind that the complaints they hear from faculty represent more than grumbling at additional work. Most people complain because their values have been violated and this perspective supports the assessment practitioner in finding a path to create transformative assessment with faculty members. This approach borrows heavily from Kegan and Lahey’s 2001 work How the Way We Talk Can Change the Way We Work and 2009’s Immunity to Change.
The approach is a two-step process that first involves a highly adapted method from The Way We Talk. Understanding what faculty members value provides the assessment practitioner an avenue to shape assessment into a process that supports the faculty members’ values rather than be a tool against their values. The second step, again highly adapted, but this time from Immunity to Change, provides the assessment practitioner tools to move assessment forward with a group of faculty.
Complaint to Commitment
The first step is in determining what the violated values are. Sometimes assessment practitioners may resist listening to faculty complaints about assessment because it seems to be a waste to time; rather than moving assessment processes forward, the complaints seem to be a way to avoid assessment altogether. This approach, however, asserts that the faculty complaints are a rich resource for understanding what it is they value.
The Complaint You Heard. The table below illustrates how to listen to the faculty complaints. The first column is “The Complaint You Heard,” and refers to listening deeply when the faculty complain. In this situation, the assessment practitioner would not react to the complaints either positively or negatively, but rather listen to understand. What is the complaint—or at least how are the faculty members voicing that complaint. In the example below, the complaint is about the administration, that the administration attempts to control faculty, and finally, that assessment is a tool for that control.
The Unspoken Complaint (if it exists). The second column asks the assessment practitioner to use information he or she may already have. For example, it may be that the faculty members in the conversation are not tenured or that the department is about to be merged with another. It could also be that what the faculty members are complaining about cover another complaint. In the example below, the unspoken complaint is also about control, but this time, administration’s control is not the issue, but the faculty members’ lack of it. Immediately, the assessment practitioner can see that dismissing the concern about the administration’s control will not necessarily ease the concern about the faculty member’s lack of control. Of course, the faculty members may well be voicing their complaints specifically and there is no unspoken complaint.
Figure 1: From Complaint to Commitment
The Underlying Commitment. The third column requires the assessment practitioner to re-phrase the complaint in a positive way. If what the faculty members’ stand against is clear, what do they stand for? In this example, the assessment practitioner determines that the issue at play is autonomy or academic freedom. Checking this assumption with the faculty members is important. In this instance, the assessment practitioner might say, “It sounds like faculty retaining control over the curriculum and teaching methods important. Would you agree that’s the primary issue?” Faculty may be vociferous in their correction of the assumption, but at least stating what the assessment practitioner assumes is going on will provide an opportunity for clarity.
Connection to Assessment. This column represents an opportunity for the assessment specialist to connect what the underlying commitment or value is directly to assessment. This can be challenging, especially if the assessment practitioner has limited assessment experience, but with experience, the connection becomes clearer. In this example, the assessment practitioner perhaps overstates the case, insisting that assessment is academic freedom, but connecting assessment to a defense of academic freedom moves to align faculty and assessment on the same side.
Once the faculty—or at least a critical mass of faculty—align with working on assessment, the next step is to guide the faculty through a process to create transformative assessment. Kegan and Lahey (2009), as in their 2001 work, have created a system in the form of a table that encourages people to reflect on their own goals and obstacles to those goals. In fact, Immunity to Change(2009) uses an actual academic department as a case study to examine how their techniques can bring together a struggling faculty as team. Such a use is the intended use as described by Kegan and Lahey. Many assessment practitioners, however, are not in a position to comment on the cohesion of a faculty or to suggest a process to improve their relationships. If the possibility presents itself and the faculty are willing, such a process would likely make the assessment process even better.
Given the limitations of what assessment practitioners may have available to them, Kegan and Lahey’s 2009 work has been adapted below as a series of steps to assist in a transformative assessment cycle, but not necessarily to support faculty in examining the role they play in the department’s community. Please note that these steps are not necessarily hierarchical; an assessment practitioner should be flexible enough to sense what the faculty or departments needs at any given time and use that appropriate step.
Transformative Assessment Process
Student learning outcomes. The first step in the process, as in traditional assessment cycles, is to get faculty to focus on what outcomes they want for their students. Outcomes can grow out of the above Complaint to Commitment process where the assessment practitioner aligns assessment to the values of the faculty. To do so, we need to know what faculty value, what are the goals they are attempting to achieve. A prepared assessment practitioner would also investigate outcomes recommended or required by a professional accreditor (if applicable) or a regional accreditor, and come with those in hand to the conversation. Additionally, information about the learning outcomes for the same discipline at peer institutions, can give the faculty a starting place in creating their own. All of this additional information provides a context for the faculty to consider, and prevents the need for faculty to revisit the outcomes after brainstorming.
What do we already know about what students are learning? This step is a survey of what is already happening in the department that gives the faculty or leadership any indication of what students are learning. This can include surveys, exams, presentations, faculty observations of student performance or other measures. The idea is to discover the strengths and weaknesses of the students, as the faculty currently understand them, as well as to gain an overview of what measures if any are in place. From this conversation, the faculty can choose what areas or questions are most important to address.
This conversation can also include the department’s discipline and what kind of data makes sense for the faculty. For example, what kinds of data exist? Are they direct or indirect measures? Are those measures quantitative or qualitative? The attempt here is to determine whether the measures speak to faculty. If an English department has quantitative data about students’ ability to analyze a poem, this is unlikely to resonate with the faculty. What kinds of measures are already in place that the faculty value?
Figure 2. Developing Transformative Assessment
What advantages and disadvantages face us? This question allows faculty to look at what might be working against them. For example, it may be that the accreditor may have such restrictive requirements that faculty cannot create their own learning outcomes, measures, or other assessment elements. If this is the case, this does not mean necessarily that the faculty do not support the requirements. There may be wiggle room for faculty to interpret the requirement to more closely align with their own values and processes. Alternatively, there may also be advantages. Students in the department may already be taking a professional exam, but that the department has not done much with the data. This is a rich source that with some analysis can provide assessment results. Taking a moment to take stock of what is already in place—even if faulty—and what is slowing the process down will support the assessment cycle.
This can also be a time for faculty to talk about their fears. If they are worried that any transformational assessment might be frowned upon by the accreditor, institution, state or other entity, now is a good time to talk about it. What is most important is that the assessment is authentic to the faculty experience. Faculty who leave this process rolling their eyes may not have been heard.
Developing assessment tools. Once a faculty, departmental, and discipline context has been established, the traditional six elements of assessment can be continued. Since learning outcomes will already have been established, the work can focus on continuing the assessment measures, if more conversation is needed. What measures, with guidance from the accreditors, peer institutions, and relevant others, should be developed or furthered? A curriculum map can support the faculty in deciding when in the program of study the assessment measure should be taken. A faculty meeting is also a good time to discuss what individual faculty members can each do as this supports ensuring the workload is equitably, if not equally, spread. It is also useful at this time if the assessment specialist is well informed about resources his or her office can provide including staff time and resources.
There are any number of processes that make an assessment process transformative for the faculty. In fact, once data is collected and analyzed, transformation of teaching and learning (and not just of the faculty experience of assessment) begins. Finding a process, however, that speaks to the faculty, makes them central to the experience and respects their perspectives (even as it views those perspectives are likely to change),
Kegan, R., & Lahey, L. L. (2009). Immunity to change: How to overcome it and unlock potential in yourself and your organization. Boston, Mass: Harvard Business Press.
Kegan, R., & Lahey, L. L. (2001). How the way we talk can change the way we work: Seven languages for transformation. San Francisco, CA: Jossey-Bass.
King, K. P. (2002). A journey of transformation: A model of educators’ learning experiences in educational technology. In J. M. Pettit & R. P. Francis (Eds.). Proceedings of the 43rd Annual Adult Education Research Conference, (pp. 195- 200). Available: http://www.adulterc.org/Proceedings/2002/papers/King.pdf
Dr. Becky Dueben has worked at Washington State University for the past 18 years in several different positions including instructional designer, faculty member, and assessment specialist. She currently works in the Voiland College of Engineering and Architecture on college-level assessment. She also directs student programs (such as the Tutoring Center and the Ambassador Program) while investigating how those support programs influence student learning at the program level. Becky finished her Ph.D. in December 2015, focusing on faculty members attitudes toward assessment.