By David Dirlam
In March’s wonderful dialogue about SLOs and grades, there was a truly elegant tension going on between assessing for institutions and assessing for teaching and learning. The money and power seem always to be on the institutional side, but great teachers know that the excitement of being where the insights and enduring commitments happen belongs to teachers and learners. Accreditation is an institutional issue that sometimes reaches up, like Joan Hawthorne notes, to touch teachers and learners. But like she implied, the real action occurs when a teacher and student collaborate to create more than either could create by themselves.
Collaborative action can sometimes reach down to the institutional level and even transform it. The problem occurs when institutions try to impose simplified solutions that undermine the teacher-student relationships rather than enhance them. Standardized tests are harrowing exemplars of such undermining, since no one can use the results except to judge rather than collaborate with others.
As Ephraim Schechter so carefully pointed out, the too-easy dictum that the loop must be closed every year for every program is another such simplified solution. Perhaps we should start calling it “tightening the noose” rather than “closing the loop.”
A third way of undermining teacher-learner relationships is the whole idea that we must aggregate to know. We do not know any planet with life other than the earth, but does that mean that there can be no science of earth? Unique sets of bones, unique mutations, and unique geologic formations have been staples of science since it first emerged as a unique human enterprise. Not only is history riddled with unique events that have transformed culture but from a chaos theory point of view, there is hardly a dynamic event that does not lead to unique outcomes.
Some statisticians say we must have more than 30 individuals in a sample. But if a person flips a coin 15 times in a row and it always comes up heads, another person would be utterly foolish not to conclude that it is a biased coin. The problem with uniqueness or even with small-sample studies is as much a problem with interpretation as it is with statistics. But how often would a statistician (or psychologist or educational researcher) consult a good historian or text analyst before he or she draws a conclusion about a small sample study? Not often enough!
This does not mean a diminished role for the statistician. The great social scientist of psychology, Kurt Danziger, wrestled with the dilemma of how to interpret social science results for years and finally concluded that the best answer had to be converging observations. When those with differing forms of expertise engage in careful analysis and agree with each other from independent points of view, we can cautiously feel more confident in their conclusions. Until we understand what others know that we do not, we will make narrow interpretations of our experiences, no matter how paradigmatic our methodology is.
Perhaps our language customs are part of the problem. Grades, and worse SLOs, (Student Learning Outcomes), address learning as if it is some fait accompli. Would Student Learning Commitments help us to see ourselves and our students as works in progress? And what would be the implications for accreditation if we started to use such a conception? At a recent on-site review, a reviewer asked what the faculty learned from me. Even though I had talked with the faculty a lot about closing the loop, and created things like Course Design Surveys to help then do it, none mentioned that. Rather, all those in the room told different stories about how I had helped them use concepts of development in their classroom. Though fear of accreditation may have funded the budget for my assessment office, I believe that using development in the classroom is both more difficult and more worth our dedication to achieve than closing the loop.
So, how could an accreditation agency or AALHE change to be more effective in supporting teacher-learner interaction? I believe the key for accreditation agencies is to ask faculty to document unique student commitments and keep track of the duration and diversity of contexts that those commitments reveal. Many individual stories combined with aggregated data should be encouraged. A paradigm for story records is the program created by David Eubanks and David Gliem at Eckerd College in which students produce real-world impacts with unique projects. Furthermore, individual stories about students should include some record of things teachers learned from their students as well as the reverse. For only with such a record do we learn about and encourage students to contribute to society. We do not know how long what we teach will last or when our own learning will return to haunt or inspire us. Aggregating our stories in time and space helps us to give context to learning, while context without uniqueness should bore accreditors as much as it would any good text analyst.
As for AALHE, we have a great opportunity for documenting both aggregated and unique data at both the institutional and the learner-teacher levels. No other organization, to my knowledge, embraces the diversity of methodology as well as AALHE. And this elegant dialogue is a powerful source of evidence in support of that statement.