Internal Validity

Yesterday, I had the pleasure of attending my first (online) class meeting of “Project Management and Oversight for Evaluators,” a course taught by Tessie Catsambas through The Evaluators' Institute (TEI). I appreciate so many things about TEI courses, but, as someone who usually works as a team of one, I especially enjoy meeting and learning from other evaluators who work in very different contexts.

Beyond the fact that we all work in such different environments, one of the things that struck me in class was how consistently articulated the concept of internal validity is in the work of my colleagues. All evaluators center internal validity, but when you work predominately with clients who don't use the language of social science research (as opposed to spending a lot of time with evaluation colleagues), you find yourself using different words to express similar concepts, drawing less on the vernacular of design and measurement.

As I listened to my new colleagues in class, I was reminded that one of the big risks of working mostly alone and not engaging in daily conversation with other research scientists is the possibility of accidentally backgrounding "out of sight, out of mind" research fundamentals, perhaps even such key requirements internal validity. But I also noticed that folks in class had questions for me, and I had a breadth of examples to draw from. It occurred to me that there is a risk, also, for working with a team of others who share so much of your training and experience – the kind of myopia that we've seen demonstrated many times and which communication specialists refer to as "groupthink."

Educators know that engaging in multiple perspectives is the backbone of critical thinking and deliberative problem solving, not to mention strong policy making. Yet, it seems like our educational policies and our societal institutions take the possibility of deeply considering multiple perspectives away from us. Neighborhoods and schools are becoming increasingly segregated, in some cases re-segregated, by race and class. Teaching and learning in grade schools and high schools are constrained by educational outcomes that leave little time for students to authentically probe multiple viewpoints for authentic research tasks that lead to forming one's own informed opinion.

Earlier in the week, I asked my students, "When was the last time you intentionally read something that you knew in advance you would strongly disagree with?" I got thirty-five blank stares. And I wasn't surprised. Under what circumstances in our contemporary American society would they come across a reason to do that? When they see something they don't like, they swipe left and it's gone.

I recalled one summer when I was about their age and working as a student research assistant on a project at Texas Tech University, and the professor I was working with got a sudden job offer and was packing up his office. He had a wall of books he'd decided weren't worth the trouble to take, and he offered me first choice. He pulled a couple of tattered books out, recommending that I read them, but I reached for another book -- the dust jacket still glossy and new -- and asked if I could have it. In the vernacular of our region, he replied, "Sure, but that's total bullshit."

The book was Allan Bloom's The Closing of the American Mind: How Higher Education Has Failed Democracy and Impoverished the Souls of Today's Students, and from not just the title alone but also the looks of the bio on the back, Bloom had some serious academic chops, the very kind I wanted for myself. I took the book home and started in, trying to discover what was so bullshitty about what looked to me like a probable masterwork. 

I can say now that I read that book more critically than I had up to that point read any other book in my life. On the surface (for me, at 21, taking courses in classical rhetoric and 19th century art history, history of the British Empire and contemporary American poetry, while serving buckets of beer to frat boys at night to make ends meet) Bloom's central claim made complete sense – if anybody was up for a Socratic quest, it was me. I had been working so hard to redirect myself from a vocational/agricultural path and make my way to the ivory tower that it seemed like Bloom was telling me I was doing things right. 

A lot has changed since the 1980s. I finally got my braces off, for instance.

At the same time, my professor was my mentor, had seen something in me that I wasn't even sure was there, had expected more of me than I expected of myself, had introduced me to feminist theory even. And he was disgusted by Allan Bloom's thesis, and apparently by Bloom himself. So, I read the book, wooed by Bloom on the top of the sentences but digging as deep as I could to find out what lurked beneath the surface of those words, and eventually, by the end of the book, I found it.

In the final analysis (or, around page 316), I realized that Bloom was an elitist, his stance against relativism was an argument for not just a parental university, but in particular one that doesn't see the necessity of growing with society, where in fact universities should be leading societal evolution toward equity and inclusion in the first place. That tension between my reading of Bloom and what I realized that I actually believe through the act of reading him has been pretty much the defining motivation of all of my work since -- both in terms of teaching and in terms of advocacy. Allan Bloom, in fact, inadvertently convinced me to let my freak flag fly.

I really cannot sell this idea strongly enough to my students -- if you want to be a critical thinker, you have to read EVERYTHING, particularly the stuff you don't want to read. You have to understand the nuances of all of the perspectives in order to really understand your own, to sharpen and clarify it. You cannot get smart by subsisting on a regurgitated diet of your own ideas. Plus, yuck.  

My husband Gordon (middle) is largely unchanged since the 1980s, however.

After walking down memory lane with a few folks (including my husband, who graduated from Cornell shortly after The Closing of the American Mind was published and whom, I'm assuming, was Bloom's prototype for the quintessential, MTV-watching, masturbatory teenage boy) and imagining my current students reading Bloom, I realized that I should be a bit suspicious of myself. Was I really going to rely on my memory of a book I read thirty years ago? Was I really going to blog about it, based on that ancient memory? Haven't I grown a bit since I was 21?

Early this morning, I started re-reading excerpts from the book, interviews with Bloom, and even fairly recent reviews (opinion pieces on the university as a contested site, often by right-wing pundits). One thing that surprised me is that, by contemporary standards, Bloom was not all that conservative. Electorally, he was a moderate Democrat, and a bit more to the left on social issues. He wasn't a champion of the free market or the ugly grubbing he noted accompanied it.  Bloom, in fact, was not in any way a movement conservative such as the kind we seem to not be able to escape today.

Another surprise was that I kind of like Allan Bloom (it's easier because he's dead, naturally).  For instance, in one interview, when Bloom is asked, "What’s your prognosis for American education and society?" he answered: 

We are like ignorant shepherds living on a site where great civilizations once flourished. The shepherds play with the fragments that pop up to the surface, having no notion of the beautiful structures of which they were once a part. All that is necessary is a careful excavation to provide them with life-enhancing models. We need history, not to tell us what happened or to explain the past, but to make the past alive so that it can explain us and make a future possible. This is our educational crisis and opportunity.

I mean, have I not said basically that (without the weird shepherd metaphor) pretty much every day for the last couple of years? Now that I'm quite close to the age Bloom was when he wrote The Closing of the American Mind, it turns out I'm more susceptible to his worldview, softer on the idea that, as Mark Grief noted in his 2001 essay in The American Prospect, "aspiring to be better is … not entirely separable from believing in an elite of better people." Isn't that what I'm doing as I insist on pushing my students so hard to think critically that a couple of them have melted in tears? (Okay, only one really. The other one's eyes brimmed, but no tears actually came out.)

Plus, in the time since his book was published, I've learned a bit more about Bloom -- about his quietly gay life, his deep friendship with Saul Bellow, and his unnecessarily tortured death by AIDS. Reviewers at the time described him as "shrill." Maybe he was just adorably bitchy, like so many men that we are lucky to grow older with here in San Francisco.

I look across my desk and I see the text for another TEI course I'm taking this weekend – Foundations of Mixed Methods Research. Why am I taking a methods course when I completed an advanced degree in sociology that was chocked full of research methods just a decade ago? Because even from 40 to 50, I know that I have grown and learned so much. I already know it will be exciting to revisit the content of the course with my "improved" brain, to have more concrete experiences to hang those methods on. In the same way that Bloom looks different to me thirty years later (not right or wrong, just different), so too will these ideas, even ten years later. And so there's another lesson for my students -- one about context, internal and external. How the world changes, and how we do by living in it, and how fleeting knowing really is.

I can't wait to see them next week.

Laurie Jones Neighbors is an independent consultant and educator who specializes in developing, implementing, and assessing programs and educational experiences in support of equitable political representation and local, regional, and national decision making by low-income communities and communities of color.