Reforming my thinking about Education and Research

I've been doing some reading recently in preparation for this fall's classes and figured I'd write a little about what I'd been thinking. Last year I had attempted to write about the research I'd come across. Throughout the summer courses, I kept pace with some of the interesting articles and concepts but once the core classes for the MA began in the fall, I found myself with little to write about. Now, that's not to say I didn't do a lot of writing but very little was about research - or at least what I thought of as research.

In the universe of education, as I have lamented before, there isn't a lot of quantitative research which trickles down into the classroom. Both this MA and my prior MAT focused heavily on "teacher research" and on research methods drawn from sociology and anthropology. When we do find quantitative research, it's often coming from economics and political science departments or, more often, policy oriented non-profits, NGOs, and foundations. My friends and many long time educators recognize the problems in taking quantitative research from outside education as a means for driving school, district, and state policy. That's more or less how we got NCLB, Common Core, and RTTT which have not been the panacea of reforms that were initially hoped for. That's not to say all all quantitative research is bad or even that quantitative research from outside schools of education is somehow always disconnected from the reality of schooling. Indeed, I can't think of a better example than economist and data scientist Raj Chetty's recent work with the Census Bureau's Maggie Jones and Sonya Porter.

In fact, as a fan of Chetty's work since his 2015 look at income mobility, I was struck by the granularity of the conclusions. More so than any other large dataset work I am aware of, Chetty, Jones, and Porter exposed the out and out racism of our society and, especially, our school system. Of particular interest are the results for black boys growing up in the US, but what stood out to me was something Chetty said repeatedly in several interviews. I'll excerpt a large section here from an interview he gave to Talk Poverty:

We really don’t think differences in ability explain the gaps that we’re documenting, and there are two simple reasons for that. The first is the pattern that I just described of downward mobility across generations. It’s really only there for black boys. Black women do just about as well as white women once you control for their parental income. And that suggests first of all, if you look at most prior theories of differences in cognitive ability, The Bell Curve book for example, it does not present evidence that you’d expect these differences to vary by gender. Furthermore, if you look at test score data, which is the basis for most prior theories about differences in ability, the fact that black kids when they’re in school tend to score lower on standardized tests than white kids, that actually is true for both black boys and for black girls to the same extent. In contrast when you look at earnings there are dramatic gender differences.
And so that suggests that these tests are actually not really capturing in a very accurate way differences in ability as they matter for long-term outcomes, which casts doubt on that whole body of evidence. So, based on that type of reasoning, we really think this is not about differences in ability. One final piece of evidence that echoes that is if you look at kids who move to different areas, areas where we see better outcomes for black kids, you see that they do much better themselves, which again demonstrates that environment seems to be important. This is not about immutable factors like differences in ability.

Let me emphasize a particular part again: "that suggests that these tests are actually not really capturing in a very accurate way differences in ability as they matter for long-term outcomes, which casts doubt on that whole body of evidence."

What Chetty's done here is confirm with data what people in education have been saying for decades. Teachers responded to the post-2001 testing regime (followed by Common Core standardized testing) by pointing out that tests are often racially biased and often fail to adequately capture students' real-world abilities. I've written before that schools have fallen under a bastardized view of constructivism which posits that access to the content standards is all that's required for a student to succeed. So, after almost two decades of reorganizing our system of schooling behind a testing regime, we have high quality empirical evidence that the whole thing is directly harmful to black males.

Why is this "reforming" my thinking, as the post's title suggests? It's because that conclusion was something that had already been reached. I figured it out during my teacher training. I saw that view confirmed in my teacher research in 2009 and again in my teacher research this year. (It's just a draft, be kind!) My friends in education and my own experiences as a teacher indicated the system was deeply flawed, racist, and classist. Systemic racism something that ethnographic researchers have been documenting in US education since the late 1800s. Why, then, was I so dismissive of qualitative research's findings and role in the classroom, in curriculum, and in educational policy? Why wait for someone like Chetty, who appears to be somewhat unique in the field, to publish findings pulled from large data sets? Moreover, if the vast majority of data used to make educational policy is pulled from the standardized tests which are deeply flawed, what good is the data?

After reviewing a few of my old posts, I came across something I'd written:

Education, in general, dislikes quantitative research. Maybe this is a response to NCLB/RTTT and the current incarnation of the reform movement? Standardized testing is widely misused by states and districts and is a tool used by politicians to break the political power of teachers unions. This has bred distrust of any quantitative measures. In turn, any data which relies on high-stakes testing is seen as illegitimate by many educators whether the use of that data is accurate/valid or not. There are elements of social justice here because testing is seen as a proxy for race, socioeconomic status, and other factors. Any quantitative measure is potentially racist, sexist, classist, or otherwise heavily biased. 

Well, it turns out that standardized testing is racist. Data which relies on high-stakes testing is illegitimate. The critics of testing that I was so quick to doubt were right, and I should have given them more credence. Even my attempts to be even handed and consider multiple sides of issues were still filtered through my perception that data = test scores or something numerical and data != observations, interviews, or analysis of work samples, etc. What Chetty's work and my own work this past year have helped me to see is that I need to take a wider view of the qualitative side of things. I'm actually quite proud of what I wrote as part of my case studies for the Master's Project even if it was "only qualitative".

I've learned some other interesting things recently about the history of education and education research. Stay tuned for further posts because I'd like to tackle a pair of books: Kliebard's The Struggle for the American Curriculum and Lageman's An Elusive Science. What I've come to realize from reading these books is that education never forgets. Once an idea enters the educational universe, it sticks around and reemerges over and over. This is true of the supposedly scientific approach championed by the "reformers" who implemented our modern standardized testing regime.