Over the years, I have read a lot of books, articles, and blogs and listened to many podcasts about assessment and research. I find it enlightening how terminology has evolved as the student affairs/higher education assessment field has matured. Language is important and nuanced.
I have heard/used the terms data-driven, data-informed, evidence-based and more to link the use of assessment to decision making and change. I have been reflecting on that as I think about how we not only develop narratives about what we do and who our students are, but also improvement and accountability. I also think about how student affairs assessment is inherently about relationships—about the relationship of multiple data points to each other, about data in relation to the people who create, analyze, and disseminate data, and the relationship between people in how data is used to improve stakeholders’ experiences and learning. It’s not always clear-cut.
These days, I’m less likely to use the term data-driven because it implies that data tells a story that we must follow in our decisions. On the other hand, data-informed and evidence-based indicate that data is one piece of the decision-making process. It allows for people to also include their own experiences, education, and biases, as well as other environmental factors that go into decisions.
In his podcast, ReThinking, Adam Grant had an episode called “Why ‘data don’t talk’ with data scientist and comedian Andrea Jones-Rooy.” It was based on Andrea’s essay, “I’m a data scientist who is skeptical about data.” They agreed that data actually don’t talk; data itself doesn’t say anything. Andrea made the analogy that data are like maps. We look at a map, but we decide which route to take. When I look at a map, I think about the shortest route (in time or distance), road construction, highway vs. roads, time of day, weather conditions, stops along the way, etc. I then decide about which route to take.
Getting back to assessment, people are the ones who say something—they collect and interpret data and then communicate and use the information. People make decisions about what data to collect when, what statistical or qualitative analysis to perform, and what changes are made. Staff also operate in an everchanging context: budgets, leadership, priorities, and policies inform actions. Assessment data is only one piece of the puzzle. I used to hear a lot of student affairs staff say something like, “I know that the students in my program are learning/satisfied/successful; I don’t need assessment to tell me that.” They are using their experience and anecdotal evidence to know that, and I don’t doubt they have a positive impact. And, I think that’s not the whole picture. I still hear that statement on occasion, but it is also usually accompanied by “I was able to use the results from my assessment to tweak a couple of things to make it even better.” I get really excited if someone tells me that they shared an assessment report with students and together they all interpreted the results and created collective change. Now, we’re really talking!
In summary, language gives meaning, and data doesn’t/shouldn’t take away your power to use your own wisdom and other pieces of information to make the best decision possible. Don’t be afraid to use your data-informed voice, since data don’t do the talking.