Connecting Data to the Stories they Represent

Connecting Data to the Stories they Represent

On these eves of upcoming quarterlies, most of us are doing some profound thinking about L4 and L5 and exactly what our data sources tell us about the impact we are having on student learning.  For most of us, our approach lies in the “data rush”, which like a gold, technology or housing “rush” has put us into a sort of frenzy of grabbing and amassing large amounts of data in hopes that these data “place holders” reveal a story that confirms that the decisions we have made were and are the right choices.  Our data kind of serves as the oracle telling the prophecy of student learning.

But how data is analyzed is just or more important that what the data says at all.  And unlike housing, gold and technology, our analysis of data for a growing demographic of students has real, and generational consequences for students if our prophecies are not getting us deeper to addressing actual root causes.  So what are some things to keep in mind?

“... experts say…..[access to achievement data]....have mostly resulted in small pockets of innovation or incremental shifts to existing practices, rather than systemic transformation.

One big reason: Big chunks of the data currently in use are either stored on paper or in teachers' heads. And much of the digital information in use is generated via students' …[scheduled exams]..., which even those in the ed-tech world acknowledge can capture only a limited slice of what constitutes real learning.

Other barriers exist, too. Even when new technologies have been introduced into classrooms, teachers have been slow to change the ways they teach. Districts have struggled for years to integrate data housed in separate silos. The education sector is embroiled in debates over how student information should be appropriately collected, shared, and used. Cite”

After watching a few TED talks on the subject of BIG data analysis, I came across an article that offered a few simple suggestions in terms of positioning our thinking around data analytics.  

1. First, what I think I am learning from this is that the most important thing about data is that its potential to tell the story of our organization is risky if we are not ahead of it by asking the the right, rather focused, questions.  One article said very explicitly that “The decision is in the question” Cite.  If we start with the question first, there is a higher likelihood that we will have a sharper focus in our data mining and analysis and this may lead us to using our data with greater value.  The question will also help us to mine that data that is going to be truly meaningful to us, meaning it will confirm that the choices we make are impacting the learners we serve (this helps us affirm our decisions are the right ones).
2. Second, using data that is simply available, may not tell the stories of the experiences of the learners we serve and we run the risk of the data determining the questions, leading us to narratives and root causes that may not be capable of, or won't, tell the whole story.  A local example of this scenario might be a school with a healthy State Report Card, students are growing-so everything we are doing is good to great and we celebrate.  Deep within the data, there may be other experiences we are missing.  Again, if the question leads the inquiry, the data has the opportunity to reveal.  Using available data may lead us down a path that does not confirm causality but only correlation….BIG data on medical trials must respect this truth about using “available data” for correlation rather than causation.  Here is an example, drowning deaths increase when ice-cream consumption increases-not causal, it is summer after all.
3. Third, our data is mostly limited to or positioned around the "IQ type" rather than the "EQ type" (emotional intelligence).  Most of what we are learning about learning is that is it a highly relational, voluntary process and that a student’s wellbeing, extent to which basic needs are met, their ability to persevere, the extent to which they believe their teacher believes in them, and self regulation etc. are the actual characteristics that determine long term success we call agency, yet our data sources fall short in measuring these realities for students.  Could our subgroup ACT scores reveal, rather than academic achievement, perhaps emotional thriving and other indicators of wellbeing?  Check out an interesting article on EQ

Well, if you have gotten this far, my purpose of writing this blog is to provide some opportunity for reflection on some of these points as we move into a season of telling our stories with data and positioning the work of teams to build capacities in our schools to do this work the best we can-happy questioning! (DIY Hacks for Data Story Telling)