What Netflix Would Look Like if an Edtech Company Built It

Here is how Netflix works:

• You do a bunch of stuff you want to do—watch movies, watch trailers, etc.

• Netflix’s computers think about the stuff you’ve been doing.

• Netflix recommends stuff you might want to do next.

Here is how Netflix would work if it was built by an edtech company:

• You would stop doing stuff you want to do and take a quiz about it.

• Netflix would display some graphs of what you did in your quizzes.

• Netflix would require you to decide what to do with that data or even how to make sense of it.

Something I don’t like admitting because I feel like I’m Someone Who is Supposed To Know Stuff is that I have no idea what I would do with most edtech data displays.

The typical approach in edtech to data display is a rare combination of overwhelming and passive. It offers you loads of data and doesn’t help you figure out what to do with it.

By contrast, Netflix’s approach is invisible and active, meaning Netflix doesn’t show you any data but it makes some active recommendations that I, personally, find pretty accurate. (Why yes, I would like to watch another paranoid political action-thriller from the late 1990s.)

I feel pretty dumb looking at graphs in edtech because, absent any recommendations from the software, I feel like it’s supposed to be obvious what I’m supposed to do with them. But it isn’t obvious to me. 

I don’t think I’m alone in feeling more confusion than clarity with edtech data displays. Heather Hill reviewed studies of how those displays get used and reported that, “Studying student data seems to not at all improve student outcomes in most of the evaluations I’ve seen.”

Why doesn’t data analysis work? All three researchers explained that while data is helpful in pinpointing students’ weaknesses, mistakes and gaps, it doesn’t tell teachers what to do about them. Most commonly, teachers review or re-teach the topic the way they did the first time or they give a student a worksheet for more practice drills.

So here is one question I’m thinking about with some collaborators at Amplify:

A teacher gives a digital assessment. How can we help that teacher make the most out of a 15-minute review of that assessment afterwards?

The 15-minute review of an assessment is already a part of the routine of many classes. We are not asking for extra time here. 

What if we threw away the pie charts, 3D scatter plots, and all the other over-determined representations of student learning and, like Netflix, made our most confident recommendation for those 15 minutes? 

“Here’s a movie you might like” could then become “Here’s a question you might ask.” Or, perhaps more specifically, “Here is the most common wrong answer from your last assessment. Ask your students to write a question where it is the right answer.”

One of the best outcomes of Desmos Classroom merging with Amplify has been the pressure to think about supporting a huge group of teachers with very different capacities instead of a smaller group of very high-capacity early adopters. That change, more than any other, accounts for my recent heel-turn away from depictions of teaching as martyrdom, and over-determined frameworks for teaching.

There are too many teachers working too hard right now and those of us who consider ourselves teacher supporters should ask ourselves at every moment, “Are we making life easier or harder for teachers? Are we taking weight off their shoulders or adding to it?”

The way Netflix uses data takes weight off my shoulders. The majority of edtech data displays only adds to it.

BTW

I put the question out on Twitter, “How do you use these things?” and got some great responses. Some people described a lack of faith in the visualizations themselves.

Heidi Echternacht ♥️ @hechternacht

@ddmeyer And a lot of times they are completely wrong!

Math ANEX is doing interesting work in this space and replied with a relevant blog post asking, “What does 2 grades below grade level really mean?”

Scott Davidson, a math coach in Clovis, CA, offered a visualization of assessment data that struck me as unambiguously helpful, a visualization that takes weight off the teacher’s shoulders, offering the teacher recommendations for what to do next depending on the problems students struggled with on a pre-assessment.

​​