OKP Learning Experience Bingo 1.0

After getting a lot of really helpful feedback on the Open Knowledge Practices Learning Experience Rubric 1.1 (OKPLER 1.1), I’ve tried to transform it into a resource that incorporates the fantastic contributions from other folks and still provides a tool we might use to think about the “openness” of learning experiences. The biggest change is from the format of a rubric to more of a mapping tool, that I’ve been thinking of as a sort of “bingo card”.

TL;DR: The new bingo card for open learning experiences is quite a bit different than the rubric, both simpler and, underneath, more complex. Read on to learn more.

I got clear feedback that a rubric model itself has issues — especially in the open education context — and I’ll admit I have a conflicted relationship with rubrics anyway. On one hand, I like the way rubrics can be used to surface expected outcomes and levels of demonstration/proficiency. But on the other hand, I don’t like the way rubrics can foreclose outcomes — what if something cool happens the rubric doesn’t expect? — and can attach questionably quantified values to participation — is doing this really worth more than doing that? People who actually know how to make good rubrics probably avoid these pitfalls, but I did not with the OKPLER 1.1 and so I’ve attempted to recast it as a bingo card.

While the rubric was maybe attempting to answer a question like: “How open is this learning experience?” I’m hoping this bingo card and its future augmentations can help people answer a fundamentally different question: “What dimensions of openness does this learning experience incorporate and generate?” If the bingo card does help answer that kind of question, then I’m hoping it can be used to help people think how experiences can be opened more rather than just cataloging and judging their openness. I may have created a monster, so I’m looking forward to your feedback on how to make the bingo card better or maybe move to yet another model that could be even more helpful.

Yes, the bingo card looks complicated. Let me explain how it got this way, and also how there’s a simpler model embedded in all the words and lines.

Please comment and contribute! Highlight any text on this page to add Hypothesis annotations, add comments or webmentions below, and/or contact me. I’ll be including all those who contribute in the acknowledgements (unless they prefer to remain unacknowledged).

You can access a Google Slides version of the full bingo card that can be downloaded in various other formats, including a PDF suitable for printing.

Learning experience ingredients

I started this whole project with the idea that learning experiences include ingredients, like materials, tools, people, assessments, etc. I also started with the idea that, in general, open educational practices attempt to open up the ways in which those ingredients might be boxed in.

For example, a materials ingredient in a learning experience might be an educational resource like a textbook. If you apply open practices to the materials box containing a textbook, you might “open” the box by using an openly-licensed textbook. You might also add openness to the materials in the box by having the participants revise or create materials. You might add openness to the box by having participants share materials they’ve worked on more broadly. I’m not so interested in which of these practices is more open or better than the others, but I am interested in being able to identify which open practices are involved and imagining how others might be added.

So that’s how the bingo card ends up having all these boxes: materials, activities, skills, tools, people, roles, places, times, goals, feedback, evaluation, recognition. These are some ingredients one might find in a learning experience. There may be others too. That’s where the bingo card is great because you could just keep adding boxes for other ingredients. The simple version of the bingo card is just these boxes, where you consider the openness of a learning experience in relation to each box with the answer to a simple question: Does this experience open this box of ingredients?

Access a Google Slides version of this simplified bingo card that can be downloaded in various other formats, including a PDF suitable for printing.

Dimensions of openness

That’s the simple version, but what’s with all the little words inside each box? Beyond the basic question of whether any particular box IS being “opened” in a learning experience, there’s the question of HOW it’s being opened.

Based on the early work of the OKPLER 1.1 and the feedback it received, certain “dimensions” of openness started to crystallize and many folks suggested also including dimensions of learning design, open or not. The pie slices radiating out from each ingredient box represent these dimensions of openness/learning design. The dimensions radiate from the center — again — to avoid the idea of hierarchy. An ingredient might be expanding along one dimension and not another (eg, a skill in a learning experience might be developed but not reflected upon), but the bingo card does not seek to measure whether one dimension of openness/learning design is better or more important than another. You could also add other dimensions as new pie slices to expand beyond the eight dimensions in version 1.0.

I’d especially like to thank Apurva Ashok, Maha Bali, Rajiv Jhangiani, Remi Kalir, and Robin DeRosa for helping me think through how to include dimensions of inclusiveness and design. These dimensions are still perhaps more abstract than some contributors might want, but I’m hoping this bingo card foregrounds the centrality of thinking about who is included in educational experiences and how educational experiences are designed.

The dimensions in the boxes are abbreviated, so here’s a 1.0 draft of a key. I’ve also presented some examples, but I’m not entirely sure every dimension works in every ingredient box, so I welcome new ingredients, dimensions, and examples:

• Surface: Are ingredients in the box surfaced in the experience (which I see as a prerequisite for their opening)? For example: Are the roles people play in the experience clearly articulated?
• Share: Are ingredients in the box shared with others? (Note: I’m not building in any judgement here about how ingredients are shared (eg, with an open license) as I believe the nuances of sharing are complex, depend on context, and sharing in any way is an opening). For example:
  • Are the materials or tools used and/or created in the experience made public?
  • Are the materials or tools used and/or created in the experience openly licensed?
• Create: Do participants create/revise/remix ingredients in the box? For example:
  • Do participants change the places where the experience happens?
  • Do participants create/remix/revise materials in the experience?
  • Are participants involved in creating the processes used to evaluate their work?
• Develop: Are ingredients in the box developed by the participants? For example: Do participants modify/extend activities in the experience?
• Reflect: Do participants reflect on ingredients in the box during the experience? This is the meta-cognitive dimension: Does the experience enable participants to think about how they are participating? (Special thanks to Rajiv Jhangiani for augmenting this dimension.) For example:
  • Do participants reflect on the goals of the experience, how they engaged in them, and/or how the goals might be different/better for themselves or others?
• Design: Is the design process of the ingredients in the box itself open? This is the dimension that explores the openness of how the experience was designed. (Special thanks to Apurva Ashok and Maha Bali for helping me think through this dimension.) For example: Did the people designing the experience represent different experiences and points of view?
• Include: Do ingredients in the box engage other people? (Special thanks to Maha Bali and Remi Kalir for helping me think through this dimension.) For example:
  • Are marginalized people engaged purposefully as a part of the experience?
  • Do materials provide space for perspectives they do not already represent?
  • Does the timing of the experience enable participation by people in other timezones?
• Connect: Do ingredients in the box connect outward, increasing their audience, impact, lifespan, intertextuality, etc? (Special thanks to Rajiv Jhangiani for pointing to the work of Seraphin et al to help augment this dimension.) For example:
  • Do the materials created by participants have value for a wider audience outside the context of the experience?
  • Might activities disseminated by the participants fruitfully engage other people for some time outside the context of the experience itself?
  • Do the goals of the experience connect to participants’ other activities/goals beyond the experience?

Putting bingo into practice

I imagine people using this bingo card by holding it up to a learning experience and checking off the areas where the experience is opening. I’ve tried it myself with a few of the open learning experiences collected in the Open Pedagogy Notebook and found it to be productive. With a specific experience, I’m able to check off dimensions of openness it includes for certain ingredients, but am also able to see other ingredients and dimensions where it could explore openness further.

I hope that as this project of rubrics and bingo cards matures, we may get to a place where holding it up to learning experiences helps increase our understandings and thoughtfulness about how openness happens.

Contributors: OKP Learning Experience Bingo 1.0

• Angela Gunder
• Apurva Ashok
• Christina Riehman-Murphy
• David Wiley
• Greg McVerry
• Heather Angell
• Maha Bali
• Nate Angell
• Paul Bond
• Rajiv Jhangiani
• Remi Kalir
• Robin DeRosa
• Twyla Angell

Bibliography: Open Learning Experience Design

Resources listed below are items in the larger, public Open Knowledge Practices Zotero group bibliography that appear in the subcollection, Learning Experience Bingo.

Let me know if you have a resource to add to this bibliography or send me your email address if you would like to join the collaborative Zotero group to add and tag resources yourself.

Bibliography Contributors

• David Wiley
• Maha Bali
• Nate Angell