Good honest reflection of what goes wrong with predictions, with a nod to Nate Silver. One thing that struck me in the column was this criticism of Silver's methodology: "O’Neil argues that Silver assumes that 'the only goal of a modeler is to produce an accurate model,' something that might hold true for some topics — topics in which Silver happens to have expertise, like baseball, gambling, and polling — but that doesn’t hold true for other areas he covers in his book, including medical research and financial markets."

But - from my perspective - this is just the same as saying that medical research and financial markets are unscientific, because that's how science is practised these days: we don't make one-off predictions, we make models. This isn't just metaphorically true, it's literally true. When I tinker with gRSShopper (which I am constantly doing) I'm making a model of what I think learning looks like. And you'll find models deeply embedded in medical and financial sciences. Now these models are not perfect - they are not reality, and we cannot infer from them (the evidentiary basis for science is still evidence; that was the core proposition in my Master's thesis, Models and Modality). And there is the intractable problem of selecting between models. But if you're not making models, you're not doing science. [Comment] [Direct Link] [Tags: Research, RSS]

Excellent overview of 2012, "The Yera of the MOOC". Audrey Watters writes, "It was students — two from India and one from Canada — who created what I think is the among most important MOOC innovations this year — 6.003z... MITx had no plans to offer the follow-up class to 6.002x. So Bhave took matters into his own hands, creating his own open online course with help from two other members of the 6.002 learning community – a class based on a blend of MIT OpenCourseWare and student-created materials. 'Taking matters into your own hands' (and 'taking learning into your own hands') is one of the most empowering things that the MOOCs can offer. But while they do offer the chance for anyone to sign up and learn, the ease with which you can drop in is echoed in the ease with which you can drop out."

See also Audrey Watters's other four top trends in eduation for 2012:

• Flipped classrooms
• Learning to Code
• The Maker Movement
• The Business of Ed Tech

Great well-researched seried; don't miss any of the articles.

  [Comment] [Direct Link] [Tags: Learning Communities, Traditional and Online Courses, Online Learning Communities, OpenCourseWare, Canada, Attrition]

Audrey Watters in one: "I think the question 'is there a bubble in ed-tech?' asks us to assess the wrong metrics of 'value' in response. Is the 'value' only what matters to investors and entrepreneurs in terms of financial return? What then is the value of ed-tech to schools? What value does ed-tech offer learners?" Quite so. [Comment] [Direct Link] [Tags: Schools]

I'm using this post in a talk I'm giving next week so it seems only fair to list it here. Audrey Watters lists four major obstacles to using OERs: discoverability, supplementary materials, licensing issues, and technological formats. These are problems from the perspective of designers and instructors; it seems to me in addition relevant to list difficulties faced by students as well, ranging from the presentation of OERs in a static (traditional course) format, to compatibility issues (bandwidth, format, language), to relevance, to the emphasis on consumption (as opposed to creativity). These obstacles, interestingly (and this is what I will say next week) lead toward the design of the loose network of resources constituting a MOOC, as opposed to the tight integration of resources characterizing a traditional course. [Comment] [Direct Link] [Tags: Open Educational Resources, Traditional and Online Courses, Networks]

As is so often the case, there's something at the core that is right with this post, and something more near the surface that needs to be corrected. At the core, there's this, summarized from a Gardner Campbell talk: "It’s not so much the what we learn but the how and the who with and the why we do so... it’s not so much about 'open' as an adjective to describe education; rather it's 'opening' as a verb to describe what we must do." Quite so: the value is in the sharing, not (typically) in what is shared. But then there's the surface feature: "Jam. (And that requires others -- a community, a network -- in turn.)" Well - no. You are the network. You can share with nobody and still learn. I frequently run Ed Radio live, with my best FM voice, to an audience of zero listeners. I posted my newsletter articles to an unread website for three years (1998-2001) before sharing them by email. Yes, feedback is important, and yes, feedback helps you learn. But sometimes, time distance and nature provide you with all the feedback you need. You don't have to have a community - not to learn, at least. (Image: Ecology of Yearning [visual notes] @gardnercampbell keynote #opened12 giulia.forsythe) [Comment] [Direct Link] [Tags: Networks, Online Learning, Newsletters]

I encounter this a lot in my day job. "It’s Week 4 of Ed Startup 101, and the class is moving on to tackle “The Pain Test.” That is, you might’ve identified your idea for an education startup, but does this idea really address a problem?" The idea of course is that your innovation - your startup - needs to address an area of genuine need. I get that, but the pain test isn't it. Before the iPod, people didn't feel the pain of not having an iPod. Before MOOCs, there was no burning need for a MOOC. The best innovations create demands for things people didn't realize they needed. But these invariably fail the pain test. [Comment] [Direct Link] [Tags: Apple Inc.]

I'm rapidly losing patience with people who read my stuff for free and then complain that it isn't free enough. I've been writing and posting openly since long before there was even a Creative Commons, much less commerical publishers who want to redefine 'free' and 'open' as meaning they put up a fence around it and charge access. The Creative Commons license generator says to me "This is not a Free Culture License" and as Audrey Watters says "and that’s a whole other can of worms, a judgmental one at that." As she says, "I think the important thing is to recognize a continuum of openness and restrictions -- licensing, access, source code, transparency, reusability -- and to think about the context in which 'open' is invoked." As for myself, I'll continue to define the words the way any competent speaker of the language would: 'open' means anyone can get in, and 'free' means you don't have to pay. If you want to do something different, that's fine, just don't give me the attitude. For me, for my courses, "free and open" is the way I roll. [Comment] [Direct Link] [Tags: Open Content, Books, Usability]

Coursera plans to handle 'soft' courses, such as those in the humanities, using peer assessment. Research shows peer assessments result in reliable grading. But as noted here, "peer assessment in a class of thirty is very different than peer assessment in a class of several thousand." The account posted by Laura Gibbs is especially telling. "There is going to be a whole range of feedback, from the very zealous people who give feedback longer than the essay itself, to the grammar police (yes, they are everywhere), to the ill-informed grammar police, and on down to the 'good job!' people with their two-word comments, and finally the people who commented not in English or who offered incomprehensible comments that had been translated by Google Translate." Which just tells me that free-form 'pretend you are a teacher' peer feedback isn't going to work for MOOCs. [Comment] [Direct Link] [Tags: Google, Assessment]

Audrey Watters, like other critics before her, takes MOOCs to task for their high drop-out rate. But let's keep this firmly in perspective. If people quit work, moved to a new city, invested thousands of dollars to pay tuition, bought books and then attended a class, then yes, I would be concerned about high rates of drop-outs. But if a person fills in a form, reads a few posts, and perhaps offers up a comment, then the drop-out rate isn't anything like the same sort of problem. [Comment] [Direct Link] [Tags: Tuition and Student Fees]

I think it's interesting that Edmodo has raised a total of $47.5 million in seed capital for a free tool. "It’s clear that the real value of Edmodo lies in user data. And here I’ll repeat the oft-quoted “if you aren’t paying for the product, you are the product.” An acquisition of Edmodo wouldn’t be about the software (which frankly isn’t particularly spectacular and whose functionality has been replicated by others); it wouldn’t necessarily be about the team – not in terms of engineering talent, at least. It would be about an 8 million userbase." A good object lesson for the people around here. [Comment] [Direct Link] [Tags: Schools]

The idea of a 'GitHub' for education is an idea that has some merit. It would be good to have a way to find learning materials and create one's own version of them. But I think that educators - who sometimes need instructions on how to use Twitter - and GitHub are a poor match. Even using software designed for the purpose, it is difficult to navigate the Git environment. Let's see what happens with browser-based programming environments and code libraries - if we can put the whole Git experience into the background, then I thgink we'll have a case where we can create a GitHub for educators. [Comment] [Direct Link] [Tags: Twitter, Experience]

Audrey Watters writes, "I think MTT2K, the Mystery Science Theater 3000-style video campaign satirizing Khan Academy, is one of the most interesting developments in education technology so far this year." I'm inclined to agree. They idea here is that people (presumably students) have created videos wherein they watch Hkan Academy videos and make, um, comments, simular to the style made famous by Mystery Science Theater 3000. It's not just that they're laughing at the videos, though there is that - and in so doing, they are undermining the idea that "Sal Khan is education’s 'Moses.'" But more, in my view, they are fundamentally creative acts, which is the antithesis of learning-by-Khan. [Comment] [Direct Link] [Tags: Video, Wikipedia]

As Watters writes, "Edshelf hopes to become a go-to site where teachers can recommend to one another what’s worked for them, and it’s building a directory of educational materials that have been reviewed for educators by educators." In this Edshelf addresses the age-old problem attempted by learning objects, repositories, Dewey Decimal, and Good Housekeeping: how to find the good stuff in a sea of dross. Tapping into social networks is a good idea, but it trades the overabundance of anonymous recommendations for a dearth of recommendations from people you actually know. [Comment] [Direct Link] [Tags: Networks, Learning Objects, Learning Object Repositories]

I suppose it was inevitable that as the popularity of online learning resources increased there would be more and more of an association between making money and making resources. Hence the latest trend to use Kickstarter to fund the development of things like Mathalicious, a project to create 52 math videos in 52 weeks. They're asking for $164,000 Audrey Watters is asking - reasonably - why? With only $20K raised, the campaign is far from a success. "I have to wonder," says Watters, "if it's just another indication that how we fund education (or how we fail to fund education) -- whether it's with crowdfunding or with venture capital or with property taxes or with philanthropic donations -- is so very deeply flawed." [Comment] [Direct Link] [Tags: Project Based Learning, Video, Online Learning]

I'm running this item just for the headline, which is the most fun headline I've seen this year. But there's also some indication that even the Coursera's Daphne Koller is seeing beyond content and testing: “There’s a growing amount of content out there on the Web,” says Koller, “and so the value proposition for the university is no longer simply getting their content out there. Rather, it’s fostering that personal interaction between faculty and students and students and students.” Promoting this interaction, of course, has been the object of the connectivist MOOCs, and so the Stanford model drifts ever closer to our own, as they gain experience. Here's the Chronicle coverage, which of course does not link to Coursera, the site it's talking about. [Comment] [Direct Link] [Tags: Interaction, Experience, Tests and Testing]

The word 'failure' must definitely be taken in context here, despite the breathless reports circulating through the media. The story is that "One Laptop Per Child doesn't increase test scores," and sites like Mashable and (naturally) the Economist can't wait to trumpet the failure. But as Audrey Watters notes, OLPC never set out to increase test scores (and that would have been a ridiculous ambition). "The best preparation for children," according to the OLPC website isn't test prep. It is "to develop the passion for learning and the ability to learn how to learn." [Comment] [Direct Link] [Tags: Portable Computers]

I guess it's a short jump from the idea that 'you are the product' of a social network to 'you are the product' of an LMS. And so it seems Blackboard has made the leap, suggests Audrey Watters. "Blackboard already has an Analytics platform which it launched last year. Its focus isn't on just "learning management" as the LMS has narrowly defined it in the past... Blackboard could have a big head-start on a lot of other newcomers to the learning analytics market. This head-start comes, in no small part, from the data the company already has/has access to." [Comment] [Direct Link] [Tags: Blackboard Inc., Networks, Online Learning]

So the doors are open at MITx and we can see the business model on the wall - for this pilot version certification will be free, but we all know that this won't last. "According to the course website, the class will demand approximately 10 hours a week from those enrolled. There will be video lectures and demonstrations, homework exercises and an "online interactive lab specifically designed to replicate its real-world counterpart." All of the assignments and exams will be graded by "robots," or rather artificial intelligence software." [Comment] [Direct Link] [Tags: Traditional and Online Courses, Video, Assessment]

We may invent these things in Canada, but its people at places like Stanford who really know how to draw out that investment dollar. Hence we see another pair of Stanford professors, Daphne Koller and Andrew Ng, starting a massive open online course enterprise. Their startup, Coursera, looks a lot like Sebastian Thrun's Udacity. "We see a future where world-leading educators are at the center of the education conversation," says Coursera, "and their reach is limitless, bounded only by the curiosity of those who seek their knowledge; where universities such as Stanford, Harvard, and Yale serve millions instead of thousands." See also the Chronicle's Jeffrey R. Young's interview with MITx's L. Rafael Reif and Anant Agarwal. [Comment] [Direct Link] [Tags: Traditional and Online Courses, Canada, Online Learning]

All over the edublogosphere today is the news that Sebastian Thrun is leaving Stanford to form a new open online leanring venture. "The Artificial Intelligence class was run by Know Labs in partnership with the university. Know Labs has now rebranded to Udacity, and this will be site where Thrun will offer his online CS courses, separate from the Stanford University umbrella." There's more on this from Inside Higher Ed. This article acknowledges that there were some other things done prior to the Stanford course and correctly picks out Thrun and Norvig's major addition to the form, an automated assignment grading system: "The option of submitting coursework that will be acknowledged by the professors -- rather than just reading a syllabus and watching lectures -- 'forces you to exercise.'" There's more from the Chronicle as well.

What's more interesting to me - and apparently also to George Siemens - was the impact on the instructors. Siemens writes, "Perhaps Thrun’s move shouldn’t be surprising. I’ve interacted with many learners in the open courses we’ve done, and I frequently hear the experience described as 'transformative' or 'life changing'. When the education system is synchronized with the interests and passions of learners, the process is invigorating and tremendously motivating. However, when learners and educators have to fight the existing education system in order to learn and teach, it’s time for dramatic change. Thrun has recognized that tomorrow’s education system will be a function of large-scale teaching and personalized, social, participative learning. Even then, it’s still surprising to hear him state that 'I can’t teach at Stanford again.'" [Comment] [Direct Link] [Tags: Connectivism, Personalization, Experience, Online Learning]

I'm of two minds. I'm actually quite supportive of 'pay what you want' pricing plans. On the other hand, I can't imagine supporting myself through such a mechanism. [Comment] [Direct Link] [Tags: none]

Audrey Watters has wrapped up her 'top 10 tech trends of 2011' series. I'm not going to do very much 'year in retrospect' coverage this year (almost none, actually) but I will do this one, at least with links.
- the iPad
- Social media - adoption & crackdown
- Text messaging
- Data (which still means mostly 'standardized testing')
- The digital library
- Khan Academy
- STEM education's Sputnik moment
- The higher education bubble
- Open
- The business of ed tech
Just for the record, I think Audrey Watters is one of the best things to come along in education technology in 2011, her style of journalism sharp and analytic, yet still a fresh change from the constant wingeing and negativity found on so many other sites. [Comment] [Direct Link] [Tags: Tests and Testing]

I'm pretty sympathic with what Audrey Watters says here, especially the bit about a blog being a labour of love, and the bit about hating advertisements. "It isn’t simply that ads are ugly and that they destroy the readability of the written word. I hate the way it’s reduced much of online writing to a race for the most page-views." I remember the day Google+ launched (yes, I was one of the first-day users) and it was war in there as the commercial blogs tried to round up followers. Ugly. Now there's also this bit about being paid. I don't need the money - like Watters, my pay comes from other gigs. But the respect sometimes still feels elusive. "It’s not simply a matter of financial sustainability for academic bloggers — although I suppose you could interpret it as such. Rather it’s about how academia still fails to recognize a lot of the online work that scholars and students create — such are the demands of tenure and the expectations of a well-rounded CV– and how that in turn points to a broader 'crisis in digital sustainability' — people opting to set aside their blogging for other projects." [Comment] [Direct Link] [Tags: Project Based Learning, Marketing, Web Logs, Google, Blogger, Academia]

Audrey Watters folows up Maciej Ceglowski’s post The Social Graph Is Neither, which I covered here a few days ago, by looking at the education graph. It is, as she notes, "partially a response to a recent Forbes article that likens the “social graph” to crude oil — as in “Drill, baby, drill” when it comes to the perceived value of the personal data." Watters argues "should we ask the same set of questions about the education or student graph? Is it a graph? Does it capture education? Does it represent 'the student' and her or his connection to learning?" These are good questions - of course, a student's Facebook activity would not represent their learning, and we would never expect this to be the case. But one wonders whether there isn't some other representation that could do so. [Comment] [Direct Link] [Tags: Books, Online Learning, Privacy Issues]

If you can get past Audrey Watters's potty-mouth there's a good point buried in this post. It's this: "those who love Codecademy and see it as the great new way to learn how to code already know how to code." This is a deficiency not limited to online learning - I remember complaining about logic textbooks that would skip steps - they were written by people who see these steps as obvious, when to those who don't know the subject well, they just seem mysterious. Watters also points to the core debate surrounding badges (and, for that matter, credentials): "the people who want to learn to code want to learn to code and the reward should be that knowledge, not some virtual item." [Comment] [Direct Link] [Tags: Online Learning]

I saw this is a couple of places. I am a but hesitant to comment, because it's not like I write lock-down secure software myself. But then again, I don't sell my stuff for hundreds of thousands of dollars. "The Australian university that requested the initial security investigation into Blackboard says that it has been trying to get the LMS to address the issues for months, but with no response. Blackboard refutes those claims." Watters should maybe say 'rebuts', rather than 'refutes', because the LMS company's response is by no means a QED. [Comment] [Direct Link] [Tags: Blackboard Inc., Australia, Security Issues]

I've signed up for something that might be interesting, this course or "experience". G+ thread here. Steve Hargadon writes, "in Mightybell for using the web for personal and professional development." It was Mightybell that caught my eye. Created by Ning co-founder Gina Bianchini, the idea of MightyBell is that you can join courses (or, as the site names them, "experiences", in which there are activities. Mightybell displays visually how many experiences you have joined and completed, and how many activities you have undertaken. As Audrey Watters writes, "As you participate in an experience on Mightybell and move through the various steps, you can view personal analytics as well as see the progress of others — 'Fellow Travelers' who are working toward the same goal." It's an interesting experiment - but it feels very linear, and I wanted to skip steps almost immediately. And there are way too many prods to connect with my social networking buddies. [Comment] [Direct Link] [Tags: Google, Networks, Experience]

"Thanks, Aaron Swartz," writes Audrey Watters as JSTOR announces that it will open access to the academic journals in its library that are public domain (Aaron Swartz was arrested for attempting to download these public domain materials - coverage here and here). The Chronicle also covers the story, citing a JSTOR representative as saying this had nothing to do with Swartz (yeah, right, they were just accidentally enclosing public domain content all along and just happened to realize that maybe the content belonged to everyone, not just them). "We are taking this step as part of our continuous effort to provide the widest possible access to the content on JSTOR while ensuring the long-term preservation of this important material," Ms. Brown wrote. "We considered whether to delay or accelerate this action, largely out of concern that people might draw incorrect conclusions about our motivations." [Comment] [Direct Link] [Tags: Academic Journals, Open Access, Academia, Academic Publications]

With the mass purge of fake (or allegedly fake) user identities on Google+ over the weekend the question of anonymity and identity has come to the fore once again. I pretty much summarized my own view here, with an addendum here, and if you really want, a remark yesterday in here. Anil Dash came out with a profanity-filled post blaming site owners for the abuses of their comment sections. Flickr founder Caterina Fake, meanwhile, came out with a defense of anonymity. "AKA or “Also Known As” is a common use case. It’s like a stage name or a nom de plume. Say your Nom de Web is Kryptyk Physh. It’s not your 'real name', but you’ve staked your claim to it." Some writers, like the Guardian's Tim Adams, suggest that everybody on the net should use their real names. In this article, Audrey Watters comes out in favour of anonymity in education and notes "At the time, I was a graduate student, and it was safer for me to not reveal my identity. In an infamous op-ed that appeared in The Chronicle of Higher Education, 'Ivan Tribble' made it clear that 'bloggers need not apply' for academic jobs." So why is Google so interested in identity? Dave Winer explains, "Simply put, a real name is worth more than a fake one... It means it's possible to cross-relate your account with your buying behavior with their partners, who might be banks, retailers, supermarkets, hospitals, airlines. To connect with your use of cell phones that might be running their mobile operating system. To provide identity in a commerce-ready way." Which is why Skud is not acceptable to Google. Which - in my mind - is wrong. [Comment] [Direct Link] [Tags: Flickr, Operating Systems, Google, Academia]

This is a good summary of some of the discussion around the Khan Academy after Wired's post on the learning initiative last week. Though most known for the videos, there's three major parts: the videos, learning games (that give 'badges' for success), and a dashboard for teachers and parents. This summary blurs some of the sharp divide between Khan and his critics. As the Wired article has it, Stager responds that "The videos and software modules... are just a high tech version of that most hoary of teaching techniques—lecturing and drilling. Schools have become 'joyless test-prep factories.'" Well maybe he says that. But as Watters says, Wired author Clive Thompson "waters down Constructivism (or constructionism, as the article says), the learning theory supported by these two Khan-objectors, to the 'idea that students won’t really understand math unless they discover each principle on their own.'" Which is a caricature of the position. I think there's a lot that Khan could do better. But I also really like the bare-bones low-tech approach that can function as a structure we can link to other more MOOC-like more constructivist or constructionist-like learning activities. [Comment] [Direct Link] [Tags: Schools, Video, Constructivism, Constructionism, Online Learning]