Donald Clark Plan B

...this guy's a fuckin' mooc.

  But I didn't say nothin'.

And we don't pay moocs.

  A mook? I'm a mooc?

Yeah.

  What's a mooc?

  What's a mooc?

  I don't know.

  What's a mooc?

  You can't call me a mooc.

I can't?

  No!                  

PUNCH THROWN - ALL HELL BREAKS OUT….

Scorcese's Mean Streets (1973)

“The future is already here, it’s just not very evenly distributed” said William Gibson, that is certainly true of MOOCs. We have MOOC mania but ‘all MOOCs are not created equal’ and there’s lots of species of MOOC. This is good and we must learn from these experiments to move forward and not get bogged down in old traditionalist v modernist arguments. MOOCs will inform and shape what we do within and without institutions. What is important is to focus on the real needs of real learners.

Taxonomy based on pedagogy

To this end, it is important to define a taxonomy of MOOCs not from the institutional but the pedagogic perspective, by their learning functionality, not by their origins. So here's a starting list of eight:

transferMOOCs

madeMOOCs

synchMOOCs

asynchMOOCs

adaptiveMOOCs

groupMOOCs

connectivistMOOCS

miniMOOCSs

1. transferMOOCs

Transfer MOOCs literally take existing courses and decant them into a MOOC platform, on the pedagogic assumption that they are teacher-led and many rely on a ‘name’ of the institution or academic to attract learners. The pedagogic assumption is that of transfer from teacher and course content to learner. Many mimic the traditional academic course with lectures, short quizzes, set texts and assessments. You could describe them as being on the cutting edge of tradition. Coursera courses largely fall into this category.

2. madeMOOCs

Made MOOCs tend to more innovative in their use of video, avoiding talking heads in favour of Khan Academy or Udacity hand on board sequences. They also tend to have more of a formal, quality driven approach to the creation of material and more crafted and challenging assignments, problem solving and various levels of sophisticated software-driven interactive experiences. Peer work and peer-assessment, used to cope with the high teacher-student ratios. These tend to be more vocational in nature, VOOCs (Vocational Open ONine Courses), where the aim is to acquire a skill or skills. Udacity take this approach. Remember that Thrun and Norvig were not academics but corporate researchers working for Google.

3. synchMOOCs

Synchronous MOOCs have a fixed start date, tend to have fixed deadlines for assignments and assessments and a clear end date. They often around the agricultural, academic calendar. For example, Coursera offer courses on strict startand end dates with clear deadlines for assignment. Udacity started with their ‘hexamester’ 7 week courses with fixed start dates. Many argue that this helps motivation and aligns teacher availability and student cohort work.

4. asynchMOOCs

Asynchronous MOOCs have no or frequent start dates, tend to have no or looser deadlines for assignments and assessments and no final end date. The pedagogic advantages of asynchronous MOOCs is that they can literally be taken anytime, anywhere and clearly work better over different time zones. Interestingly, Udacity have relaxed their courses to enrol and proceed at user’s own pace. Some sceptics point towards this as being a tactic to reduce drop-out rates due to missed assignment deadlines. Note that Coursera offers a completely open self-study option but this does not warrant a certificate of completion.

5. adaptiveMOOCs

Adaptive MOOCs use adaptive algorithms to present personalised learning experiences, based on dynamic assessment and data gathering on the course and courses. They rely on networks of pre-requisites and take learners on different, personalised paths through the content. This has been identified by the Gates Foundation as an important new area for large scale productivity in online courses. These MOOCs tend not to deliver flat, linear structured knowledge but leaning experiences driven by back-end algorithms. Analytics are also used to change and improve the course in the future. Cogbooks is a leading example of this type of MOOC.

6. groupMOOCs

Group MOOCs start with small, collaborative groups of students. The aim is to increase student retention. Stanford, the MOOC manufacturing factory, has spun out NovoEd (formerly Venture Lab) which offers both MOOCs and closed, limited number, internal courses. They argue that some subjects and courses, such as entrepreneurship and business courses, lose a lot in looses, open MOOC structures and need a more focussed approach to groupwork. The groups are software selected by geography, ability and type. They have mentors and rate each others commitment and progress. Groups are also dissolved and reformed during the course.

7. connectivistMOOCS

Pioneered by Geperge Siemens and Stephen Downes, these connectivist MOOCs rely on the connections across a network rather than pre-defined content. Siemen’s famously  said “cMOOCs focus on knowledge creation and generation whereas xMOOCs focus on knowledge duplication”. More simply, Smith says “in an xMOOC you watch videos, in a cMOOC you make videos”. The whole point is to harvest and share knowledge that is contributed by the participants and not see the ‘course’ as a diet of fairly, fixed knowledge. These course tend to create their own trajectory, rather than follow a linear path.

8. miniMOOCSs

So far, MOOCs tend to be associated with Universities, whose courses last many weeks and often fit the semester structure and timetable of traditional institutions. We have also seem=n the emergence of shorter MOOCs for content and skills that do not require such long timescales. This is mpore typical of commercial e-learning courses, which tend to be more intense experiences that last for hours and days, not weeks. They are more suitable for precise domains and tasks with clear learning objectives. The Open Badges movement tends to be more aligned with this type of MOOC.

Conclusion

Note that these are not mutually exclusive categories, as one can have a transfer MOOC that is synchronous or asynchronous. What’s important here is that we see MOOCs as informing the debate around learning to get over the obvious problems of relevance, access and cost. This is by no means a definitive taxonomy but it’s a start. I’d really appreciate any comments, critiques or new categories. 

There’s been a lots of different reactions to MOOCs and a few fixed narratives have emerged:

1. ‘US Valley’ narrative around Khan, Stanford, not-for-profits, investors, Coursera, Udacity, NovoED and so on.

2. ‘Canadian connectivist’ narrative that MOOCs originated with Siemens & Downes and have been usurped by the ‘US valley’ folks.

3. ‘Out of OER’ narrative, where MOOCs are seen as building upon the Open Educational Resource culture.

4. ‘Traditional backlash’ narrative, that MOOCs dangerously undermine the traditional values and funding of Universities. 

5. ‘Silver bullet’ narrative where MOOCs are seen as the future saviour for higher education.

In my view, none are wholly true, yet all have a degree of truth. What we have to do is stop seeing all of these as mutually exclusive and look to the future not the past. This is a phenomenon or movement that, whatever its origins has the momentum that none of the past initiatives seemed to gather. It’s a time to drive forward with debate and discussion, not constantly checking the wing mirror.

New MOOC narrative

My own position is that we need a future-looking narrative that lies beyond all of these. Here’s a thought. MOOCs will not replace or even undermine Universities. In fact, they are likely to make our Universities even more important as the future keepers of cultural capital. No one wants to see our University system fail or crumble. Then again, many want to see aspects of the closed ‘ivory tower’ reshaped into something a little flatter, more open and accessible. There are genuine worries about insularity, quality of teaching, cost, access and relevance. If we can reposition academe as more open, transparent and relevant, that could be to the benefit of us all. There are seven components to this narrative:

1. Open

Being more open, through MOOCs, will engage and re-engage potential school leavers, parents, alumni, adult learners and the majority of people worldwide who may see it as a realistic aspiration. Just as important are those who,frankly, have no chance of ever seeing the inside of a University. The data from MOOCs already show a huge appetite from an untapped audience around the world for knowledge and learning. I suspect that academics, research and reputations of Universities would be enhanced of that knowledge were seen as more open and accessible

2. Scalability

Higher Education does face the problem of increasing costs. In most other areas of human endeavour, increased volume leads to decreased costs. Along comes a solution that promises to ease that problem. Sure the business models have yet to be refined, but they will. Sure there may be less teacher-student face-to-face contact but this is the ‘trade-off’, namely that a MOOC may have less student/professor contact but some of that may be worth sacrificing for openness and access. Sebastian Thrun was teaching 200 students at Stanford, on his MOOC it was 169,000. That would have taken 800 years at his old teaching rate. Even with the 26,000 that completed, it’s 130 years. The benefits of scale and literally ‘massive’.

3. Diversity

The philosophy Professors at San Jose, who recently wrote an ‘open letter’, complained that MOOCs undermine the ‘diversity’  of the student mix. How they came to that conclusion beggars belief. MOOCs are massively diverse in terms of age, nationality, ethnic origins and background. This is precisely is a consequence of them being Massive, Open and Online. This is an important point in learning, as critical thinking may well be enhanced by having a larger, more diverse set of globally-based, learners engaged on courses. It shifts us out of our cultural groupthink and brings in a wider range of experience, example and perspectives.

4. Academic status

Rather than the occasional academic making an appearance through a TV series on art of history, we could see a renaissance of interest in knowledge and learning if they engaged more directly and openly with society. A good example in the UK are Classical scholars, such as Mary Beard and Robin Lane-Fox, who have headed up TV series on Roman history. With MOOCs, many more talented academics will have a chance to reach out to audiences beyond their own yearly intake of students.

5. Relevance

This may also realign university subjects and activities more closely with the needs of their communities, economies and student needs. I live in a relatively small town, Brighton, with two large Universities, yet there is precious little engagement between them and the local population. The vast majority would be hard pressed to name the Vice-chancellors or even a single academic at either institution. As a local employer , who employed many students from both Universities, it worried me over many years how disinterested they were in even minor curriculum tweaks or the fate of students beyond graduation date. Engagement with the local community through the arts, debates, public lectures and reuse of low-occupancy buildings and sports facilities would make Universities more loved.

 6. Giving

Rather than the educational colonialism of setting up shop in the developing world with new-build campuses, the developed world could funnel educational aid through MOOCs. This would have greater impact through scale and lower costs. The evidence from MOCCs so far is that huge numbers of people are accessing them from countries where HE is not affordable or even remotely accessible for the majority of citizens. I’d like to see some foreign aid budgets go to MOOCs, especially further down the educational ladder into schools.

7. Reframe away from ’18 year-old undergrad

When something new, and let’s even use that word ‘disruptive’, hits a sector, debate erupts, especially on social media and blogs. This is all good as it helps us think through the many issues that emerge, some predictable, some not so predictable. But one thing has happened that surprises me in the debate is the framing of this new phenomenon (MOOCs) into the old, restrictive model of the 18 year old undergraduate course.

If you believe that the purpose of a MOOC is to mimic the standard undergraduate course, you will be disappointed as many of the participants in MOOCs are not young undergraduates. You will also see drop-out, rather than drop-in, a category mistake that sees anything other than passing the final exam as failure (a BIG mistake). There is also a false assumption that face-to-face teaching is a necessary condition for learning. It is not. We learn most of what we learn, not from direct teaching but informally from all sorts of sources and interactions. This is not to say that teaching is unimportant. In practice, on MOOCs, human contact takes all sorts of forms, from teacher to student, student to student, content to student, peer assessment, physical meetups among students, forums, social media. This is a rich blend of human interaction and, in connectivist MOOCs, it is this very feature that, their connectivist founders claim, makes them work so well. There are demands for more rigour in summative assessment, despite the fact that many learners may not want summative assessment at all and others lighter forms of assessment. MOOCs are taken for all sorts of reasons by all sorts of people from all sorts of places. For many it’s not a paper-chase. Squeezing the debate back into the ‘do I get a credit for this course – if not it’s a waste of time’ is wrong-headed.

God’s in the detail

Sure, there’s the old world that has to adjust to new ideas but we can’t hang on to old practices just because they’ve been around for a long time - we’d never have got rid of slavery! On the other hand we must be careful not to totally abandon old practice and look for readjustments, for example, the recording or inclusion of active learning within lectures. We can surely borrow from the work that’s been done on OER, connectivist MOOCs, adaptive learning and so on. MOOCs are not the preserve of one group, country or group of elite Universities.

MOOCs make everyone reflect, discuss and experiment with pedagogy in way that is far more agile than the slow and ponderous ‘research’ route. Let’s face it, HE accreditation is odd. You get a two numbers with a dot between them. What use is that? We need far more innovation on what we assess, when we assess and how we assess. MOOCs are starting to give us real answers.

So what models have emerged?

1. No certification

First up, MOOCs are NOT, fundamentally, about summative assessment. It is clear than huge numbers of learners don’t give a toss about accreditation. For them, and I’m one of them, it’s not a paper chase but a learning experience. Many will choose to learn without wanting to sit a final exam or get any form of certification. Don’t assume that everyone is gagging for a certificate from the University of ‘somewhere’ – they’re not. To be honest, as someone who spent years delivering massive learning projects to employers, few of them care a jot about certificates. We need to separate the MOOC movement from the idea of summative assessment being a necessary condition for success. Some free MOOCs offer no certification at all, seeing it as a pure learning experience. Carnegie Mellon have a whole rack of such courses on language learning, science and maths.

For many, however, certification will be desirable. This may be important for students who want to use these courses for progression, jobs, even personal motivation and satisfaction. Certification also matters as a revenue model for the platform providers and Universities. This is where they hope to make money.

2. Certificate of completion

Certification is for completion, the norm in Coursera, simply recognises that the student has stuck with the course, got through all of the formative assessments and assignments and, well, completed the course. This is fine for those who simply want some recognition at the end, without a need for official accreditation.

3. Certificate of mastery

Some edX courses from Harvard and MIT have Certificates of Mastery. They come with a grade but not an official credit. EdX offer a certificate of mastery issued at the discretion of edX and the University that offered the course. These certificates have been free but they plan to charge a modest fee in the future. In an interestingly footnote, edX hold certificates for learners from Cuba, Iran, Syria and Sudan in line with US embargoes!

4. Certificates of distinction

Different levels of accomplishments are being offered by many MOOC providers. With Udacity, this is the core model, with the following different grades; completion, distinction, high distinction, highest distinction. This is not far off the 3^rd, 2.2, 2.1dn 1^st model. Udacity also offer a "testing kit" to any institution for a low fee if they are interested in providing proctored exams on our courses.

5. University credits

On selected courses for San Jose State University (transferable within the California State University system), where credits are available, you pay $150 and this buys you the course, course support, direct comms with instructors/staff and online proctored exams with credited transcript. There are different grades; completion, distinction, high distinction, highest distinction and a service where resumes are sent to prospective employers.

How and when are these exams managed?

Proctored online

Huge efforts are being made to allow learners to sit summative exams online. It’s a complex but not insurmountable problem. Identity, cheating, security and other issues have to be addressed. Iris, fingerprint and voice recognition are just some of the digital identity methods used. Motion sensing and camera identification are also used. Progress is being made. Note that almost all exam methods are subject to cheating. Even proctored offline paper exams do suffer from distributed leaks, teacher and student cheating. One of the advantages of online testing is that questions can be drawn from randomised banks or different numbers laced into test items, and answer options randomised, to prevent the straight copying cheating that exists in physical, paper exams.

Udacity and Coursera both offer online proctored exams at home (a cost of $60–$90) through ProctorU. With ProctorU, you make an appointment, log in to the website and speak to a live proctor who talks you through the process via webcam. You can select a date, time and you are ready to go. At the appointed time, the proctor gets control of your screen and IDs you by requesting photo ID. The proctor will snap photos of you and ask you personal questions, using public databases. They will also make the student do a 360 degree scan of the room with the webcam and ask to see the monitor and its surroundings on the webcam, mirror or CD, left and right. During the exam, the proctor watches the student’s body and eye movement through the webcam.

Proctored test centres

Udacity and edX both offer proctored exams at Pearson VUE test centres. There’s lots of angst around Pearson’s involvement in proctored exams, through Pearson VUE, but why? They have invested in test centres and can deliver this stuff to large numbers of people at low cost. This is how we pass our driving test. We pay for a course to learn the theory and practice (increasingly learning the theory online), then book a test. National networks of centres allow students and adult learners to sit exams at place and time of their own convenience. This frees learners from the tyranny of time and place. 

Pearson VUE has test centres in every US state and over 4400 test centres in 160 countries. These centres have surveillance and biometric systems, in particular a digital fingerprinting system, used by the FBI, that has an almost zero rejection rate.

Innovations

This flurry of activity in MOOCs has produced summative assessment that takes us forward in our thinking:

1. Has different degrees of certification based on demand

2. Caters for different types of learner

3. Offers anytime assessment

4. Offers anywhere exams at home

5. Offers network of test centre exams

6. Sees education funded by volume certification

7. Can be cheaper

8. Pushes Innovation in online testing, like essay marking

9. Makes us see that certification is not always desirable

When people say, there’s nothing new in MOOCs, think again and look at the detail. When we do, there’s some radical changes taking place, not least in exams and certification. The main benefit is in loosening up the whole process and not regarding certification as some sort of one-off, end-of-year, binary pass or fail activity. We can expect more experimentation and innovation, and more is good.

One final note, and this is radical. Why can’t we separate accreditation and testing from the institutions that deliver the learning? It avoids the obvious conflict of interest. Why can’t we have a free Google like service for accreditation?  Wouldn’t that be great for learners?

MOOCs: taxonomy of 8 types of MOOC
MOOCs: Who’s using MOOCs? 10 different target audiences
MOOCs: a breath of fresh air, albeit the same air

MOOCs: more action in 1 year than last 1000 years

What happened here? MOOC mania seemed to come from nowhere. Faster than Facebook and here to stay, in just a year MOOCs emerged from a unique mix of entrepreneurial spirit, a few leading US Universities, supported by not-for-profits and venture capital. It’s an ecosystem that can take an idea and support it through to a sustainable business. That’s impressive.

Big Bang Khan

Whatever the obscure origin of the word or examples of previous HE online courses, MOOCs mania has its origin in one, big-bang source – Salman Khan. Khan was a necessary condition for MOOC mania. It was he who popularised the short video where the lecturer was literally taken out of the picture. Forget all of that YouTube EDU and iTunes U stuff, basically dumps of dull lectures, it was Khan who got the big numbers by doing something different. OER had also stalled with MITOpenCourseWare languishing and OpenLearn an also-ran; resource dump that simply mimicked all that was lazy and bad about internal HE courses. They were within the paradigm. To be fair, Downes and Siemens were different, and certainly deserve praise for avoiding this old-school approach, but I don’t see any real causal influence on Khan and subsequent MOOC mania. Sebastian Thrun has already paid his dues to Khan.

Why was Khan the catalyst?

This is interesting, as it’s yet another example of innovation coming from outside of the bubble. It took a hedge fund manager to shake up education because he didn’t have any HE baggage. Khan’s developed his ideas through direct contact with learners, not through research project or grants. It was in dealing with his young relatives that he suddenly thought – guess what, I can save time here by doing cool videos. Neither was he hung up on the whole ‘academic hubris’ thing. It didn’t matter that his face wasn’t on screen. He understood that learning maths was about the maths, not the face; semantic memory, not episodic. In short, he dumped the long-form lecture. This was about the learner, not the teacher. His second masterstroke was to slam them up on YouTube. He intrinsically understood scalability, first, in terms of the rapid production of short videos, secondly by making them available on an already existing free platform. Then, something crucial happened, funding from the Gates Foundation ($5.5m), Ann and John Doerr ($110k) and Google ($2m). Lastly, remember the ecosystem here – Khan is a Harvard  MIT guy – the institutions that subsequently funded edX.

Universities

One University stands out as MOOC central, that’s Stanford. Paul Hennessey is easily the most visionary leader of any Uni8versity on the planet. The man who categorically does NOT want to build any more lecture theatres (that’s counts as a radical position in HE). What Stanford does, and we in Europe should be envious, is understand how to turn students into aspirational autonomous achievers. It’s in the DNA of that organisation. Udacity owes its very existence to Stanford, in that Sebastian Thrun, inspired by Khan, set up his Stanford course online and that led to the founding of Udacity. Ng and Koller, both Stanford academics, set up Coursera. NovoED was also a Stanford product, originally Venture Lab, started by Amin Saberi and Farnaz Ronaghi. Let’s not forget Class2go,another open source product out of Stanford, that has merged with edX. Harvard and MIT have each put $30m into edX (other money coming from the liks of Jonathan Grayer and Philipe Lafont).

Not-for-profits

Another key player has been inspirational in all this - the Gates Foundation. I’ve dealt with these guys and they’re good. They do their research, identify the sweet spots and take action. It was a matter of weeks that a company I had invested in, Cogbooks, who have real adaptive learning and MOOC product, had been spotted through research, invited to the US and put in front of potential customers. They put a cool $5.5m into Khan, $4m into edX. Then we have the MacArthur Foundation, Hewlett and the National Science foundation weighing in with other supportive initiatives.

Money

The third ingredient is, of course, capital. When you have a world class institution, like Stanford, producing students and academics with ambition, it attracts capital. Capital matters, as it allows you to keep up product development, while keeping your promise on delivery. It also allows you to bring on business expertise and support. Coursera has had $16m from Kleiner Perkins Caufield & Byers and NewEnterprise Associates. Andreesen Horowitz put $15 into Udacity.

Companies

Lastly, don’t forget existing companies such as Google who put $2m into the Khan Academy and Pearson who have teamed up with both Udacity and edX to offer proctored examinations through Pearson VUE. Desire2learn, a player in the HE VLE market raised $80 million and acquired Wiggio, a group collaboration tool, and Degree Compass, a student support tool. They have entered the MOOC market, with the venerable Siemens and Downes.

Conclusion

It took a drop-out like Gates to turbo-charge the PC industry, a maverick like Jobs to take it much further, Bezos to transform book selling, Torvalds open source and subsequently OER, Jimmy Wales to give us Wikipedia and Khan, then Thrun, to give us MOOCs. As I keep saying, we’ve had more pedagogic change over the last 10 years than the last 1000 years because of these outsiders and technology. It happened because the time is right. HE is in a mess with spiralling costs, old agrarian timetables and old pedagogies. Outside pressure, in the form of entrepreneurial spirit, some leading Universities, with support from not-for-profits and that all important ingredient, capital, has given us, in a year, an alternative to something that has been around for nearly 1000 years. MOOCs are a powerful force for good. They promise to break down the barriers between higher education and the rest of the world, to the benefit of both.
MOOCs: taxonomy of 8 types of MOOC
MOOCs: Who’s using MOOCs? 10 different target audiences
MOOCs: a breath of fresh air, albeit the same air

AI fallacy 1: dystopia
Let’s not be misled by dystopian Hollywood visions of AI. Movies like Her, Ex Machina and Chappie are fiction and this is about fact. Robots have certainly had an impact in manufacturing (as did machines in agriculture when labour moved from fields to factories), where their speed, precision and ability to deliver 24/7 have led to massive increases in productivity. The cost has been the elimination of dull, monotonous and repetitive jobs. But AI is a broad and complex area of endeavour, only one of which is robotics.
AI fallacy 2: mimics the brain

Neither should we see AI as simply analogous with the human brain. This is another AI fallacy. We didn’t succeed in the airline business by aping birds, nor did we make much progress in going faster by copying the legs of a cheetah – we invented the wheel. So it is with AI. It’s about doing things well, more consistently, faster, more accurately than the human brain. Our brains have several drawbacks when it comes to some real world tasks. It likes to spend one third of its time asleep, the other third in leisure. It is also full of biases, gets tired, inattentive, has emotional swings, even suffers from mental illness.

Similarly, in the world of learning, AI is not about dystopian fantasies or aping teachers. AI is already being used by almost every learner on the planet, through that algorithmic tool Google. It is already being used in predictive analytics and already being used in adaptive learning.

5 Level taxonomy of AI in learning
To untangle some of the complexity I propose a five level taxonomy for AI in learning. My taxonomy is similar to the five level taxonomy developed for automated vehicles, where the driver is in complete and sole control of a vehicle, with only some interal algorithmic fucntions obvious on the dashboard, through assistive power steering, predictive satnav tech, therough degrees of autonomy, to full self-driving automation. At the top level, vehicles are designed to perform all safety-critical driving functions and can safely operate without any driver intervention.

Level 1  Tech

Level 2  Assistive

Level 3  Analytic

Level 4  Hybrid

Level 5  Autonomous

Level 1  Tech

You’re reading this from a network, using software, on a device, all of which rely fundamentally on algorithms. These include; Public Key Cryptograph, Error Correcting Codes, Pattern Recognition, Database use and Data Compression – to name but a few. With data compression, we when we use files, they are compressed for transmission, decompressed for use. Lossless and lossy compression and decompression magically squeeze big files into little files for transfer.

These, and many other algorithms, enable the tech to work and shape the software and online behaviours of people when they are online. These algorithms really are works of art that have been designed, tweaked and finessed in response to experiment with real hardware and users. They work because they’ve been proven to work in the real world. Of course, what’s seen as an algorithm is likely to be multiple algorithms with all sorts of fixes and tricks. These ‘tricks’ of the trade, such as checksum, prepare then commit, random surfer, hyperlink, leave it out, nearest neighbour, repetition, shorter symbol, pinpoint, same as earlier, padlock,, these make algorithms really sing. Every time you go online all files you use, audio you hear, images and videos you watch, are only possible because of an array of compression algorithms. These are so deeply embedded in the systems we use they are all but invisible. The personal computer you use is essentially a personal assistant that helps you on your learning journey. With mobile you now have a PA in your pocket. These are examples of AI and algorithms deeply embedded in the technology and tools.

Level 2  Assistive

Google was a massive pedagogic shift, giving instant access to a vast amount of human knowledge teaching and learning resources. Yet Google is still simply an algorithmic service that finds and sorts data. Every time you enter a letter into that letterbox it brings huge algorithmic power to bear on trying to find what you personally are looking for. Search Engine Indexing is like finding needles in the world’s biggest haystack. Search for something on the web and you’re ‘indexing’ billions of documents and images. Not a trivial task and it needs smart algorithms to do it at all in a tiny fraction of a second. Then there’s Pagerank, now superseded, the technology that made Google one of the biggest companies in the world. Google has moved on, or at least greatly refined, the original algorithm(s), nevertheless, the multiple algorithms that rank results when you search are very smart.

Other forms of assistive, algorithmic power in learning include; unique typing and facial recognition in online assessment. Pattern Recognition is just one species of algorithms used in learning. Learning from large data sets in translation, identifying meaning in speech recognition – pattern matching plucks out meaning from data. Mobile devices especially need to use these algorithms when you type on virtual keyboards or use handwriting software. Facial and typing recognition are now being used to authenticate learners in online assessment.
A nice example of assitive AI in learning is PhotoMaths, which uses the mobile phone camera to 'read' maths problems and not only provide the answer, but break down the steps to that answer. Algorithms are therefore increasingly used to directly assist learners in the process of learning.

Level 3  Analytic

Using algorithmic power to analyse student, course, admission or other forms of educational data, is now commonplace. Here, an institution can mine its own, and other, data to make decisions about what it should do in the future. This could be increasing levels of attainment, identifying weaknesses in courses, lowering student dropout and so on.

Beyond the institution, on MOOCs, for example EdX have identified useful pedagogic techniques, such as keeping video at 6 minutes or less, based on an analysis of aggregated data across many courses and many thousands of students. Smart algorithmic analysis an also identify weak spots in courses, such as ambiguous or too difficult questions.

Level 4  Hybrid

Technology enhanced teaching where algorithmic power is applied to the tasks of teaching and learning. Here the AI powered system works in tandem with the teacher to deliver content, monitor progress and work with the system to improve outcomes.

A good example is automated essay marking, where the system is trained using a large number of professionally marked essays. These marking behaviours are then used to mark other student essays. For more detail see Automated essay marking - kick-ass assessment.

Another example would be spaced practice tools, that often use algorithms such as SuperMemo, to determine the pattern and frequency of spaced practice events. See an example here as used by real students.

However, the most common use is in adaptive learning systems, where the software uses student and aggregated student data to guide the learner, in a personalised fashion, through a course or learning experience. This is still in the context of a human teacher, who uses the system to deliver learning but also as a tool to identify progress among large numbers of students and take appropriate action. We are educating everyone uniquely but it is still technology enhanced teaching. A good example is CogBooks. This is where we are at the moment with AI in learning. Our evidence from courses at ASU suggest that good teachers plus good adaptive learning produces optimum results.

Level 5  Autonomous

Autonomous tutoring is the application of AI to the issue of teaching without the participation, even intervention, of a teacher, lecturer or trainer. The aim is to provide scalable, personalised solutions to many thousand, if not millions of learners, at very low cost. The software needs to be able to deliver personalised content based on user data and behaviour, as well as assess. In some cases autonomy can rise to a level where the system learns how to deliver better learning experiences, on its own, to produce self-improvement, through machine learning. There are already online systems that attempt to do this, such as Duolingo, used by over one hundred million learners and other platforms are on their way to performing at this level.
At this level, one could argue that the concept of teaching collapses. There is only learning. In the same way that Google collapsed the idea of the person looking through shelves on libraries or card indexes tos earch and find information, autonomous AI will disintermediate teaching.

Other dimensions

This taxonomy looks at AI from the educators perspective and works back from the learning task. Another perspective is the different types of AI that can be applied in learning. Looking at our taxonomy again, one can identify algorithmic power that delivers, technical functionality, speed/accuracy of learning task, speed/efficacy of predictive analytics, algorithms that embody learning theory, Natural Language Processing, genetic algorithms, neural networks, machine learning and many other species of algorithm.

Within this there’s a plethora of different techniques using data mining, cluster theory, semantic analysis, probability theory and decision making that takes things down to the next level of analysis. This, in my view is too reductive. Yet this is where the real work is being done. This is very much a field where real progress is being made at a blistering pace, fuelled by massive amounts of data from the internet. But this approach to taxonomy is of little use to professional educators who want to understand and apply this technology in real learning. Contexts.

Conclusion

AI in learning is not without its problems, in terms of privacy, false positives, errors, over-learning and potential unemployment. Some of these problems can be overcome through progress in the maths and design, others lie on the regulatory, cultural and political sphere. But given the fact that productivity has stalled in education and costs still rising, this seems like a sensible way forward. If we can deliver scalable technology, assistance, analysis, learning and teaching, at a much lower costs than at present, we will be solving one of the great problems of our age. Macines helped move us on through the industrial revolution, where machines replaced manula labour. They will now move us on replacing some forms of mental work. When a famous economist standing at a huge building site asked of a government official “Why are all these people digging with shovels?” the official proudly said “It’s our jobs programme”. The economist replied “So then, why not give the workers spoons instead of shovels?” We are still, in education, using spoons to educate.

Big Data, at all sorts oflevels in learning, reveals secrets we never imagined we could discover. It reveals things to you the user, searcher, buyer and learner. It also reveals thing about you to the seller, ad vendors, tech giants and educational institutions. Big data is now big business, where megabytes mean megabucks. Given that less 2% of all information is now non-digital, it is clear where the data mining will unearth its treasure- online. As we do more online, searching, buying, selling, communicating, dating, banking, socializing and learning, we create more and more data that provides fuel for algorithms that improve with big numbers. The more you feed these algorithms the more useful they become.

Among the fascinating examples, is Google’s success with big data in their translation service, where a trillion word data-set provides the feed for translations between over a dozen languages. Amazon’s recommendation engine looks at what you bought, what you didn’t buy, how long you looked at things and what books are bought together. This big data driven engine accounts for a third of all Amazon sales. With Netflix, their recommendation engine accounts for an astonishing three quarters of all new orders. Target, the US retailer, know (creepily) when someone is pregnant without the mother-to-be telling them. This led to an irate father threatening legal action when his daughter received a mail voucher for baby clothes. He returned a few days later, sheepishly apologizing!

Why is Big Data such a big deal in learning?

Online learning, by definition, is data, it can also produce data. This is one of the great advantages of being online, that it is a two-way form of communication. For many years data has been gathered and used in online learning. De facto standards even emerged making this data interoperable, namely SCORM and now TinCan.

However, something new has happened, the awareness that the data produced by online learning is much more powerful than we ever imagined. It can be gathered and used to solve all sorts of difficult problems in learning, problems that have plagued education and training – formative assessment, drop-out, course improvement, productivity, cost reduction and so on.

Learning = Large data

So how relevant is big data to learning? We need to start with an admission, that big data in learning is really just ‘Large data’. We’re not dealing with the unimaginable amounts of relevant data that Google bring to bear when you search or translate. The datasets we’re talking about come from individual learners, courses, individual institutions and sometimes, but rarely from groups of institutions, national tests and examinations and rarer still, from international tests or large complexes of institutions.

Ten level taxonomy of data

Data can be harvested at 10 different levels:

1. Data on brain

We’ve seen the commercial launch of some primitive toys using brain sensors (see my previous post) but we’ve yet to see brain and situation really hit the world of learning. Learning is wholly about changing the brain, so one would expect, at some time, for brain research to accelerate learning through cheap, consumer brain and body based technology. S Korea is developing software and hardware that may profoundly change the way we learn. With the development of an ’emotional sensor set’ that measures EEG, EKG and, in total, 7 kinds of biosignals, along with a situational sensor set that measures temperature, acceleration, Gyro and GPS, they want to literally read our brains and bodies to accelerate learning. There are problems with this approach as it’s not yet clear that the EEG and other brain data, gathered by sensors measure much more than ‘cognitive noise’ and general increases in attention or stress, and how do we causally relate these physiological states to learning, other than the simple reduction of stress. The measures are like simple temperature gauges that go up and down. However, the promise is that a combination of these variables does the job.

2. Data on learner

This is perhaps the most fruitful type of data as it is the foundation for both learners and teachers to improve the speed and efficacy of learning. At the simplest level one can have conditional branches that take input from the learner and other data sources to branch the course and provide routes and feedback to the learner (and teacher). Beyond this rule-sets and algorithms can be used to provide much more sophisticated systems that present, screen-by-screen, the content of the learning experience. There are many ways in which adaptive learning can be executed. See this paper from Jim Thompson on Types on Adaptive Learning. In adaptive learning systems, the software acts as a sort of satnav, in that it knows who you are, what you know, what you don’t know, where you’re having difficulty and a host of data about other, useful learner-specific variables. These variables can be used by the software, learner or teacher to improve the learning journey.

3. Data on course components

One can look at specific learning experiences components in a course, such as video, use of forums, specific assessment items and so on.  Peter Kese of Viidea is an expert in the analytics from recorded lectures and his results are fascinating. Gathering data from recorded lectures improves lectures, as one can spot the points at which attention drops and where key images, points and slides raise attention and keep the learners engaged. When Andrew Ng, the founder of Coursera, looked at the data from his ‘Machine Learning’ MOOC, he noticed that around 2000 students had all given the same wrong answer – they had inverted two algebraic equations. What was wrong, of course, was the question. This is a simple example of an anomaly in a relatively small but complete data set that can be used to improve a course. The next stage is to look for weaknesses in the course in a more systematic way using algorithms designed to look specifically for repeatedly failed test items. At this level we can pinpoint learner disengagement, weak and even erroneous test items, leading to course improvement. At a more sophisticated level, in a networked learning solution where the learning experiences are presented to the learner based on algorithms, screen-by-screen, items can be promoted or demoted within the network.

4. Data on course

A course can produce data that also shows weak spots. It can also show dropout rates and perhaps indications of the cause of those dropouts. One can gather pre-course data about the nature of the learners (age, gender, ethnicity, geographical location, educational background, employment profile and existing competences). During the course time taken on tasks, note taking, when learning takes place and for how long. Physiological data such as eye tracking and signals from the brain. This pre-course or initial diagnostic data can be used to determine what is presented in the course. At a more sophisticated level, it can be used as the course progresses, much as a satnav provides continuous data when you drive. Course output data from summative assessment is also useful, however, the big data approach pushes us towards not relying solely on this as was so often the case in the past. This is important for two reasons the learner themselves, knowing what they’ve achieved, not achieved, and the tutor, teacher, trainer, who can use personal data to provide formative assessment, interventions and advice based on such data. In this sales course for a major US retailer, sales staff are given sales training in a 3D simulation which delivers sales scenarios with a wide range of customers and customer needs. Individual competences are taught, practiced and tracked, so that the actual performance of the learners is measured within the simulation. Sales in the stores where staff received the simulation training were 6% greater than the control group who did traditional training. This is a good example of fine-grained data being gathered

5. Data on groups of courses

MOOCs, in particular, have raised the stakes in data-driven design and delivery of courses. In truth, less data is gathered about learners than one would imagine by the likes of Coursera and Udacity but MOOC mania has accelerated the interest in data-driven reflection. The University of Edinburgh have produced a data-heavy report on their six 2013 Coursera MOOCs taken by over 300,000 learners. The report has good data, tries to separate out active learners from window shoppers and not short on surprises. It’s a rich resource and a follow up report is promised. This is in the true spirit of Higher Education – open, transparent and looking to innovate and improve. Rather than summarise the report, I’ve plucked out the Top Ten surprises, that point towards the future development of MOOCs. If I were looking at MOOCs, I’d pour over this data carefully. That, combined with the useful information on resources expended by the University, is an invaluable business planning tool. Lori Breslow, Director of MIT Teaching and Learning Laboratory has looked at data generated by MOOC users provide clues on how to design the future of learning using massive data from “Circuits and Electronics” (6.002x), edX’s MOOC, launched in March 2012 which includes IP addresses of 155,000 enrolled students, clickstream data on each of the 230 million interactions students had with platform, scores on homework assignments, labs, and exams, 96,000 individual posts on a discussion forum and an end-of-course survey to which over 7,000 students responded.

6. Data on institution

At this organizational level, it is vital that institutions gather data that is much more fine-grained than just assessment scores and numbers of students who leave. Many institutions, arguably most have problems with drop-outs, either across the institution or on specific courses. One way to tackle this issue is to gather data to identify deep root causes, as well as spot points at which interventions can be planned.

7. Data on groups of institutions

Perhaps we should be a bit realistic about the word ‘big’ in an educational context, as it is unlikely that many, other than a few large multinational, private companies will have the truly ‘big’ data. Skillsoft, Blackboard, Laureate and others may be able to muster massive data sets, but a typical school, college or university may not.  The MOOC providers, such as Coursera and Udacity are another group that have the ability and reach to gather significantly large amounts of data about learners.

8. Data on national

National data is gathered by Governments and organisations to diagnose problems and successes and reflect on whether policies are working. This is most often input data, such as numbers of students applying for courses and who those students are and so on. Then there’s output data, usually measured in terms of exams and certification. This misses much, in terms of actual improvement and often leads to an obsession with testing that takes attention away from the more useful data about the processes of learning and teaching.

9. Data on international

At international leve the United Nations, UNESCO and others collect data, such as PISA, PIAC and OECD data, produced to compare countries performance. It is not at all clear that this data is as reliable as its authors claim. Within countries politicians then take these statistics, exaggerate their significance, cherry-pick the comparative countries (Singapore but not Finland) and use it to design and implement policies that can, potentially do great harm. PISA, for example, has huge differences in demographics, socio-economic ranges and linguistic diversity within the tested nations. The skews in the data, include the selection of one flagship city (Shanghai) to compare against entire nations. Immigration skews include numbers of immigrants, effect of selective immigration, migration towards English speaking nations, and first-generation language issues. There’s also the issue of taking longer to read irregular languages and selectivity in the curriculum. (see Leaning Tower ofPISA: 7 skews)

10. Data on web

Google, Amazon, Wikipedia, YouTube, Facebook and others gather huge amounts of data from users of their services, This data is then used to improve the service. Indeed, I have argued that Google search, Google translate, Wikipedia Amazon and other services now play an important pedagogic role in real learning. There are lessons here for education in terms of the importance of data. One should always be looking to gather data on online learning and Google Analytics is a wonderful tool.

Conclusion

Big data is changing learning by providing a sound basis for learners, teachers, managers and policy makers to improve their systems. Too much is hidden so more and more open data is needed.  Data must be open. Data must be searchable. Data must also be governed and managed. There is also the issue of visualization. Big data is about decision making by the learner, teacher or at an organizational, national or international level and must be understood through visualization. However, data is also being used to do great harm. Big data in the hands of small minds can be dangerous (see When Big Data goes bad: 6 Epic fails).