I just finished teaching a Massive Online Open Class (MOOC) titled “Computational Investing, Part I” via coursera.org. 53,000 people “enrolled,” which is to say they clicked a “sign up” button. How many finished?
related post regarding lessons learned
Completion rates are low, but that statistic is misleading
Much of the criticism of MOOCs centers on supposedly low completion rates. And these rates do seem low when compared to completion rates of regular university courses. But the comparison isn’t apples to apples. Let’s dive in by considering what does it mean to start a course.
One of the 53,000 students in my class watches a lecture video.
What does it cost a student to enroll in a course?
The financial economics are significantly different for a student at a traditional university than for a student starting a MOOC where they teach you how to trade precious metals.
At a regular university all of the students starting a course have paid tuition, they have moved to an apartment or dorm near the university, and they’ve set aside time to complete the course. They have changed their lives significantly in preparation for this course. Most MOOC students are simply adding the MOOC to their existing lifestyle. They’re not moving, they’re not paying a significant fee.
Also, at most universities, students may withdraw from a course early in the semester with no penalty. This enables, perhaps encourages, them to sample a few courses before settling on the set they’ll engage in for the semester. Completion rates for university courses do not include those students who enroll in a course, then withdraw early. On the other hand, completion rates for MOOCs may include these students.
Compare the costs above to the cost of enrolling in a course at one of the MOOC providers such as Coursera, Udacity, or EdX. The cost for a MOOC is zero. All a student need do is provide an email address, and click a button labeled “sign me up.”
- Enrolling in a course at a university is expensive monetarily and intellectually.
- Enrolling in a MOOC costs nothing.
What’s the cost of failure or withdrawal?
Failing a course at a university is costly in many ways for a student. Besides the time and funds lost, there’s the cost of that “F” on the transcript. There are no such costs associated with MOOCs
- Failing a course at a university is expensive both monetarily and intellectually.
- Failing a MOOC costs nothing.
What are the implications for completion rates?
There’s not a lot of data yet, but my hypothesis is that students are significantly more likely to enroll in an interesting MOOC than in a comparable course at a university. And importantly they are also substantially more likely to withdraw. This combination of factors points to a low completion rate for MOOCs.
That’s not necessarily a bad thing. More on that later.
Here are some numbers
For the course I recently wrapped up:
- Enrolled (clicked “sign me up”): 53,205
- Watched a video: 53% of those who enrolled
- Took a quiz: 26% of those who enrolled
- Submitted first homework: 12% of those who enrolled
- Completed the course:
- 4.8% of those who enrolled
- 18% of those who took a quiz.
- 39% of those who submitted the first project.
Note that the more investment we see from the students, the higher the completion rate. Overall I suggest that the definition of “enrolled” for a MOOC should probably not include those who simply click “sign me up.” Those who took a quiz are at least a bit invested, those who completed a project even more so.
Commentary: Drawing some conclusions and suggesting future directions
First of all, we need to recognize that completion rates for MOOCs really have a different meaning than those for regular university courses. This is mainly because of the differing level of investment the students make from the start. In other words “skin in the game” matters.
But MOOC completion rates aren’t really low in the context of Internet engagement. A click through rate of 5% for a google ad is considered a strong success. Convincing 5% to engage intellectually for 8 weeks is, I think, a big deal.
If we continue to keep the barrier to entry low, we’ll enable students to taste many many courses, and that may be a good thing for education.
On the other hand if we want to boost completion rates, and perhaps boost engagement in MOOCs we should consider forcing the students to invest. A small enrollment fee might make a big difference. I’d be very interested to see how this would affect engagement, learning and completion rates.
Related Articles
An article by Amy Bruckman about the consequences of costs for courses.
the augmented trader 
Amy Bruckman
January 6, 2013
Interesting data–thanks for posting. Totally agree with your analysis. It’ll be interesting to see what happens when it’s not free. (I wrote about this a while back–http://nextbison.wordpress.com/2012/10/01/what-happens-when-free-courses-arent-free/)
Andrew Skinner
January 6, 2013
Hi Tucker,
Firstly, thanks! Your course was excellent. I was one of those who never submitted an assignment and to be honest I knew this would be the likely outcome due to my work commitments. So… one point in the ‘good thing for education’ column.
I think you’re probably on the right track drawing parallels to search engine results.
A University validates students based on location, funding, ability, prior education and a proven commitment. Online courses simply throw the net out too wide to reasonably expect the same results.
If completion rates are the aim, I’d venture that the same techniques used in other online businesses and communities would be worthwhile to increase engagement with the subscribed user base.
I’d be interested to know why completion rate is a key metric when providing these courses beyond the fact that the courses are provided by Universities – I would have thought for free education participation metrics and content consumption metrics may be more appropriate.
Thanks again!
Tucker Balch
January 7, 2013
Good point about appropriate metrics. I agree with you. I think completion rates are over emphasized for MOOCs, and that other metrics are probably more appropriate.
M Stepenaskie
January 7, 2013
I took the course and learned. If the goal is learning I succeeded thought I did not complete the course. I watched the videos and tried each exercise. It was my first effort with programming in python , if not for that struggle I would have completed the course. I enjoyed and learned. An investment would not have made a difference to me. If offered again I will enroll and likely complete. 2500+ certificates must more than 40 times the number in a live class so this course seems a real success, completion was secondary to me.
Thanks
Dimitar Margaritov
January 7, 2013
Hi mr. Tucker,
I was one of those who watched the course and I wish to thank you very much for this! I watched all the videos but did not complete the final exam nor I submitted any of the intermediate assignments. However I am deeply engaged and I made all the analysis (SR, event study etc.) for the local stock exchange in my country (Bulgaria). I am also seriously considering extending knowledge in this area (e.g. first I would read any of the books you mentioned. do you have any better proposal?). Shortly said statistic would probably classify me as attendee who only clicked “sign up” and then forget about the course but it is not the case definitely. Just balance between professional duties, kids at home ;-)) and this stuff prevented me from being more active.
For me introducing an enrollment fee would not make any difference as I guess I would behave in the same way i.e. I won’t appear as very active during the course but will make sure I take the most out of the course.
My point is that there could me many professionals who do not appear engaged in the course but are quite much in fact and this should be taken into account.
I was disappointed to learn (from your wiki) that Computational Investing II will be offered only on site at Georgia Tech. Let us know if that change in the future.
Thanks again and best regards.
Tucker Balch
January 7, 2013
Dimitar,
Thanks for the kind words. Really glad to see that you got value from the course even without submitting assignments. I was hoping folks would be able to do that.
I believe we’ll eventually offer two versions of Part II (online and in person). I’m going to produce the in person version first though.
Tucker
Benedict
January 7, 2013
Hi Mr. Balch!
I enjoyed your course a lot (and completed it as well) – thank you! Looking forward to part II.
I also don’t think MOOCs shouldn’t be compared to university courses in measures like completion rates. For me it’s a great opportunity to dive into fields I don’t get in touch during my regular studies. So I sometimes enroll courses just because of an interesting intro-video… I thinks thats even the point with MOOCs. You can have a closer look without pressure… and if you don’t like it you don’t have to put any work into it.
Best regards,
Benedict
Eric
January 8, 2013
MOOC’s blur the line between education (a huge expense and serious endeavor) and entertainment (supported by advertisers or supported by massive numbers each paying small amounts; and just for fun). Maybe MOOC’s are similar to college football?
mgozaydinMuvaffak GOZAYDIN
January 12, 2013
The reasons of high enrollment to MOOCs are
1.- Curiosity
2.- Free. Since it is free curiosity can be satisfied easily. If it was not, curiosity would stop there .
3- The name of the university . Curiosity for big name schools . Nice to know what they are .
I agree with your suggestion.
A small fee such as $ 10 is good enough .
Then schools will not deal with unnecessary population .
Serious, at least a little serious people will enroll.
Schools will make some money.
I wonder what the completion rate will be then . I say 50 % . Any guess .
plerudulier
January 12, 2013
Reblogged this on Things I grab, motley collection .
Danny
January 12, 2013
I recently graduated from a small state university. Not a week after finishing I signed up for three Coursera courses. One was an algebra course because I want to brush up on my skills, and and take higher level classes (most likely on Courera). I enrolled in a microeconomics class simply to keep learning in my degree field, and a course a course on online education because I feel it might be helpful knowledge to have for my current work project. There is no doubt in my mind that MOOCs offer an invaluable service. However, I don’t think many people will be willing to pay for courses unless a degree is offered, or colleges and universities start accepting MOOC courses as transfer hours. Of course, if higher completion rates are the goal, then yes, those who make a financial investment are more likely to finish the course.
BTW, I’m sure teaching MOOCs can’t provide much, if any, income, so thank you for sharing your knowledge with those who are willing to learn, but don’t otherwise have a means to obtain a first class education.
Tucker Balch
January 12, 2013
Thanks Danny. See my more recent post about a small charge for a verified certificate. Would you pay $30 for that?
mgozaydinMuvaffak GOZAYDIN
January 13, 2013
Tucker
Yes most people can pay $ 30 per course.
My more attractive fee is $ 10 per course.
Please do some multiplications
One online course is taken by 1,000 students for 10 quarters or semesters at $ 10
then income is 1,000 x 10 x 10 = $ 100,000 Many professor confessed that development of a course is $ 50,000 or so .
If fee is $ 30 then school collects $ 300,000
The more the merrier . Assume 10,000 per semester, income is $ 1,000,000 and
$ 3,000,000
That is the beauty of online. But ONLINE must be made by the best schools of the world in order to be worthile to take the courses + to attract many students . Then globalisation is a must . look up MITx courses 15 % Americans 85 % all foreigners .
Also do not forget the quality of online courses from elite universities will improve everyday since they are non profit and they do not need money . If they just cover the cost that is enough for them . They are after accomplishments, success.
Tucker Balch
January 13, 2013
@mgozaydin, there are some aspects of your math that aren’t quite right. There are significant costs, especially when you factor in ID verification, and grading of qualitative material. Also the statement “…are non profit and they do not need money” isn’t right. Institutions like Georgia Tech and MIT have huge budgets and employee bases (as high as $1B/year). They want to grow just like any large company. Non profit doesn’t mean no excess revenue.
mgozaydinMuvaffak GOZAYDIN
January 13, 2013
Dear Tucker
Thanks billion for your clarification and comments .
ID and assessment software are done only once may be at $ 10 million even . Those software can be used by all online providers . And it can be amortised very easily with 10 or 20 courses or say 50 courses within several years . And share of amortisation per course is nill .
Cost of the platform is also less than $ 1 per person .
When I say they do not need money I meant they do not do it for profit .
They do it for accomplishment, staying leader in the education technology and education
One can make including MIT and Harvard huge profits even if they charge only $ 10 since the cost will be less than $ 1 per course when they reach the right volume . There is nothing in the world free . But since the cost is nill MIT and Harvard does need to charge more than cost plus a small margin . Budget of MIT is $ 2.5 billion . Most of the material needed for online is also ready for 2,200 courses due to Opencourseware project .
EDX is very ambitious they talk 1 billion students . Imagine then their profit is about $ 10 billion per year .
Let me give another example.
One course 5,000 students ( it is possible globally + or if many schools adopt the same course, like Antioch Uni. State Uni of Colorado plus some more ) for 10 semesters at only $ 20 collected income is
5,000 x 10 x $ 20 = $ 1,000,000 No online course in the world costs more than $ 1 million .
ONLINE means scale .
We , engineers , know that very well . That is the beauty of online. Best teachers of the world can be accessed by millions in the world .A normal teacher accesses to only 100-200 students per year .
Kajal
January 18, 2013
A point which every body is missing is that 4.8% of number enrolled comes out about 2500. It shows that about 2500 learners completed the course. If we compare this with numbers completing through offline mode then there should not be any complain about the discrepancy.
Tucker Balch
January 18, 2013
Yes, Kajal. With this course I have, in one semester more than doubled the total number of students who completed a course with me.
mgozaydin
November 5, 2013
4.8 % is not low . Most MOOCs end up with 1-2 %
Rob
January 19, 2013
Interesting analysis. While I find this encouraging as a supporter & enthusiast of MOOCs, these data you cite–e.g. that 18% of students who complete the first quiz complete a course–aren’t that meaningful without knowing WHY those initial drop-out students dropped out, especially once we take in the broad educational picture: namely, a situation of escalating higher education costs where MOOCs may eventually take over large portions of what is currently offered by brick-and-mortar universities. That’s a different context from “Internet engagement.”
Perhaps these drop-out students were highly motivated but simply did not have the requisite knowledge, or know how to get it. If MOOCs are truly going to be a mass educational solution someday, we’re going to have to find out why folks drop out, even early on in the course–and then we’ll have to figure out a way to get completion rates beyond the range between, say, the number of people in the US with a PhD and those with a Master’s degree. Hopefully the results of the experiment with San Jose State & Udacity (http://www.nytimes.com/2013/01/15/technology/california-to-give-web-courses-a-big-trial.html?_r=0) will provide some answers.
Tucker Balch
January 19, 2013
Take a look at my related blog: https://augmentedtrader.wordpress.com/2013/01/12/mooc-teaching-lessons-learned-best-balch-practices/
mgozaydinMuvaffak GOZAYDIN
January 29, 2013
People just register with a click out of curiosity not for really learning.
Also name of the schoool attracts them .
So far I clicked 12 courses non I had attended either .
But if there is credit to be transferred or degrees at the end real students will register and they will not be thousands . Only 1,000 – 5,000 at most globally .
But even that number is good for efficiency of online.
Imagine if they charge $ 100 per course the income is
1,000 x 100 = $ 100,000 within 12 – 14 weeks
5,000 x 100 = $ 500,000 not bad .
In 10 semesters that is $ 5 million It is a money machine .
Therefore elite schools can even afford $ 10 per course .
Marketing companies are using massive numbers for their ads . That means they mislead people again . No good .
Bob C
January 28, 2013
I have signed up for many of MOOCs. And dropped most of them.
By signing up, there is usually some access to materials that give you a better sense of the course itself. If a course loks interesting then I can sign up, look at the forums if they are available, any intro material or videos that have been left up to view.
Another issue is time management. I work full time and have other demands on my time. I sign up to a course hoping that the time constraints will allow me participate once the class goes live. Because there is no penalty for dropping, being in the course makes me think a little harder about dropping it. Especially if I watch a few lectures or take a quiz. But if the time management gets out of control, I can cut my losses, glad for what I was able to learn.
If these courses were paid in advance and carried college credits, then this would change completely. The flexibility this system offers is huge. I hope it doesn’t change, even as they seek ways to monetize it.
mgozaydinMuvaffak GOZAYDIN
January 29, 2013
That is my wish and I work very hard on that.
MOOCs should provide credits and degrees as well soon . Then those high drop out will end .real learner will come to sien.
Then only 20-30 elite universities in the world can be sufficient for 250 milliopn people of the world .
mgozaydin
February 1, 2013
Yes. For the sake of the world
please do not try MONETISE online courses from elite universities
Brian
January 29, 2013
Andrew Ng (Coursera) recently made a point about funding for MOOCs in that the minute you start charging fees you start excluding people, regardless of the fee. I believe it would be far better to charge for course certificates, validation, verification and accreditation etc, at the end of the course. Therefore those who had completed (and passed) would pay so that they could use the certificate (in the labour market or other educational arena) and those who wanted just to learn would have been able to. This till may leave some excluded but steps could be found to alleviate this and would be moving in the right direction in the funding/fee dichotomy.
Bob C
January 29, 2013
Paying at the end of the course is an interesting idea. Some form of tiers to the pricing would be nice too. Or possible “upgrades”. A base level completion certificate for free with other levels, maybe including an additional project or final exam.
The free part is really attractive. It gives people an opportunity to explore without taking financial risks. It encourages curiosity and that can never be bad. Adding in accreditation and fees makes a lot of sense, but the open door at the beginning is fabulous.
mgozaydinMuvaffak GOZAYDIN
January 29, 2013
Free is very wrong. Including me ( I enrolled 12 MOOCs so far I never showed up even in the first lesson ) Then by one click you think people are registered , no they just made a click .
It is very misleading that announcing Couırsera has 150,000 enrollments . It is very bad practise and misleading. Worse, I interpret that as it means cheating on people .
But charge only $ 10 to enroll.
Charge $ 100 for a credit and degree . But no more than that .
Please do a multiplication
Only 5,000 students x 2 semesters x $ 100 = $ 1,000,000 what an income
With $ 10 fee you can have even 10,000 students x 2 x $ 10 = $ 200,000 still not bad .
It is waste of time and money to make it free and clicks of 150,000 .
But for profits like to make it as an advertisement . Be careful .
mgozaydinMuvaffak GOZAYDIN
January 29, 2013
Read this http://www.educationdive.com
Davos Switzerland World Economic FORUM, top leaders said
” long – timed university – held monopply is over.
10 elite universities can survive.
Many will bankrupt ”
So MOOCS are advancing very fast . But be careful GOOD MOOCS by non profit elite universities .
mgozaydin
February 1, 2013
Yes if Ng charges $ 10 per course no 100,000 clicks but only 4,000 registration at $ 10 . Then completion is still 2,000 .
This is my theory . I suggest Ng experiment that with one elite schools such as harvard MIT Stanford Uni of Penns Princeton . Let us see the result .
If elite university provides credits and degrees at $ 10 per course I claim registration will be 5,000 and completion will be 4,000 that is 80 % excellent number .
Let us try .
Brian
February 1, 2013
Why does free have to be so very wrong? It seems to me that we have a societal condition or belief that things are only good if we pay directly for them.
However, even on the math, in terms of funding, I don’t see any logic in charging fees as this would simply exclude many.
I recently completed a Venture Lab course that had around 12,000 enrolled. I discover now that just under 1,000 completed. Poor percentages eh! However, this is a volume business. The expectation is for under 10% completion. The important thing for Venture Lab here is the 1,000 that completed. How many face to face courses see 1,000 students completing, and at what cost?
mgozaydin
February 1, 2013
Tucker
I am 100 % with you .
I am glad you do not advertise yourselves with so many thousands clicks.
You just confes that .
But still completion of 1,000 students is really still something .
I fight for the idea of credits and degrees .
I do not understand while
many no good colleges can provide no good online courses and credits and degrees .
Be careful , whomever allows , no good colleges can award credits and degrees for the last 20 years . Now there are 7 million onliner. probably 2 million online degrees .
Now many people object online credits and degrees
FROM MOST ELITE UNIVERSITIES AT A SMALL FEE
Why? Please somebody tells me ?
Many teachers say these elite universities online courses and degrees cannot replace the traditional HE.
But now 7 million onliners already replaced .
Why do you object ? I guess the coverage of elite universities perfect online degrees will leave many teachers jobless , many schools closed . Those selfish teachers never think of the betterment of the USA and thye world . Shame . Any comment from teachers .
Even at DAVOS leaders of the world say that .
mgozaydin
February 1, 2013
What would be wrong to charge $ 10 only and real registration of 1,000
Would not that be better ? Clicks , please be aware not registration , of 100,000 and next day first lecture 2,000 . Shame . It is cheating yourself and the people too .
Bob C
February 1, 2013
You seem to be fixated against “free”.
First, nothing is actually free. Somewhere, somehow, someone pays for it.
Second, and more importantly, you discount the marketing potential of “free”. Sign up for free. Complete a course, At the end pay $10 for the certificate and for an extra exam/project pay $1000 to get college credit. If there is an up front fee, then instead of starting with 50,000 potential customers, you end up with 5,000. Get the 50,000 into the door. Then close the deal. Sales and Marketing 101.
A third “Professional” certificate could be offered for $100. Something in between the basic certificate and college credit.
mgozaydin
February 1, 2013
Bob
You are amazing .
How do you cheat yourself saying
” 50,000 clicks before course start,
then course is started with 2,000 only ”
And ended up with even 1,000 .
Does that mean you ever had 50,000 customers.
It is not a marketing potential at all . Potential means you can make use of some of that potentials. But you do not make use of anything .
You cheat yourself saying that
” Get 50,000 into the door . Then close the deal ”
Is that closing a deal while you know 50,000 clicks will be nothing next day .
It seems some MOOCs are thinking your way too . Then you can use this argument as saying people that your course has been enrolled by 50,000 students. It is a complete lie.
Unfortunately that is what they say .
Yes I have fixed idea about free things .
You say yourself nothing is free.
Then if you say people free it is a lie , it is not ethical at all, it is a marketing gimmicks to decieve people .
In Marketing courses they teach that too .
I am so sorry to see these unethical behaviour at the educational sector.
Bob we are 100 % disagree. And MOOOCs will be harmed from that too .
Bob C
February 1, 2013
Unethical? What is unethical about having people come to your site? If there was coercion once you are there, then you start down the road to perdition. But getting people to come and look around? Where is the ethical dilemma in that?
I submit that it is unethical to put up ANY obstacles to knowledge. Hiding all knowledge behind pay walls and onerous copyright laws is the worst form of “ethics”.
What needs to happen is the creation of a viable model to pay for the infrastructure and talent it take to run these type of courses. NOTHING is free. Having an easy entry point that scales up in cost depending on the value you expect is certainly on viable option.
mgozaydin
February 2, 2013
Bob
You agree with me that nothing is free.
What is unethical :
To say our courses are followed by 50,000 students and to try to attract the people as most marketers would do .
Actually clickers are 50,000 follower are only 2-4 %
In fact I say lately marketing = to convince people your product is good while you know it is not . Please think philosophically . Please think deeply .
But I agree that marketing is needed for illetirate people .
Bob C
February 2, 2013
Are you suggesting that I am a shallow thinker?
Is the capacity for deep thought correlated in any way to the extent one agrees with you?
If Coursera uses a revenue model that depends on click through rates, you might be able to make a case that dissemination of knowledge is only a veneer over their true intent and that they simply wish to derive profits by page views and placement of advertisements. But the evidence suggests otherwise. If your experience of the Coursera web site is filled with banner ads and insidious plug-ins, then perhaps your computer has been compromised.
Tucker Balch
February 2, 2013
Hey guys, I think the conversation has veered off topic.
mgozaydin
February 1, 2013
Tucker
Please keep it up.
Let us have 100 comments .
I declare to all :
There are good MOOCs and there are bad MOOCs as
There are good schools
There are bad schools .
So please do not defend MOOCs as it is . Defend good MOOCs .
My good MOOC is :
1.- Online courses are provided by the best universities of the world by the best teachers of the world .
2.- Universities are non profit. They do not need to cheat you for your money .
3.- Online courses are same as oncampus courses so that academic integrity obtained.
4.- Elite universities provide credits and degrees such as MITx Harvardx berkeley x
Not the same name in order to protect $ 50,00 tuition paying students .
5.- Price would be $10 to $ 100 per coursedepending upon enrollment
since the cost is less than $ 1 if there is enough enrollment .
6.- Be global to attract millions
7.- Elite universities make a consortium for the world such as edx .
8.- I even say
” federal Government should license such consortiums “
waswer
April 24, 2013
Excellent Course ! The topic was really interesting for me.
However, I am still stuck at Homework 1 :)
i’m taking my time to complete the course . It might be 6mths time before I can finished the course.
I need to learn Python, Pandas, Numpy first. The learning curve is very steep for a non-programmer.
Marloes
November 5, 2013
Reblogged this on books and bananas and commented:
The high dropout rates of online courses receive a lot of attention. But how big of a deal is it really? Excellent article by Tucker Balch based on data from the 2013 MOOC on Computationel Investing.
mgozaydin
November 5, 2013
It is great, people are still responding .
Lately finishers ( certificate hıolders ) are aeround 1-2 % . Still not bad .
But instead of 100-150,000 enrollment we have mostly 20-40,000 enrollment .
There is a big push for degrees. I also force for a degree .
mgozaydin
November 5, 2013
Tucker
What is your latest comments about MOOCs .
Please look up European MOOCs . They are ahead of USA
Niranjan Agnihotri
March 28, 2018
Great One Sir. It takes a great deal to enroll on self into a MOOC and get through it. I myself have completed around 20 MOOCs alongside my job. MOOCs are professors like you who take effort in creating them are a real blessing for students like us.