Content-type: text/html ~ Stephen's Web ~ Cost and Quality Metric for Different Forms of Learning

Stephen Downes

Knowledge, Learning, Community

Feb 21, 2002

1. Why people use distance or online learning

As with any explanation of human actions, no single explanation is sufficient. People use - or do not use - distance or online learning for a variety of reasons. It's a statistical thing.

You may recall the recent survey from Britain's Open University that I posted in OLDaily in which students gave the following reasons for studying by distance:
- It was more convenient - they could study and still keep their jobs
- it was cheaper
- the university campus is a poor place to study seriously

Of these, the third was the most surprising; the first two reasons have been showing up on surveys time and time again.

An insight into why people study online as opposed by traditional distance or correspondance may be obtained by looking at what people use when they use the internet. The most frequently used applications are, in order:
- instant messaging - I forget the exact number, but billions of messages daily
- email - Again, it's something like a billion messages daily
- web browsing

This shows us that communication is a least as important for internet users as reading content. Substantially more so, if we are to take the volume of messages as compared to page views as indicative.

Now again, let me emphasize that there is no reason why a person would take an online course as opposed to a traditional course or distance learning course. But I think we can identify some reasons common to many people who take an online course (your statistical results will vary within a normal range of error):
- convenience
- cost
- communication
- content

2. The Cost of Content

It makes no sense to say that content is expensive or that content is cheap. There are many varieties of content, some of which are expensive and some of which are considerably less so.

To get at the costs of content, it is important to draw a number of distinctions.

First, we must distinguish between the cost of producing content and the cost of obtaining that same content. The cost of producing content is the total of the cost of authoring or designing the content and the cost of distributing that content. The cost of obtaining content is the total of any costs paid to the content producer of the content, the cost of retrieving the content, and the cost of using the content. There is no necessary correlation between the cost of producing the content and the cost of consuming the content.

Second, we must distinguish between what may be called static content and dynamic content. Static content is content that is produced as an artifact. Static content may include a web page, an online course, or a textbook. Dynamic content is content that is produced for an occasion. Dynamic content includes email messages, discussion board posts and telephone conversations.

Now, when people write that the cost of content is high, they are usually writing from the perspective of a publisher or educational institution. The are looking at the cost of producing static content. Because most static content is designed for a specific purpose, because it must meet the needs of many consumers at once, and because it requires a greater degree of sophistication, the cost of producing static content can be high.

But when people write that the cost of content is low, they are usually writing from the perspective of the consumers of content. The consumer may opt for either static or dynamic content. The cost of dynamic content is very low, so low that the consumer is frequently unwilling to pay costs for expensive static content unless required to in order to obtain a degree, access specific expertise, or read otherwise unavailable information.

3. Dynamic Content

There is a general and common understanding of what constitutes static content. Most of our discussions of content center around static content, around books, web pages, videos, simulations and interactive environments, images and animations, and other artifacts. But despite the importance of dynamic content in education, there is little discussion of dynamic content as content.

Most dynamic content is discussed in other contexts, using different terminology. It is 'discussion' for example or 'interaction'. In an online environment it may be described as 'customization' or 'personalization'. In more traditional educational settings it is described as 'localization' or even 'cultural sensitivity'. In a traditional university environment it often amounts to 'questions' and 'office hours'. Dynamic content, then, is what would be classified under the heading of 'communication'.

Specific examples of modes dynamic content in online learning include:
- email messages
- discussion list posts
- SMS or instant messages
- dynamically generated web pages

Specific examples of the nature of dynamic content include:
- course calendars
- comments or evaluations of submitted work
- answers to questions
- comments relative to a particular context - a reaction to a recent event, say, or
a use of a locally or culturally specific example

It is important to keep in mind that dynamic content is not necessarily dynamic content. The same content, dynamic when delivered as a personal email message, may become static when encoded as a web page. Not all dynamic content may appropriately become static content; the practice is to extract the static elements from dynamic content. In online FAQ (Frequently Asked Questions) is an example. A FAQ represents the static componment of a set of dynamically posed questions and answers.

4. The Cost of Learning

The cost of learning may be divided into three major components:
- the cost of content - for example, textbooks or online course materials
- the cost of the instructor - for example, a lecturer or online tutor
- the cost of infrastructure - for example, buildings and sidewalks, or an internet
server plus bandwidth

In a traditional setting, the cost of the content is frequently conflated with the cost of the instructor because the instructor typically produces and delivers, in a classroom, much of the content. Thus, in a traditional setting, the costs of content are restricted to the costs of static content, while the costs of dynamic content are included as part of the cost of the instructor.

In an online environment, we need to look at these central issues:
- how much of the content traditionally delivered by the instructor can be shifted
to content contained in online course materials? In other words, how much of the
content delivered by the instructor is really static content delivered dynamically?
- How much of the static content for a course nedds to be produced? In other words,
can the content be obtained for free or low cost from existing sources, or must
it be produced specifically for the course?
- how much of the remaining content - that is, how much of the content delivered by
the instructor that is genuinely dynamic content - can be delivered by persons
other than the instructor?

The first question is important because it is more expensive to deliver static content dynamically that it is to delivery it statically. If delivered dynamically, the same content must be produced over and over again, while if delivered statically, the same content needs to be produced only once. It may be less expensive to produce the content
once than to produce it anew for each course offering. That is, over the long run,
books (or online course materials) are cheaper than lectures.

The second question is important because a great deal of static content common to
many courses already exists either for free or low cost on the internet. And while the
current buzz about such content centers around the discussion of learning objects,
it is important to keep in mind that the cost of obtaining such content may be (and
usually is) significantly lower than the cost of producing the same content. That is,
over the long run, it's cheaper to buy a book than to write one.

The third question is important because dynamic content produced by instructors is
more expensive to produce than dynamic content produced by students, friends, peers
and other people in a non-instructional role. If, for example, a student's question
can be answered by another student, then the cost of that answer is zero, but if an
instructor must answer the question, then the cost of that content may be several
hundred dollars. That is, over the long run, it's cheaper to ask a friend who has read
the book than it is to buy the book.

5. The Quality of Learning

The previous section dealt with identifying and reducing the cost of learning. In each of the three stages - identified by the three questions - there are issues related to the quality of learning.

The first issue considers the question of whether static content can be as high quality as an in person lecture (or discussion, or similar dynamic form of content creation). The clear answer is: not in all cases. That's why we meet with lawyers rather than obtaining written case studies from them, say. Therefore, the first requirement is that dynamic content converted to static content must be genuinely static; in some cases, dynamic content must remain dynamic.

The second issue considers the question of the quality of third party content when compared to content created speficially for a given course. One of the reasons instructors use textbooks, rather than student essays, in their classes, for example, is that the veracity and presentation of the textbook content is likely to be of greater quality. This leads to a number of subsidiary questions: does content quaity have to be of the highest possible quality? If not, what is the lowest acceptable quality? How does one determine quality? How does one locate quality content?

The third question is the question of whether dynamic content produced by third parties, such as peers, other students, or strangers on the internet is of equal quality to that produced by a dedicated instructor, and if not, whether it is nonetheless sufficient for the purposes of an online course. A student's answer to another student's question may not be as good as what the professor would write, for example, but it may be sufficient to meet the needs of the questioner.

In each of these three cases, it seems clear that an arbitrator is required to mediate between the content and the content consumer. Usually the role of the arbitrator is fulfilled by a course instructor. The instructor determines what can be read from a text and what must be delivered in class. The instructor selects the textbook. The instructor listens to class discussion and offers prompts or corrections as necessary. This arbitration function appears to be a significant and non-eliminable part of learning.

6. Knowledge Networks and Communities of Practice

All I want to say here for now is that a community of practice can produce high quality static contents at lower cost. And example of such a material is the FAQ, alluded to above.

7. A Cost versus Quality Metric

Employing the terms and discussion from above, it is now possible to construct a metric comparing the relative benefits with respect to content and quality for the different forms of learning. For the purposes of this metric, infrastructure costs and the convenience factor are not counted. Moreover, the numerical values employed below are 'off the cuff' and ought over time to be replaced with measured values.

We consider four major forms of learning:
- traditional classroom based learning
- distance learning with some teleconferencing or telephone tutoring
- online learning
- network learning
Network learning may (yet) be defined variously. For the purposes of the metric, we will depict network learning as online learning in which:
- less reliance is placed on static content produced by publishers
- greater reliance is placed on static content produced by a community of practice

We consider two types of content:
- static content
- dynamic content

And we are measuring for the overall value of a form of learning, where the value of the learning is measured as follows: The percentage of dynamic content used in the form of learning times the value of the dynamic content plus the percentage of the static content times the value of the static content. Expressed as a formula:

%D(D) + %S(S)

To establish the quality of dynamic and static content in the different forms of learning, we first identify the various sources of the content. Then, for each source:
- we then determine how much of the course content is derived from that source
- we evaluate the cost of the content from that source, and
- we evaluate the quality of that content from that source
From these, we derive an overall value of the content from that source; the total value is the sum of the values from each source.

The value of the content from a given source is obtained by multiplying the quantity used in the course times the quality of the content divided by the cost of the content. Thus (using 'E' to represent the 'sum of' operator), the value of the content from all sources may be expressed as:


And thus the overall value of the form of learning is:

%D ( E(quant*qual/cost) + %S ( E(quant*qual/cost))

8. Cost versus Quality in a Three-Valued Metric

For the purposes of illustrating the relative values of the different forms of learning (again with infrastructure and convenience factored out) we can use a three-valued metric to provide an intuitive and (probably) roughly average approximation. The three valued metric is defined as follows:

The percentages of static versus dynamic content employed in a form of learning is calculated as either 30 percent, 50 percent, or 70 percent, so long as the percentages add up to 100 percent.

The quantity of content provided by a source is given a numerical value:
3 - most of the course content is provided from this source
2 - some of the content is provided from this source
1 - a tiny bit of content is provided from this source

The quality of the content given by a source is given a numerical value:
3 - the quality is high
2 - the quality is medium
1 = the quality is low

And the cost is given a numerical value:
3 - the cost is high
2 - the cost is medium
1 - the cost is low

I have calculated the metric and placed it online at

Stephen Downes Stephen Downes, Casselman, Canada

Copyright 2024
Last Updated: Jul 21, 2024 06:37 a.m.

Canadian Flag Creative Commons License.