Reflections on CETIS’s “Future of Interoperability Standards” Meeting
Posted by Brian Kelly on 14 Jan 2010
On Tuesday I attended a “Future of Interoperability Standards” meeting which was organised by JISC CETIS. The interest in the subject area can be gauged by the popularity of the meeting with about 40 people managing to arrive at Bolton despite the problems with the snow, with attendees travelling from as far as Belgium, Norway, Spain, Greece and the US. And the participants were willing to contribute actively in helping to identify limitations with the processes for the development of interoperability standards and approaches for addressing such limitations. The active participation took place not only on the day but also in advance of the meeting, with 20 participants having submitted a position paper prior to the meeting.
In my position paper I described “An Opportunities and Risks Framework For Standards“. In the position paper, which was published on this blog, I described some of the failings of open standards to live up to their expectations – ideas which I have previously described in several peer-reviewed papers dating back to 2003:
- Ideology Or Pragmatism? Open Standards And Cultural Heritage Web Sites, B. Kelly, A. Dunning, M. Guy and L. Phipps, ichim03
- A Standards Framework For Digital Library Programmes, B. Kelly, R. Russell, P. Johnston, A. Dunning, P. Hollins and L. Phipps, ichim05
- A Contextual Framework For Standards, B. Kelly, A. Dunning, S. Rahtz, P. Hollins and L. Phipps, WWW 2006
- Addressing The Limitations Of Open Standards, B. Kelly, MW 2007
- Openness in Higher Education: Open Source, Open Standards, Open Access, B. Kelly, S. Wilson and R. Metcalfe, elPub 2008
- What Does Openness Mean To The Museum Community?, B. Kelly, M. Ellis and R. Gardler, MW 2008.
These papers were co-authored with colleagues from other JISC-funded services including AHDS, JISC TechDis, JISC CETIS and JISC OSS Watch together with Eduserv, as well as with colleagues from UKOLN.
But despite the limitations of open standards and the dangers of an uncritical belief in their benefits which experts from a number of JISC-funded and related organisations have identified there is a danger, I feel, that policy-makers are unaware of such limitations and seek to apply pressure to encourage (or perhaps even mandate) adoption of open standards far too early in their life cycle.
I was really pleased to discover that we were not alone in such views. The focus of the CETIS meeting was exploring ways in which more informal approaches to standardisation processes can address the limitations of the more formal approaches. The limitations of the traditional approaches to the development of standards in an e-learning context did not need to be addressed as many of the participants, most of whom had been involved in standardisation activities (in some case for several decades) , were well aware of the failings. Tore Hoel summarised the concerns succinctly in his position paper:
… the interoperability standards in the LET domain failed miserably. Second, the ICT developed more to the benefit of Learning, Education and Training than anybody could dream of. All of sudden, anybody (well, so we claim) can do almost anything with technology to support what they want in learning, e.g., finding information, expressing views from different perspectives, building communities, etc. Who asks any more for standards? Well, the enduser shouldn’t anyway, but then the ones that should ask for LET standards are not very enthusiastic either!
That’s right – ‘interoperability standards in the Learning, Education and Training domain have failed miserably’ (and in other domains, as I pointed out recently in the context of W3C standards). And we have seen a huge range of technological innovations which are being adopted enthusiastically by many in the user community where there hasn’t been a significant focus placed in the development of new standards. And many developers are now also engaging enthusiastically in exploiting the opportunities which are now available which don’t require support for slow-moving and possibly complex standards.
So there was broad agreement on the need for an alternative approach to the development of interoperability standards. The afternoon session explored ways in which informal approaches to the development of standards might help – and I should mention the position paper on “An agile approach to the development of Dublin Core Application Profiles” by my colleague Paul Walk which illustrates an example of an agile approach to the development of Application profiles (with an embedded video clip which illustrates the approaches which have been taken).
The discussions also addressed possible limitations of such approaches and ways in which such limitations could be addressed. The concerns I highlighted focussed on the policy-makers, including the need to ensure that policy-makers were aware of the limitations of standards-making processes, the dangers of mandating standards prematurely, the dangers that mandating procurement of IT systems based on open standards would inhibit the take-up of emerging new standards and the dangers that a view that there was a preferred hierarchy for standards making organisations would be a barrier to the take-up of standards which have been developed through more agile processes.
I’m looking forward to reading the synthesis of the discussions which staff at JISC CETIS will be publishing shortly.
UK Web Focus (Brian Kelly) 
Paul Walk said
Hi Brian,
at the meeting you raised the important point that we need to understand what the success criteria might be for an open standard. I agree with this point, and I think we should probably do more to articulate these.
So when you say “That’s right – interoperability standards in the Learning, Education and Training domain have failed miserably” could you expand a little – which standards, and according to which criteria?
Paul
Brian Kelly (UK Web Focus) said
Hi Paul – in that comment I was simply reiterating the point that Tore Hoel made in his position paper: “the interoperability standards in the LET domain failed miserably” (note I failed to include a link to his paper – I’ve now added that link).
I agree with you that there is a need to identify success criteria. As discussed at the meeting MP3 might be regarded as a success story – MP3 files can be played on all music players, unlike the more open Ogg format.
Scott Wilson said
I think it may just be the case that the most successful interop standards in education are easier to forget about – e.g. IMS Enterprise has been extremely successful and is used in pretty much all LMS and MIS products today but was never really hyped much. Also SIF has been a big success in streamlining education logistics in the US. Again pretty low on the radar for most people compared to Learning Objects, ePortfolios, etc.
erikduval said
I certainly strongly agree that policy makers sometimes have a somewhat naive view of the standards process – but then so did we when we started this?
Whether or not standards for learning technologies have been a success is more debatable though, as I try to explain in my own blog post on this meeting at http://erikduval.wordpress.com/2010/01/14/future-of-interoperability-food-for-thought/ …
In any case, it would be very useful to have more explicit criteria for the succes (and, as pointed out in the meeting, for the failure!) of open standards: I’d love to see that idea further developed!
Brian Kelly (UK Web Focus) said
Thanks for the response. I would agree with you that if policy makers do have a naive expectations of standards then the development community must accept a certain responsibility.
I will be looking at the factors which can be used to identify the success and failure of standards.
Martin Morrey said
SCORM has also been an extremely successful (i.e. popular) “standard” in the LET sector.
SCORM eclipsed its various IMS and AICC components because it was better documented, better promoted, better supported, had a testing and conformance programme, addressed the needs end-users, and (not least) backed by a major purchaser with lots of $$$!
You can argue about its pedagogical value, whether it truly is a standard etc, but it clearly did not “fail miserably”.
Brian Kelly (UK Web Focus) said
Thanks for the response. I’ve just had a read of a page on the Benefits of SCORM on the SCORM Web site. I’m pleased to see that this does have a list of the benefits of SCORM and also a summary of “When is SCORM Not a Good Fit”.
David Massart said
You can also have a look at the aspect website where Warwick Bailey from Icodeon has prepared two short videos that present the pros and cons of SCORM and IMS Common Cartridge.
Tore Hoel said
Yes, we should come up with some criteria for failure and success of standards. And “failed miserably” might be too harsh words. But if you have too high expectations it is easy to fail. SCORM might be successful standard from a marketing perspective (brand recognition). But pedagogically? When SCORM was supposed to be developed by the new LETSI organisation the reference model was position as “the de facto international standard for eLearning” (LETSI Working Meeting, Orlando, October 21, 2007). As “one size fits all” SCORM has failed miserably and has discredited learning technology standards in pedaogical communities.
Furthemore, there are reports showing that interoperability is not really working in practice, as there are a number of problems with packaging of the content, problems with defining metadata and vocabularies, and problems with runtime of the SCORM packages. (The main conclusions have been conveyed to ISO/IEC JTC1 SC36 experts, but as far as I know, the reports were never released by the pubic organisation that initiated the study.)
I agree with Scott that there are successful interoperability standards that are easy to forget because they to their job. But IMS Enterprise was one of the first IMS specifications. The focus has been elsewhere these last years, and nobody will really call LOM, ePortfolio, Learning Design for big successes. They all have different types of problems. It is perhaps less meaninfull to label these problems as failures. What we need is a good framework that lets us discuss how standards could succeed.
Structured Methods › links for 2010-01-17 said
[…] Reflections on CETIS’s “Future of Interoperability Standards” Meeting Brian Kelly wrote: […]
Techno-fil (20/01/10) « pintiniblog said
[…] Reflections on CETIS’s Future of Interoperability Standards Meeting (source: UK Web Focus, […]
iPad, Flash, HTML 5 and Standards « UK Web Focus said
[…] whilst bringing down costs by avoiding reliance of vendors of proprietary formats. In response Erik Duval pointed out that “I certainly strongly agree that policy makers sometimes have a somewhat naive view of […]
iPad, Flash, HTML 5 and Standards « ArchivePress + APrints said
[…] bringing down costs by avoiding reliance of vendors of proprietary formats. In response Erik Duval pointed out that “I certainly strongly agree that policy makers sometimes have a somewhat naive view of […]
Criteria for Sucessful and Unsuccessful Web Standards « UK Web Focus said
[…] on CETIS… on An Opportunities and Risks Fra…iPad, Flash, HTML 5 … on Reflections on CETIS’s …iPad, Flash, HTML 5 … on Why Did SMIL and […]
Weekly Research Index | January 15, 2010 « The Xplanation said
[…] Reflections on CETIS’s “Future of Interoperability Standards” Meeting « UK Web Focus — From the article: “That’s right – interoperability standards in the Learning, Education and Training domain have failed miserably (and in other domains, as I pointed out recently in the context of W3C standards). And we have seen a huge range of technological innovations which are being adopted enthusiastically by many in the user community where there hasn’t been a significant focus placed in the development of new standards. And many developers are now also engaging enthusiastically in exploiting the opportunities which are now available which don’t require support for slow-moving and possibly complex standards.” […]