We Need Evidence – But What If We Don’t Like The Findings?
Posted by Brian Kelly on 28 Sep 2009
The Need For Evidence
We know that technologies have the potential to provide many benefits, but this potential is not necessarily also realised. We therefore need to gather evidence in order to inform our policies – perhaps to help us recognise that what seemed to be a great idea has actually not been delivered in practice, perhaps to make us aware of a need for greater advocacy and user engagement or perhaps for refining the approaches we initially took.
Usage Statistics For Mobile Devices
Such issues came to mind following a recent discussion on the website-info-mgt JISCMail list. The discussion began by addressing the question of whether institutions should be developing iPhone applications providing, for example, resources of interest to new students.
Following a discussion as to whether we should be developing generic applications for mobile devices and whether this could fail to exploit device specific features, especially features which might be particularly valuable for students with disabilities, David Bailey (Bath Spa University) put the discussion into context by providing statistics on access to his institutional Web site from various platforms.
His statistics revealed that 80.55% of visits to the Web site in the past month came from an MS Windows platform, 17.84% from the Apple Macintosh and 0.66% from a Linux platform, The figures for mobile devices were iPhone (0.44%), iPod (0.11%) and Symbian (0.10%) with the figures for mobile devices such as the Palm, Blackberry and Android and gaming devices such as the Wii and Playstation being less than 0.1%.
In response to this sharing of evidence a number of follow-up posts provided additional statistics:
Heriot-Watt: MS Windows (93.51%), Apple Macintosh (5.05%), Linux (0.67%), iPhone (0.34%), Symbian (012%) and iPod (0.11%) (see email).
Sunderland: MS Windows (92.4%), Apple Macintosh (5.7%) and Linux (0.7%). The figures for other devices were all less than 0.1% (see email).
Imperial College: MS Windows (91.69%), Apple Macintosh (6.9%), Linux (0.87%), iPhone (0.3%), Symbian (012%). The figures for other devices were all less than 0.1% (see email).
University of Warwick: MS Windows (89.19%), Apple Macintosh (8.4%), Linux (1.85%) and iPhone (0.25%). The figures for other devices were all less than 0.1% (see email).
Before reflecting on the implications of this evidence we need to be aware of the limitations of these figures: it reflects the experiences of only four institutions; the data is not necessarily based on institutional data and may reflect usage for departmental Web servers and the data reflects usage in the summer vacation. But having acknowledged these caveats, what might the implications be if this evidence does prove to be indicative of the wider higher educational community?
Discussion
Ironically although the discussion on the website-info-mgt list began over access to institutional Web sites from mobile devices the data provides little evidence of significant usage by mobile devices. But the data does reveal patterns of desktop usage which are worthy of further consideration.
I suspect many of the Web and IT developers and support staff who have been critical of Microsoft over the years will be disappointed at the overwhelming popularity of the MS Windows platform for accessing the institutional Web sites described above. Should we now accept that MS Windows has won the battle for the desktop operating system environment? And at a time when, if the predictions are correct, we may see a reduction in staffing levels, do these figures suggest that the time and effort in testing Web sites on the Linux platform may not be justified? This isn’t to suggest that Web sites should be designed for the MS Windows platform, rather that the effort in testing and tweaking for little-used platforms may not be justified.
Of course an argument could be made that the figures suggest that there is no point in developing services for the mobile Web as the current levels of usage are very low. But the difference is that the desktop and laptop computer environment is now mature, whereas the mobile environment is new.
I think there is a debate to be had – and there is also, perhaps, the need to ask “Where did it go wrong? What happened to the diversity of operating systems? Where have the Mac users and Linux users gone?” Or perhaps they are still around, and simply aren’t visiting institutional Web sites. What do you think?
UK Web Focus (Brian Kelly) 
Phil Barker said
“Where have the Mac users and Linux users gone?” — are you sure they were ever there? Think about where people buy computers from. How many high streets, shopping centres, retails parks have you seen where the number of Windows-based PCs on sale doesn’t outnumber the number of Macs on sale by 10 to 1? See also http://arstechnica.com/apple/news/2009/02/mac-market-share.ars
Michael Webb said
Our stats are pretty similar – 88.17% Windows, 10.04% Mac. 0.68% Linux, 0.55% iPhone/iPod 0.16% Symbian. Interestingly on the browser front, Chrome is starting to make inroads, with 3.6%, so the browser wars are far from over.
Ian Cooper said
In a sector where it’s probably difficult to get anything but a Windows PC on our desk and where the vast majority of student PC rooms will be set up with Windows PCs is it at all surprising that the vast majority of accesses are from the Windows platform?
Is there any breakdown between on-campus access and off- ? Anything between corporate information (e.g. staff finance, directory and the like), marketing (to the outside world), VLE, or student support areas? My gut instinct tells me that you’re likely to see some differences if you actually focus on areas people might be using on their own devices.
On the subject of those universities that developed iApps, while they may be nice (and no-doubt generate press) I wonder if the developers are missing a trick. Does the content really need to be inside an App (limited to one manufacturer’s devices) or should there be “mobile” content that can be viewed on any limited-bandwidth, limited-screen devices? Or are we still constrained by form over function, with it being difficult to bling-up mobile content unless bundled for specific platforms?
Mike Nolan said
There’s still massive variations between HEIs. Bath Spa’s 17.84% Mac OS share shows that in some markets Apple still performs very strongly.
I don’t think I’ve ever actively tested on Linux machines – there’s not really the need to because they generally run modern browsers (or the users accept that things won’t look 100% accurate if they’re doing odd things). On the contrary, if there are cuts in staffing levels then I would be looking to the Windows platform for cost savings. Supporting IE6 takes a disproportionate amount of time.
I’ve got lots of thoughts about mobile websites but I’ll save them for our own blog!
Liam Green-Hughes said
I fear there is a bit of circular logic going on here. If an institution doesn’t support anything but Windows officially, and then finds the majority of its visitors are Windows users, is this very surprising? It’s a bit like going into a Ford dealership and being surprised nobody is buying Nissans.
A good exercise would be for education web people to compare their figures with figures in other sectors. If the proportion of non-Windows platforms is much lower in the educational field this may be indicative of an issue. This could be support, or maybe the institution’s offerings don’t appeal to the non-Windows using demographic.
Brian Kelly (UK Web Focus) said
@Liam Green-Hughes Good point. It would be interesting to see if UK Universities make greater use of MS Windows than other sectors. In which case hasn’t the advocacy work for greater use of Linux, in particular, failed?
Liam Green-Hughes said
What advocacy work are you thinking of? I think it is important to note though that an increase in the number of Linux users won’t necessarily lead to an increase in the number of Linux users visiting institutional web sites. There are many factors involved.
Nick Skelton said
I have survey data from 10 years of students at Bristol – surveys of mainly 1st year students asking them what OS they have on their computer. We currently have 10% Mac users, which is the highest it has ever been. In this context it isn’t “Where did all the Mac users go?” but “where did they all come from?”.
2009 10%
2008 10%
2007 5%
2006 3%
2005 4%
2004 3%
2003 1%
2002 4%
2001 1%
2000 4%
We shouldn’t be advocating use of any particular platform to our students. They won’t listen to us anyway – there are many more important factors driving their purchasing decisions. We must accept that there will be a diversity of platforms, and then provide services to whatever hardware they bring. It is reasonable to put more effort into the most common platfroms but we can’t neglect the others completely. Thankfully the growth in standards-based webapps makes this easier.
For mobile platfroms this is even more important as there are so many platforms. Have a look at Gartner’s Managed Diversity strategy for ideas on supporting mobile devices.
I’ve got more data at http://www.bristol.ac.uk/is/computing/survey09/ and http://www.bristol.ac.uk/is/computing/advice/homeusers/resnet/about/survey/ if anyone wants to trawl through it.
Nicole Harris said
Agreed with all of the above. A lot of institutions will not support macs on campus at all, so self-perpetuating logic.
AJ Cann said
The number of students pitching up here with Mac laptops is growing year by year (but still limited by price). They are shut out of many UoL services (such as Outlook), so they go elsewhere (e.g. GMail).
Emma said
Fully agree with previous points – including the need to separate out what is in use on campus by student vs staff (and possibly academic vs administrative staff); what is in use off campus; what’s in use via Wireless on campus / wired on campus.
A general look round a (computing) lecture theatre would suggest a higher proportion of Macbook owners than some of the stats above might suggest – but, they are computing students & they’re the ones that have brought their laptops to the lecture theatre. I don’t know what those without them have!
What Do I Need To Do To Lose 20 Pounds In 2 Weeks? said
[…] We Need Evidence – But What If We Don't Like The Findings? « UK … […]
If It’s Not “All About The Technology” Then What Else Is It Not About? « UK Web Focus said
[…] We Need Evidence – But What If We Don’t Like The Findings? […]
David Cunningham said
As others have indicated, apparent popularity of operating system does not mean that users are making a conscious decision to use any particular system. In organisations most users have no choice about their desktop platform, and outside the workplace the majority of people in my experience consider that personal computers are things that run Windows, and would give no thought to the possibility of using an alternative. I checked Analytics here at the University of Bath for the period since Saturday (when new students started arriving) and get the figures: Windows 85.5%, Mac 7.8%, SunOS 3.7%, Linux 1.5%, iPhone 1.0%. Some of our public desktops are thin clients (Sun Rays) with a front menu giving a choice of Windows, Linux, and Firefox (running on SunOS as it happens although users are not aware of this). For the same period Firefox usage is running at 27.9%, which seems quite high considering that most of our desktops have IE only.