#H807 Case studies in elearning innovation (1): formative e-assessment

This case study description is part of Activity 2 – Week 2 in the H807 course at the OU.

The first case study describes the introduction of formative e-assessment in an first-year undergraduate micro-economics course at the University of Derby.  It’s a course attracting 200 to 300 students with a wide range of academic backgrounds.

The main objective of the e-assessment was to increase retention rates by improving students’ skills for self-regulated learning.  The e-assessment was organized as a complement to face-to-face lectures and tutorials.  The formative assessment included a weekly range of exercises such as simulations, drag and pull diagrams, recognition exercises, calculation activities and concept identification activities.  Extensive feedback loops and links to course and other materials were integrated into the exercises. Student access was monitored and students failing to login by week 3 were contacted and invited for remedial clinics.

Results were largely positive.  There were increases in pass rates and grades achieved and the drop out rate declined.  Students liked the materials,  practising course concepts in between lectures. They appreciated the feedback loops and indicated the material increased their confidence to engage during the tutorials.  Also, students treated the activity as a social activity, sharing their progress on social networks.  For the institution the materials enabled identifying weaker students at an early stage of the course, so they could be given additional support.  Moreover, the materials triggered other elearning uptakes as lecturers learned from the experiment.

There were some disadvantages though. Due to memory constraints materials could only be accessed within the institution (would be addressed for later runs).  More seriously, making and regularly updating the materials proved very time-intensive. Remedial clinics were dropped in the second run.  For courses with fewer students the development cost may be too high. The type of subject, which requires regular practice and development of knowledge and skills in a short time frame, is likely more suitable to this type of formative  e-assessment than others.

The case study shows nicely how e-assessment is aligned with the face-to-face component of the course, making new things possible.  Feedback was more personalized. The on-line activities made it possible to use lecture time and tutorials more efficiently.  Links to course materials were embedded in the e-assessment.  Better monitoring allowed reducing the drop out rate.  

The case study illustrates some guiding principles for successful formative assessment such as immediate feedback and links to course materials.  Also, limitations and drawbacks are discussed


#H807 Elearning Concepts: What’s New?

This week the clarification of terms continues with an exercise locating concepts related to elearning on a grid.  

Elearning concepts on a 2-dimensional grid
  1. What pattern was formed when you placed these concepts on the grid? Were they evenly spaced or in a cluster?
I could place most concepts on the axis ‘Existing – New’.  It made me think about whether these concepts represented something really new you can do with technology, or whether they described an existing concept that is affected by technology.  It’s still somewhat arbitrary however, as personalisation, virtual communities and flexibility arguably are not new concepts and might be relevant in a technology-poor learning environment as well.  I placed most concept half-way on the axis between formal and informal learning, as I found that distinction quite problematic to make for most concepts.  As argued below, the distinction between formal and informal learning is blurring with learners develop Personal Learning Networks (PLNs) in a formal learning environment and informal learning initiatives are looking for ways of more formally recognizing learning.
As an alternative placing the concepts on a timeline could be interesting, as most concepts drift in and out of favour.  Google’s nGram viewer for GoogleBooks shows how collaborative learning and personalisation and peer assessment are already mentioned before the widespread adoption of the internet.  Other terms such as blended learning, elearning and just-in-time learning seem more connected to the rise of the Internet.  It’s a pity that the timeline only runs until 2008, as there appears some indication that the term ‘elearning’ is slightly shifting out of favour.
Occurrence of elearning concepts with Google’s Ngram viewer

  1. 2.  
    To what extent should we think of elearning as a distinct discipline with a need for its own concepts and vocabulary?
I don’t think we should consider elearning as a distinct discipline.  Theories on how people learn draw on ideas from psychology, neuroscience, sociology, economy etc.  Technology has an impact on all these elements, changing the nature of information, altering business models, impacting modes of delivery, arguably affecting our brains etc., but basically the research domain is still the same, optimizing the way people learn. The profound impact technology has on learning makes it certainly worthwhile to study, but I don’t see why it should form a distinct discipline.  I prefer the term “technology-enhanced’ learning to elearning, because it doesn’t seem to imply a separate discipline.
  1. 3.    Do you think the formal–informal divide is more or less evident in elearning than in more traditional forms of learning?
I think elearning blurs the distinction between formal and informal learning, mainly by offering more opportunities for informal learning. In the OU courses there is the formal part with the course guide, the forum activities etc. But there is also an important informal element with reading and commenting on others’ blog posts, writing on my own blog, discussing course concepts with friends and engaging with my Twitter network.  Not all these elements are limited to technology, but the internet has certainly made it easier to create an online network of people with similar interests and objectives.
The divide is related to what makes learning formal. There are the accreditation, the guidance, the selection and making available of course materials.  However, for each of these aspects of formal education there are initiatives challenging the traditional institutions, such as various MOOCs, Stanford’s AI course, MITx, CodeYear and Livemocha.
All these initiatives offer free access to information, guidance and support and (sometimes) some way of accreditation, as a letter of notification (in the Stanford course), free or non-free badges (CodeYear, MITx) or course credits (Livemocha).  It seems that, after content and guidance, accreditation is the next (and last?) monopoly of traditional educational institutions that is increasingly coming under attack.
  1. 4.    Did you find any of the concepts difficult to place on the grid provided? If so, why was this?
Some concepts are difficult to locate because they are not clearly defined. For example, I find ‘mobile learning’ a fuzzy and rather meaningless term, in particular with the growth of tablets, netbooks, e-readers etc. If a read an article in the train on an e-reader or mobile phone, I’m engaging in mobile learning.  If I print the same article, print it out and read it on the same train, It’s not mobile learning. That doesn’t seem to make a lot of sense.

#H807 Innovations in learning

In a course on “innovations” in e-learning it seems sensible to spend some thoughts on what we understand as innovation in education.  Two (rather old) articles were included to help us frame our ideas on innovation.  I jotted down some thoughts on what I understand as innovation, then read the articles to complement my ideas, reflected on my own context and finally read the comments on the tutor group forum.

Source: miketidd.com

1. My Initial Thoughts

An innovation in learning means doing something differently as before, not necessarily better.  This can be related to technology, like the introduction of mobile phones or tablet computers in the learning process.  It can relate to the content of learning such as updating the curriculum with recent developments in biotechnology or physics, or shifting focus to various literacy skills. Innovation can lie in the pedagogy, such as student-centred approaches or “flipping the classroom” using clickers.  Furthermore, innovation can lie in the organisational structures, like challenges to the traditional classroom, school and university models of education delivery (MOOCs) or changes in copyright models (OERs).

Second, innovation is context-related.  Introducing a blog or wiki would be highly innovative in Cambodian classrooms, but much less so in western countries.  Introducing (sometimes ancient) elements from foreign cultures may be innovations in educational systems, like countries are analysing the education systems of Finland, Singapore and South Korea, high-flyers in the OECD’s PISA tests.  I believe that innovation can be both incremental and disruptive.  

Innovations can be useful or not useful.  Often it’s difficult to know.  It takes some time to measure their effects, the technology might not yet be mature and some effects only come into play when many people adopt the innovation (like social media).

As the 5 most important recent innovations in learning, I would list:

1. Widespread availability of internet (increasing bandwidths, lower costs, handheld devices)
2. OERs
3. Social Media
4. New ways of education delivery (MOOCs, OERs…)
5. Learning Analytics

2. The Articles
The  first article from Rich and Holtham (2005) recounts the introduction of a range of technologies in the course of a decade in a MBA Programme in the UK.

First, they highlight that every innovation meets resistance.  Not everyone is an early-adopter and, recalling the “digital native” literature from H800, we should certainly not assume that all students are early technology adopters.

Second, they focus on the importance of creating a climate of innovation.  Elements of such a climate include the presence of sufficient awareness (I would call it ‘information’, or ‘connections’) of new development or alternatives, the willingness (with the leadership) to make fundamental changes and a decentralised organisation.  I would add allowing innovations to fail.

The second article, from Luck and Laurence in Innovate (2005) looks at the introduction of video-conferencing for delivering guest lectures in a tourism undergraduate course.  They highlight that the innovation was complementary to the course, allowing to do things that were not possible or more expensive before.  Second, they point to the importance of content (the ability to invite higher-quality guest speakers) and pedagogy (the need for interaction during video-lectures).

3. In my own context

Innovation is an important topic in development as we usually try to introduce new practices and often deal with resistance to change current behaviour .  Some important elements that I would list are:

Select carefully your target group. You need preferably people who are open to change, will benefit from change and are in a position to influence others.

Allow for failure.  Include a pilot phase in the programme design where various things can be tried out, rigorously evaluated and adapted, scaled up or discarded.

Focus not only on personal factors but also on the environment.  For example, when we want to change a teacher’s behaviour we tend to focus on activities (workshops, follow-up for teachers).  However, it’s equally important to design activities for school directors, government officials or parents as they affect the environment in which teachers will work.

4. Forum discussion

One element of discussion is whether an innovation should be useful.  I think not as argued before in this post.  Second, there are some thoughts what makes students accept some innovations and reject others.  Some factors I can quickly think of are:
– the innovation should benefit the student.
– the innovation should leave some flexibility to the student.
– the innovation should be introduced with support and information.
– the reason for the innovation should be clear to the student.