#ELESIG London - Evaluation By Numbers - 7 July RVC

On Thursday 7th July, the ELESIG London local group met for the third time, again somewhat-organised by Mira Vogel and me. This time we were at the Royal Veterinary College in Camden.

As with the previous meetings, the group was small but perfectly formed. With no shortage of institutions and people in the London region, there is potential to grow the size of the meetings but I was delighted with the mix of people and the group dynamic, and the fact that it had more of a 'working group' feeling, with everyone participating and no one just being an 'audience'. So this format seems to be working for us for now.

Mira's write up of the event plus links to various resources is here: ELESIG London 3rd Meeting – Evaluation By Numbers. I just wanted to add a personal thank you to our presenters Michele Milner (UEL) and Andreas Konstantinidis (KCL), our hosts the RVC (especially Sarah Sherman for meeting and greeting), Mira, and everyone who attended and contributed. Looking forward to the next one.

The EMA Nightmare at #M25LTG

Last Wednesday the M25 Learning Technology Group met at UCL for an afternoon of presentations and interactive sessions on the theme of 'Dreams and Nightmares'. While some presentations were really quite dreamy, Sarah Sherman and I decided we could present some of our ongoing work on assessment and feedback in Bloomsbury, specifically considering issues with electronic management of assessment. The slides below give a flavour of our presentation.

Storified tweets of the event below, courtesy of Vicki Dale.

ELESIG London: Come Evaluate With Me! 11 November 2015, UCL

The second meeting of ELESIG London took place at UCL yesterday. Mira Vogel and I facilitated an interactive morning considering the nature of evaluation, why do it, and how it can be done. Our aim was for participants to move forward with their own evaluation plans, as well as hear about some interesting work, so plenty of time for discussion and networking was built into the schedule. The room was (as hoped) a hive of activity as participants discussed their own evaluation plans and gave each other suggestions. We also welcomed some special guest stars, with a couple of great presentations from Tunde Varga-Atkins from the University of Liverpool (unfortunately only via Skype, rather than in person, but it worked well), who discussed the use of nominal focus group technique, and Professor Amanda Jefferies from the University of Hertfordshire, explaining how to get the best results from learner video diaries. It was also great to have London-escapee Vicki Dale down from Glasgow for the meeting.

All in all I think it was an informative and enjoyable meeting, and has taken us a step further towards a local (but extended), networked, community of practice in learner experience research. Many thanks again to Tunde and Amanda for presenting and everyone who attended for participating.

Links for presentation slides:

• Mira Vogel & Leo Havemann - Come Evaluate with Me

• Tunde Varga-Atkins: Nominal Focus Group Technique

• Amanda Jefferies: Video Diaries

A Storify of the tweets from the event should display below.

Farewell to the things I never wrote - ambition is good, but realistic goals are better (some thoughts for #acwrimo)

It's already been a week since I realised that November is #AcWriMo (academic writing month) and read some excellent pieces of reflection, advice and encouragement such as the writing is never done – a post for #acwrimo from Pat Thomson and Why I changed my mind about #acwrimo from Inger Mewburn, and lots of other great stuff on Twitter via the hashtag. 

And so, I thought (as I often do), I should write something about this. And then I didn't, because to be honest, I was writing too many other things. In fact what I have been discovering lately is that having finished one quite major writing (and editing) project (an independently published, open access book I've edited with Javiera Atenas), I now want to encourage people to discover and read it, and that means writing more about it. And that made me reflect on another idea for a piece of reflective writing I never wrote, which was about how I have really, really intended to get a lot better about posting on this blog more frequently (are you noticing a theme developing here?).

The problem is not a lack of things to write about. I have ideas, sometimes even paragraphs. But that's where I get stuck; when I have got somewhere that is already too long for a tweet, but not long enough to be an adequate blog post, or so I feel anyway. Maybe I should be more relaxed about these short posts and think of them as more akin to a workshop, rather than a lecture, where the purpose is to generate or engage in conversation, rather than deliver an argument. And I should also give myself permission to say farewell to the things I never wrote, and enjoy completing things a bit more. 

What I really want to do though, and this #acwrimo phenomenon has definitely made me think about this, is set realistic, achievable goals, rather than just have an ever increasing list of stuff I need to get done 'now'. Yes I realise I was supposed to start the month by doing that. But it's harder than it sounds. I am going to get on to that straight away. Well, after the next thing I need to get written and sent off urgently.

Turnitin User Group meeting, London, October 2015

I recenty attended the 2015 Turnitin User Group meeting, with the same idea as many in the audience, it seems, of finding out how things are moving along in relation to some long-promised developments, and some further information regarding the mysterious forthcoming iteration of the platform, codenamed Turnitin Next.  

The ‘backchannel’ of the conference on Twitter tended to reveal the frustration of the user community with the slow progress Turnitin are making on many of the more UK specific requests, such as non-numeric grading scales and multiple marking layers that would support combinations such as double blind marking. 

The UK HE community is putting a lot of effort and faith into the Jisc EMA project, and we are really hoping Turnitin as our key supplier in this space is watching closely and engaging. Worryingly this wasn't mentioned during the many presentations from Turnitin staff. We were advised that they have been extensively reorganised in both staffing and in their move from ‘waterfall’ to ‘agile scrum’ development, which should help them to clear their ‘technical debt’ (it was a bit like being at a party conference). As they remain the only company with a product that covers submission, originality and marking, they continue to be the default option for the sector. But I must say that the sense that markets such as the UK are not seen as high priority was not dispelled at this event.

There is a good capture of main points discussed at the meeting in this blog post from Dewi Parry Cardiff Uni TEL team.

Tweets from the event Storified below.

Today's Reading: Online Teaching Manifesto / Digital Capability / Open Access Journals / Transforming Assessment and Feedback

For those of you out there who have blogs AND actually regularly write on them, I salute you. Today I thought I would try a new strategy. Rather than think, interesting, I ought to write something about that too, and never getting around to it as usual, I am just going to point you to some things I think are worth reading.

*
First up on my way to work this morning I read a thought provoking post from Jenny Mackness, reflecting on Uni of Edinburgh's update to their Manifesto for Teaching Online:

Great synthesis of Edinburgh Uni’s updated Manifesto for Teaching Online with relevant research https://t.co/swCPdGU9Do via @jennymackness

— Mary Loftus (@marloft) October 22, 2015

Next up Sheila McNeill (whose rants I always value) on digital capability:

A few thoughts aka mini rant from me on institutional digital capability https://t.co/RZcHpr0XZG

— Sheila MacNeill (@sheilmcn) October 22, 2015

Sheila's mention of the notion that "technology alone will somehow wave some magical digital fairy dust" particularly struck a chord with me.

*
As it's Open Access Week there is a lot of discussion taking place around what we want from open access, and how maybe transforming the business model of commercial academic publishers to APC-based rather than subscription-based (but actually how about we pay both for now), is rather an impoverished (and impoverishing) goal. See for example, Opening Up Open Access: Moving beyond business models and towards cooperative, scholar-organized, open networks. Javiera Atenas has also posted regarding this, arguing that we should be selective about where we publish, and identify and praise honest and reliable journals in our respective disciplines:

Honest and reliable Open Access Journals in Open and Distance Education https://t.co/SvfD6wvmXw pic.twitter.com/LqvuhH9ira

— Javiera Atenas (@jatenas) October 21, 2015

And Robert Farrow has followed this up with a post asking what other journals we could add to the list:

Honest and reliable Open Access Journals in Open and Distance Education https://t.co/SOBcaQpOCq pic.twitter.com/TP6XZGby9F

— Dr. Robert Farrow (@philosopher1978) October 22, 2015

Replies welcomed by @jatenas and @philosopher1978.

*
I am cheating by including this, because I haven't read that much of it yet, but some useful work-in-progress on Transforming Assessment and Feedback has emerged from the Jisc EMA Project that is definitely worth a look for anyone who is interested in assessment practices:

New Jisc guide 'Transforming assessment and feedback' available http://t.co/SdfWWfDIuj, feedback welcome #jiscexperts15 #jiscassess

— Lisa Gray (@ljanegray) October 14, 2015

They are looking for feedback on the guide which you can provide via this form.

*
Anyway, I'd better get back to today's other reading, emails...

Step up to Postgraduate Study in Arts, v 3.0

The third and latest iteration of the summer blended module Step up to Postgraduate Study in Arts is drawing to a close (today is the second deadline or as we like to say, 'assessment opportunity', for students to submit the assessments, so there will still be marking to do). This year we signed up over 70 students, so the module continues to grow year on year, though of course as it is a free and non-credit bearing module, some people sign up but don't really get engaged with it. And on one level that's fine, because nothing about this is compulsory, but on another, it's a shame, because the feedback we are getting from the students who do work their way through the module, whether they complete everything, or just have a go at certain topics, is that they are gaining insights into academic skills and gaining confidence to begin their new chapter as postgrad students. 

It would have been great to have had a module like this available when I was starting my MA (back in the mid 90s!). But then, reflecting on that makes me think how much higher education has changed in the intervening decades - for example, as part of Step Up we encourage students to evaluate and develop their information and digital literacies, and to interrogate the concept of plagiarism by deliberately plagiarizing, as well as writing an unplagiarized version, and submitting both versions to Turnitin. And of course, the module is studied almost completely online (though supported by two engaging face-to-face events). For these reasons the module has been recognized by Jisc as an exemplar in preparing and supporting students to study successfully with digital technologies (see: http://digitalstudent.jiscinvolve.org/wp/exemplars). 

Open Data as OER update

A few months ago (and yet, somehow, only a couple of posts ago, 'here'!), I posted the link to a piece I co-wrote for the Open Education Working Group blog with Javiera Atenas and Ernesto Priego considering the potential of open data to be used as open educational resources. We shared a survey and we got some responses.

Javiera and I spoke at the 7th Open Education Working Group call on the topic and introduced our proposal to gather case studies of pedagogic practice in this area.

Open Data as Open Educational Resources 

See also:
Minutes and recording from the call
Reflections on the call from Lorna Campbell

Crowdsourcing Quality in Open Education

How can we assure quality in the context of Open Education? This question lies at the heart of the challenge of gaining mainstream acceptance of open content. It perhaps brings to mind the idea of some kind of gatekeeping process that would vet content before it is released for 'consumption' - sort of the way academic peer review is supposed to work. But there are questions over the efficacy and sustainability of the traditional peer review model, even for research publications. 

Traditional peer review is labour intensive, relying on a donation of free labour/time from reviewers (and academic time seems to become an ever scarcer resource). Crucially, it also operates in a 'closed' space, where authors and reviewers are anonymised, and editors pass messages back and forth, eventually resulting in a finished product which may later be cited, and possibly (but not necessarily!) freely distributed, but most likely never revised or updated. 

It appears inadvisable to transplant this model into an open space, where open licenses encourage reuse and modification (rather than simply consumption). Never mind the question of who exactly would be sourcing and providing the labour to peer review these open learning resources. Instead Javiera Atenas and I, in a new post for Open Education Europa, suggest that we must leverage openness itself, taking advantage of open educational practices to improve the quality of open content.

Our post about this is here: 

Crowdsourcing Quality (Or, Why Openness Matters)

Open Data as OER: Blog Post and Survey

Recently in conjunction with Open Education Week, the Open Education blog over at Open Knowledge featured a guest post by Javiera Atenas, Ernesto Priego and me, in which we propose that Open Data is under-explored as an educational resource. See:

• THE 21ST CENTURY’S RAW MATERIAL: USING OPEN DATA AS OPEN EDUCATIONAL RESOURCES

The piece ends with a link to a very short survey, through which we are trying to capture examples of OD as OER use. Many thanks to everyone who already completed the survey - please do/share if you haven't!

Breaking down barriers: Open Educational Practices as an emerging academic literacy

A conference presentation with Jo Stroud and Javiera Atenas at Connected Learning in an Open World: Academic Practice and Technology conference, University of Greenwich; 8 July 2014. Links for our accompanying Prezi plus our speaking notes are below.

Abstract: 
This paper considers barriers to the uptake of Open Educational Practices (OEP) and how these barriers might potentially be overcome. ‘Open education’ has become an increasingly important topic in TEL and wider educational debates, partly driven by the rapid emergence of MOOCs. Various activities, such as sharing of teaching resources, open access publishing, and delivery of free, online courses, are being promoted under the umbrella of ‘openness’, yet the majority of scholars do not appear to be embracing the concept wholeheartedly. Amidst dramatic changes to the current learning, teaching, and research landscape, academics are increasingly called upon to become ‘connected’, ‘digital’, and ‘open’ scholars, requiring engagement with a kaleidoscope of interconnected open social and digital practices. Potentially the most powerful and compelling argument for any version of ‘open education’ is the claim that academics should adopt Open Educational Practices (OEP). For example, Andrade et al. (2011) assert that “OER and OEP are changing learning scenarios” and that “the use of OER and the implementation of OEP lead to innovations in pedagogical terms”. OEP consist not only of creating and reusing OER, but also of other forms of transparency around academic practice, such as blogging, tweeting, presenting, and debating scholarly and pedagogic activities, in ways that promote reusability, revision, and collaboration (Ehlers, 2011; Jacobi & Woert, 2012). OEP are therefore tactics for increasing access to knowledge, developing and strengthening communities of practice, disseminating positive ways of working, leveraging high-quality existing resources, and promoting innovative pedagogies (Andrade et al., 2011). In spite of this, there appears to be a lack of engagement with OEP. It’s possible that these theoretical and practical benefits have not been widely understood by the academic community, but it also seems to be the case that real or perceived regulatory, technical, cultural, and skills barriers exist. In our presentation we will consider the nature of these barriers in greater detail and suggest potential strategies for their mitigation. 

References: 
Andrade, A., Ehlers, U.-D., Caine, A., Carneiro, R., & Conole, G. (2011). Beyond OER: Shifting Focus from Resources to Practices (pp. 1–191). Duisburg-Essen. Retrieved from http://www.oerasia.org/OERResources/8.pdf  
Ehlers, U., & Conole, G. (2010). Open Educational Practices : Unleashing the power of OER (pp. 1–9). Retrieved from http://efquel.org/wp-content/uploads/2012/03/OEP_Unleashing-the-power-of-OER.pdf  
Ehlers, U-D. (2011). From Open Educational Resources to open educational practices. eLearning Papers, (23), 1–8. Retrieved from http://www.elearningeuropa.info/nl/node/71328  
Jacobi, R., & Woert, N. van der. (2012). Trend Report on Open Educational Resources 2012. SURF, Utrecht. 


Prezi available: http://prezi.com/yenbduyyzsjy/greenwich-apt 
Slide notes available: https://docs.google.com/presentation/d/1kEmbPQKuvtIm9r84aTJ8raJnfEPJIw6WvrJdBguJe4s/edit?usp=sharing

Seminar review: Practical considerations of running a MOOC

The University of London’s (UoL) Centre for Distance Education (CDE) provides thought-provoking lunchtime seminars on topics of interest to learning technology practitioners and staff who teach or administer online courses. A recent well-attended seminar on the practicalities of Massive Open Online Courses (MOOCs) included presentations on MOOCs in English Common Law (ECL) and Creative Programming (CP). 

Julie Voce, E-Learning Services Manager at Imperial College and I wrote up our notes of this event and they are available on the ALT Online Newsletter. 

Is it time for MOOCs to 'open up'?

A piece I co-wrote with Javiera Atenas is now live on the LSE Impact of Social Sciences blog: MOOCs must move beyond open enrolment and demonstrate a true commitment to reuse and long-term redistribution.

MOOCs must move beyond open enrolment and demonstrate a true commitment to reuse and redistribution. http://t.co/QpqAYJmSya #OER #edchat
— LSEImpactBlog (@LSEImpactBlog) March 7, 2014

Literacies for open practice: open up and say OER?

A presentation at Bloomsbury TEL event on Digital Literacy for Teaching and Research, 19 February 2014, at the Institute of Education, University of London.

Literacies for open practice: Open up and say OER? 

Media Cloud forecast: in-video quizzes and more

Recently the Bloomsbury Media Cloud User Group attended a presentation from MediaCore, the company who provide the platform and hosting that powers the service. There are some interesting developments in the pipeline.

In-video quiz prototype

• It is good to see research and thought has gone into this. The goal is that it should be designed and prototyped in accordance with educational theory and research and user community engagement. Reference to literature arguing that in an educational setting, videos shouldn’t just be passively consumed – you want the students actively engaging.
• They have also surveyed academics and technical staff regarding what they would want to do if there was a quizzing functionality. An interesting result was ‘discuss the responses in class’. Also teachers felt they would change the way they create videos if they had the quizzing functionality, so seen as having a strong pedagogical affordance.
• Also reviewed literature on quizzes/MCQs. Creating good ‘distractors’ (plausible wrong answers) and provision of response-contingent feedback (i.e. tells you why you are right or wrong) both seen as key.
• Design is also based on Bloom’s taxonomy with a view to enabling and encouraging teachers to consciously apply Bloom’s taxonomy in design of questions and feedback.
• Questions can be textual or image-based.
• Getting it working smoothly and making it easy to use is the priority. Integration with Moodle gradebook and similar via SCORM or other methods is further down the track, but the group generally felt this functionality will be more valuable for formative/reflective purposes anyway.

Now & next for the platform

• We are now using the new player.
• You can now embed whole collections (not from the Chooser in Moodle though).
• They now have epub support (i.e. you can upload an epub file as the main media item).
• Capture app already existed for iOS devices, and now exists for Android and Windows 8 (PC & tablet).
• Snagit integration allows upload of Snagit screencapture videos directly to MediaCore but there is a bug that is not displaying all the collections (this is currently with TechSmith to resolve).
• Plan for everything to be more responsive to device/screen size so it works and looks better.
• Planned new ‘super embed’ will allow a bunch of functionality from the site (e.g. collections, attached docs, download main media item) to be accessible from the embeds.
• User group members Javiera Atenas, Elizabeth Charles and Leo Havemann supplied recommendations regarding inclusion of licensing options (including the range of Creative Commons licences) and these will be incorporated into  the platform soon.
• Sharing assets across more than one collection is known to be our current major issue.
  
  

  This post was originally written for the Bloomsbury Learning Environment Blog

Birkbeck Special Blend Project Presented at HEA/SEEC Conference 2013

The HEA and SEEC held a combined conference on Flexible Learning at the University of Westminster in July. Along with Liz Drew who was part of the project team with me, I presented an overview of Special Blend, our Changing the Learning Landscape -assisted project which aimed to come up with a Birkbeck-flavoured approach to blended learning. The project morphed into a second phase which involved developing and then delivering a pilot module, Step Up to Postgraduate Study in Arts. The presentation slides are below.

Special Blend: Developing a Model for Technology-Enhanced, Flexible Learning 

Are Repositories Holding Back OER?

Are repositories holding back OER? - Presented at M25 Learning Technology Group, July 2013

Turnitin User Group meeting: Senate House, London, 15 February 2013

A meeting of the Turnitin UK User Group was held in London last week and the following represents my capture of points of interest from that meeting rather than a full record of all that was presented, but I hope this is of some use to the community.

1. Service Updates

Communication

Status updates: various options to be updated on this including email, Twitter, RSS. You can also enable mobile notification from the Twitter feed. They send updates on the US and UK services via the same Twitter account (TurnitinStatus) which many felt was unhelpful so perhaps this will be changed.

Product updates: there is now a Twitter feed for this as well (TurnitinProduct).

Support Portal

They have never had one but will now be implementing this – it will enable us to see all open cases and any progress they are making. This would enable us to have a Bloomsbury-wide overview of the service status as well as better visibility at the more granular level of individual support tickets.

Currently their plan is that there will be an additional charge as this has never been included in our licence cost. At the moment they are seeing this as something some clients will opt in to and pay for but I wonder if it wouldn’t be better to provide this for all customers as the cost would then be spread widely. Possibly HELF (the Heads of e-Learning Forum) could make this suggestion on behalf of UK HE if there is general agreement across the sector.

There is an opportunity to join the pilot of the system which seems like a good idea for Bloomsbury.

Plagiarism Website

Relaunched and improved with a lot of resources worth checking out – the conference proceedings from the last 10 years are now up. 

See: www.plagiarismadvice.org

2. Product Roadmap - General

Grademark

We were reminded to promote the online tutorial for Grademark. There are free/public rubrics available now (from the US) which we can download and adapt.

iPad App

This is due in beta in late February, with full launch in May – this allows you to sync the papers and mark offline, which is a very useful feature that is possible to implement for iPad but unfortunately not for Android (or for use on Windows / Mac laptops  either) at this time. Android may become a possibility (or at least certain devices). 

One point that concerned me was that it may not be totally straightforward for staff who use Turnitin integrated with Moodle (as ours exclusively do) unfortunately as they will need to find a one time access code in the Document Viewer to get them in to the relevant class via the iPad app.

Admin account interface (via the submit.ac.uk website)

There will be improvements to the interface and tools available to us.

Document viewer

This is the main interface Turnitin users ever see, which contains the Originality Report and Grademark views of the submissions. Version 2 is now in progress for release by the end of this year and should solve some of the persistent issues (but will be even less compatible with older browsers than the current one).

3. Product Roadmap - Integrations

Legacy API

(i.e. the current API all integrations are connecting to.) There are a lot of issues with the legacy API and a new one is under construction. This will resolve the prevalent 423 errors and will support use of Peermark by integration users.

Moodle Direct Integration

This is what we use in Bloomsbury. The new version of MD will connect via the new SOAP API. This will provide support for using Turnitin in the same way as users of the current ‘Dan Marsden’ plugin, which lets you use the normal Moodle assignment tool but still run an originality report. This may suit staff/departments which want the OR but don’t want to mark online.

It will be possible to download anonymous assignments (at the moment this function is switched off when anonymity is enabled, because Turnitin appends the student’s name when you download, so a code change will allow for downloading with names removed). The beta version of Moodle Direct v 2 is expected in May.

4. Guest Speaker 

Cheryl Reynolds (University of Huddersfield) - The JISC EBEAM Project

Cheryl presented an interesting case study of the implementation of Turnitin at Huddersfield under the auspices of EBEAM, which sought to drive uptake of online submission and marking. 

The emphasis was put on the benefits to staff. Lessons included: compulsion caused hostility. People will make the move in their own time, especially once colleagues are doing it.

Some tutors started using the new audio feedback option which students are responding very positively to – students in general felt the audio feedback was more personal and ‘human’ but it was also especially useful because of the greater accessibility of audio compared with text for visually impaired and dyslexic students.  

See more information about EBEAM here.

Creative uses of Moodle at MUGGL

Creative uses of Moodle? - MUGGL presentation and feedback from Leo Havemann

Lecture Capture: More Info Needed?

Lecture capture is quite a hot topic in HE these days and we recently had a showcase event in Bloomsbury to look at how some of the BLE member colleges are making use of it as well as hear from other universities.

In Bloomsbury the technology which has been adopted for this (especially at the RVC and LSHTM) is Echo360 (formerly known as Apreso). Echo can record a video stream of the lecturer, another video stream of whatever is shown on the big screen (this is usually used to record the steady advance of PowerPoint slides on the teaching PC, but can also capture a feed from a visualiser or DVD player), plus the audio of the lecturer (provided they speak into the mic).

The event included an especially useful presentation from Graham McElearney of Sheffield University, who discussed the intimidating copyright and IPR issues and implications of lecture capture but also offered frameworks from which to develop sound institutional policies. For more info from the event click here (BLE login required).

Another event was recently held by JISC Legal and recordings can be accessed here: http://www.jisclegal.ac.uk/RecordingLectures.

Upcoming events

Echo360 are holding a conference at the LSE on the 25th of May - for more information see http://echo360.com/community/community-conference-europe/.

A platform-agnostic ALT event entitled "Lecture Capture - doing it well and at scale" is coming up at QMUL on 16 June, and will be covering:

• Case studies of successful implementation including from the Institute of Theoretical Physics in Trieste, Italy
• Lecture recording in the sciences
• AV issues when working at scale and solutions
• Encouraging adoption
• Pedagogic and learning design issues
• Supporting teachers and students in their use of lecture capture
• Institutional policies and obstacles
• Lecture capture myths and misconceptions and how to tackle these
• IPR/Copyright issues

Click here to sign up for further information about the event.