Journal Club Article: Cook, D. A., & Ellaway, R. H. (2015). Evaluating technology-enhanced learning: a comprehensive framework. Medical Teacher, 37(10), 961-970.
Aim: To outline a comprehensive framework for Technology Enhanced Learning evaluation, which is the process of judging the value of education.
Rationale: Lack of clear focus of evaluation for difitical technology educational activities, including online learning.
“Technology Enhanced Learning can involve web-based technologies, mobile devices and apps, computers, tablets, and other digital devices, and may include activities that are entirely digitally-mediated or those that integrate technology into hybrid or blended activities.
Cook & Ellaway, 2015
General Principles of Evaluation:
- Objectives Orientated Focus – how well were course objectives met?
- Participant Orientated Approach – the learners experience
- Process Orientated Evaluation – consider the life cycle of a course from implentation and delivery.
Special Considerations in Technology Enhanced Learning:
- Conduct needs analysis and environmental scan – detrmine the need for change at the start before educational change.
- Document processes, decisions, and final product – educational design and framework.
- Test usability – “Learnability refers to how easily users can accomplish basic tasks the first time they encounter the technology.”
- Document key events during implementation and final product.
- Assess participant experience and satisfaction – the value and meaning of the learning.
- Assess learning outcomes – Kirkpatrick Model of Evaluation
- Estimate cost, reusability, and sustainability
Practical Approaches to Principles of Evaluation:
Three new instruments were developed to assess perceptions and experiences following participation in a Technology Enhanced Learning course for both learners and instructors. The evaluations allows use of a a consistent instrument in evaluation which allows comparisons within and across courses.
Some challenges with evaluating Technology Enhanced Learning:
- Interactions between teachers and learners are different in Technology Enhanced Learning when compared to face-to-face learning envrionments.
- Technology Enhanced Learning can generate a lot of data, and different types of data, there is a need to consider how this will be analysed.
- Is the learning technology as advanced as the technology used in the healthcare workplace?