Design of evaluation of content sharing solutions in medical education

Abstract

The problem that the mEducator Best Practice Network (BPN) focuses on refers to the abundance of medical educational content in individual EU academic institutions, which is not widely available or easy to retrieve due to the lack of standardized content sharing mechanisms. The mEducator BPN aims to implement and critically evaluate existing standards and reference models in the field of e-learning in order to enable specialized state-of-the-art medical educational content to be discovered, retrieved, shared and re-used across European institutions. mEducator includes both traditional and user-generated content and addresses several learning contexts ranging from traditional instructional teaching to active learning and experiential teaching/studying approaches. It furthermore includes many different content types, ranging from text to exam sheets, algorithms, teaching files, computer programs (simulators, serious games) and interactive objects (like virtual patients and digital tracings of anatomies), while it covers a variety of tools. The project currently focuses on the development of two contemporary ways of achieving content sharing, mEducator2.0, a solution based on Web2.0 technologies and mEducator3.0, a solution based on Semantic Web Services, and their evaluation.

mEducator2.0 develops a brokerage mechanism based on RSS feeds, “mashup” technology and other Web 2.0 technologies that allow the exchange of clinical cases, medical images and videos. mEducator3.0 is a sharing platform using the Semantic Web and its most recent development, the Linked Data approach. Both mEducator solutions are in their beta-version and they are currently being tested by the target user groups of the project such as students, educators, doctors, and health professionals. Their evaluation is based on an evaluation framework for assessing the effectiveness of the platforms that will be implemented through the mEducator BPN and any future platforms that attempt to address the same issues. Our proposed evaluation framework draws upon systems that face similar challenges, such as digital libraries, peer-to-peer and social networks, LCMSs where adaptation issues and courseware reuse are addressed, information retrieval systems, e-learning systems, search engines and public libraries, such as PubMed. It is expected that the resulting solutions will address the needs of different types of users and they will be easily transferable to other domains and disciplines. This paper presents preliminary evaluation results comparing the overall usability of the two solutions and comments on the challenges associated with implementing and carrying out a complex evaluation model.