Table 1 Categorization of evaluation criteria within TPACK dimensions and mapping to MARS and MARuL sub-criteria.
From: Evaluation of medical education mobile apps with MARS and MARuL in a TPACK informed model
TPACK Dimensions | Metric | MARS | MARuL | Definition | |
|---|---|---|---|---|---|
Technology | Options for user support | MuD6 | Provide technical support to enhance user experience with the app, including tutorials, search functionality, help resources, and automatic updates7,46,47,48 | ||
Accessibility | Ensure the mobile app is accessible across smartphones with all essential features, requiring minimal cognitive effort, while complying with web and multimedia accessibility standards48,49. | ||||
Design | Layout | MaC1 | MuD1 | Ensure an aesthetically pleasing and user-friendly design with a clear, consistent layout, high-quality media, and intuitive navigation. Use bright colors for learners, appropriate fonts and sizes, well-structured touch-screen menus, colorful animated buttons, and icons that accurately represent their functions45,46,47,49,50 | |
Graphics | MaC2 | MuD1 | |||
Visual appeal | MaC3 | MuD1 | |||
Ease of use | MaB2 | MuD4, MuD1 | |||
Purpose | MaA5 | MuA1 | Ensure well-defined objectives with coherence between goals, skills, and target audience, supported by appropriate content and guidance for teachers and students46,49. | ||
User control | Users should be able to select and sequence tasks within the mobile app, with the freedom to make choices throughout the mobile app47,51. | ||||
Stability | Assess whether the mobile app experiences freezing or crashes46. | ||||
Portability | MuD7 | The mobile app should function seamlessly across platforms52. | |||
Multimodal options | Evaluate the role of embedded visual and verbal features in motivating and sustaining student engagement, while ensuring they do not distract from learning or serve merely as entertainment53 | ||||
Functionality | MaB4 | MuB5 | We can fully utilize all of its features | ||
Communication | Enable learners to share mobile app-generated content, as well as their learning progress, issues, or concerns, with others via email or print at any time47,50,51,54,55,56. | ||||
Performance | MaB1 | MuD2 | Measure the mobile app’s loading speed, responsiveness, and task completion efficiency47 | ||
Gamification | MaA1 | The process of integrating game-like elements into mobile apps46. | |||
Interoperability | Developed in standard formats for universal compatibility, including complete metadata and accessibility features, and packaged for easy transfer and reuse49. | ||||
Navigations | MaB3 | MuD4 | The ease with which students can navigate the app independently, supported by clearly labeled menus and icons, functional status links, and clear instructions46,47,49,53,54,55,56 | ||
Working mode | Determine whether the mobile app requires an internet connection to function46. | ||||
Error prevention | MaB1 | MuD2 | Facilitate error prevention and recovery by providing concise error messages47,51. | ||
Reusability | High modularity, enabling reuse across disciplines and learning environments, including face-to-face, online, and blended settings49. | ||||
UI customization | MaA3 | MuA7 | Customization of voice, color, layout, and other user interface elements7,57. | ||
Pedagogy | Teaching | MuA6 | Determine whether the mobile app presents, explains, or models subject content, rather than solely testing it46. | ||
Feedback | MuA8 | The mobile app’s ability to track progress and provide meaningful, specific, and timely feedback that reinforces correct responses, enabling learners to conduct self-evaluation46,47,50,54,55,56. | |||
Engagement | MaA2 | MuB7 | Evaluates whether the mobile app maintains student engagement and attention, or if it is perceived negatively by the learner46,47,54. | ||
Learning values | MuA9 | Typically evaluated through teacher or student perceptions, focusing on learner control, knowledge construction, information-seeking skills, and overall achievement. Effective mobile apps structure learning into manageable, connected steps and offer strategies to support task completion58. | |||
Cognitive development | Emphasize recognition over recall, using appropriate language and content grounded in familiar concepts47 | ||||
Practice targets | A mobile app that enables users to practice learning targets either in a realistic, problem-based environment or in isolation, such as through flashcards50,54 | ||||
Motivation | The mobile app delivers pedagogical feedback from instructors through activities or content designed to engage learners’ interests45,49,51,55. | ||||
Goal orientation adaption | Evaluate mobile app flexibility based on applicability to diverse curricular areas54 | ||||
Effective scaffolding | Evaluate whether the app’s activities progress in difficulty in a manner that supports the learner46 | ||||
Self-Directedness | Educational mobile apps are recommended to provide learners with personalization options52 | ||||
Frequency | MaE2 | MuB6 | The frequency of mobile app use by teachers and learners varies by purpose; mobile apps with potential for daily use are considered more valuable than those used occasionally48 | ||
Content | Customization | MuD3 | Mobile apps should allow personalization and individual profiles to match learners’ abilities and needs45,46,47,54,55. | ||
Authenticity | Include classroom activities that provide real-life practice through simulations, such as exploring spacecraft or planets, or constructing atoms using electrons, protons, and neutrons49,51 | ||||
Interactive elements | MaA2, MaA4 | MuA7 | Engages users and supports learning by allowing content manipulation for creativity and expression, with appropriate gestures and easy interaction47,49,53. | ||
Content quality | MaD3, MaD5 | Content should be balanced, up-to-date, accurate, objective, respectful of copyright, and presented with clear instructions46,49. | |||
Critical thinking | The mobile app should promote higher-order thinking by encouraging questioning, analysis, evaluation, and openness. It should help students connect prior experiences with in-app activities and provide opportunities for scientific inquiry through observation, experience, reflection, reasoning, and communication45,49,50,54,56,59 | ||||
Relevance | MaD2 | MuA5 | The mobile app’s content should be aligned with and reinforce the targeted goals and skills of the lesson45,48,49,56. | ||
Accuracy | MaD1, MaD4, MaD6 | MuB3, MuC3 | Content should be accurate and complete, with graphics that enhance understanding of scientific concepts and correctly represent experimental procedures and measurements56. | ||
Curriculum oriented | Mobile apps should align with curriculum skills or concepts and effectively incorporate them into meaningful learning experiences45,50,52 | ||||
Up-to-date | MuC1 | Delivering information and representations in a timely manner51 | |||
Credibility of app | MaD6 | MuC2 | Evaluate whether the mobile app was developed by experienced experts in the relevant learning domain7,57 | ||
Evidence base | MaD7 | MuC1 | Check for evaluation and validation in academic sources57 | ||
General | Satisfaction | MaE4 | MuB2 | The evaluator’s level of satisfaction with the mobile app7,57 | |
Perceived importance | MaE3 | MuB4 | |||
Advertisements | MuD5 | Evaluate whether ads affect usability and if a paid version eliminates them7 | |||
Recommend to other | MaE1, | MuB1 | |||
