This project explores how tangible Augmented Reality (AR) can be used to teach computer programming. We have developed TARPLE, A Tangible Augmented Reality Programming Learning Environment, and are studying its efficacy for teaching text-based programming languages to novice learners. TARPLE uses physical blocks to represent programming functions and overlays virtual imagery on the blocks to show the programming code. Use can arrange the blocks by moving them with their hands, and see the AR content either through the Microsoft Hololens2 AR display, or a handheld tablet.
This research project expands upon the broader question of educational AR as well as on the questions of tangible programming languages and tangible learning mediums. When supported by the embodied learning and natural interaction affordances of AR, physical objects may hold the key to developing fundamental knowledge of abstract, complex subjects for younger learners in particular. It may also serve as a powerful future tool in advancing early computational thinking skills in novices. Evaluation of such learning environments addresses the hypothesis that hybrid tangible AR mediums are able to support an extended learning taxonomy both within the classroom and without.
We are currently conducting user studies to explore how learning with TARPLE is different from traditional programming tutorials, and the benefits that tangible AR could bring to an educational environment. The initial target for the application is high school children with little experience of programming, but the project could be expanded to support a wide range of other people who are programming novices.Project Video(s):
Resnyansky, D. (2020, December). Augmented reality-supported tangible gamification for debugging learning. In 2020 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE) (pp. 377-383). IEEE.
Resnyansky, D., Billinghurst, M., & Dey, A. (2019, December). An AR/TUI-supported debugging teaching environment. In Proceedings of the 31st Australian Conference on Human-Computer-Interaction (pp. 590-594).
Resnyansky, D., Ibili, E., & Billinghurst, M. (2018, December). The potential of augmented reality for computer science education. In 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE) (pp. 350-356). IEEE.