Pre-Conference Talk by POH Emran Bin Elias | ‘Show it, Don’t Just Say It’: The Complementary Effects of Instruction Multimodality for Software Guidance

Designing adaptive tutoring systems for software learning presents challenges in determining appropriate instructional modalities. To inform the design of such systems, we conducted an observational study of ten human teacher-student pairs (N=10), where experienced design software users taught novices two new graphic design software features through multi-step procedures. These lessons were limited to three communication channels (speech, visual annotations, and remote screen control) to mimic possible AI tutor modalities. We found that annotations complement speech with spatial precision and remote control complements it with spatial and temporal precision but both of them cause intrusion to learner agency. Teachers adaptively select modalities to balance the need for instruction progress with students' cognitive engagement and sense of digital territory ownership. Our results provide further support to the contiguity principles and the value of agency in learning, while suggesting precision-agency trade-off and digital territoriality as new design constraints for adaptive software guidance.

This is a Pre-Conference talk for The ACM (Association for Computing Machinery) CHI Conference on Human Factors in Computing Systems (CHI 2026).

Emran Poh is a first-year PhD student, where he studies how people learn with and around AI. Advised by Asst. Prof. Jiannan Li, his current work examines AI intervention timing in software learning and agentic systems — particularly the tension between a tutor that's precise and one that leaves room for agency. He will be presenting one paper and two workshop papers at CHI 2026, including work on multimodal instruction and the role of restraint in AI tutoring design.