Most learning systems still teach in a “one-size-fits-all” way: the same explanations, the same hints, the same pacing - regardless of who is learning or how they’re progressing. That can overwhelm novices, bore advanced learners, or deliver help at the wrong time.

AI offers a unique opportunity here: it can approximate the role of a human teacher or learning companion - one that not only reacts to success/failure, but tries to understand what the learner needs next. Compared to basic performance metrics (like task time or error counts), AI can enable a more detailed, context-aware assessment and more sophisticated adaptation.

In our recently published paper, we outline a multi-agent framework that orchestrates these capabilities into a structured OODA loop (observe, orient, decide, act). Different agents handle sensing and preprocessing, pedagogical reasoning, and the generation of concrete, bounded interventions. The broader idea is to treat the learning system as an AI platform: it can combine signals from multiple data sources and, at the same time, serve as a multimedia interface for multimodal feedback and actuation (text, visuals, guidance, etc.).

We use robot training in Augmented Reality as a practical example - not because the concept is limited to robotics, but because it’s an ideal environment to study real-time, in-context adaptation.

To get a more insights into the framework, I recommend reading the full paper (arXiv).


Multi-agent framework

Multi-agent framework