Invited Talk: Dr. Anaïs Tack on 'Generating Teacher Responses in Educational Dialogues: The AI Teacher Test & BEA 2023 Shared Task'

We like to invite you to our upcoming invited talk by Dr. Anaïs Tack from the KU Leuven, Belgium . The talk will take place on Thursday, June 1, 2023 at 16:00, as part of our graduate seminar in Room 3408-1630 at Appelstraße 9A:

Speaker

Dr. Anaïs Tack

KU Leuven, Belgium

https://anaistack.github.io

Time and location

Thursday, June 1, 2023 at 16:00 in Room 3408-1630 (Appelstr. 9A)

Title

Generating Teacher Responses in Educational Dialogues: The AI Teacher Test & BEA 2023 Shared Task

Abstract

How can we test whether state-of-the-art generative models, such as Blender and GPT-3, are good AI teachers, capable of replying to a student in an educational dialogue? Designing an AI teacher test is challenging: although evaluation methods are much-needed, there is no off-the-shelf solution to measuring pedagogical ability. In the first part of this talk, I will describe our paper "The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialogues" presented at EDM 2022. The paper reported on a first attempt at an AI teacher test. We built a solution around the insight that you can run conversational agents in parallel to human teachers in real-world dialogues, simulate how different agents would respond to a student, and compare these counterpart responses in terms of three abilities: speak like a teacher, understand a student, help a student. Our method builds on the reliability of comparative judgments in education and uses a probabilistic model and Bayesian sampling to infer estimates of pedagogical ability. We find that, even though conversational agents (Blender in particular) perform well on conversational uptake, they are quantifiably worse than real teachers on several pedagogical dimensions, especially with regard to helpfulness. In the second part of this talk, I will describe the results of the "BEA 2023 Shared Task on Generating AI Teacher Responses in Educational Dialogues", which was a continuation of our EDM paper.

External participants (i.e. neither students nor employees of LUH) are kindly requested to send a short, informal email to office@ai.uni-hannover.de to register