Where does frontier multimodal training data actually come from? In this hands-on demonstration, attendees step into the role of an expert AI trainer on a live production platform and try to make frontier vision-language and audio models fail.

Station 1, Stump the Model: attendees pick an image, chart, document, video clip, or audio clip, or bring their own from their phone, and write a question designed to break a frontier multimodal model. The model answers live. If it fails, the attendee writes a short rationale and a grading rubric. A live quality-scoring system, which ranks contributors with a Bradley-Terry rating model calibrated against expert reviewers, scores each submission in real time, and the hardest verified questions climb a public leaderboard over the two hours.

Station 2, Failure Gallery: a running wall of the failures collected so far, grouped by failure type (fine-grained visual detail, chart and table reading, spatial reasoning, temporal order in video, speech and sound understanding). Next to it, sample multimodal datasets show the collection and QC pipeline that turns a one-off failure into a verified training and evaluation example.

Everything runs live. The aim is to show researchers, through their own participation, where today's multimodal models still break and what separates a trustworthy data point from a noisy one.