Narratron: A Storytelling AI for Shadow Play

Shadow puppetry or shadow play, allows bodily participation into the process of linguistic storytelling, while the potential of multi-modal interaction through shadow plays in existing large-language-model-based creative tools has not been fully discovered. We propose Narratron, a generative story-making tool that co-creates and co-performs children stories from shadow using Claude 2 model. To achieve Narratron, our system is designed to recognize hand gestural inputs as main character and to develop story plot in accordance with character change. Through our system, we seek to stimulate creativity in shadow play storytelling and to facilitate a multi-modal human-AI collaboration.