Generation and Human-Expert Evaluation of Interesting Research Ideas using Knowledge Graphs and Large Language Models

Advanced artificial intelligence (AI) systems with access to millions of research papers could inspire new research ideas that may not be conceived by humans alone. However, how interesting are these AI-generated ideas, and how can we improve their quality? Here, we introduce SciMuse, a system that uses an evolving knowledge graph built from more than 58 million scientific papers to generate personalized research ideas via an interface to GPT-4. In a large-scale evaluation, over 100 research group leaders from the Max Planck Society ranked more than 4,000 ideas by their level of interest. This allowed us to understand the relationship between scientific interest and the core properties of the knowledge graph. We found that data-efficient machine learning can predict research interest with high precision, thus optimizing the generated ideas' interest-level. Our work represents a step toward an artificial scientific muse that could catalyze unexpected collaborations and suggest avenues avenues for scientists.