Do LLM Agents Manufacture Interaction? A Data-Anchored Spin Model as a Benchmark for Language Agents Simulating Hostile Online Discourse
Abstract
LLM agents are increasingly used as social simulators: populate a network, run an intervention, read off what would happen. That is only valid if the agents reproduce the interaction structure of the data they stand in for. We test this on 5637 human-annotated comments from the comment sections of 22 Croatian news articles on a socially sensitive topic, where every comment is labelled both for hostility and for whether it addresses the article or another commenter. The second label yields the reply network, and that network is a broadcast regime: 62% of users comment once and are never replied to, only 777 are connected to anyone, and there is no stance homophily among those who are. We calibrate a three-state Blume-Capel spin model (hostile, silent, supportive) to the same annotations, with no free parameter other than the cost of speaking, and use it as a benchmark that makes quantitative predictions for three counterfactual interventions before any language model is run. We then place EuroLLM-9B agents on the same network, each agent's persona drawn from that user's own comments, silence read off the model's own log-odds of speaking, and every utterance mapped back to the spin states by classifiers fine-tuned on the human labels, so the generator never grades itself. Humans addressed another commenter in 21% of comments; agents shown their neighbours do so in 49%, and agents shown nobody at all still do so in 37%. Every LLM run, from any initial condition, at any cost of speaking, with or without injected supporters, converges to a mildly supportive consensus where the humans sit at a hostile one. The agents manufacture both the conversation and the ambient stance, and they agree with the spin model only where agreement is trivial: a system with one strong attractor has no tipping point either.