The era of proliferation of large language and image generation models begs the question of what happens if models are trained on the synthesized outputs of other models. The phenomenon of "model collapse" refers to the situation whereby as a model is trained recursively on data generated from previous generations of itself over time, its performance degrades until the model eventually becomes completely useless, i.e.~the model collapses. In this work, we study this phenomenon in the setting of high-dimensional regression, under low- and high-dimensional asymptotics, and obtain analytic formulae which quantitatively outline this phenomenon in a broad range of regimes. We show how test error increases linearly in the number of model iterations in terms of all problem hyperparameters (covariance spectrum, regularization, label noise level, dataset size) and further isolate how model collapse affects both bias and variance terms. We show that even in the noise-free case, catastrophic (exponentially fast) model-collapse can happen in the over-parametrized regime.In the special case of polynomial decaying spectral and source conditions, we obtain modified scaling laws which exhibit new crossover phenomena from fast to slow rates. We also propose a simple strategy based on adaptive regularization to mitigate model collapse. Our theoretical results are validated with experiments.