Deep neural networks (DNNs) have showcased their remarkable precision in approximating smooth functions. However, they suffer from the {\it spectral bias}, wherein DNNs typically exhibit a tendency to prioritize learning of lower-frequency components of a function, struggling to effectively capture its high-frequency features. This paper is to address this issue. By observing that a function having only low frequency components can be well-represented by a shallow neural network (SNN), a network having only a few layers, and that composition of low frequency functions can effectively approximate a high-frequency function, we propose to learn a function containing high-frequency components by compositing several SNNs, each of which learns certain low-frequency information from the given data. We implement the proposed idea by exploiting the multi-grade deep learning (MGDL) model, a recently introduced model that trains DNNs incrementally, grade by grade, with each grade training an SNN from the residue of the previous grade. We apply MGDL to synthetic, manifold, and MNIST datasets, all characterized by presence of high-frequency features. Our study reveals that MGDL excels at representing functions containing high-frequency information. Specifically, the neural network learned in each grade adeptly capture some low-frequency information, allowing its compositions with neural networks learned in previous grades effectively represent the high-frequency features. Our experimental results underscore the efficacy of MGDL in addressing the spectral bias inherent in DNNs. By leveraging MGDL, we offer insights into overcoming spectral bias limitation of DNNs, thereby enhancing the performance and applicability of deep learning models in tasks requiring the representation of high-frequency information. This study confirms that the proposed method offers a promising solution to address the spectral bias of DNNs.