Global sustainability has become an urgent call. Garments and textiles are ubiquitous in our daily lives, however, tons of garments end up in landfill every year. The fashion industry is undergoing a significant change to help textile materials to be reused, repaired and recycled in a sustainable manner. Textiles need to be traced back to their original forms so that recycling can be guaranteed. Yet, textiles are mostly sorted manually as automatic identification of textile materials is challenging and we lack a low-cost and effective technique for identifying textiles. Our proposed model looks at this textile classification problem from a very different angle, we use simple ubiquitous RGB textiles garments images.