Timezone: »

 
Overcoming Transformer Fine-Tuning process to improve Twitter Sentiment Analysis for Spanish Dialects
Daniel Palomino

Mon Dec 07 03:21 PM -- 03:31 PM (PST) @

Is there an effective Spanish Sentiment Analysis algorithm? The aim of this paper is to answer this question. The task is challenging because there are several dialects for the Spanish Language. Thus, identically written words could have several meanings and polarities regarding Spanish speaking countries. To tackle this multidialect issue we rely on a transfer learning approach. To do so, we train a BERT language model to ``transfer'' general features of the Spanish language. Then, we fine-tune the language model to specific dialects. BERT is also used to generate contextual data augmentation aimed to prevent overfitting. Finally, we build the polarity classifier and propose a fine-tuning step using groups of layers. Our design choices allow us to achieve state-of-the-art results regarding multidialect benchmark datasets.

Author Information

Daniel Palomino (Universidad Católica San Pablo)

I'm working on the development of Deep Learning Techniques and Language Models using GPU distributed architecture that allow a more efficient extraction of data in problems related to Natural Language Processing, especially those related to the analysis of unstructured data like as opinion analysis or sentiment analysis remain in a low average development especially for languages such as Spanish and Portuguese. This work is enhanced with Cuda programming and Linux HPC environment with excellent results.

More from the Same Authors