Taking "Big Data" evidence to policy: Experiences from the Global South

LIRNEasia has been working on leveraging big data for public purposes since 2012. As an organization situated in a developing country, we have experienced challenges in developing new insights, and informing policy and government processes. When leveraging big data and machine learning for development purposes, developing countries face three main inter-related challenges: 1. Skills: data scientists are in short supply and developing skills to make use of these new data sources become paramount. How should we build these skills? What should be the composition of research teams? 2. Data: accessing private sector data as well as government data can both be challenging. In an imperfect, often inconsistent regulatory environment, how can we facilitate responsible data access and use? 3. Policy impact and mainstreaming: Except in extreme cases most policy domains already have pre-existing established processes for generating and incorporating evidence in policy planning and implementation. How do we disrupt these ‘sticky’ processes with new forms of data and techniques? This talk will address these three sets of challenges and our experiences in tackling them.