DaME: Data Mapping Engine for Financial Services

In modern financial services, data engineers seek access to the end-to-end integrated data flow to take business decisions, which otherwise exists in silos, and hence cannot be used for effective analysis and inferencing. This work introduces a system called Data Mapping Engine (DaME) to realize the full potential across data sources and provide a comprehensive analysis of their relatedness. In the case of financial services, knowing the data mapping between contracts and invoices can provide insights into potential risks. However, based on the client engagements we have observed key challenges in data mapping across various industries. First, there is a Lack of Standardization across multiple organizations within the same industry different entity definitions are used for synonymous terms with similar meanings. Second, Manual Mapping and Maintenance as data engineers manually define mappings between data warehouses, which is prone to errors and counterproductive. Lastly, due to Lack of Governance, over time different departments may digress from the business process that are essential for maintaining and monitoring contract data. Hence, there is a need for a trustful, automated data mapping engine that can connect the different data types across tables using natural language processing and Human-in-the-Loop (HIL).