`

Timezone: »

 
Contributed Talk 5: NUBIA: NeUral Based Interchangeability Assessor for Text Generation
Hassan Kane

Mon Dec 07 10:05 AM -- 10:15 AM (PST) @ None

We present NUBIA, a methodology to build automatic evaluation metrics for text generation using only machine learning models as core components. A typical NUBIA model is composed of three modules: a neural feature extractor, an aggregator and a calibrator. We demonstrate an implementation of NUBIA which outperforms metrics currently used to evaluate machine translation, summaries and slightly exceeds/matches state of the art metrics on correlation with human judgement on the WMT segment-level Direct Assessment task, sentence-level ranking and image captioning evaluation. The model implemented is modular, explainable and set to continuously improve over time.

Author Information

Hassan Kane (General Motors)