Answering Open domain questions by respecting Power Law artifacts

Abstract

It has been an evolution in the last few years of machine learning methods, in which a part of this family is the deep learning models. Neural Attention (NA) is the most recent field in this area, which various methods are implemented or still in progress. One of many models that are based on NA, is sequence-to-sequence (seq2seq); an architecture of Natural Language Processing, used mainly to process data in text format and uncover useful insights. In this paper, we will focus on NA and show what are the challenges with it. We aim to examine a Question Answer (QA) model, whether addressing Power Law artifacts could facilitate modern performance; this is a plausible hypothesis, since we know that language understanding does exhibit such behavior. Also, we examine the performance of the resulting model using the benchmarks datasets.