by Fatma Tarlaci
https://fatmatarlaci.wordpress.com/2019/05/08/gpt2qa/
This is part of a series on the research exhibited recently by OpenAI Scholars Spring 2019: Final Projects. I will be focusing on research that interests me, as opposed to covering all the research as listed on OpenAI’s website.
Background
Question Answering (QA) is a popular subdomain of Natural Language Processing (NLP). While humans find it easy to link questions to certain contexts, machines have a much harder time as they do not understand natural language as humans do. Many research has been done on factoid based questions- given a passage, find the answer to a fact-based question.
Novel approach
Tarlaci attempted to use GPT-2, OpenAI’s language model, as a pre-trained model for supervised learning for QA. While Tarlaci appears to not have much success thus far with this approach, it is still worth considering the benefits of such an approach. I am not quite sure of Tarlaci’s intuition, but a possible hunch is that by paraphrasing the question or selecting key words in the question, it may be possible to use GPT-2’s language modelling ability to generate multiple possible answers and look for matches in the given text.
As there is insufficient details provided in the blog post, it is hard to look at the details of the architecture that was employed. Nevertheless, it is helpful as a machine learning practitioner to come up with ideas that may be different than intended, in the spirit of trying new things and seeing their results.
If you are more familiar in this field/approach, I am keen to hear from you. Please leave a comment below for discussion.
ML Quant Pok3r 