Date of Award
8-1-2018
Document Type
Masters Thesis
Degree Name
M.S.
Organizational Unit
Daniel Felix Ritchie School of Engineering and Computer Science, Computer Science
First Advisor
Mohammad H. Mahoor, Ph.D.
Second Advisor
Ryan Elmore
Third Advisor
Matthew Rutherford
Keywords
Bi-Encoder, Learning, LSTM, Long short term memory, Machine, RNN, Recurrent neural network, Ubuntu
Abstract
Creating a data-driven model that is trained on a large dataset of unstructured dialogs is a crucial step in developing a Retrieval-based Chatbot systems. This thesis presents a Long Short Term Memory (LSTM) based Recurrent Neural Network architecture that learns unstructured multi-turn dialogs and provides implementation results on the task of selecting the best response from a collection of given responses. Ubuntu Dialog Corpus Version 2 (UDCv2) was used as the corpus for training. Ryan et al. (2015) explored learning models such as TF-IDF (Term Frequency-Inverse Document Frequency), Recurrent Neural Network (RNN) and a Dual Encoder (DE) based on Long Short Term Memory (LSTM) model suitable to learn from the Ubuntu Dialog Corpus Version 1 (UDCv1). We use this same architecture but on UDCv2 as a benchmark and introduce a new LSTM based architecture called the Bi-Encoder LSTM model (BE) that achieves 0.8%, 1.0% and 0.3% higher accuracy for Recall@1, Recall@2 and Recall@5 respectively than the DE model. In contrast to the DE model, the proposed BE model has separate encodings for utterances and responses. The BE model also has a different similarity measure for utterance and response matching than that of the benchmark model. We further explore the BE model by performing various experiments. We also show results on experiments performed by using several similarity functions, model hyper-parameters and word embeddings on the proposed architecture.
Copyright Date
August 2018
Publication Statement
Copyright is held by the author. User is responsible for all copyright compliance.
Rights Holder
Diwanshu Shekhar
Provenance
Received from ProQuest
File Format
application/pdf
Language
en
File Size
87 p.
Recommended Citation
Shekhar, Diwanshu, "A Bi-Encoder LSTM Model for Learning Unstructured Dialogs" (2018). Electronic Theses and Dissertations. 1508.
https://digitalcommons.du.edu/etd/1508
Discipline
Computer science