Date of Award
8-1-2018
Document Type
Thesis
Degree Name
M.S.
Department
Computer Science
First Advisor
Mohammad H. Mahoor, Ph.D.
Second Advisor
Ryan Elmore
Third Advisor
Matthew Rutherford
Keywords
Bi-Encoder, Learning, LSTM, Long short term memory, Machine, RNN, Recurrent neural network, Ubuntu
Abstract
Creating a data-driven model that is trained on a large dataset of unstructured dialogs is a crucial step in developing a Retrieval-based Chatbot systems. This thesis presents a Long Short Term Memory (LSTM) based Recurrent Neural Network architecture that learns unstructured multi-turn dialogs and provides implementation results on the task of selecting the best response from a collection of given responses. Ubuntu Dialog Corpus Version 2 (UDCv2) was used as the corpus for training. Ryan et al. (2015) explored learning models such as TF-IDF (Term Frequency-Inverse Document Frequency), Recurrent Neural Network (RNN) and a Dual Encoder (DE) based on Long Short Term Memory (LSTM) model suitable to learn from the Ubuntu Dialog Corpus Version 1 (UDCv1). We use this same architecture but on UDCv2 as a benchmark and introduce a new LSTM based architecture called the Bi-Encoder LSTM model (BE) that achieves 0.8%, 1.0% and 0.3% higher accuracy for Recall@1, Recall@2 and Recall@5 respectively than the DE model. In contrast to the DE model, the proposed BE model has separate encodings for utterances and responses. The BE model also has a different similarity measure for utterance and response matching than that of the benchmark model. We further explore the BE model by performing various experiments. We also show results on experiments performed by using several similarity functions, model hyper-parameters and word embeddings on the proposed architecture.
Publication Statement
Copyright is held by the author. User is responsible for all copyright compliance.
Recommended Citation
Shekhar, Diwanshu, "A Bi-Encoder LSTM Model for Learning Unstructured Dialogs" (2018). Electronic Theses and Dissertations. 1508.
https://digitalcommons.du.edu/etd/1508
Provenance
Received from ProQuest
Rights holder
Diwanshu Shekhar
File size
87 p.
Copyright date
2018
File format
application/pdf
Language
en
Discipline
Computer science