Date of Award

8-1-2018

Document Type

Masters Thesis

Degree Name

M.S.

Organizational Unit

Daniel Felix Ritchie School of Engineering and Computer Science, Computer Science

First Advisor

Mohammad H. Mahoor, Ph.D.

Second Advisor

Ryan Elmore

Third Advisor

Matthew Rutherford

Keywords

Bi-Encoder, Learning, LSTM, Long short term memory, Machine, RNN, Recurrent neural network, Ubuntu

Abstract

Creating a data-driven model that is trained on a large dataset of unstructured dialogs is a crucial step in developing a Retrieval-based Chatbot systems. This thesis presents a Long Short Term Memory (LSTM) based Recurrent Neural Network architecture that learns unstructured multi-turn dialogs and provides implementation results on the task of selecting the best response from a collection of given responses. Ubuntu Dialog Corpus Version 2 (UDCv2) was used as the corpus for training. Ryan et al. (2015) explored learning models such as TF-IDF (Term Frequency-Inverse Document Frequency), Recurrent Neural Network (RNN) and a Dual Encoder (DE) based on Long Short Term Memory (LSTM) model suitable to learn from the Ubuntu Dialog Corpus Version 1 (UDCv1). We use this same architecture but on UDCv2 as a benchmark and introduce a new LSTM based architecture called the Bi-Encoder LSTM model (BE) that achieves 0.8%, 1.0% and 0.3% higher accuracy for Recall@1, Recall@2 and Recall@5 respectively than the DE model. In contrast to the DE model, the proposed BE model has separate encodings for utterances and responses. The BE model also has a different similarity measure for utterance and response matching than that of the benchmark model. We further explore the BE model by performing various experiments. We also show results on experiments performed by using several similarity functions, model hyper-parameters and word embeddings on the proposed architecture.

Publication Statement

Copyright is held by the author. User is responsible for all copyright compliance.

Rights Holder

Diwanshu Shekhar

Provenance

Received from ProQuest

File Format

application/pdf

Language

en

File Size

87 p.

Discipline

Computer science

Download



Share

COinS