Day #130 - Chatbot Architecture

Goal - Oriented Bots

• Narrow Domain
• Specific tasks
• Example Call Center
• Model - Retrieval Based
• Use Predefined responses

General Chat bots

• General Conversation
• Generative Models
• For general entertainment
• Generate new responses

Sequence to Sequence

• Incoming message - Encoder
• Decoder for response
• Attention or least reserved input
• Have fixed length to Padding is done

Padding

• EOS - End of Sentence
• PAD - Filler
• GO - Start Encoding
• UNK - Unknown word not in vocabulary

Bucketizing

• Opportunity to avoid padding by bucketizing
• Place them in different batches for RNN
• RNN to keep track of intent of conversations

Cons

• To Dramatic responses
• Based on Domain of data

Intents Clustering

• Graph of different responses
• Labels to cluster them
• Propagate the knowledge to other labels of graph
• Expander library is used for this purpose

Updated - Jule 2022

Interesting Reads - LaMDA: Language Models for Dialog Applications

LaMDA: Language Models for Dialog Applications

Key Notes

• Language Models for Dialog Applications
• Metrics - (sensibleness, specificity, and interestingness)

LaMDA: our breakthrough conversation technology

• BERT and GPT-3, it’s built on Transformer
• Meena, a 2.6 billion parameter end-to-end trained neural conversational mode
• At its heart lies the Evolved Transformer seq2seq architecture, a Transformer architecture discovered by evolutionary neural architecture search to improve perplexity.

Towards a Conversational Agent that Can Chat About…Anything

Happy Learning!!!