Klara Chatbot - Devil in Details

• It recites exact docs and passes me on to human support fast.
• Good job on the team for making hallucination not possible - because it seems to spit out the same responses however I ask it, and refuses to go “out of bounds.”
• As soon as I ask or instruct anything that is not a doc, I’m *boom* talking with a human agent.
• Also, almost all questions I ask about payment terms or problems the chatbot tells me - in various ways - to talk to the merchant if I have a problem, not to Klarna
• These assistants turn docs into chat text, that people read!
• Klarna is a middleman. The customer buys from the merchant and Klarna sells defaulted payments to collections agencies!!
• Klarna wants potential investors to believe they are buying into an “AI edge” company

Ref - Link

Keep Exploring!!!

Custom Chatbot vs OpenAI Chatbot

Build vs Develop on LLM

Custom Chatbot

Time- Data collection, labeling, classification, NER models

Build

• Preprocessing
• Lower case
• Stemming, Lemmatization with POS Tags
• Entities, NER

Inference

• Intent
• Topic Classification
• Frame a response

Context - Limited to corpus

LLM Chatbot

Time - Prompts / Responses / Store / Retrieve

OpenAI

• Prompt
• Inference answers

Challenge

• Air Canada hallucinations use cases
• Context vs Hallucination

Keep Exploring!!!

Chatbot Lessons

Interesting Read

GenAI Customer Care Project Now Saving $4M/yr for Car Insurance Firm, Jerry

Key Lessons

• Leverage Large Language Models (LLMs) to create a chatbot 
• Open AI GPT-4 for complex queries
• Open AI GPT-3.5 for initial sorting
• Dataset - Messages are captured from chat and SMS through Twilio and stored on Jerry's servers
• Routing Agent - Route based on requests - "Payments," "Policy," and "Opt-out" agents
• Webhook / API - Handlebars to insert data from their database into the chatbot's responses

Key Learning's

• Prompt engineering
• Rapid iteration
• Investment in testing
• Version control

Webhooks in Python with Flask

Webhook

How to Create a Webhook in Python

Python Flask Webhook Receiver

Drawback of LLM

• Exposes your corporate data to the provider of your LLM 
• LLMs have been shown to suffer from “hallucinations

Quick Lessons

• FAQ from Cache
• VectorDB for queries where answers can be located in docs
• Intent recognition and call APIs based on OrderNumber#
• Knowledge Graph if we have some meta data loaded
• If all attempt fails LLM answer
• Add guard rails wherever possible

Ref - Link

Dialogflow Notes Link

Keep Exploring!!!!

Voice Powered SQL Assistant

SQLBot - I am your Query Assistant what do you want me to do?
User - I want a query to join few tables

SQLBot - Tell the tables
User - Employee, Payment, JobDetails tables

SQLBot - Based on my analysis these are join columns EmployeeId for Employee-JobDetails, JobId for Job and Payment Table
User - Give me the query

SQLBot - There are four indexes available which indexes do you want me to use, any inputs
User - Give best possible query

SQLBot - I tried this query on 10K records it took 2.3 seconds, Is it fine? Do you want me to populate for 100K and try again?
User - I will do it in next sprint, Until then this is fine

SQLBot - Thank you, Small Stats - Other uses who worked on this similar query spent 40% more time analyzing than how you spent time
User - Time to go, bye, Check-in the code 

Use Technology to add value on top of human intelligence :)

Happy Learning!!!