Building LLM Models

Notes from article 

• Open-source tools like LangChain and LlamaIndex are current tools for developers to build apps that rely on external services and sources

The tools part is significant

• Tools - LlamaIndex, LangChain
• Vector databases - Pinecode, Milvus 
• Milvus - A cloud-native vector database, storage for next generation AI applications (by milvus-io)

Experiments to do

• Run Milvus using Python
• llama-llm
• Marketing Copy Generation [Public Documentation] - llama-llm
• Zilliz Cloud managed vector database.
• Langchain - Question Answering over Docs
• Langchain Alternatives

• gpt_index
• llama_index
• llama
• visual-chatgpt

Recent Models -  BloombergGPT, an LLM for finance.

Key Training Process

Reinforcement Learning with Human Feedback

2 - Training Process - Ref Link

The LLM Three-Pronged Strategy: Tune, Prompt, Reward

Cerebras-GPT: A Family of Open, Compute-efficient, Large Language Models

Knowledge Graph - provide an interface for querying the index and obtaining knowledge-augmented output

Ref - Link1, Link2

RLHF helps to build state-of-the-art models like ChatGPT.

Did you know that training RLHF LLMs involve 4 key steps?

Here’s an illustrated guide to the process:

1 of 5

— Surge AI (@HelloSurgeAI) March 27, 2023

xTuring is an open-source AI personalization software.

Koala: A Dialogue Model for Academic Research

Eight Things to Know about Large Language Models

Ref - Link

Keep Exploring!!!

Paper Read - Gradient Descent: The Ultimate Optimizer

Gradient Descent: The Ultimate Optimizer

• When we train deep neural networks by gradient descent, we have to select a step size α for our optimizer
• If α is too small, the optimizer runs very slowly, whereas if α is too large, the optimizer fails to converge
• First, recall the standard weight update rule at step i for SGD, using some fixed step size α:

• Update rule for SGD is simply a multiplication by a constant, whose derivative is trivial.
• Adam optimizer which has a much more sophisticated update rule involving the four hyperparameters α, β1, β2, e (though e| is typically not tuned).

• AdaGrad/AdaGrad hyperoptimizer increases α to make up for this effect. 

Keep Exploring!!!

Storytelling Tips

Tips

• SOS - Sense, Observe, Store
• RSI - Real, Specific, Incidents
• Stories of Self
• Stories of Outsiders

Perspectives

• Connect with person
• Then build your story
• Build your brand / Narration
• The hero of the story is the audience
• The story connects more with data

Personality

• Self-awareness / Brand Building
• Self-awareness
• Key takeaway - Two styles 

Aristotle Principles

• Ethos - Credibility
• Pathos - Relate
• Logos - Data

Keep Learning!!!

Advanced Cloud Computing 2013 Notes

Advanced Cloud Computing 2013 Notes. Yesterday I attended ACC2013 held @ Nimhans. Every conference provides a lot of inspiration and motivation to try out new things.

Session #1 - Inauguration Talk

Inauguration was done by Padma Bushan  Rajaram. He explained cloud computing in very simple terms. Cloud computing is utility computing in simple terms. It was coined by a management professor named Chellapa.

He had earlier written a paper in 2005 on challenges in utility computing. He recollected his experiences with new jargon's/ technical abbreviations. He provided several examples on BYOD keyword. BYOD - Bring Your Own Drinks, BYOD - Bring Your Own Dope, BYOD - Bring Your Own Device (Recent usage). Storage and processing costs have come down. This has become a business potential for Amazon and Google to leverage it by outsourcing their excess storage and processing infrastructure.

He stressed on several areas to standardize the cloud for leverage the complete potential of it. Example- SLA to provide the required performance while hosting / sharing the infrastructure, Developing a universally usable cloud, interoperability between cloud providers

Session #2 - This was Talk by Karanataka's IT secretary VidyaShankar.IAS

He mentioned on developing trends Virtualization, Cloud and 3D Technologies. He mentioned
couple of products cloudmagic, cubby.

Session #3 - Connected Systems, Cloud beginner tech talk by Vikas Agarwal (Tally)

This talk pretty much focused on evolution of cloud computing. He tracked from the very beginning PC Era to cloud computing.

•  Stage 1 - Mainframe Systems
•  Stage 2 - PC Era (Moving data to personal systems)
•  Stage 3 - LAN (Locally connected systems) - Intranets
•  Stage 4 - Connected Era, WANs
•  Stage 4 - Evolution of Internet (Globally Connected)
•  Stage 5 - Cloud (Shared computing, storage) - Access anytime / anywhere

Challenges / Features in Cloud

•  Pooling Optimization
•  Elasticity
•  Efficiency

Session #4 - Big Data in Safety & Security Domain, Tech talk by Bob Brewin (Tyco)

This talk focused on basics of cloud computing, challenges and applications in Fire and Security Domain. Key notes covered were

• Fallacies in Distributed computing
• Current Challenges in Fire & Security Domain are
• Identifying False Positives
• Predictive Analytics to identify and isolate false positives
• Real time monitoring

Session #5 - Cloud Services in Yahoo by Jothi Padmanabhan

Yahoo has its own private cloud, Author provided details on Yahoo infrastructure and their software stack
 Challenges

• Scaling systems as per growing data
• Data Partitioning
• Data Consistency
• Hardware provisioning

Benefits of Private Cloud

• Developers can focus on Application logic instead of designing for crash / recovery scenarios
• Focus on appealing content for users (UX Exp)

Requirements for Cloud

Multitenancy

• Several applications will share the same hardware and software
• Resources can be shared but there should not be performance conflict between resources
• Multiple Apps will be running in parallel
• Spike in resource consumption of one app should not affect other application's performance
• SLA defined for performance need to be met for all hosted apps

Elasticity

• Applications will have projected capacity vs actual capacity
• Based on a ball park figure but actual load will be measure when the product is implemented
• Scale as you need

Scalable

• Process several requests, Store Huge data, Analytics on top of data are offerings

Other key aspects include Availability, Security, Metering, Global APIs, Load Balancing, Simple API's

More Detailed Architecture is explained in paper link

• Overview of Open Stack
• Apache Traffic Server used as caching proxy server
• Proxy (Route Traffic through intermediate steps)
• Reverse proxy vs Forward Proxy (Several Variations)
• Yahoo has 25K Clusters and 40K Servers
• Mobstor (Storage for large unstructured files)
• Sherpa (NOSQL solution from Yahoo)

Happy Learning!!!

Day1 - pyCon2013 Notes

Hello World!!

After a long time updating my notes, learning’s in this post.

Some good one liner’s from Pycon2013

• WLT – We Love Typing
• DRY – Don’t Repeat Yourself

Today I attended Pycon 2013. This was my first python focussed learning session. Sessions were pretty good. Summarizing my notes of today's session

Session #1 - The first session was log management in inmobi 

• They have a framework built to store, manage all kinds of logs (Web Server Logs, DB Logs, Application logs, Cron Tab Logs) 
• Three steps were followed in collecting logs
• Collection of Logs – Apply Patterns – Fetch Logs
• Tools used include logstash, grok (ships with parse patterns) - http://logstash.net/ 

Author also provided other alternatives and tools for implementation

• Transport – flume, rsyslog, conduit, scribe
• Search and Analytics – hadoop, graylogz, elsa
• Storage – HDFS, Cassandra, Elastic Search
• Apache Falcon

Jordan Sissel is the brain behind logstash. His video session is also available in link

Session #2 – Django Framework beyond Basics

It was a well presented session. Although this was my first session on Django I could get the feel of the framework. Presenter Arun (http://Arunrocks.com)

Django – Web Framework. This is built on principles of Rapid App Development. How a typical web request is handled in Django framework 

• Models – Matter
• Views – Thinkers
• Templates – Should be dumb

Good Things about Django Framework

• Good Admin Interface
• Security
• Great Documentation
• Stable
• Batteries Included

Basic Definitions of Query Set (Object that interfaces with DB), Media Separate etc. Python implements class based views. Earlier it was function based generic views

Tools Discussed

• IDE – Pychan, PyDev, Emacs, Vi, Sublimetext
• Deployment Tools – Chef, Puppet, fabric
• Security Checks – ponycheckup.com for Django based sites
• ORM – SQLAlchemy

Session #3 – Third Session was Rapid development & integration of Real Time Communication in Websites

The session as pretty interactive. Demonstrating realtime video chat using google webrtc. https://github.com/cjgiridhar

Demonstrated the setup

• End User Requests sent to Tornado Web Server
• Using Web Socket / Ajax communication happens (video chat / live chat)
• Chat messages saved in Redis database

Tools

• Webserver – flask, bottle, tornado
• Javascript and webRTC

Chetan’s website - link

More Links - pythontutor

Happy Learning!!!