2023 - Lessons - AI - GenAI

After every year learning extends Data, AI, Products, and Domain. 2023 had a blend of experiences. Still figuring out answers for every dimension #2023 #Learnings

→ How you've adapted to industry shifts, and GenAI's meaningful adoption. Possible use cases vs relevant, meaningful production-ready use cases. Example - Newly launched section in Amazon reviews, What customers say. 

→ How you've overcome engineering challenges balancing business goals. New ways to solve old problems with Foundation models. Time vs building a production-grade solution. Example - Moving away from custom NER vs Leveraging LLM Embeddings, Blend of both custom embedding + RAG, New ways of solving.

→ How your skills align with the company's vision, Learning to predict the future. New approaches and papers evolve faster than certifications. A blend of tech + and domain is key. Segment Anything model, Visual QnA, Intructpix2pix have made more vision use cases feasible Tryon, etc..

→ How you bridge the gap between tech and business, Fast yet impactful use cases, Get the basics right. Demos / New offerings vs making it to production need a careful selection of use cases / applying past experiences to get things right in the first iteration. Balance the tradeoff between creativity vs innovation vs build a product strategy vs solve a real need vs fancy demos. #learning #perspectives #solutions #datascience #MachineLearning #AI #DeepLearning

Keep Exploring!!!

My Learning moments

In the past 8 years Skill is a blend of product + domain + data + AI lens. Learning is more from a customer point of view vs a technology point of view. How well you build your MVP depends on product and domain clarity. With a frugal setup often it means you end up playing product, domain, data, and AI roles. Many projects have to pick up and answer the below questions as part of successful strategy and MVP.

→ How you've adapted to industry shifts, GenAI's meaningful adoption 

→ How you've overcome engineering challenges balancing business goals

→ How your skills align with the company's vision, Learning to predict the future

→ How you bridge the gap between tech and business, Fast yet impactful use cases, Get the basics right

Keep Exploring!!!

Stable Diffusion - Basics

Dataset - LAION-5B (5 billion text-image pairs) 

Dataset from - Pinterest and DeviantArt, e-commerce services like Shopify, cloud services like Amazon Web Services, thumbnails from YouTube, and images from news sites.

CNN vs Diffusion

• CNN – Feature Extraction, Error calculation, Weights update
• Diffusion – Noise Addition in the forward step, Denoising in the second step

Key Steps in Implementation 

• Method of learning to generate new stuff - Forward/reverse diffusion
• Way to link text and images - Text-image representation model, Word as vectors, CLIP
• Way to compress images retain features - Autoencoder - imposes a bottleneck in the network which forces a compressed knowledge representation of the original input
• Priors built into the algorithm, Diffusion for Images – UNet architecture  - U-net architecture + ‘attention’
• ControlNet  - Control diffusion models by adding extra conditions, a "locked" copy, and a "trainable" weights copy

The guide to fine-tuning Stable Diffusion with your own images

From DALL·E to Stable Diffusion: how do text-to-image generation models work?

Keep Exploring!!!

Stable Diffusion Internals

Stable Diffusion Key Steps

• Method of learning to generate new stuff - Forward/reverse diffusion
• Way to link text and images - Text-image representation model
• Way to compress images - Autoencoder
• Way to add in good inductive biases - U-net  architecture + ‘attention’

Build Stable Diffusion “from Scratch”

• Principle of Diffusion models (sampling, learning)
• Diffusion for Images – UNet architecture
• Understanding prompts – Word as vectors, CLIP
• Let words modulate diffusion – Conditional Diffusion, Cross Attention
• Diffusion in latent space – AutoEncoderKL
• Training on Massive Dataset. – LAION 5Billion

GAN

• One shot generation. Fast. 
• Harder to control in one pass. 
• Adversarial min-max objective. Can collapse.

Diffusion

• Multi-iteration generation. Slow.
• Easier to control during generation. 
• Simple objective, no adversary in training. 

Key Ingredients of UNet

• Convolution operation 
• Save parameter, spatial invariant

Down/Up sampling

• Multiscale / Hierarchy 
• Learn modulation at multi scale and multi-abstraction levels.

Skip connection 

• No bottleneck
• Route feature of the same scaledirectly. 
• Cf. AutoEncoder has bottleneck

Autoencoder 

• Autoencoder - impose a bottleneck in the network which forces a compressed knowledge representation of the original input
• An autoencoder can learn non-linear transformations with a non-linear activation function and multiple layers
• An ideal autoencoder will learn descriptive attributes of faces such as skin color, whether or not the person is wearing glasses, etc. in an attempt to describe an observation in some compressed representation.
• For variational autoencoders, the encoder model is sometimes referred to as the recognition model whereas the decoder model is sometimes referred to as the generative model

Applications of Autoencoders

• Image Coloring, Feature variation, Dimensionality,  Reduction, Denoising Image, Watermark Removal

PCA vs Autoencoder

• PCA attempts to discover a lower dimensional hyperplane which describes the original data
• Autoencoders are capable of learning nonlinear manifolds (a manifold is defined in simple terms as a continuous, non-intersecting surface)

ControlNet 

• ControlNet is a neural network structure to control diffusion models by adding extra conditions.
• It copys the weights of neural network blocks into a "locked" copy and a "trainable" copy.
• The "trainable" one learns your condition. The "locked" one preserves your model.

Keep Exploring!!!

Planogram Inspirations

Planogram Inspirations

• 3D detailing (wheels, palms, pineapples and the awning)
• QR code to enter a holiday contest
• Information postcards on the side of the display

• An exercise in beautiful simplicity that creates a valuable solution for the consumer/shopper

• ‘Refill & Save’ communication very effectively

• Great way of communicating value beyond price

• Epic festive display from Lindt in Budgens. The palletised display houses a bountiful array of SKUs and really benefits from the integral LED lighting. First class. 

Ref - Link

Keep Exploring!!!

My 2023 Lessons from three lens

The "Been there, Done that" Opinion

• GenAI Adoption is Real
• AI-Assisted Design is here to stay but the balance of autonomy and assistance is evolving
• Text is promising, Image is Evolving
• SpeedyPrototyping is here to Supercharge your innovation 

Industry Lens

• Domain-specific LLMs and use case adoption in Fintech, Healthcare - BloombergGPT, ClimateBERT, KAI-GPT, ChatLAW, FinGPT, BioMedLM 
• Upcoming new opportunities with MultimodalKnowledge - Images + Text + Data for personalization

Functional Domain Lens

• Retail Seems adopting Fast from field work
• RAG is going to replace lot of FAQ, Support Systems
• Healthcare is showing promising ambitions to adopt AI

Keep Thinking!!

My principles for next independent consulting role

Next roles - AI Advisory

The "BEEN THERE, DONE THAT"

• Consulting across Retail, Fashion, FMCG
• POV - POC - Product Implementation

Industry Expert

• Trends to Focus
• Upcoming new opportunities

Functional Expert

• Bridge Domin + Tech
• Iterative product building

Consulting Skillsets / Focus

• Get better at one thing / Future-oriented learning 
• Partner on complimenting skills
• Expertise is Domain + Data + Tech
• It's ok to stop earning but not stop learning
• Be result oriented not task oriented
• Do not pick something if you don't like to
• Don't bother being an expert of all, Be good at a few
• Build Vision + NLP + Data skills + Strategy skills
• Build meaningful connections and help whenever needed / possible

Teaching

Learn by Domain knowledge - Learn within the context of a specific domain

• I will teach you how to solve with AI - Providing personalized instruction and practical experience.
• Build prototype / Solution options - First tangible step in transforming your ideas into reality
• Tools / Data / Key drivers - Combine Expertise with Data
• Build a production-grade app - Advance your prototype to a production-grade application

Enable them to solve by connecting things they don't know with things they know.

Empower your problem-solving capabilities by learning to connect new concepts with your existing knowledge, fostering innovation and independent thinking

20% Efforts for AI for social good

10% Efforts to contribute back learning

Keep Exploring!!!

Gemini Multimodel - Video to Ads Creation

Downloaded Kia Sonet video from link

Experimented with Google Multimodal Gemini 

These are specs

**Introducing the all-new Kia Sorento:** a mid-size SUV that's perfect for families and adventurers alike. With its rugged good looks, spacious interior, and impressive performance, the Sorento is sure to turn heads wherever you go.

**Here are just a few of the things that make the Sorento so great:**

* A powerful 3.3-liter V6 engine that produces 290 horsepower and 252 lb-ft of torque
* An 8-speed automatic transmission that provides smooth and responsive shifting
* All-wheel drive for superior traction in all weather conditions
* A spacious interior with seating for up to seven passengers
* A long list of standard features, including a panoramic sunroof, heated front seats, and a premium audio system

**If you're looking for a mid-size SUV that offers the perfect blend of comfort, performance, and style, the Kia Sorento is the perfect choice for you.**

**Visit your local Kia dealer today to learn more about the all-new Sorento and take it for a test drive.**

Seems very much a lot of own creativity :) 

Way to go, Good start but long way to goto production, Multiple human in loop iterations

Key Points

“AI-ASSISTED DESIGN CAN SAVE ONE-THIRD OF A DESIGNER’S EFFORT”

A BALANCE OF AUTONOMY AND ASSISTANCE IS KEY TO KEEPING HUMAN AGENCY WHILE BENEFITING FROM THE SPEED AND EFFICIENCY OF AI

Recommendation systems offer more of the same, exploiting what is already known, but good design needs exploration

Keep Exploring!!!

Reflecting back on Consulting Projects - Product - Program - Architect - Solution

• In the past 8 years, independently navigated 80% of projects without product managers, leading from concept to production.
• Managed plans and schedules for a 3-5 member AI/ML team, showcasing leadership in streamlined project execution.
• Specialized in strategy and vision realization across multiple domains, with practical experience in point-of-view (POV) framework and minimum viable product (MVP) execution.
• Successfully developed and deployed innovative products, reflecting a record of tangible achievements in product development.

The role in a few words is Product and Innovation Architect with a focus on Applied AI, reflecting expertise in driving technology solutions and product innovation.

Keep Exploring!!!

Startup vs Big Companies

In a startup, everyone works toward a shared goal using the best web, data, AI, and APIs, focusing on coding a cohesive user journey. Compared to big companies, where five separate teams with mixed abilities might work on overlapping projects, a startup operates with 'X' effort, whereas a large company expends '5X' with the risk of internal competition leading to inefficiencies. By the time the larger company overcomes these challenges, the startup likely has rolled out a robust set of production features.

Rate:

• Efficiency and focus: Startups - 8, Big Companies - 4
• Resource utilization: Startups - 9, Big Companies - 3

The ratings reflect the agility and clear focus of startups against the potential for redundancy and internal competition at large corporations.

Why a great product needs alignment at all levels is required

Keep Exploring!!!

Coding before / After ChatGPT

Coding before ChatGPT:

• Read instructional blogs and official documents.
• Select a relevant helpful tutorial post.
• Set up coding environment needed.
• Execute available blog example code.
• Search StackOverflow for bug solutions.
• Locate and use functioning examples.

Coding after ChatGPT:

• Ask specific coding-related questions.
• Receive comprehensive, detailed explanations.
• Arrange and configure necessary infrastructure.
• Implement and test example code.
• Query errors, and receive alternative solutions.

Observations with ChatGPT:

• Google searches for information are reduced.
• StackOverflow usage noticeably decreased.
• Overall, I experienced time savings.

Keep Exploring!!!

Streamlit + Docker + App Engine + IAP = Production Grade App

GCP App Engine

• Fully Managed Platform
• Deployment, Scaling, yaml based

Streamlit

• Opensource framework
• User interface for interactive AI / ML web Applications
• Easier for Data processing / Visualization

Docker

• Containerize Apps
• Build - Test - Dockerize - Test - Push - Deploy

Google Identity Aware Proxy

• Authenticated users gain access
• Control over App Deployed
• Google Identity and access management services

Keep Learning!!!

Valuation Basics

Key Lessons

• Focus needs to change by age
• Values need to change by age
• Consultants - Plastic Surgeons
• Growth assets - Contribute to growth
• Startup - Peak of Glory - Mature Growth - Google / FB
• Middle Ages - Decline - Nature of process
• How little capital for New Businesses
• Compressed Lifecycle
• Nokia, Yahoo, Blackberry - Short life cycles
• Valuation - Outside inn
• Finance - Inside out view
• More degrees of free give more experimentation
• Identify good projects - Fund them well
• Companies refuse to act their age

The Laws of Valuation

From "The Dean of Valuation" Aswath Damodaran

He reveals the fundamentals, myths & misconceptions: pic.twitter.com/nV1iR5iCtv

— Value Theory (@ValueInvestorAc) December 10, 2023

Keep Learning!!!! 

AI use case development for 2025

 AI use case development for 2025

Genetics

• Molecular modelling in drug discovery
• Enhanced chatbots for employee and customer interaction
• Accelerating clinical trials

Retail

• Micro-fulfillment centers powered by AI and robotics
• More precise prediction of inventory needs using analysis of omnichannel transaction data
• Expanded product personalization for omnichannel experiences
• Optimizing promotions and markdown
• Personalization of customer experience
• Next-gen retailing platform, including offers and dynamic pricing

Energy

• Climate change: Optimizing energy and water consumption in manufacturing

Insurance

• Reducing risk in claims assessment relating to natural disasters 
• Refining underwriting through monitoring and analysis of driver behavior

Automotive

• Prognostics: predicting failure of engine parts to streamline service and reduce warranty costs
• Improving product design and engineering

AI + SaaS

• SaaS was the play of the internet era
• SaaS innovated the distribution of software and pricing model
• AI-SaaS could be the next golden age
• All major lucrative enterprise SaaS horizontal opportunities are taken - Salesforce, Workday, Netsuite, Zendesk, ServiceNow
• AI SaaS is the good way to go forward for startup founders - AI marks a pivotal shift, distinctly different from SaaS
• Owning data will be key to creating fine-tuned task specific models for enterprise use cases
• The innovation seems to have hit escape velocity but needs more stability at the bottom layers

Keep Exploring!!!

Flooded car repair options

I had to encounter total flooding. The repair process is more like "Replace All" not "Fix what is repaired"

What perplexes me the most is the hefty repair bills quantified in figures such as 8L and 9L. I had fuel injector issues and I was informed due to fuel contamination. I had serviced/repaired whatever best possible option. 

Based on whatever I learned/understood

• Dismantle parts
• Dry up / Clean up
• Remove any rusts and apply WD40
• Replace Engine, Coolant oils
• Replace Spark plugs
• Strip whatever was possible. The seats, floor trim and finally the carpet.
• Clean up the circuit boards thoroughly by iso propyl alcohol the damages will be minimized. 
• Circuit repair guys help if needed
• Blew hot air to all nooks and crannies which was not accessible by hand

Some electrical parts may need effort due to flooding and water stagnation

• ECM
• Gearbox
• Electric panel
• Wiring
• Sparkplugs

Problem

• In reality, not all cars would have suffered the same extent of damage that they slap a predetermined estimate to owners
• Since every part of the car should be touched, it will take too much time, whereas these days service just means replacing whatever is possible and getting the job done within no time
• If we spend some time to gauge the extent of the damage, then it will give a better picture of the whole scene.

Not sure

• Injector
• Gearbox changes
• Airbags

We subject our vehicles to a wide range of conditions, from potholes to torrential rain. All cars, whether new or old, are susceptible to issues. Somehow did not feel this answer was reasonable not well thought through :)

Patience and knowledge would be key.

Keep Exploring!!!

Good Read - Kubernetes - Basics - Connect the Dots

Basic Kubernetes building blocks.

• Containerization (like Docker) - Dockerization, Runtime, Commands, Code - Build Image
• Cloud Basics - Managed solution kubernetes cluster, Identity / Access. Cloud managed solution. Monitor / rebalance pods. Spot instance automation. Optimization features
• YAML - Write config files, Resources in Kubernetes, Desired state of kubernetes deployment object
• Networking Basics - pods communication, clusterip, node ports
• Linux - Command line kubectl tools proficiency, vi

CAST AI is the leading all-in-one platform for Kubernetes automation, optimization, security, and cost management. 

Keep Exploring!!!

Simplifying Neural Network Training Under Class Imbalance

Simplifying Neural Network Training Under Class Imbalance

• Small batch size - Class-imbalanced - settings, where small batch sizes shine.
• Data augmentations have an amplified impact on performance under class imbalance, especially on minority-class accuracy
• Adding a self-supervised loss during training can improve feature representations
• Label smoothing, especially on minority class examples, helps prevent overfitting. We adapt label smoothing for the class-imbalanced setting by applying more smoothing to minorityclass examples than to majority-class  examples
• A small modification of Sharpness-Aware Minimization (SAM) pulls decision boundaries away from minority samples and significantly improves minority-group accuracy
• Loss reweighting. Reweighting methods assign different weights to majority and minority class loss functions, increasing the influence of minority samples which would otherwise play little role in the loss function

Label smoothing is a technique often used in training deep learning models, particularly for classification tasks. It modifies the target labels, making them a blend of the original hard labels and some uniform or prior distribution. This can lead to better generalization by preventing the model from becoming too confident about its predictions. In a class-imbalanced setting, where some classes have significantly more examples than others, label smoothing can help by reducing the model's bias towards the more frequent classes.

Label smoothing for the class-imbalanced setting python example

In practice, label smoothing does not change the dataset's inherent imbalance but softens the target distributions by moving a portion of the mass from the peak (corresponding to the hard label) to other classes, which can help during the training of a model, preventing it from becoming overly confident on the majority class.

Loss reweighting for the class-imbalanced setting python example

class_weight: Optional dictionary mapping class indices (integers) to a weight (float) value, used for weighting the loss function (during training only).

Let's import the module first

from sklearn.utils import class_weight

In order to calculate the class weight do the following

class_weights = class_weight.compute_class_weight('balanced', np.unique(y_train), y_train)

Thirdly and lastly add it to the model fitting

model.fit(X_train, y_train, class_weight=class_weights)

Keep Exploring!!!

Work vs Satisfaction

Customer Work vs POC Work

The happy moments,

• It took me 4 years to rewrite the warranty engine and make an impact that saved 1 billion warranty extension
• Data migration and improving performance by 100%
• Meeting bundle recommendations 100%

I am not happy with POC work, satisfaction comes from product work, Product work takes time.

Keep Exploring!!!

Transformer Notes

• CNN - all the outputs are self dependent 
• CNN/FF-Nets all the outputs are self dependent Feed-forward nets don’t remember historic input data
• RNN - Hidden state memory, correlation between previous input to the next input, Cell state, Forget Gate
• RNN - learn to keep only relevant information to make predictions and forget non relevant data RNN - Conveyer belt
• RNN Perform well when the input data is interdependent in a sequential pattern correlation between previous input to the next input introduce bias based on your previous output

Transformer

• Positional embeddings - the order and position of words in a sequence 
• Self attention - allows each token to dynamically weigh and integrate information from all other positions 
• The self-attention mechanism is a type of attention mechanism which allows every element of a sequence to interact with every others and find out who they should pay more attention to.
• Multi-head attention runs multiple self-attention processes in parallel, capturing diverse aspects of the data

Keep Exploring!!!

2024 Predictions - LLM Series - Domain-Specific GPT, Multimodal World, SpeedyPrototyping, ResponsibleAI

2024, More products / Innovation with Improved Benchmarks for Domain-Specific GPT, Multimodal World, SpeedyPrototyping, ResponsibleAI and Privacy assured personalized LLM recommendations for Mental Health, Healthcare Tips

1. Improved Benchmarks from Domain-Specific GPT - Go niche with a Domain-Specific GPT. Bring your data, and domain knowledge and Build your LLM. Why settle for generic when you can bring your data and craft a model that knows your field inside out? More benchmarks and newer models both custom / open source across domains #CustomizedAI #CustomGPT

2. #Personalized answers with privacy - Welcome Personal Data + LLM for mental health, healthcare, and suggestions. Your privacy remains intact while you enjoy tailor-made advice crafted just for you. #PersonalizedAdvice #PrivacyMatters.

3. Integrated Multimodal World - It's a #Multimodal World! Knowledge isn't just text; it's Images + Text + Data. Embrace the power of combined data forms to receive enriched, multimodal insights that tell the complete story for recommendations, personalization. #MultimodalKnowledge #HolisticAI

4. #SpeedyPrototyping - Supercharge your innovation engine with Ideas an ensemble of Models. Watch your prototyping speed take off as diverse AI models converge to refine your visions faster than ever! #SpeedyPrototyping #AIEnsemble

5. #ResponsibleAI Model building and solutioning - It's not just about building models; it's about building them right—robust, reliable, and fair. For all infrastructure providers, LLM training vendors, evaluators, and advocates for responsible AI, your insights are invaluable. #AIEthics #ResponsibleAI #GenAI #LLM #Domains #2024 #Predictions #perspectives

Domain Specific GPT

• BloombergGPT
• ClimateBERT
• KAI-GPT
• ChatLAW
• FinGPT
• BioMedLM

Ref - Link1, Link2

Custom Training Models

• Customizing GPT Models for Domain-Specific NLP Tasks: Techniques for Fine-tuning and Adaptation
• Training Stable Diffusion from Scratch Costs <$160k
• Training Stable Diffusion from Scratch for <$50k with MosaicML (Part 2)
• How We Trained Stable Diffusion for Less than $50k (Part 3)
• HOW TO TRAIN AN OPEN-SOURCE FOUNDATION MODEL INTO A DOMAIN-SPECIFIC LLM?

The future is Intelligence as a Service (Commodity)

• LLM OS
• LLM Super App
• LLM based content creation tools

Keep Exploring!!!

GCP Experiments

• Retail interactive tutorials
• Vertex Pipeline
• MinkKF
• KF Pipeline
• MiniKF Kale

Keep Exploring!!!