Data Science, Database, AI Startups and Domain Learning's (Video-Image-Text-Data-Database)

November 12, 2023

Vector database - Research Paper Read

Vector database

Key Summary

Support for high dimensionality and sparsity
Describe rich data such as texts, images and video in various domains such as recommender systems, similarity search, and chatbots.

Vector data is in geospatial applications
Two dimensional points such as the location of the end-user and points-of-interest may be represented as vectors
High-dimensional vectors can be used to represent more complex data such as text, image, audio and video features

VDBMSs typically support similarity search through indexing methods that enable rapid and accurate searching of similar vectors
Search for vectors that closely resemble a given query vector based on specific distance metrics such as Euclidean distance or cosine similarity.

In natural language processing, words and phrases are vectorized into vectors in such a way that similar words have similar vector representations.
Word2vec [7], FastText, and Doc2vec [8] are examples of techniques that create vector embeddings for words in natural language
From a developer perspective, queries in VDBMSs are more closely related to simple document or keyvalue store queries than to complex queries in relational databases
Vectors are retrieved using one or several query vectors

Use-cases

Similarity search in general
Image and video similarity search
Voice recognition
Chatbots and long-term memory

Current challenges

Balancing between speed and accuracy
Growing dimensionality and sparsity
Information security

Keep Exploring!!!

Vertex AI Vision

Some key steps to experiment in coming weeks. This low-code vision platform has been in my to-do list. Bookmarking some references

Stream registration

Open the Streams tab of the Vertex AI Vision dashboard.

Go to the Streams tab
Click addRegister.
Enter the stream name and select a region. You can click Add Row to register multiple streams at the same time.
Click the Register button to create one or more streams.

# This command streams a video file to a stream. Streaming ends when the video ends.

vaictl -p PROJECT_ID \

-l LOCATION_ID \

-c application-cluster-0 \

--service-endpoint visionai.googleapis.com \

send video-file to streams STREAM_ID --file-path LOCAL_FILE.EXT

# This command streams a video file to a stream. Video is looped into the stream until you stop the command.

vaictl -p PROJECT_ID \

-l LOCATION_ID \

-c application-cluster-0 \

--service-endpoint visionai.googleapis.com \

send video-file to streams STREAM_ID --file-path LOCAL_FILE.EXT --loop

Traffic Monitoring Solution

export SOURCE=gs://cloud-samples-data/vertex-ai-vision/street_vehicles_people.mp4
gsutil cp $SOURCE .

export PROJECT_ID=<Your Google Cloud project ID>
export LOCATION_ID=us-central1
export LOCAL_FILE=street_vehicles_people.mp4

nohup vaictl -p $PROJECT_ID \
-l $LOCATION_ID \
-c application-cluster-0 \
--service-endpoint visionai.googleapis.com \
send video-file to streams 'traffic-stream' --file-path $LOCAL_FILE --loop &

Vertex AI Vision Queue Detection App

Vertex Vision AI

Keep Exploring!!!

November 09, 2023

Good Read - Things I've learned as a Sr Engineer

Things I've learned as a Sr Engineer

Key things I liked

All fields have about 10-20 core principles
Be authentic at work.
Good code is code that can be understood by a junior engineer. Great code can be understood by a first-year CS freshman. The best code is no code at all.
Writing good proposals for changes is a great skill
We should hire more interns, they're awesome
What did you do and what did you accomplish. That's all people care about
Be kind to everyone. Not because it'll help your career (it will), but because being kind is rewarding by itself.
Being a good engineer means knowing best practices. Being a senior engineer means knowing when to break best practices.

Learning vs Doing vs Influencing

Walk me through a project: what you think, what you ask, what you do, what tools you use and WHY! What you don’t use and WHY? That’s more valuable for me.
Make me think! Hands-on also needs to be brains-on
Working with humans is a complex process. We are not logical creatures.

5 Coding Interview Tips From A Senior Engineer

Capacity of criticism
Opinions and reasons to back them up
Willing to learn outside the field
Hire for attitude, teachable people

Product delivery framework

Eliminate repetitive strain of sprint planning
Scheduled regular check-ins
More time to delve into customer issues and develop well-defined proposals

Keep Exploring!!!

November 08, 2023

Why Legacy companies Fail to Innovate :)

Well summarized - Vision needs implementation strategy

#Tesla - build one #product with a #vision

Legacy companies - Modules developed by component providers without integration/vision

Lesson - Write Software yourself for 2.0 products

Keep Exploring!!!

November 07, 2023

REST APIs

microservices showcasing Kubernetes, Istio, and gRPC.

Code Review Checklist

The rules are:

𝟭. 𝗔𝘃𝗼𝗶𝗱 𝗖𝗼𝗺𝗽𝗹𝗲𝘅 𝗙𝗹𝗼𝘄: Steer clear of tricky control structures; stick to simple loops and conditionals.

𝟮. 𝗕𝗼𝘂𝗻𝗱 𝗟𝗼𝗼𝗽𝘀: Ensure loops have a clear exit point to prevent endless looping.

𝟯. 𝗔𝘃𝗼𝗶𝗱 𝗛𝗲𝗮𝗽 𝗔𝗹𝗹𝗼𝗰𝗮𝘁𝗶𝗼𝗻: Favor stack or static memory allocation to dodge memory leaks.

𝟰. 𝗨𝘀𝗲 𝗦𝗵𝗼𝗿𝘁 𝗙𝘂𝗻𝗰𝘁𝗶𝗼𝗻𝘀: Keep functions concise, handling a single task. This goes along well with Clean Code practices (Signe responsibility principle).

𝟱. 𝗥𝘂𝗻𝘁𝗶𝗺𝗲 𝗔𝘀𝘀𝗲𝗿𝘁𝗶𝗼𝗻𝘀: Utilize assertions to catch unexpected conditions.

𝟲. 𝗟𝗶𝗺𝗶𝘁𝗲𝗱 𝗗𝗮𝘁𝗮 𝗦𝗰𝗼𝗽𝗲: Keep the scope narrow to maintain clarity. Use the smallest scope for your variables (e.g., private or protected in C#).

𝟳. 𝗖𝗵𝗲𝗰𝗸 𝗥𝗲𝘁𝘂𝗿𝗻 𝗩𝗮𝗹𝘂𝗲𝘀: Always check the return values of functions, handling any errors.

𝟴. 𝗦𝗽𝗮𝗿𝘀𝗲 𝗣𝗿𝗲𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗼𝗿 𝗨𝘀𝗲: Minimize preprocessor directives for readability.

𝟵. 𝗟𝗶𝗺𝗶𝘁 𝗣𝗼𝗶𝗻𝘁𝗲𝗿 𝗨𝘀𝗲: Simplify pointer use and avoid function pointers for clearer code.

𝟭𝟬. 𝗖𝗼𝗺𝗽𝗶𝗹𝗲 𝗪𝗶𝘁𝗵 𝗔𝗹𝗹 𝗪𝗮𝗿𝗻𝗶𝗻𝗴𝘀 𝗘𝗻𝗮𝗯𝗹𝗲𝗱: Address all compiler warnings to catch potential issues early. This is often neglected in many projects!

Coding Guidelines

How I spend my time as a developer:

10% writing code
20% refactoring
70% reading code

This is why I optimize my code for readability while still in the writing stage.

It always pays off in the long run.

And I know the next engineer reading that code will be thankful.

So think about this next time you're hurrying to finish a task.

Here's a checklist in no particular order:

Variables defined close to where they are used
Fluent syntax structured vertically
Names are descriptive
80 characters per line
Early return principle

Keep Exploring!!!

YouTube recommendation systems

Some papers need more iterations to learn the basics. Summary from link

Patterns

Episodic series are usually watched sequentially
Users often discover artists in a genre beginning with the most broadly popular before focusing on smaller niches
Balancing new content exploration/exploitation perspective.
If a user was recently recommended a video but did not watch it then the model will naturally demote this impression on the next page load

Approach

YouTube videos, split into two distinct problems: candidate generation and ranking
YouTube activity history
Shortlist small subset (hundreds) of videos from a large corpus
Broad personalization via collaborative filtering
Similarity between users - video watches, search query tokens, demographics
Similarity of Features describing the video and user
Ranking network accomplishes this task by assigning a score to each video according to a desired objective function

Recommendation as extreme multiclass classification where the prediction problem
Embeddings of each candidate video
Deep neural network is to learn user embeddings u as a function of the user’s history and context that are useful for discriminating among videos with a softmax classifier
Explicit feedback mechanisms exist on YouTube (thumbs up/down) + implicit feedback of watches
Magnitude of implicit user history / feedback is extremely sparse
Traditional softmax vs hierarchical softmax

Features used

Video watches, search query tokens, demographics
Feedback mechanisms
implicit user history when feedback is extremely sparse
Demographic features
User’s geographic region and device
User’s gender, logged-in state and age

Scaling Lessons

Previous systems at YouTube relied on hashing
Scoring problem reduces to a nearest neighbor search in the dot product space
Previous linear and tree-based methods for watch time prediction

Feature Engineering

A user’s watch history is represented by a variable-length sequence of sparse video IDs which is
mapped to a dense vector representation via the embeddings
Network requires fixed-sized dense inputs

Algorithms

If users are discovering videos through means other than our recommendations, we want to be able to quickly propagate this discovery to others via collaborative filtering.

Ranking

The primary role of ranking is to use impression data to specialize and calibrate candidate predictions for the particular user interface.
Assign an independent score to each video impression using logistic regression
Ranking with watch time better captures engagement

Feature Representation

Binary (e.g. whether the user is logged-in)
millions of possible values (e.g. the user’s last search query)
Features are further split according to whether they
Contribute only a single value (“univalent”) or a set of values (“multivalent”)
An example of a univalent categorical feature is the video ID of the impression being scored
multivalent feature might be a bag of the last N video IDs the user has watched
A univalent feature in machine learning would be a feature that can assume only one value or has only one dimension
If you have a dataset with a feature called "IsEmailVerified," it could potentially be a binary feature signifying whether a user's email is verified (1) or not (0)
A multivalent feature would be a feature that can have multiple values or dimensions.
For example, a feature representing "VehicleType" could have multiple categories like "Car," "Truck," "Bike," and "Bus," each representing a different mode of transportation.
"Color," which could encode the color of an item as an RGB tuple (Red, Green, Blue).

Deep ranking network architecture

univalent and multivalent features
All layers are fully connected. In practice, hundreds of features are fed into the network.

Embedding

Categorical features in the same ID space also share underlying emeddings.
Out-of-vocabulary values are simply mapped to the zero embedding

CHATGPT4 Support

Define the user and video embeddings (simulating the learned embeddings from the actual system).
Simulate the function recommend_videos that scores and ranks videos for a user based on their embedding.
Implement the softmax function for the recommendation score.
Provide sample usage of the recommendation function using dummy data.

softmax vs hierarchical softmax with python example

The key takeaway is that while softmax calculates probabilities across all possible classes, hierarchical softmax does it for a specific path in a binary tree, reducing the computation when dealing with a vast number of classes.

Softmax Probabilities: The softmax function has converted the logits ([2.5, 1.0, 0.5]) into probabilities ([0.73612472, 0.16425163, 0.09962365]), which sum to 1.

Hierarchical Softmax Probability of a Class: The hierarchical softmax has calculated the probability for the class represented by the path 'LL' (left, left), which is (0.6 \times 0.8 = 0.48).

Keep Exploring!!!