Dark side of profits

• The dark side of  analytics - mobile apps - facebook - youtube - amplify the #engagement for the sake of profit 
• If you aren't the paying customer, you are the product. #google #facebook #android
• Anger/ Hate / Excitement / Drugs creates dopamine addiction and keeps the conversation going
• For the cab sharing, delivery partners - The illusion of guaranteed income while the variable incentives seem attractive initially but mental and physical costs would take a big toll soon
• High dopamine low effort entertainment (video games, drugs, porn, Netflix), it becomes the default way to spend leisure time really quickly

I will let this happen to my own kid vs can I leverage everything outside my home as an untapped market.

As an end-user think

• How much time does Zuck spend on FB every day
• Will they let their kids spend so much time FB that a typical teen does

Until we recognize we are in this trap of low-cost internet, virtual addiction you will never come out of this virtual trap - low-cost mobile phone, free internet, engagement vs leaving away your goals in life.

The same happens in every other domain, Why do restaurants don't hesitate to use outdated/expired products in their food. It boils down to one's own integrity vs profits. 

How I Met & Surpassed My Career Goals While Following One Actionable Rule

Keep Thinking!!!

Perspective of Learning

During schools days

• Why should I learn?
• Life will be same, I will become driver / cleaner ?
• What are ways to quit education, All my friends started working
• I don't apply anything I learn why should I learn ?
• My dad work vs what he learns is not connected

Now

• Learn to know state of art
• Learn to design things 
• Solve business problems in your own way
• Learn to review others work
• Learn to have good domain knowledge
• Learn to be employable, do better contributions

Now I learn more sincerely than my school days :) :) :)

NLP - NER - Papers

Paper #1 - Recent Trends in Named Entity Recognition (NER)

Key Notes

• ‘Named Entity Recognition’ refers to identifying person, organization, location
• NER belongs to a general class of problems in NLP called sequence tagging 
• Prominent supervised learning methods - Hidden Markov Models (HMM), Decision Trees, Maximum Entropy Models (ME)

• Unsupervised clustering method using lexical resources eg. Wordnet

Paper #2 - A Survey on Deep Learning for Named Entity Recognition

Key Notes

• Rule-based approaches, which do not need annotated data as they rely on hand-crafted rules
• Unsupervised learning approaches, which rely on unsupervised algorithms Feature-based supervised learning approaches, which rely on supervised learning algorithms

• 71% of search queries contain at least one named entity

• word-level representation  - continuous bagof-words (CBOW) and continuous skip-gram models
• Commonly used word embeddings include Google Word2Vec, Stanford
• GloVe, Facebook fastText and SENNA.
• CharNER considers a sentence as a sequence of characters and utilizes LSTMs to extract characterlevel representations.
• Besides word-level and character-level representations, some studies also incorporate additional information (e.g., gazetteers [18], [108], lexical similarity [109], linguistic dependency [110] and visual features [111]) into the final representations of words

Paper #3 - Document Ranking for Curated Document Databases using BERT and Knowledge Graph Embeddings: Introducing GRAB-Rank

• Key Notes
• Curated Document Databases (CDD) play an important role in helping researchers find relevant articles in scientific literature
• Document ranking has been extensively used in the context of document retrieval
• Recent work on Learning to Rank (LETOR) has used word embeddings of various kind as the input
• Word embeddings can be learnt from scratch or a pre-trained embedding model can be adopted
• A popular algorithm for generating vector representations of words is GloVE (Global Vectors for Word Representation), an unsupervised learning algorithm that operates by aggregating global word-word co-occurrence statistics
• Semantic document ranking models take into account the context of terms in relation to their neighbouring terms
• Context of the word “bank”, either as: (i) an organisation for investing and borrowing money, (ii) the side of a river or lake, (iii) a long heap of some substance
• A popular choice of pre-trained contextual model is the Bidirectional Encoder Representations from Transformer (BERT)
• An alternative contextual model that can be used is the embeddings from Language Model ELMo
• A knowledge graph is a collection of vertices and edges where the vertices represent entities or concepts, and the edges represent a relationship between entities and/or concepts. 

• OIE4KGC (Open Information Extraction for Knowledge Graph Construction)

More Reads

• Ad-hoc retrieval with BERT
• Multi-Stage Document Ranking with BERT
• Pretrained Transformers for Text Ranking: BERT and Beyond
• Composite Re-Ranking for Efficient Document Search with BERT
• CEDR: Contextualized Embeddings for Document Ranking

Code Experiments

Compare documents similarity using Python | NLP

Measuring the Document Similarity in Python

How to compute the similarity between two text documents?

How to do semantic document similarity using BERT

How to cluster text documents using BERT

Question answering using transformers and BERT

Calculating Document Similarities using BERT, word2vec, and other models

Pykg2vec representation of entities and relations in Knowledge Graphs

Pykg2vec: Python Library for KGE Methods

Python NLP Tutorial: Building A Knowledge Graph using Python and SpaCy

NER with spacy

Keep Exploring!!!

Time series variables and Insights

Time series variables and Insights

Good read - Link

• Discrete variables - Discrete data is information that can only take certain values. Discrete data refers to individual and countable items (discrete variables). Countable, Point in time data (Bank balance). Looks like clusters, points. The number of customers who bought different items. The number of computers in each department. The number of items you buy at the grocery store each week
• Continuous Variables - Continuous data is data that can take any value. Takes any measured value within a specific range. Height, weight, temperature and length are all examples of continuous data. Some continuous data will change over time. Looks like line graphs, continuous.
• Univariate analysis is the simplest form of data analysis where the data being analyzed contains only one variable
• Bivariate data – This type of data involves two different variables. The analysis of this type of data deals with causes and relationships and the analysis is done to find out the relationship among the two variables.
• Multivariate analysis is the analysis of three or more variables.
• Multiple linear regression
• Multiple logistic regression
• Multivariate analysis of variance (MANOVA)
• Factor analysis
• Cluster analysis
• The aim of multivariate analysis is to find patterns and correlations between several variables simultaneously
• Simple regression pertains to one dependent variable and one independent variable
• Multiple regression (aka multivariable regression) pertains to one dependent variable and multiple independent variables
• Multivariate regression pertains to multiple dependent variables and multiple independent variables
• A stationary (time) series is one whose statistical properties such as the mean, variance and autocorrelation are all constant over time. Hence, a non-stationary series is one whose statistical properties change over time.

Keep Exploring!!!

Dark Side of Social Media

I was expecting such evidence-based insights to understand social media manipulation.

Key Notes from Link1, Link2

• There were conflicts of interest between what was good for the public and what was good for Facebook. And Facebook, over and over again, chose to optimize for its own interests, like making more money
• Facebook has realised that if they change the algorithm to be safer, people will spend less time on the site, they'll click on less ads, they'll make less money
• The version of Facebook that exists today is tearing our societies apart and causing ethnic violence around the world,” says former Facebook employee France Haugen.

Paper #1 - THE WELFARE EFFECTS OF SOCIAL MEDIA

• Adverse outcomes such as suicide and depression appear to have risen sharply over the same period that the use of smartphones and social media
• Social media may create ideological “echo chambers” among like-minded friend groups, thereby increasing political polarization
• Deactivating Facebook freed up 60 minutes per day for the average person in our Treatment group
• Facebook deactivation significantly reduced news knowledge and attention to politics. 

More Read

POVERTY, DEPRESSION, AND ANXIETY: CAUSAL EVIDENCE AND MECHANISMS

Social Media can be addiction, It can make you feel lonely. We are not monitored for mental health. Talk more, Walk more. Quit Social Media!!!!

Data Curation paper - Reads

Paper #1 - A Survey on Data Cleaning Methods for Improved Machine Learning Model Performance

• Two aspects of data cleaning: what to clean and how to clean

Key Notes

• SampleClean: Simulated Clean Data Instances - SampleClean suggests a solution to sampling the raw data that can better present clean data instances.
• Approximate Query Processing (AQP). The AQP consists of two steps: first, in Direct Estimate (DE), a set of k rows is sampled randomly and cleaned, and the training result is returned independently of the dirty data. The correction step is used to reweight the sample based on the contribution of the cleaned data
• ActiveClean: Incremental Data Cleaning in Convex Models. ActiveClean gradually cleans a dirty dataset to learn a convex-loss model, such as Logistic Regression and Support Vector Machine (SVM).
• HoloClean: Holistic Data Repairs With Probabilistic Inference
• AlphaClean: Generate-Then-Search Parallel Data Cleaning
• CPClean: Reusable Computation in Data Cleaning

ML Papers - Learning-with-Label-Noise

Paper #2 - Advancing Data Curation With Metadata and Statistical Relational Learning

Key Notes

• We refer to data science as an umbrella term gathering algorithms and techniques from several disciplines, such as statistics, software engineering, and machine learning
• Data is inconsistent, duplicated, stale, incomplete, and/or inaccurate. Data errors, such as outliers, duplicates, missing values, and inconsistencies.
• Mapping Metadata to Data Quality Issues
• Error Detection
• Joint Error Detection and Repair Suggestion

Data Quality fundamentals

• The Consistency dimension refers to the validity and integrity of values and tuples with respect to defined inter- and intra-relational constraints that exist within either single or multiple relations
• The accuracy dimension identifies correct and true values of the entities presented by data.
• Completeness is a degree to which values are included in a data collection
• Timeliness dimension reflects the change and update of data by identifying the most current value of an entity in a database
• Core data quality dimensions, the violation of Accuracy, Consistency,
• Uniqueness, Completeness and Timeliness lead to data quality issues

• Metadata is "structured information that describes, explains, locates, or otherwise makes it easier to retrieve, use, or manage an information resource"

Single-Column Profiling Tasks

• Cardinalities refers to the counts of values
• Number of rows: the number of entities which are available in the table;
• Distinctness: the number of distinct values of the single attribute;
• Uniqueness: the ratio of the number of distinct values to the number of rows

Value Distribution refers to the distribution of values on the column. This category includes:

• Constancy: the ratio between the most frequent value count and the number of rows;
• Extreme values: minimum and maximum values in numeric columns; shortest and
• longest strings in categorical, alphanumeric or text columns;
• Histogram: values distribution summary on an attribute
• Quartiles: three points that divide numeric distribution into four equal groups;
• Inverse distribution: an inverse frequency distribution (a distribution of the frequency distribution);

Patterns

• Patterns refers to the syntactic properties on the values of the individual column.
• Lengths, which specifies the descriptive statistics of the column value lengths
• Decimals, which determines the number of decimals in numeric columns

Multi-Column Profiling Tasks

• Functional dependencies
• What. The first dimension captures common data quality issues and typical data cleaning tasks, which had been found in the literature.
• How. The second dimension reflects differently focused data cleaning approaches.

Rule-Based Approaches

• Data cleaning rules or integrity constraints to detect and repair various error types in the dataset.

Statistical Approaches

• DEC (DetectExplore-Clean) framework [22] uses statistical and other analytical techniques, such as the Fleiss’ kappa measure, to compute the glitch score, which identifies and scores the data glitches

Probabilistic and Machine Learning-Based Approaches

• The BoostClean system [141] addresses the domain value violations while cleaning training data for predictive models
• The HoloClean system [202] considers error detection as a black-box component and expects the specification of integrity constraints-aligned data quality rules to make probabilistic suggestions on how to repair erroneous data values.
• Interactive Data Cleaning
• Numerous data cleaning systems use crowdsourcing for duplicate detection and resolution

Supervised Error Detection with Metadata

1) an Error Detection Suite, which includes pluggable error detection systems that function as black boxes to our system.

2) a Metadata Profiler Suite, which extracts various metadata categories, and 

3) an Aggregation Suite, which combines the output of the error detection suite and the profiler. In the following, we describe each of the components.

Keep Exploring!!!

Probabilistic Forecasting Reads

Paper - Master's Thesis : Comparison of probabilistic forecasting deep learning models in the context of renewable energy production

• DeepAR
• Wavenet
• Transformer
• Temporal Fusion Transformer
• Prophet

Awesome Reads

Timeseries ML

Code - Link

• Naive forecasting models (Naive, Seasonal Naive, Moving Average, etc)
• MXNet [10], developed by Amazon Web Services
• GluonTS has been developed by a Amazon Web Service team to fill the gap of time series modeling toolkit
• MQCNN, MQRNN, NBEATS and Wavenet does not outputs samples of a distribution function, but quantiles of the distribution itself
• NPTS is the implementation of the “Non-Parametric Time Series Forecaster” model
• MQCNN is the implementation of one variant of the model described in paper ”A Multi-Horizon Quantile Recurrent Forecaster”
• The model Transformer is the implementation of “Transformer” model architecture, as it was defined in paper [22]. It is described in this paper as ”The first sequence transduction model based entirely on attention, replacing the recurrent layers most commonly used in encoder-decoder architectures with multi-headed self-attention”
• The model Wavenet is the implementation of ”Wavenet” model architecture, as it was defined in paper [23], with a quantized target. This model network is composed of dilated causal convolutional layers. Both residual and parameterised skip connections are used throughout the network,to speed up convergence and enable training of much deeper models
• DeepAR - global model from historical data of all time series. Similar to LSTM-based recurrent neural network architecture to the probabilistic forecasting problem
• Binomial distribution - Two possible outcomes (the prefix “bi” means two, or twice)
• Assumptions - Each trial is independent. The probability of success (tails, heads, fail or pass) is exactly the same for each trial
• Poisson distribution - Gives us the probability of a given number of events happening in a fixed interval of time
• Continuous distribution - data can take on any value within a specified range
• Discrete distribution is one in which the data can only take on certain values, for example integers
• RNN architecture for probabilistic forecasting, incorporating a negative Binomial likelihood
• Monte Carlo (MCMC) methods comprise a class of algorithms for sampling from a probability distribution. 

Spacetimeformer Multivariate Forecasting

Multivariate Time Series Forecasting with Transformers

Top 5 Forecasting Demos

Demand forecasting using RNN with LSTM on PyTorch

Anomaly Detection with PyOD

NGBoost: Natural Gradient Boosting for Probabilistic Prediction

Keep Exploring!!!!

Siamese Networks

There is always a pleasure in reading, typing line by line. Staying focused and trying to understand. Siamese with my own comments and observations added for my reference

• Siamese Network Keras for Image and Text similarity.
• Multi-class Classification using Deep Neural Networks
• Siamese Networks
• Image Similarity with Siamese Networks
• machine-learning-experiments
• Generating Images of Clothes Using Deep Convolutional Generative Adversarial Network (DCGAN)
• Plant Disease Using Siamese Network - Keras
• Face Recognition Using Siamese Network
• Image similarity estimation using a Siamese Network with a contrastive loss
• Image similarity estimation using a Siamese Network with a triplet loss
• Near-duplicate image search
• Siamese Neural Networks for Few Shot Learning

Keep Exploring!!!

My Perspectives

The core of ML work in Top companies is only 5%, The same applies to all startups. The vision vs showcase of selective 5% is one part of the story. Every company is struggling to build a data lake, archive vs store date. 5% of problems solved vs 95% of vision. 

Do not consider external talks as a representation of internal data science growth. 

ML Vision vs Use cases with measurable benefits vs AI vision will reflect the maturity of the organization

Keep Thinking!!!

Leaf Detection - Top K Techniques - Reads

Paper #1 PL@NTNET APP IN THE ERA OF DEEP LEARNING

Key Notes

• Observations are stored within a NoSQL document storage called CouchDb
• CNN architecture is the inception model
• The number of species (i.e. classes) in January 2017 was about 10K and the number of training images about 332K
• Species filtering is applied based on the checklist of species
• activated within the app (e.g. West Europe, North Africa, South America, etc.)
• By default, this checklist is automatically chosen according to the geo-location of the mobile device, but the user also has the possibility to select another one manually
• Similarity Search - This content-based image retrieval is performed through a hashing-based approximate nearest neighbors search algorithm applied on top of the 1024-dimensional feature vectors extracted by the last hidden layer of the fine-tuned CNN

Paper #2 - Fine-grained recognition of plants from images

Key Notes

• A number of approaches is based on the popular local binary patterns (LBP)
• Fast Features Invariant to Rotation and Scale of Texture (Ffrst)
• "One versus All" classifcation scheme is used for multi-class classifcation

• The Foliage leaf dataset by Kadir
• The Swedish leaf dataset
• The Leafsnap dataset

Paper #3 - Neural Network Application on Foliage Plant Identification

Key Notes

• Polar Fourier Transform that proposed by Zhang [18] has properties that are very useful for represents shape of objects, including leaf of plants
• There are three kinds of geometric features involved as shapes features: slimness ratio, roundness ratio, and dispersion. 
• Color features on a leaf can be extracted by using statistical calculations such as mean, standard deviation, skewness, and kurtosis

Leaf Type Classification

Paper #4 - An Online Algorithm for Large Scale Image Similarity Learning

Key Notes

• Here we focus on a weaker supervision signal: the relative similarity of different pairs
• we extract similarity information from pairs of images that share a common label
• we extract similarity information from pairs of images that share a common label or are retrieved in response to a common text query in an image search engine

Paper #5 - Large Scale Local Online Similarity/Distance Learning Framework based on Passive/Aggressive

Paper #6 - Comparison of Image Matching Techniques 

• Blob detection technique
• Template matching
• SURF feature extraction 

Paper #7 - Large Scale Online Learning of Image Similarity Through Ranking

Key Notes

• OASIS is both fast and accurate at a wide range of scales
• Similarity information is extracted from pairs of images that share a common label or are retrieved in response to a common text query
• OASIS can be trained on more than two million images within three days on a single CPU
• OASIS learned similarity show that 35% of the ten nearest neighbors of a given image are semantically relevant to that image
• Algorithm that uses triplets of images
• Color histograms are obtained by K-means clustering
• Local Binary Pattern
• LMNN - Large Margin Nearest Neighbor Classification

Paper #8 - Sparse online learning of image similarity

tf2_semantic_approximate_nearest_neighbors

More Reads

• Simple and fast method to compare images for similarity
• Sampling Wisely: Deep Image Embedding by Top-k Precision Optimization
• image-match
• Image Deduplicator (imagededup)
• Large Scale Local Online Similarity/Distance Learning Framework based on Passive/Aggressive
• Image similarity model
• Hashing Learning
• Fast Near-Duplicate Image Search using Locality Sensitive Hashing
• Querying Similar Images with TensorFlow
• Image classification with Keras and deep learning
• K-Nearest Neighbors Hashing
• A Survey on Deep Hashing Methods
• Image similarity estimation using a Siamese Network with a triplet loss
• Finding similar images using Deep learning and Locality Sensitive Hashing
• Fingerprinting Images for Near-Duplicate Detection
• LSH for near-duplicate image detection

Keep Exploring!!!