"No one is harder on a talented person than the person themselves" - Linda Wilkinson ; "Trust your guts and don't follow the herd" ; "Validate direction not destination" ;

January 26, 2022

Hiring perspectives

I have two candidates

  • Candidate #1 - Cleared Data Science Test, Good Deep Learning, Has not worked in Pharma domain but good at ML
  • Candidate #2 - Just passed ML Test, No Deep Learning Expertise but worked in Pharma, Has domain and insights, Data knowledge

Both performed well in other behavioral rounds. If we end up hiring multiple types of Candidate 1 effectively you will end up with #duplicate skill set. I would prefer to have a mix of both in a team. Skills need to complement and add different perspectives to the problem. In a team, we need a combination of both #1 and #2. We need to have different #proportions of #skills and assessments to get a good mix of talent that can look at #same #problem in multiple #perspectives

Keep Thinking!!

January 25, 2022

What questions we need to ask Facebook

  1. What Affirmative action is taken to reduce extensive user amplification/engagement
  2. What plans do you have to engage kids towards creativity/thinking, a platform for self-development vs content engagement
  3. 50% of the world's population still being offline, How has FB impacted in positive ways vs migrating/working for #metaverse

Keep Thinking!!

Setting up cockroachDB

  1. Signup for cockroachDB
  2. Install DBweaver
  3. From cockroachDB signup get user, host, database, port details
  4. Set password for user it must be 12 digits minimum
  5. Took existing SQL scripts of table creation ported it to postgres with online converter link
  6. Data generation script does not work as is need to tweak it in postgres. 
  7. Tables created, Data need to be populated, run some load tests :)

Keep Exploring!!!


January 23, 2022

30K, 10K feet to Building Algo perspectives

30K, 10K feet to Building Algo perspectives

  • Get the 30K Feet Big Picture of Algo 
  • Know the functions/methods implement
  • Map it do applicable domains / use cases
  • Apply First principles to learn basics
  • Find gaps in fundamental assumptions vs algos vs intermediate gaps in learning
  • Start Filling the gaps iteratively
  • Build your own version of algo with first principles + domain + data + algo implementation

#KnowledgeBuilding #MLBytes

Keep Exploring!!!


January 22, 2022

A / B Testing Revisions

 Session #1

Notes

#A/BTesting = #Randomizedcontrolledtrials of two versions of same application

  • Running experiments
  • Used in Testing search, pricing algos
  • Origins of A/B testing 100 years back
  • Testing two fertilizers
  • Commit to sample size, Number of pots to use
  • More pots, Higher sample size honest assessments
  • Analyze by hypothesis test
  • The difference between two options

  • Binary classification false positives
  • What might have happened if no difference



  • Current state
  • Adjust test lengths in realtime
  • Adjust duration of test

  • How long test running
  • Variations of page
  • Views / Example clicks
  • Compared to baseline comparison
  • A-A Testing both variations are same
  • 5% False positive probability
  • Number of experiments, Sample size, 50% of actual size of customerbase
  • Optimal procedure - Monitoring and Stopping early

  • Statistical significance of confidence level

  • Staging trials
  • Wait to see p value < 5
  • Secondary diagnostics
  • Sample size calculation
  • How many samples to wait for

Session #2




  • Conversions / clicks measure
  • Prior Beleif + Update prior belif and Probability distribution








  • Some marketing experts even recommend sample sizes of up to 5,000 people.
  • One way is to run A/B tests separately for specific devices and browsers.
  • statistical significance of 95%

P > 0.05 is the probability that the null hypothesis is true
P ≤ 0.05 means that the test hypothesis is false or should be rejected


Keep Exploring!!!!

AI Solutions in SaaS model

AI solutions in saas model, #Data as #service, #Insights as #service #Forecasting, #Recommendations as #service #quantumics.ai #abacus.ai #graphext.com #deepstack.cc


Keep Exploring!!!

January 20, 2022

Research Reads - Markdown pricing

Paper #1 - Markdowns in E-Commerce Fresh Retail: A Counterfactual Prediction and Multi-Period Optimization Approach

Key Notes

  • Due to the limited shelf life of perishable products and the limited opportunity of price changes, it is difficult to predict
  • sales of a product at a counterfactual price, and therefore it is hard to determine the optimal discount price to control inventory and to maximize future revenue.
  • Sequential pricing strategy by Markov decision process, and design a two-stage algorithm to solve it
  • Many perishable products, such as vegetables, meat, milk, eggs, bread, have a limited shelf life promotional markdown is a common approach for e-commerce fresh retails
  • The normal channel, where goods are sold by no-discount retail price
  • Markdown channel, where customers can buy goods by discount under the condition that their total purchase has reached a certain amount

Key Questions

  • First, can goods be sold out with the retail price before its expiry date?
  • Second, if not, what is the optimal discount price for promotional markdown to ensure the goods being sold out while maximizing the profit?
  • The first problem is about sales forecasting 
  • Second problem is about price-demand curve fitting
  • We observe the sales of a product with price A and B, we aim to predict the sales of a product with price C, which is counterfactual

  • To avoid price discrimination, the discounts of the same product in different stores within the same region should be all equal
  • To optimize the discount price, we need to take all stores in a region into consideration
  • We collect a set of observable covariate features 𝒙𝑖 ∈ R, including categories, holidays, event information, inherent properties and historical sales of products and shops
  • The key of pricing decision making is to accurately predict the demand of products at different discount prices
  • We aggregate data of all products by using the category information and learn the causal effect of each product jointly
  • The price elasticity is daily updated once the new transaction data is collected

Paper #2 - Markdown Pricing Under Unknown Demand

  • Unimodal Multi-Armed Bandit problem where the goal is to find the optimal price under an unknown unimodal reward function
  • “optimal” solutions exist under numerous variations on (a) the set of demand functions allowed, on (b) how inventory is treated, and on (c) the frequency at which prices are allowed to change, just to name a few. 
  • A Markdown Policy and Performance Guarantee: We introduce a policy which satisfies the markdown constraint
  • Optimality via a Minimax Lower Bound: We prove that our policy is in fact orderoptimal by showing

Paper #3 - Markdown Pricing Under Unknown Parametric Demand Models

  • Markdown Policies with Theoretical Guarantees
  • Tight Minimax Lower Bound
  • Impact of Smoothness
  • In the Discrete Multi-armed Bandit problem, the player is o↵ered a finite set of arms, with each arm providing a random revenue from an unknown probability distribution specific to that arm. The objective of the player is to maximize the total revenue earned by pulling a sequence of arms 

Keep Exploring!!!

January 17, 2022

Validation Loss vs Accuracy

 I see both of them a bit differently, Both represent different aspects. 

Loss

  • A loss function is used to optimize a machine learning algorithm. 
  • Validation loss is measure of how much our predictions differ from what they should be before we put them through the threshold.

Accuracy

  • An accuracy metric is used to measure the algorithm’s performance (accuracy) in an interpretable way.
  • Empirically, accuracy seems like quite a limited measure of quality of predictions. To predict whether an example belongs to some class, our model outputs a number (whatever we put through sigmoid or softmax) between 0 and 1.

Ref - Link

Keep Exploring!!!

Vision Lessons

Some use cases convey how we simplify implementation with the setup/environment



Vision Lessons

  • Plate as a base and black background
  • The black background will reduce External noise
  • Spread uniformly
  • Easy to identify/report

Keep Exploring!!!

January 16, 2022

#keras #experiments #ParallelNetworks #Merge

Experiments to build hybrid approach of models. Leverage different convolutions, activation functions.

For custom training vision tasks. Get Features from both vggnet, resnet
  • Resnet - 224 x 224 x 3
  • Vgg16  - 224 x 224 x 3
Feature Vectors
  • VGG16 feature shape — (1L, 7L, 7L, 512L)
  • VGG19 feature shape — (1L, 7L, 7L, 512L)
  • InceptionV3 feature shape — (1L, 5L, 5L, 2048L)
  • ResNet50 feature shape — (1L, 1L, 1L, 2048L)
Inputs
  • Sobel, Laplace Transformations
  • Shareped X / Y Axis edges
  • Multiple inputs
Further Techniques
  • Apply different convolution filters
  • Apply different activation functions
  • Append different weights and analyze
Keep Exploring!!!