"No one is harder on a talented person than the person themselves" - Linda Wilkinson ; "Trust your guts and don't follow the herd" ; "Validate direction not destination" ;

May 31, 2021

Concepts - Heuristics vs ML, Deterministic vs. stochastic models

Heuristics vs ML

  • A heuristic is a way to find the solution to some problem without exhaustively trying all possible solutions, or without knowing the answer ahead of time.
  • A heuristic is a strategy for finding a solution to a problem faster or approximating a solution for it
  • The term heuristic is used for algorithms that find solutions among all possible ones,but they do not guarantee that the best will be found

Ref - Link

  • Heuristic is any approach to problem-solving, learning, or discovery that employs a practical method not guaranteed to be optimal or perfect, but sufficient for the immediate goals.

Ref - Link

Machine Learning is different, in that instead of teaching the computer the technique that you found for solving the problem 

Sampling-based techniques

  • Do not require first or second derivative
  • Exhaustive search
  • Simulated annealing
  • Genetic Algos

Gradient-based techniques

  • Branch and bound

Algorithms for Decision Making: Optimization, Heuristics and Machine Learning

Deterministic vs. stochastic models

  • In deterministic models, the output of the model is fully determined by the parameter values and the initial conditions.
  • Stochastic models possess some inherent randomness. The same set of parameter values and initial conditions will lead to an ensemble of different outputs
  • Demographic stochasticity describes the randomness that results from the inherently discrete nature of individuals
  • In deterministic algorithm, for a given particular input, the computer will always produce the same output going through the same states
  • non-deterministic algorithm, for the same input, the compiler may produce different output in different runs

Ref Link

P, NP, NP-Hard & NP-complete problems

It takes time to connect all the dots and plot the big picture. Keep Plotting!!!

No comments: