Heuristics vs ML
- A heuristic is a way to find the solution to some problem without exhaustively trying all possible solutions, or without knowing the answer ahead of time.
- A heuristic is a strategy for finding a solution to a problem faster or approximating a solution for it
- The term heuristic is used for algorithms that find solutions among all possible ones,but they do not guarantee that the best will be found
Ref - Link
- Heuristic is any approach to problem-solving, learning, or discovery that employs a practical method not guaranteed to be optimal or perfect, but sufficient for the immediate goals.
Ref - Link
Machine Learning is different, in that instead of teaching the computer the technique that you found for solving the problem
Sampling-based techniques
- Do not require first or second derivative
- Exhaustive search
- Simulated annealing
- Genetic Algos
Gradient-based techniques
- Branch and bound
Algorithms for Decision Making: Optimization, Heuristics and Machine Learning
Deterministic vs. stochastic models
- In deterministic models, the output of the model is fully determined by the parameter values and the initial conditions.
- Stochastic models possess some inherent randomness. The same set of parameter values and initial conditions will lead to an ensemble of different outputs
- Demographic stochasticity describes the randomness that results from the inherently discrete nature of individuals
- In deterministic algorithm, for a given particular input, the computer will always produce the same output going through the same states
- non-deterministic algorithm, for the same input, the compiler may produce different output in different runs
Ref Link
P, NP, NP-Hard & NP-complete problems
It takes time to connect all the dots and plot the big picture. Keep Plotting!!!
