Weekend Learning - Convex Optimization - Stephen Boyd, Professor, Stanford University

Key Notes
Mathematical Optimization

• Choices of a vector/numbers
• Constraint - Legal / Technical / Physics
• Judged by objectives
• Examine on profit/utility

Purpose

• Make good actions
• Reduce risk/ cost is objective / action
• Constraint come from the manufacturing process

Variables

• Vector x could be trades, schedule 
• Resource allocation 
• Optimize signals

AI / Stats / ML

• X - parameters to model
• Constraints (impose requirements)
• Optimization used for worst-case analysis

Optimization-based models

• Aggregate small number of agents
• Simplistic assumptions and formulate the problem
• Predictive ability of models

Convex Optimization

• Minimize objective
• Constraint to hold
• Linear constraints
• Constraints and linear functions will curve up

Why?

• Methods available to solve them

Different application areas

• Spacex landing is effort of optimization
• Optimal trajectory to landing path
• 10 times a second
• Networking / Circuit design

How to use ?

• Formulate as convex problem

Examples

Example #1 - Radiation treatment planning

• Things decided are actions
• linear y = Ax
• options - beam diverges / tissues / hits bone scatters
• Overcharge / Undercharge

Example #2 - Image in painting

• Guess the lost parts
• Minimize function / Convex problem
• Remove 5% of pixels

SVM

• Predict boolean outcome
• spam/ fraud 
• Old school - gradient method
• Convex Optimization - Differentiability irrelevant

Lasso

• Methods for sparse model construction
• With 1/5th measurements analyze

Solving

• Define in High Level language
• Solved by solver
• Helps in rapid prototyping

Large Scale Distributed Optimization

• Grid Updates
• Image / Video processing

Read Ferenc Huszár's answer to Why is Convex Optimization such a big deal in Machine Learning? on Quora Happy Learning!!!