CMSC764-Advance-Numerical-Optimization

Homework 1 : Linear Algebra

1. Problem 1 : Prove the dual norm a norm.
2. Problem 2 : Proove Convolution Theorem.
3. Problem 3 : Prove the Cooley-Tukey factorization formula.
4. problem 4 : Derive the negative log-likelihood function for x given y.

Homework 2 : More Linear Algebra

1. Problem 1 : Effect of bad condition number.
2. Problem 2 : Write a method for checking whether At is the adjoint of A.
3. Problem 3 : Implement the adjoint/transpose for convolution operators.
4. problem 4 : FFT.

Homework 3 : Gradients

1. Problem 1 : Gradient checker.
2. Problem 2 : Write a routine that evaluates the logistic loss function.
3. Problem 3 : IImplement the total-variation denoising objective.

Homework 4 : PySmorch (A machine learning library for the dregs of society)

1. Problem 1 : A linear layer.
2. Problem 2 : ReLU layer.
3. Problem 3 : Cross Entropy.
4. problem 4 : Bias layer.

Homework 5 : Convex Functions

1. Problem 1 : Check if the functions are convex.
2. Problem 2 : Verify properties of convex functions.
3. Problem 3 : Quasi convex

Homework 6 : Gradient methods and Duality

1. Problem 1 : Gradient descent: GD, Barzilai-Borwein, Nestrov
2. Problem 2 : Image denoising
3. Problem 3 : The dual
4. Problem 4 : Linear programming example

Homework 7 : Splitting Methods

1. Problem 1 : Forward-backward splitting
2. Problem 2 : Netflix problem

Homework 8: Alternating Direction Method of Multipliers (ADMM)

Homework 9: Monte Carlo Markov Chain (MCMC)

Fun Projects

1. Image recovery using Total-Variation denoising objective.