509895 - NUMERICAL OPTIMIZATION AND DATA SCIENCE

A) NUMERICAL OPTIMIZATION MODULE 1. Introduction to unconstrained optimization. Uniform grid search. 2. Iterative methods and derivative-free methods: Nelder-Mead, genetic algorithms and particle swarm optimization. 3. Linear methods: descent direction and step size; Wolfe conditions. Gradient method, Newton method, and quasi-Newton methods (DFP and BFGS). 4. Trust-Region methods. 5. Applications to machine learning and deep learning methods; stochastic gradient descent. B) DATA SCIENCE MODULE 1. Regression: loss function, linear and polynomial regression; regression trees and bagged trees; accuracy metrics and k-fold cross-validation. 2. Classification: classification trees; perceptron and support vector machines (SVM) with kernel methods; training error, generalization and overfitting, PAC-learning and the uniform convergence theorem; Occam’s theorem and regularization. Weak learners and boosting method. 3. Clustering: k-center and the farthest traversal algorithm; k-means, Lloyd’s algorithm, Ward’s algorithm, and initialization strategies. 4. Dimensionality Reduction: law of large numbers, spherical Gaussian distributions in high dimensions, Gaussian Annulus theorem; separation and fitting of spherical Gaussians; random projection theorem and the Johnson-Lindenstrauss lemma 5. Singular value decomposition (SVD), best rank-k approximation, and principal component analysis (PCA).