Mod-10 Lec-38 No Free Lunch Theorem; Model selection and model estimation; Bias-variance trade-off

Course Lectures

• Mod-01 Lec-01 Introduction to Statistical Pattern Recognition

  Prof. P.S. Sastry

  Play
• Mod-01 Lec-02 Overview of Pattern Classifiers

  Prof. P.S. Sastry

  Play
• Mod-02 Lec-03 The Bayes Classifier for minimizing Risk

  Prof. P.S. Sastry

  Play
• Mod-02 Lec-04 Estimating Bayes Error; Minimax and Neymann-Pearson classifiers

  Prof. P.S. Sastry

  Play
• Mod-03 Lec-05 Implementing Bayes Classifier; Estimation of Class Conditional Densities

  Prof. P.S. Sastry

  Play
• Mod-03 Lec-06 Maximum Likelihood estimation of different densities

  Prof. P.S. Sastry

  Play
• Mod-03 Lec-07 Bayesian estimation of parameters of density functions, MAP estimates

  Prof. P.S. Sastry

  Play
• Mod-03 Lec-08 Bayesian Estimation examples; the exponential family of densities and ML estimates

  Prof. P.S. Sastry

  Play
• Mod-03 Lec-09 Sufficient Statistics; Recursive formulation of ML and Bayesian estimates

  Prof. P.S. Sastry

  Play
• Mod-04 Lec-10 Mixture Densities, ML estimation and EM algorithm

  Prof. P.S. Sastry

  Play
• Mod-04 & 05 Lec-11 Convergence of EM algorithm; overview of Nonparametric density estimation

  Prof. P.S. Sastry

  Play
• Mod-05 Lec-12 Nonparametric estimation, Parzen Windows, nearest neighbour methods

  Prof. P.S. Sastry

  Play
• Mod-06 Lec-13 Linear Discriminant Functions; Perceptron -- Learning Algorithm and convergence proof

  Prof. P.S. Sastry

  Play
• Mod-06 Lec-14 Linear Least Squares Regression; LMS algorithm

  Prof. P.S. Sastry

  Play
• Mod-06 Lec-15 AdaLinE and LMS algorithm; General nonliner least-squares regression

  Prof. P.S. Sastry

  Play
• Mod-06 Lec-16 Logistic Regression; Statistics of least squares method; Regularized Least Squares

  Prof. P.S. Sastry

  Play
• Mod-06 Lec-17 Fisher Linear Discriminant

  Prof. P.S. Sastry

  Play
• Mod-06 Lec-18 Linear Discriminant functions for multi-class case; multi-class logistic regression

  Prof. P.S. Sastry

  Play
• Mod-07 Lec-19 Learning and Generalization; PAC learning framework

  Prof. P.S. Sastry

  Play
• Mod-07 Lec-20 Overview of Statistical Learning Theory; Empirical Risk Minimization

  Prof. P.S. Sastry

  Play
• Mod-07 Lec-21 Consistency of Empirical Risk Minimization

  Prof. P.S. Sastry

  Play
• Mod-07 Lec-22 Consistency of Empirical Risk Minimization; VC-Dimension

  Prof. P.S. Sastry

  Play
• Mod-07 Lec-23 Complexity of Learning problems and VC-Dimension

  Prof. P.S. Sastry

  Play
• Mod-07 Lec-24 VC-Dimension Examples; VC-Dimension of hyperplanes

  Prof. P.S. Sastry

  Play
• Mod-08 Lec-25 Overview of Artificial Neural Networks

  Prof. P.S. Sastry

  Play
• Mod-08 Lec-26 Multilayer Feedforward Neural networks with Sigmoidal activation functions;

  Prof. P.S. Sastry

  Play
• Mod-08 Lec-27 Backpropagation Algorithm; Representational abilities of feedforward networks

  Prof. P.S. Sastry

  Play
• Mod-08 Lec-28 Feedforward networks for Classification and Regression; Backpropagation in Practice

  Prof. P.S. Sastry

  Play
• Mod-08 Lec-29 Radial Basis Function Networks; Gaussian RBF networks

  Prof. P.S. Sastry

  Play
• Mod-08 Lec-30 Learning Weights in RBF networks; K-means clustering algorithm

  Prof. P.S. Sastry

  Play
• Mod-09 Lec-31 Support Vector Machines -- Introduction, obtaining the optimal hyperplane

  Prof. P.S. Sastry

  Play
• Mod-09 Lec-32 SVM formulation with slack variables; nonlinear SVM classifiers

  Prof. P.S. Sastry

  Play
• Mod-09 Lec-33 Kernel Functions for nonlinear SVMs; Mercer and positive definite Kernels

  Prof. P.S. Sastry

  Play
• Mod-09 Lec-34 Support Vector Regression and ?-insensitive Loss function, examples of SVM learning

  Prof. P.S. Sastry

  Play
• Mod-09 Lec-35 Overview of SMO and other algorithms for SVM; ?-SVM and ?-SVR; SVM as a risk minimizer

  Prof. P.S. Sastry

  Play
• Mod-09 Lec-36 Positive Definite Kernels; RKHS; Representer Theorem

  Prof. P.S. Sastry

  Play
• Mod-10 Lec-37 Feature Selection and Dimensionality Reduction; Principal Component Analysis

  Prof. P.S. Sastry

  Play
• Mod-10 Lec-38 No Free Lunch Theorem; Model selection and model estimation; Bias-variance trade-off

  Prof. P.S. Sastry

  Playing
• Mod-10 Lec-39 Assessing Learnt classifiers; Cross Validation;

  Prof. P.S. Sastry

  Play
• Mod-11 Lec-40 Bootstrap, Bagging and Boosting; Classifier Ensembles; AdaBoost

  Prof. P.S. Sastry

  Play
• Mod-11 Lec-41 Risk minimization view of AdaBoost

  Prof. P.S. Sastry

  Play