FAFU机器学习05-1 Model Evaluation课件VIP专享VIP免费

Foundations of Machine LearningModel EvaluationModel EvaluationLesson 4- 1Model EvaluationLesson 4- 2Q: How do we estimate the performance of these models using different machine learning algorithms?Model EvaluationLesson 4- 3Q: How do we estimate the performance of these models with different parameters?Blue: Observed dataGreen: true distributionPolynomial Curve Fitting Red: Predicted curveModel EvaluationLesson 4- 4Q: How do we estimate the performance of machine learning model?Answer:① We want to estimate the generalization performance, the predictive performance of our model on future (unseen) data.②We want to increase the predictive performance by tweaking the learning algorithm and selecting the best performing model from a given hypothesis space.③we want to compare different algorithms, selecting the best-performing one as well as the best performing model from the algorithm’s hypothesis space.Basic ConceptsModel EvaluationLesson 4- 5 i.i.d.: Independent and identically distribution means that all samples have been drawn from the same probability distribution and are statistically independent from each other.  Accuracy : the number of correct predictions a divided by the number of samples m Error Rate : the number of wrong predictions b divided by the number of samples mmaAccAccmbErr-1Basic ConceptsModel EvaluationLesson 4- 6Error （误差） : generally speaking, the difference between expected output value from the model and real sample value Training error （训练误差） : empirical error （经验误差） , is the error we get applying the model to the same data from which we trained.Test error （测试误差） : is the error that we incur on new data..Generalization error （泛化误差） : out-of-sample er...