Chapter 7AdaBoostZhi-Hua Zhou and Yang YuContents7.1Introduction........................................................... 1277.2 TheAlgorithm......................................................... 1287.2.1 Notations....................................................... 1287.2.2 A GeneralBoostingProcedure.................................. 1297.2.3 TheAdaBoostAlgorithm....................................... 1307.3 Illustrative Examples ................................................... 1337.3.1 SolvingXORProblem.......................................... 1337.3.2 PerformanceonRealData ...................................... 1347.4 RealApplication....................................................... 1367.5 Advanced Topics ....................................................... 1387.5.1 TheoreticalIssues............................................... 1387.5.2 MulticlassAdaBoost............................................ 1427.5.3 OtherAdvanced Topics ......................................... 1457.6 Software Implementations .............................................. 1457.7 Exercises .............................................................. 146References.................................................................. 1477.1 Introdu ctionGeneralization ability, which characterizes how well the result learned from a giventraining data set can be applied to unseen new data, is the most central concept inmachine learning. Researchers have devoted tremendous efforts to the pursuit of tech-niques that could lead to a learning system with strong generalization ability. Oneof the most successful paradigms is ensemble learning [32]. In contrast to ordinarymachine learning approaches which try to generate one learner from training data,ensemble methods try to construct a set of base learners and combine them. Baselearners are usu...
