Question 18
Consider the following two ensemble regression architectures trained on the same dataset:
ensemble1 computes a weighted sum of predictions where later trees have higher weights; ensemble2 takes a simple arithmetic mean of all trees.
ensemble1 reduces variance by averaging independent predictions from 100 trees, meaning increasing n_estimators to 500 will ideally not cause overfitting.
ensemble2 reduces bias sequentially, meaning increasing n_estimators to 500 carries no risk of overfitting.
Both models output predictions by calculating the majority vote across the terminal leaf nodes of all individual base estimators.