...
The gain due to the splitting of a node A into the nodes B1 and B2, which depends on the chosen cut, is given by: ΔI=I(A)-I(B)-I(B2) , where I denotes the adopted metric (G or E, in case of the Gini index or cross entropy introduced above). By varying the cut, the optimal gain may be achieved.
...
A machine learning model has two types of parameters. The first type of parameters parameter is the parameters that are learned through a machine learning model while the second type of parameters parameter is the hyperparameter that we pass to the machine learning model.
Normally we set the value for these hyperparameters by hand, as we did for our ANN, and see what parameters result in the best performance. However, randomly selecting the parameters for the algorithm can be exhaustive.
...
Grid Search algorithm basically tries all possible combinations of parameter values and returns the combination with the highest accuracy. The Grid Search algorithm can be very slow, owing to the potentially huge number of combinations to test. Furthermore, the cross-validation further increases the execution!
For these reasons, the algorithm is commented in on the following code cells and images of the outputs are left to you!
...