Back to Home
You will see two steps in this article-
But I will mostly focus on the result analysis. Before the above steps, we have to do some preprocessing to the model. Because the dataset contains missing values. As column values are repeated, therefore, missing values can be replaced by the mode values of that particular column. You may apply other preprocessing techniques to get more accurate high performance if you want. But for the time being, I will replace repeated values with mode value.
I am a lazy developer. I don’t like to write code always. So when I save some time from coding, I take a nap. The same thing happens here. Instead of writing a bunch of python code, I have used WEKA 3.8 tool to apply the AdaBoost and LogitBoost. Weka is a tool for data mining tasks. Believe me, it's gonna save your time a lot. You may watch Netflix more or take a nap like me, haha! Hey, but I am not demotivating you to write code.
It is a visualization tool that contains algorithms for data analysis. It provides tools for data pre-processing, classification, regression, clustering, association rules, and visualization. You can find the tutorials for Weka hereandhere. Andyou can download Weka from here.
I have a plan to write more details on Weka in near future.
Accuracy is the common measurement for comparison among machine learning algorithms. Accuracy is calculated by summation of true positive and true negative divided by total instances (Equation 1).
Let’s talk about TP, TN, FP, and FN a bit.
True Positive: You predicted positive and it’s true. You predicted that a woman is pregnant and she actually is.
True Negative: You predicted negative and it’s true. You predicted that a man is not pregnant and he actually is not.
False Positive: You predicted positive and it’s false. You predicted that a man is pregnant but he actually is not.
False Negative: You predicted negative and it’s false. You predicted that a woman is not pregnant but she actually is.
Both boosting algorithms AdaBoost and LogitBoost performed very well and
achieved good accuracy against the CKD dataset. Because as I said before boosting algorithms iteratively do the classification until it achieves better accuracy. Performance comparison between AdaBoost and LogitBoost based on Accuracy is shown in Table 3 and Figure 3.
Root mean squared error interpretation: It tells the error between the actual result and predicted result. Equation 2 shows the formula of RMSE. It ranges from 0 to 1. As the low value of RMSE means a better fit model, the performance of AdaBoost was lower than LogitBoost’s performance (Figure 4).
F-Measure interpretation: F-measure is a combined metric of precision and recall. Precision is defined as how many patients actually have CKD among all people who are thought to have CKD. Equation 3 describes the formula for calculating the f-measure. Recall refers to how many models can identify
among the patients who truly have CKD. Higher value of precision and recall is good. Table V contains precision and recall values which were resulted from applying AdaBoost and LogitBoost. It can be interpreted that LogitBoost performed well, as precision, recall and harmonic mean f-measure values
of LogitBoost are higher (Figure 4).
Kappa statistics interpretation: Value of Kappa ranges between 0 to 1 inclusive. When the Kappa value is above 0.75, that means there is a strong relationship between actual values and predicted values. Here, the Kappa value of LogitBoost is higher than the AdaBoost, which means predictions of LogitBoost are more accurate than the predictions of AdaBoost (Figure 5).
Both Boosting algorithms had an accuracy close to 100%, but we can come to a point that LogitBoost is the winner. As Logitboost is an updated version of Adaboost, it comes up with satisfactory results.
If you like this article, make sure to follow my Medium profile and check out other articles of mine!