Installing the libraries¶
In [1]:
pip install pandas numpy scikit-learn
Requirement already satisfied: pandas in c:\users\erhan\anaconda3\lib\site-packages (2.2.3) Requirement already satisfied: numpy in c:\users\erhan\anaconda3\lib\site-packages (2.1.3) Requirement already satisfied: scikit-learn in c:\users\erhan\anaconda3\lib\site-packages (1.6.1) Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\erhan\anaconda3\lib\site-packages (from pandas) (2.9.0.post0) Requirement already satisfied: pytz>=2020.1 in c:\users\erhan\anaconda3\lib\site-packages (from pandas) (2024.1) Requirement already satisfied: tzdata>=2022.7 in c:\users\erhan\anaconda3\lib\site-packages (from pandas) (2025.2) Requirement already satisfied: scipy>=1.6.0 in c:\users\erhan\anaconda3\lib\site-packages (from scikit-learn) (1.15.3) Requirement already satisfied: joblib>=1.2.0 in c:\users\erhan\anaconda3\lib\site-packages (from scikit-learn) (1.4.2) Requirement already satisfied: threadpoolctl>=3.1.0 in c:\users\erhan\anaconda3\lib\site-packages (from scikit-learn) (3.5.0) Requirement already satisfied: six>=1.5 in c:\users\erhan\anaconda3\lib\site-packages (from python-dateutil>=2.8.2->pandas) (1.17.0) Note: you may need to restart the kernel to use updated packages.
In [2]:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.preprocessing import LabelEncoder, StandardScaler
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix
Loading the Dataset¶
In [3]:
df=pd.read_csv('kdd_test.csv')
df.head()
Out[3]:
| duration | protocol_type | service | flag | src_bytes | dst_bytes | land | wrong_fragment | urgent | hot | ... | dst_host_srv_count | dst_host_same_srv_rate | dst_host_diff_srv_rate | dst_host_same_src_port_rate | dst_host_srv_diff_host_rate | dst_host_serror_rate | dst_host_srv_serror_rate | dst_host_rerror_rate | dst_host_srv_rerror_rate | labels | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 5 | tcp | smtp | SF | 2429 | 475 | 0 | 0 | 0 | 0 | ... | 57 | 0.11 | 0.02 | 0.01 | 0.04 | 0.0 | 0.0 | 0.87 | 0.0 | normal |
| 1 | 0 | udp | domain_u | SF | 45 | 134 | 0 | 0 | 0 | 0 | ... | 155 | 0.80 | 0.02 | 0.01 | 0.00 | 0.0 | 0.0 | 0.00 | 0.0 | normal |
| 2 | 0 | udp | domain_u | SF | 45 | 80 | 0 | 0 | 0 | 0 | ... | 255 | 1.00 | 0.00 | 0.01 | 0.00 | 0.0 | 0.0 | 0.00 | 0.0 | normal |
| 3 | 1979 | udp | other | SF | 145 | 105 | 0 | 0 | 0 | 0 | ... | 2 | 0.01 | 0.84 | 1.00 | 0.00 | 0.0 | 0.0 | 0.00 | 0.0 | normal |
| 4 | 14462 | tcp | other | RSTR | 1 | 0 | 0 | 0 | 0 | 0 | ... | 2 | 0.01 | 0.68 | 1.00 | 0.00 | 0.0 | 0.0 | 1.00 | 1.0 | portsweep |
5 rows × 42 columns
Checking the info of the dataset¶
In [4]:
df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 22544 entries, 0 to 22543 Data columns (total 42 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 duration 22544 non-null int64 1 protocol_type 22544 non-null object 2 service 22544 non-null object 3 flag 22544 non-null object 4 src_bytes 22544 non-null int64 5 dst_bytes 22544 non-null int64 6 land 22544 non-null int64 7 wrong_fragment 22544 non-null int64 8 urgent 22544 non-null int64 9 hot 22544 non-null int64 10 num_failed_logins 22544 non-null int64 11 logged_in 22544 non-null int64 12 num_compromised 22544 non-null int64 13 root_shell 22544 non-null int64 14 su_attempted 22544 non-null int64 15 num_root 22544 non-null int64 16 num_file_creations 22544 non-null int64 17 num_shells 22544 non-null int64 18 num_access_files 22544 non-null int64 19 num_outbound_cmds 22544 non-null int64 20 is_host_login 22544 non-null int64 21 is_guest_login 22544 non-null int64 22 count 22544 non-null int64 23 srv_count 22544 non-null int64 24 serror_rate 22544 non-null float64 25 srv_serror_rate 22544 non-null float64 26 rerror_rate 22544 non-null float64 27 srv_rerror_rate 22544 non-null float64 28 same_srv_rate 22544 non-null float64 29 diff_srv_rate 22544 non-null float64 30 srv_diff_host_rate 22544 non-null float64 31 dst_host_count 22544 non-null int64 32 dst_host_srv_count 22544 non-null int64 33 dst_host_same_srv_rate 22544 non-null float64 34 dst_host_diff_srv_rate 22544 non-null float64 35 dst_host_same_src_port_rate 22544 non-null float64 36 dst_host_srv_diff_host_rate 22544 non-null float64 37 dst_host_serror_rate 22544 non-null float64 38 dst_host_srv_serror_rate 22544 non-null float64 39 dst_host_rerror_rate 22544 non-null float64 40 dst_host_srv_rerror_rate 22544 non-null float64 41 labels 22544 non-null object dtypes: float64(15), int64(23), object(4) memory usage: 7.2+ MB
Finding how many missing values each column has¶
In [5]:
df.isnull().sum()
Out[5]:
duration 0 protocol_type 0 service 0 flag 0 src_bytes 0 dst_bytes 0 land 0 wrong_fragment 0 urgent 0 hot 0 num_failed_logins 0 logged_in 0 num_compromised 0 root_shell 0 su_attempted 0 num_root 0 num_file_creations 0 num_shells 0 num_access_files 0 num_outbound_cmds 0 is_host_login 0 is_guest_login 0 count 0 srv_count 0 serror_rate 0 srv_serror_rate 0 rerror_rate 0 srv_rerror_rate 0 same_srv_rate 0 diff_srv_rate 0 srv_diff_host_rate 0 dst_host_count 0 dst_host_srv_count 0 dst_host_same_srv_rate 0 dst_host_diff_srv_rate 0 dst_host_same_src_port_rate 0 dst_host_srv_diff_host_rate 0 dst_host_serror_rate 0 dst_host_srv_serror_rate 0 dst_host_rerror_rate 0 dst_host_srv_rerror_rate 0 labels 0 dtype: int64
Generates summary statistics for all numeric columns¶
In [6]:
df.describe()
Out[6]:
| duration | src_bytes | dst_bytes | land | wrong_fragment | urgent | hot | num_failed_logins | logged_in | num_compromised | ... | dst_host_count | dst_host_srv_count | dst_host_same_srv_rate | dst_host_diff_srv_rate | dst_host_same_src_port_rate | dst_host_srv_diff_host_rate | dst_host_serror_rate | dst_host_srv_serror_rate | dst_host_rerror_rate | dst_host_srv_rerror_rate | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| count | 22544.000000 | 2.254400e+04 | 2.254400e+04 | 22544.000000 | 22544.000000 | 22544.000000 | 22544.000000 | 22544.000000 | 22544.000000 | 22544.000000 | ... | 22544.000000 | 22544.000000 | 22544.000000 | 22544.000000 | 22544.000000 | 22544.000000 | 22544.000000 | 22544.000000 | 22544.000000 | 22544.000000 |
| mean | 258.858499 | 1.226873e+04 | 2.966787e+03 | 0.000089 | 0.016989 | 0.000177 | 0.155562 | 0.008783 | 0.406095 | 0.259359 | ... | 186.878549 | 123.129613 | 0.545571 | 0.084905 | 0.141117 | 0.028275 | 0.229375 | 0.225729 | 0.155008 | 0.153560 |
| std | 2184.355090 | 4.665783e+05 | 6.293205e+04 | 0.009419 | 0.217507 | 0.016313 | 1.699324 | 0.104942 | 0.491114 | 11.370701 | ... | 97.508604 | 111.577695 | 0.447079 | 0.199813 | 0.305538 | 0.105966 | 0.410665 | 0.412500 | 0.339255 | 0.350511 |
| min | 0.000000 | 0.000000e+00 | 0.000000e+00 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | ... | 1.000000 | 1.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
| 25% | 0.000000 | 0.000000e+00 | 0.000000e+00 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | ... | 95.000000 | 12.000000 | 0.050000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
| 50% | 0.000000 | 4.500000e+01 | 0.000000e+00 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | ... | 255.000000 | 85.000000 | 0.670000 | 0.020000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
| 75% | 0.000000 | 2.750000e+02 | 5.950000e+02 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 1.000000 | 0.000000 | ... | 255.000000 | 255.000000 | 1.000000 | 0.070000 | 0.050000 | 0.010000 | 0.100000 | 0.010000 | 0.010000 | 0.000000 |
| max | 54451.000000 | 6.282565e+07 | 5.153771e+06 | 1.000000 | 3.000000 | 2.000000 | 30.000000 | 5.000000 | 1.000000 | 796.000000 | ... | 255.000000 | 255.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 |
8 rows × 38 columns
Automatically identifying prediction target column¶
In [7]:
target = df.columns[-1]
Counts how many times each unique class appears¶
In [8]:
df[target].value_counts()
Out[8]:
labels normal 11245 neptune 6654 satan 698 smurf 540 ipsweep 479 guess_passwd 411 portsweep 371 mscan 310 warezmaster 279 apache2 228 back 227 processtable 211 nmap 206 warezclient 107 teardrop 104 snmpguess 99 mailbomb 94 saint 93 snmpgetattack 43 httptunnel 41 pod 35 buffer_overflow 17 multihop 9 ps 9 sendmail 8 rootkit 4 loadmodule 4 xlock 4 xterm 3 named 2 imap 2 land 2 ftp_write 2 xsnoop 1 phf 1 perl 1 Name: count, dtype: int64
Visualizes how many samples exist per class¶
In [9]:
# Plot target distribution
plt.figure(figsize=(13,11))
sns.countplot(x=df[target])
plt.title("Attack Class Distribution")
plt.xticks(rotation=45)
plt.show()
Visualizing how strongly numeric features in the dataset¶
In [10]:
# Correlation heatmap (optional for large datasets)
plt.figure(figsize=(11,8))
sns.heatmap(df.corr(numeric_only=True).abs(), cmap="magma")
plt.title("Correlation Heatmap")
plt.show()
DATA PREPROCESSING¶
In [11]:
# Separate features & labels
X = df.drop(target, axis=1)
y = df[target]
In [12]:
# Identify categorical columns
categorical_cols = X.select_dtypes(include=["object"]).columns
In [13]:
# Label encoding
le = LabelEncoder()
for col in categorical_cols:
X[col] = le.fit_transform(X[col])
In [14]:
# Encode target if needed
y = LabelEncoder().fit_transform(y)
In [15]:
# Feature Scaling
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
In [16]:
# Train-Test Split
X_train, X_test, y_train, y_test = train_test_split(
X_scaled, y, test_size=0.25, random_state=42
)
Random Forest¶
- I will build the model using RandomForestClassifier
In [17]:
# Building Random Forest model
rf = RandomForestClassifier(n_estimators=100, random_state=42)
# Fitting Random Forest model
rf.fit(X_train, y_train)
rf_pred = rf.predict(X_test)
print("\nRandom Forest Accuracy:", accuracy_score(y_test, rf_pred))
print("\nClassification Report:\n", classification_report(y_test, rf_pred))
Random Forest Accuracy: 0.9877572746628814
Classification Report:
precision recall f1-score support
0 1.00 1.00 1.00 51
1 1.00 1.00 1.00 42
2 1.00 0.14 0.25 7
4 1.00 0.98 0.99 114
5 1.00 1.00 1.00 9
6 0.00 0.00 0.00 1
7 1.00 0.95 0.97 99
8 0.00 0.00 0.00 2
9 0.00 0.00 0.00 1
10 1.00 1.00 1.00 24
11 0.97 0.99 0.98 68
12 0.00 0.00 0.00 1
13 0.00 0.00 0.00 1
14 1.00 1.00 1.00 1714
15 0.92 0.98 0.95 50
16 0.99 0.99 0.99 2796
19 1.00 1.00 1.00 7
20 0.99 0.96 0.97 94
21 1.00 1.00 1.00 48
22 0.00 0.00 0.00 1
23 0.00 0.00 0.00 2
24 0.92 0.85 0.88 27
25 0.94 0.97 0.95 178
27 1.00 1.00 1.00 131
28 0.80 0.29 0.42 14
29 1.00 1.00 1.00 26
30 0.91 1.00 0.95 31
31 0.91 0.91 0.91 22
32 0.99 0.99 0.99 71
33 0.00 0.00 0.00 2
34 0.00 0.00 0.00 1
35 0.00 0.00 0.00 1
accuracy 0.99 5636
macro avg 0.67 0.62 0.63 5636
weighted avg 0.99 0.99 0.99 5636
C:\Users\erhan\anaconda3\Lib\site-packages\sklearn\metrics\_classification.py:1565: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, f"{metric.capitalize()} is", len(result))
C:\Users\erhan\anaconda3\Lib\site-packages\sklearn\metrics\_classification.py:1565: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, f"{metric.capitalize()} is", len(result))
C:\Users\erhan\anaconda3\Lib\site-packages\sklearn\metrics\_classification.py:1565: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, f"{metric.capitalize()} is", len(result))
Creating a confusion matrix for Random Forest predictions¶
In [18]:
# Generates a confusion matrix for the Random Forest predictions.
cm_rf = confusion_matrix(y_test, rf_pred)
cm_rf = confusion_matrix(y_test, rf_pred)
plt.figure(figsize=(13,8))
# Displays the confusion matrix as a color-coded grid.
sns.heatmap(cm_rf, annot=True, fmt="d", cmap="magma")
plt.title("Random Forest - Confusion Matrix")
plt.xlabel("Predicted")
plt.ylabel("Actual")
plt.show()
Gradient Boosting¶
- I will build the model using GradientBoostingClassifier
In [19]:
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report
import seaborn as sns
import matplotlib.pyplot as plt
print("\nTraining Gradient Boosting...")
# Faster version
gb = GradientBoostingClassifier(
n_estimators=60, # default 100, reduced for speed
max_depth=2, # shallow trees, faster
learning_rate=0.1,
random_state=42
)
gb.fit(X_train, y_train)
gb_pred = gb.predict(X_test)
print("\nGradient Boosting Accuracy:", accuracy_score(y_test, gb_pred))
print("\nClassification Report:\n", classification_report(y_test, gb_pred))
Training Gradient Boosting...
Gradient Boosting Accuracy: 0.9614975159687722
Classification Report:
precision recall f1-score support
0 1.00 0.98 0.99 51
1 0.98 0.98 0.98 42
2 0.22 0.71 0.33 7
4 0.98 0.94 0.96 114
5 0.22 1.00 0.36 9
6 0.00 0.00 0.00 1
7 0.97 0.93 0.95 99
8 0.00 0.00 0.00 2
9 0.00 0.00 0.00 1
10 1.00 1.00 1.00 24
11 0.92 0.85 0.89 68
12 0.00 0.00 0.00 1
13 0.00 0.00 0.00 1
14 1.00 1.00 1.00 1714
15 0.90 0.92 0.91 50
16 0.97 0.97 0.97 2796
19 0.00 0.00 0.00 7
20 0.96 0.83 0.89 94
21 1.00 1.00 1.00 48
22 0.00 0.00 0.00 1
23 0.00 0.00 0.00 2
24 1.00 0.30 0.46 27
25 0.85 0.88 0.86 178
26 0.00 0.00 0.00 0
27 1.00 0.99 1.00 131
28 0.00 0.00 0.00 14
29 1.00 0.92 0.96 26
30 0.94 0.94 0.94 31
31 0.94 0.77 0.85 22
32 1.00 0.93 0.96 71
33 0.25 0.50 0.33 2
34 0.00 0.00 0.00 1
35 0.00 0.00 0.00 1
accuracy 0.96 5636
macro avg 0.55 0.56 0.53 5636
weighted avg 0.97 0.96 0.96 5636
C:\Users\erhan\anaconda3\Lib\site-packages\sklearn\metrics\_classification.py:1565: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, f"{metric.capitalize()} is", len(result))
C:\Users\erhan\anaconda3\Lib\site-packages\sklearn\metrics\_classification.py:1565: UndefinedMetricWarning: Recall is ill-defined and being set to 0.0 in labels with no true samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, f"{metric.capitalize()} is", len(result))
C:\Users\erhan\anaconda3\Lib\site-packages\sklearn\metrics\_classification.py:1565: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, f"{metric.capitalize()} is", len(result))
C:\Users\erhan\anaconda3\Lib\site-packages\sklearn\metrics\_classification.py:1565: UndefinedMetricWarning: Recall is ill-defined and being set to 0.0 in labels with no true samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, f"{metric.capitalize()} is", len(result))
C:\Users\erhan\anaconda3\Lib\site-packages\sklearn\metrics\_classification.py:1565: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, f"{metric.capitalize()} is", len(result))
C:\Users\erhan\anaconda3\Lib\site-packages\sklearn\metrics\_classification.py:1565: UndefinedMetricWarning: Recall is ill-defined and being set to 0.0 in labels with no true samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, f"{metric.capitalize()} is", len(result))
Creating a confusion matrix for Gradient Boosting predictions¶
In [20]:
# Generates a confusion matrix for the Gradient Boosting predictions.
cm = confusion_matrix(y_test, gb_pred)
plt.figure(figsize=(13,8))
# Displays the confusion matrix as a color-coded grid.
sns.heatmap(cm, annot=True, fmt="d", cmap="magma")
plt.title("Gradient Boosting - Confusion Matrix (Fast Version)")
plt.xlabel("Predicted")
plt.ylabel("Actual")
plt.show()
Hyperparameter Tuning¶
In [21]:
# Prints a message to indicate that hyperparameter tuning
print("\nPerforming Hyperparameter Tuning for Random Forest...")
param_grid_rf = {
"n_estimators": [100, 150],
"max_depth": [10, 20, None],
"min_samples_split": [2, 5],
}
grid_rf = GridSearchCV(
RandomForestClassifier(random_state=42),
param_grid_rf,
cv=3,
n_jobs=-1,
verbose=1
)
# Trains the GridSearch model on the training data.
grid_rf.fit(X_train, y_train)
# Prints the best hyperparameters found by GridSearchCV.
print("\nBest RF Parameters:", grid_rf.best_params_)
# Extracts the best Random Forest model and makes predictions on the test set.
rf_best = grid_rf.best_estimator_
rf_best_pred = rf_best.predict(X_test)
print("\nTuned RF Accuracy:", accuracy_score(y_test, rf_best_pred))
print("\nConfusion Matrix (Tuned RF):")
print(confusion_matrix(y_test, rf_best_pred))
Performing Hyperparameter Tuning for Random Forest... Fitting 3 folds for each of 12 candidates, totalling 36 fits
C:\Users\erhan\anaconda3\Lib\site-packages\sklearn\model_selection\_split.py:805: UserWarning: The least populated class in y has only 1 members, which is less than n_splits=3. warnings.warn(
Best RF Parameters: {'max_depth': 20, 'min_samples_split': 2, 'n_estimators': 150}
Tuned RF Accuracy: 0.9872249822569198
Confusion Matrix (Tuned RF):
[[51 0 0 ... 0 0 0]
[ 0 42 0 ... 0 0 0]
[ 0 0 1 ... 0 0 0]
...
[ 0 0 0 ... 0 0 0]
[ 0 0 0 ... 0 0 0]
[ 0 0 0 ... 0 0 0]]
Model Comparison¶
In [22]:
# Prepare results
model_names = ["Random Forest", "Gradient Boosting", "Random Forest (Tuned)"]
accuracies = [
accuracy_score(y_test, rf_pred),
accuracy_score(y_test, gb_pred),
accuracy_score(y_test, rf_best_pred)
]
# Convert to DataFrame
comparison_df = pd.DataFrame({
"Model": model_names,
"Accuracy": accuracies
})
# Visualization
plt.figure(figsize=(13,11))
sns.barplot(x="Model", y="Accuracy", data=comparison_df)
plt.title("Final Model Accuracy Comparison")
plt.ylim(0, 1)
plt.ylabel("Accuracy Score")
plt.xlabel("Machine Learning Models")
plt.xticks(rotation=15)
plt.show()
# Automatically displays the table
comparison_df
Out[22]:
| Model | Accuracy | |
|---|---|---|
| 0 | Random Forest | 0.987757 |
| 1 | Gradient Boosting | 0.961498 |
| 2 | Random Forest (Tuned) | 0.987225 |
Summary¶
In [23]:
import pandas as pd
from sklearn.metrics import accuracy_score
# Collect all model accuracies
summary_data = {
"Model": ["Random Forest", "Gradient Boosting", "Random Forest (Tuned)"],
"Accuracy": [
accuracy_score(y_test, rf_pred),
accuracy_score(y_test, gb_pred),
accuracy_score(y_test, rf_best_pred)
]
}
# Create DataFrame
summary_df = pd.DataFrame(summary_data)
# Identify the best model
best_model = summary_df.loc[summary_df["Accuracy"].idxmax()]
summary_df, best_model
Out[23]:
( Model Accuracy 0 Random Forest 0.987757 1 Gradient Boosting 0.961498 2 Random Forest (Tuned) 0.987225, Model Random Forest Accuracy 0.987757 Name: 0, dtype: object)
Observations:¶
- Random Forest model achieved a slightly higher accuracy (0.987757) compared to the tuned model (0.987225).
RESULT¶
The experimental results confirmed that ensemble learning is a highly effective strategy for intrusion detection. The Random Forest model emerged as the superior classifier and demonstrated exceptional robustness against overfitting. While Gradient Boosting provided competitive performance, it required significantly higher computational resources and training time.