Assignment ML Model

Student name: Suraj Ramesh Babu

ID No.: 2513205

Module code: ACCA7026

Program: MSc Software Engineering and AI

1) Importing required Libraries

# Importing core libraries for data manipulation and numerical operations
import pandas as pd
import pandas as pd
import numpy as np
# Libraries for data visualization
import matplotlib.pyplot as plt
import seaborn as sns
# To suppress unnecessary warnings for cleaner output
import warnings
warnings.filterwarnings("ignore")
# Tools for preprocessing: encoding categorical data and scaling numerical values
from sklearn.preprocessing import LabelEncoder, StandardScaler
# Tools for splitting data, hyperparameter tuning, and model training
from sklearn.model_selection import train_test_split, GridSearchCV
# Machine learning models used in the project
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
# Evaluation metrics to measure model performance
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report

• Importing essential libraries for data handling, visualization, preprocessing, model building, and evaluation.
• These libraries provide tools to load data, encode features, scale values, train ML models, tune hyperparameters, and measure performance

2) Reding the csv files

# Loading the training dataset from the CSV file
train_df = pd.read_csv("kdd_train.csv")
# Loading the test dataset
test_df = pd.read_csv("kdd_test.csv")

• The code allows you to deal with the dataset in Python by reading a CSV file called kdd_test.csv and loading it into a pandas DataFrame. Until alternative files are substituted, the training and testing datasets are similar since train_df and test_df load the same file. Your machine-learning workflow will be built on these DataFrames, which will subsequently be utilized for data preprocessing, model training, and evaluation.

3) Data Preprocessing

# # Displays  the first five rows of the training dataset
train_df.head()

	duration	protocol_type	service	flag	src_bytes	dst_bytes	land	wrong_fragment	urgent	hot	...	dst_host_srv_count	dst_host_same_srv_rate	dst_host_diff_srv_rate	dst_host_same_src_port_rate	dst_host_srv_diff_host_rate	dst_host_serror_rate	dst_host_srv_serror_rate	dst_host_rerror_rate	dst_host_srv_rerror_rate	labels
0	5	tcp	smtp	SF	2429	475	0	0	0	0	...	57	0.11	0.02	0.01	0.04	0.0	0.0	0.87	0.0	normal
1	0	udp	domain_u	SF	45	134	0	0	0	0	...	155	0.80	0.02	0.01	0.00	0.0	0.0	0.00	0.0	normal
2	0	udp	domain_u	SF	45	80	0	0	0	0	...	255	1.00	0.00	0.01	0.00	0.0	0.0	0.00	0.0	normal
3	1979	udp	other	SF	145	105	0	0	0	0	...	2	0.01	0.84	1.00	0.00	0.0	0.0	0.00	0.0	normal
4	14462	tcp	other	RSTR	1	0	0	0	0	0	...	2	0.01	0.68	1.00	0.00	0.0	0.0	1.00	1.0	portsweep

5 rows × 42 columns

# Used to count the number of rows
train_df.count()

duration                       125973
protocol_type                  125973
service                        125973
flag                           125973
src_bytes                      125973
dst_bytes                      125973
land                           125973
wrong_fragment                 125973
urgent                         125973
hot                            125973
num_failed_logins              125973
logged_in                      125973
num_compromised                125973
root_shell                     125973
su_attempted                   125973
num_root                       125973
num_file_creations             125973
num_shells                     125973
num_access_files               125973
num_outbound_cmds              125973
is_host_login                  125973
is_guest_login                 125973
count                          125973
srv_count                      125973
serror_rate                    125973
srv_serror_rate                125973
rerror_rate                    125973
srv_rerror_rate                125973
same_srv_rate                  125973
diff_srv_rate                  125973
srv_diff_host_rate             125973
dst_host_count                 125973
dst_host_srv_count             125973
dst_host_same_srv_rate         125973
dst_host_diff_srv_rate         125973
dst_host_same_src_port_rate    125973
dst_host_srv_diff_host_rate    125973
dst_host_serror_rate           125973
dst_host_srv_serror_rate       125973
dst_host_rerror_rate           125973
dst_host_srv_rerror_rate       125973
labels                         125973
dtype: int64

• It is helpful for preliminary examination since it lets you verify sample values, column names, and possible problems like inconsistent or missing data.
• Usually used in conjunction with exploratory data analysis (EDA), this command provides a brief summary prior to more thorough processing or cleaning.

#  Displays  the first five rows of the testing dataset
test_df.head()

	duration	protocol_type	service	flag	src_bytes	dst_bytes	land	wrong_fragment	urgent	hot	...	dst_host_srv_count	dst_host_same_srv_rate	dst_host_diff_srv_rate	dst_host_same_src_port_rate	dst_host_srv_diff_host_rate	dst_host_serror_rate	dst_host_srv_serror_rate	dst_host_rerror_rate	dst_host_srv_rerror_rate	labels
0	5	tcp	smtp	SF	2429	475	0	0	0	0	...	57	0.11	0.02	0.01	0.04	0.0	0.0	0.87	0.0	normal
1	0	udp	domain_u	SF	45	134	0	0	0	0	...	155	0.80	0.02	0.01	0.00	0.0	0.0	0.00	0.0	normal
2	0	udp	domain_u	SF	45	80	0	0	0	0	...	255	1.00	0.00	0.01	0.00	0.0	0.0	0.00	0.0	normal
3	1979	udp	other	SF	145	105	0	0	0	0	...	2	0.01	0.84	1.00	0.00	0.0	0.0	0.00	0.0	normal
4	14462	tcp	other	RSTR	1	0	0	0	0	0	...	2	0.01	0.68	1.00	0.00	0.0	0.0	1.00	1.0	portsweep

5 rows × 42 columns

# Used to count the number of rows
test_df.count()

duration                       22544
protocol_type                  22544
service                        22544
flag                           22544
src_bytes                      22544
dst_bytes                      22544
land                           22544
wrong_fragment                 22544
urgent                         22544
hot                            22544
num_failed_logins              22544
logged_in                      22544
num_compromised                22544
root_shell                     22544
su_attempted                   22544
num_root                       22544
num_file_creations             22544
num_shells                     22544
num_access_files               22544
num_outbound_cmds              22544
is_host_login                  22544
is_guest_login                 22544
count                          22544
srv_count                      22544
serror_rate                    22544
srv_serror_rate                22544
rerror_rate                    22544
srv_rerror_rate                22544
same_srv_rate                  22544
diff_srv_rate                  22544
srv_diff_host_rate             22544
dst_host_count                 22544
dst_host_srv_count             22544
dst_host_same_srv_rate         22544
dst_host_diff_srv_rate         22544
dst_host_same_src_port_rate    22544
dst_host_srv_diff_host_rate    22544
dst_host_serror_rate           22544
dst_host_srv_serror_rate       22544
dst_host_rerror_rate           22544
dst_host_srv_rerror_rate       22544
labels                         22544
dtype: int64

• the quantity of non-missing (non-NaN) values in each column, which enables you to assess the completeness of your collection.Since any column with a smaller count than others probably contains null or empty entries, it is frequently used to find missing data. Before preprocessing, cleaning, or creating a machine-learning model, this command assists you in understanding the quality of the data.

4) Combined both test and train data sets

# Combine train+test for consistent encoding
full_df = pd.concat([train_df, test_df], axis=0)

# Identify categorical columns
cat_cols = ['protocol_type', 'service', 'flag']
label_encoders = {}

# Label Encoding categorical features
for col in cat_cols:
    le = LabelEncoder()
    full_df[col] = le.fit_transform(full_df[col])
    label_encoders[col] = le

# Split processed data back
train_df = full_df.iloc[:len(train_df)]
test_df = full_df.iloc[len(train_df):]

# Features and labels
X_train = train_df.drop(['labels'], axis=1)
y_train = train_df['labels']

X_test = test_df.drop(['labels'], axis=1)
y_test = test_df['labels']

# Scale numerical values
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

• All categories are identified and encoded consistently when datasets are combined.
• Avoids missing or mismatched encoded values, which could cause the model's prediction to fail on categories that haven't been encountered.
• Ensures that the test and train feature formats are the same, enabling the model to function properly.
• enhances model stability by guaranteeing consistent preprocessing, such as scaling and encoding, which must be used consistently to prevent biased or inconsistent model behavior.

5) Exploratory Data Analysis

# Display detailed information about the dataset
# number of non-null values, and memory usage
full_df.info()

<class 'pandas.core.frame.DataFrame'>
Index: 45088 entries, 0 to 22543
Data columns (total 42 columns):
 #   Column                       Non-Null Count  Dtype  
---  ------                       --------------  -----  
 0   duration                     45088 non-null  int64  
 1   protocol_type                45088 non-null  int64  
 2   service                      45088 non-null  int64  
 3   flag                         45088 non-null  int64  
 4   src_bytes                    45088 non-null  int64  
 5   dst_bytes                    45088 non-null  int64  
 6   land                         45088 non-null  int64  
 7   wrong_fragment               45088 non-null  int64  
 8   urgent                       45088 non-null  int64  
 9   hot                          45088 non-null  int64  
 10  num_failed_logins            45088 non-null  int64  
 11  logged_in                    45088 non-null  int64  
 12  num_compromised              45088 non-null  int64  
 13  root_shell                   45088 non-null  int64  
 14  su_attempted                 45088 non-null  int64  
 15  num_root                     45088 non-null  int64  
 16  num_file_creations           45088 non-null  int64  
 17  num_shells                   45088 non-null  int64  
 18  num_access_files             45088 non-null  int64  
 19  num_outbound_cmds            45088 non-null  int64  
 20  is_host_login                45088 non-null  int64  
 21  is_guest_login               45088 non-null  int64  
 22  count                        45088 non-null  int64  
 23  srv_count                    45088 non-null  int64  
 24  serror_rate                  45088 non-null  float64
 25  srv_serror_rate              45088 non-null  float64
 26  rerror_rate                  45088 non-null  float64
 27  srv_rerror_rate              45088 non-null  float64
 28  same_srv_rate                45088 non-null  float64
 29  diff_srv_rate                45088 non-null  float64
 30  srv_diff_host_rate           45088 non-null  float64
 31  dst_host_count               45088 non-null  int64  
 32  dst_host_srv_count           45088 non-null  int64  
 33  dst_host_same_srv_rate       45088 non-null  float64
 34  dst_host_diff_srv_rate       45088 non-null  float64
 35  dst_host_same_src_port_rate  45088 non-null  float64
 36  dst_host_srv_diff_host_rate  45088 non-null  float64
 37  dst_host_serror_rate         45088 non-null  float64
 38  dst_host_srv_serror_rate     45088 non-null  float64
 39  dst_host_rerror_rate         45088 non-null  float64
 40  dst_host_srv_rerror_rate     45088 non-null  float64
 41  labels                       45088 non-null  object 
dtypes: float64(15), int64(26), object(1)
memory usage: 14.8+ MB

• Column names, data types, and the quantity of non-missing values in each column are displayed in this dataset summary. Prior to preprocessing or model construction, it assists in identifying problems with data quality, such as missing values or inaccurate data types.

# Displays both numeric and non-numeric descriptive statistics for every column
full_df.describe()

	duration	protocol_type	service	flag	src_bytes	dst_bytes	land	wrong_fragment	urgent	hot	...	dst_host_count	dst_host_srv_count	dst_host_same_srv_rate	dst_host_diff_srv_rate	dst_host_same_src_port_rate	dst_host_srv_diff_host_rate	dst_host_serror_rate	dst_host_srv_serror_rate	dst_host_rerror_rate	dst_host_srv_rerror_rate
count	45088.000000	45088.000000	45088.000000	45088.000000	4.508800e+04	4.508800e+04	45088.000000	45088.000000	45088.000000	45088.000000	...	45088.000000	45088.000000	45088.000000	45088.000000	45088.000000	45088.000000	45088.000000	45088.000000	45088.000000	45088.000000
mean	258.858499	1.060593	30.820085	6.931867	1.226873e+04	2.966787e+03	0.000089	0.016989	0.000177	0.155562	...	186.878549	123.129613	0.545571	0.084905	0.141117	0.028275	0.229375	0.225729	0.155008	0.153560
std	2184.330866	0.418760	15.093458	2.865598	4.665731e+05	6.293136e+04	0.009419	0.217505	0.016313	1.699305	...	97.507522	111.576458	0.447074	0.199811	0.305535	0.105965	0.410661	0.412495	0.339252	0.350507
min	0.000000	0.000000	0.000000	0.000000	0.000000e+00	0.000000e+00	0.000000	0.000000	0.000000	0.000000	...	1.000000	1.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
25%	0.000000	1.000000	22.000000	5.000000	0.000000e+00	0.000000e+00	0.000000	0.000000	0.000000	0.000000	...	95.000000	12.000000	0.050000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
50%	0.000000	1.000000	23.000000	9.000000	4.500000e+01	0.000000e+00	0.000000	0.000000	0.000000	0.000000	...	255.000000	85.000000	0.670000	0.020000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
75%	0.000000	1.000000	46.000000	9.000000	2.750000e+02	5.950000e+02	0.000000	0.000000	0.000000	0.000000	...	255.000000	255.000000	1.000000	0.070000	0.050000	0.010000	0.100000	0.010000	0.010000	0.000000
max	54451.000000	2.000000	65.000000	10.000000	6.282565e+07	5.153771e+06	1.000000	3.000000	2.000000	30.000000	...	255.000000	255.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000

8 rows × 41 columns

• It generates summary statistics for all numeric columns, such as mean, standard deviation, minimum, maximum, and quartiles.
• It helps you quickly understand the distribution, spread, and overall behavior of numerical features, which is essential during Exploratory Data Analysis (EDA).

# Select only the numeric columns (integer and float types) from the dataset for analysis
numeric_df = full_df.select_dtypes(include=['int64', 'float64'])

# Compute the correlation matrix to measure how strongly each numeric feature is related
corr_matrix = numeric_df.corr()

# Display the correlation matrix showing pairwise correlation values between all numeric features
corr_matrix

	duration	protocol_type	service	flag	src_bytes	dst_bytes	land	wrong_fragment	urgent	hot	...	dst_host_count	dst_host_srv_count	dst_host_same_srv_rate	dst_host_diff_srv_rate	dst_host_same_src_port_rate	dst_host_srv_diff_host_rate	dst_host_serror_rate	dst_host_srv_serror_rate	dst_host_rerror_rate	dst_host_srv_rerror_rate
duration	1.000000	0.034765	0.107219	-0.021154	0.008243	0.016966	-0.001116	-0.009257	-0.000770	-0.001580	...	0.054504	-0.085035	-0.091574	0.201141	0.176762	-0.024386	-0.040706	-0.038633	0.110226	0.113938
protocol_type	0.034765	1.000000	0.056937	0.104430	-0.003957	-0.006525	-0.001363	0.140160	-0.001574	-0.013246	...	0.216810	0.103722	0.019163	0.122548	-0.167177	-0.323542	-0.079571	-0.079183	-0.038347	-0.063393
service	0.107219	0.056937	1.000000	-0.306949	-0.027654	-0.001592	-0.008001	0.063478	0.014182	-0.063407	...	0.221458	-0.389706	-0.439143	0.253447	-0.124034	-0.161544	0.240663	0.241546	0.210313	0.202308
flag	-0.021154	0.104430	-0.306949	1.000000	0.016753	0.032367	-0.006350	0.056373	0.007850	0.063119	...	-0.319486	0.594074	0.649638	-0.259833	0.220041	0.091505	-0.345392	-0.340881	-0.740350	-0.789984
src_bytes	0.008243	-0.003957	-0.027654	0.016753	1.000000	0.066529	-0.000248	-0.002026	-0.000244	0.003826	...	-0.014590	-0.017755	-0.009657	0.001821	0.013121	0.001072	-0.011446	-0.014204	-0.010721	-0.010563
dst_bytes	0.016966	-0.006525	-0.001592	0.032367	0.066529	1.000000	-0.000444	-0.003682	0.003930	0.004253	...	-0.030615	0.001004	0.017841	-0.015270	0.012780	-0.004808	-0.024953	-0.024287	-0.013761	-0.018428
land	-0.001116	-0.001363	-0.008001	-0.006350	-0.000248	-0.000444	1.000000	-0.000736	-0.000102	-0.000862	...	-0.017956	-0.010141	0.009574	-0.004003	0.026479	0.041932	0.017676	0.017681	-0.004304	-0.004127
wrong_fragment	-0.009257	0.140160	0.063478	0.056373	-0.002026	-0.003682	-0.000736	1.000000	-0.000850	-0.007151	...	0.026277	-0.043908	-0.042782	0.036215	0.040799	-0.007485	-0.038663	-0.042744	0.012319	-0.034221
urgent	-0.000770	-0.001574	0.014182	0.007850	-0.000244	0.003930	-0.000102	-0.000850	1.000000	0.005405	...	-0.015436	-0.007251	0.000838	0.010348	0.004054	-0.001876	-0.006075	-0.005952	-0.002245	0.003303
hot	-0.001580	-0.013246	-0.063407	0.063119	0.003826	0.004253	-0.000862	-0.007151	0.005405	1.000000	...	-0.016347	-0.047259	-0.031264	-0.006534	-0.031009	-0.020947	-0.048245	-0.049140	-0.029516	-0.035869
num_failed_logins	-0.001324	-0.012110	0.107923	0.046243	-0.002172	0.074259	-0.000788	-0.006537	-0.000910	-0.005423	...	0.020561	0.001600	0.009433	-0.021115	-0.030272	-0.002387	-0.043475	-0.043454	-0.021844	-0.019506
logged_in	-0.061158	-0.119650	-0.186207	0.547180	0.006808	0.036606	-0.007789	-0.064589	0.013153	0.108849	...	-0.380702	0.608180	0.593710	-0.257662	-0.186595	-0.048515	-0.438448	-0.441688	-0.306059	-0.313711
num_compromised	0.124051	-0.003301	0.037489	0.014046	-0.000460	0.048518	-0.000215	-0.001782	0.051168	0.000974	...	-0.014792	-0.021606	-0.017622	0.009072	-0.007416	-0.002893	-0.004400	-0.008804	-0.003597	-0.006371
root_shell	0.055699	-0.006543	0.059479	0.025088	-0.001089	0.098757	-0.000426	-0.003532	0.180282	0.039246	...	-0.020086	-0.024457	-0.012876	0.001251	-0.005924	0.002777	-0.017046	-0.016666	-0.005043	-0.008676
su_attempted	0.112957	-0.003411	0.040890	0.011872	-0.000572	0.157457	-0.000222	-0.001841	-0.000256	-0.002158	...	-0.010659	-0.024158	-0.022199	0.010237	-0.009249	0.000940	-0.002715	-0.007352	0.001129	-0.006349
num_root	0.123607	-0.003199	0.036282	0.013645	-0.000535	0.045821	-0.000208	-0.001727	0.038026	-0.000484	...	-0.015342	-0.022178	-0.018498	0.009541	-0.006448	-0.002648	-0.004442	-0.008608	-0.003421	-0.006415
num_file_creations	0.070725	-0.003614	0.036434	0.016476	-0.000403	0.007868	-0.000235	-0.001951	0.021812	0.005133	...	-0.003660	-0.017947	-0.013596	0.011509	-0.006309	0.008906	0.002933	0.006649	-0.001834	-0.005365
num_shells	-0.001467	-0.003288	0.018997	0.016401	-0.000490	0.000276	-0.000214	-0.001775	-0.000247	0.012965	...	-0.018427	-0.017313	-0.008512	-0.003046	0.009726	-0.004187	-0.006710	-0.002141	-0.010090	-0.005296
num_access_files	0.085597	-0.006118	0.030836	0.029086	-0.000953	0.031256	-0.000398	-0.003302	0.111015	0.001480	...	-0.006321	-0.005644	-0.002418	0.005196	-0.011091	-0.007892	-0.013807	-0.013879	-0.011586	-0.014918
num_outbound_cmds	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
is_host_login	-0.000268	-0.000964	0.011553	0.004807	-0.000171	-0.000120	-0.000063	-0.000520	-0.000072	0.003310	...	0.004653	-0.006216	-0.007085	-0.001830	-0.003076	-0.001777	-0.003720	-0.003645	-0.003043	-0.002918
is_guest_login	0.000937	-0.017609	-0.095304	0.086797	-0.003079	-0.003206	-0.001146	-0.009506	-0.001324	0.621428	...	-0.011597	-0.084998	-0.063603	0.030483	-0.042782	-0.030553	-0.058525	-0.065959	-0.014063	-0.051659
count	-0.080469	-0.038272	0.111538	-0.471583	-0.017689	-0.031447	-0.006517	-0.017763	-0.007525	-0.062987	...	0.428367	-0.372806	-0.445387	0.240177	-0.022793	-0.180300	0.360451	0.361963	0.270653	0.261142
srv_count	-0.040086	0.048987	-0.209450	0.177907	-0.008576	-0.014201	-0.003241	0.018803	-0.003881	-0.031840	...	0.147370	0.215829	0.195390	-0.107859	0.264774	-0.058229	-0.123739	-0.122758	-0.112558	-0.117273
serror_rate	-0.062275	-0.078452	0.233362	-0.352775	-0.013810	-0.023611	0.017430	-0.032917	-0.006037	-0.049652	...	0.319274	-0.496054	-0.551683	-0.000184	-0.224818	-0.137392	0.970600	0.970645	-0.218683	-0.214369
srv_serror_rate	-0.061954	-0.079790	0.230765	-0.347388	-0.013625	-0.023836	0.017405	-0.043072	-0.005998	-0.049442	...	0.318096	-0.493821	-0.549156	-0.005394	-0.224299	-0.135169	0.962955	0.977608	-0.215527	-0.223396
rerror_rate	0.115775	-0.063531	0.193773	-0.802671	-0.008796	-0.019907	-0.004144	-0.033970	-0.004785	-0.034855	...	0.149257	-0.321443	-0.348518	0.309982	-0.063741	-0.022298	-0.226968	-0.223993	0.909549	0.950706
srv_rerror_rate	0.117674	-0.063273	0.189795	-0.804668	-0.008672	-0.019655	-0.004119	-0.034156	-0.004756	-0.035027	...	0.149833	-0.321340	-0.347095	0.310976	-0.063762	-0.023797	-0.222320	-0.232593	0.897110	0.960629
same_srv_rate	0.077493	0.036313	-0.352729	0.735933	0.018455	0.034182	0.006909	0.046954	0.007978	0.064129	...	-0.459726	0.715560	0.795421	-0.226196	0.233726	0.185600	-0.660305	-0.662643	-0.356982	-0.359469
diff_srv_rate	-0.020179	0.044073	0.207805	-0.314833	-0.005874	-0.014984	-0.003286	-0.023995	-0.003794	-0.023715	...	0.163318	-0.310190	-0.341518	0.547001	0.008288	-0.066073	0.014293	0.015783	0.274435	0.321243
srv_diff_host_rate	-0.039607	-0.182428	-0.057639	0.135188	-0.006419	-0.005206	0.033567	-0.017587	-0.004038	-0.022188	...	-0.323249	0.114362	0.217887	-0.109842	0.172757	0.292709	-0.185399	-0.187329	-0.055137	-0.036040
dst_host_count	0.054504	0.216810	0.221458	-0.319486	-0.014590	-0.030615	-0.017956	0.026277	-0.015436	-0.016347	...	1.000000	-0.245914	-0.458687	0.138572	-0.292562	-0.434364	0.331272	0.329708	0.156640	0.143175
dst_host_srv_count	-0.085035	0.103722	-0.389706	0.594074	-0.017755	0.001004	-0.010141	-0.043908	-0.007251	-0.047259	...	-0.245914	1.000000	0.902438	-0.399349	-0.047918	-0.019893	-0.514236	-0.506914	-0.342552	-0.342036
dst_host_same_srv_rate	-0.091574	0.019163	-0.439143	0.649638	-0.009657	0.017841	0.009574	-0.042782	0.000838	-0.031264	...	-0.458687	0.902438	1.000000	-0.425759	0.140692	0.169718	-0.571598	-0.562652	-0.370295	-0.366949
dst_host_diff_srv_rate	0.201141	0.122548	0.253447	-0.259833	0.001821	-0.015270	-0.004003	0.036215	0.010348	-0.006534	...	0.138572	-0.399349	-0.425759	1.000000	0.201405	-0.001158	-0.002984	-0.004977	0.371208	0.322953
dst_host_same_src_port_rate	0.176762	-0.167177	-0.124034	0.220041	0.013121	0.012780	0.026479	0.040799	0.004054	-0.031009	...	-0.292562	-0.047918	0.140692	0.201405	1.000000	0.377270	-0.227095	-0.225524	-0.060338	-0.062770
dst_host_srv_diff_host_rate	-0.024386	-0.323542	-0.161544	0.091505	0.001072	-0.004808	0.041932	-0.007485	-0.001876	-0.020947	...	-0.434364	-0.019893	0.169718	-0.001158	0.377270	1.000000	-0.138781	-0.137999	-0.017666	-0.013813
dst_host_serror_rate	-0.040706	-0.079571	0.240663	-0.345392	-0.011446	-0.024953	0.017676	-0.038663	-0.006075	-0.048245	...	0.331272	-0.514236	-0.571598	-0.002984	-0.227095	-0.138781	1.000000	0.980668	-0.231279	-0.220638
dst_host_srv_serror_rate	-0.038633	-0.079183	0.241546	-0.340881	-0.014204	-0.024287	0.017681	-0.042744	-0.005952	-0.049140	...	0.329708	-0.506914	-0.562652	-0.004977	-0.225524	-0.137999	0.980668	1.000000	-0.224010	-0.231605
dst_host_rerror_rate	0.110226	-0.038347	0.210313	-0.740350	-0.010721	-0.013761	-0.004304	0.012319	-0.002245	-0.029516	...	0.156640	-0.342552	-0.370295	0.371208	-0.060338	-0.017666	-0.231279	-0.224010	1.000000	0.920049
dst_host_srv_rerror_rate	0.113938	-0.063393	0.202308	-0.789984	-0.010563	-0.018428	-0.004127	-0.034221	0.003303	-0.035869	...	0.143175	-0.342036	-0.366949	0.322953	-0.062770	-0.013813	-0.220638	-0.231605	0.920049	1.000000

41 rows × 41 columns

• The pairwise correlation values between all numerical features, which indicate how strongly each feature is related to the others, are contained in the corr_matrix.
• Additionally, the correlation matrix aids in identifying relationships or trends in the data, which can direct model selection and enhance overall prediction performance.
• In order to choose features, eliminate multicollinearity, or comprehend linkages within the dataset, it assists in identifying highly associated characteristics.

# Select only numerical columns from the dataset for visualization
# Plot histograms for all numerical features to understand their distributions
# Display the histogram plots
numeric_df = full_df.select_dtypes(include='number')
numeric_df.hist(figsize=(10, 12))
plt.show()

• In order to guarantee that histograms are only produced for features that can be represented numerically, the code only chooses numerical columns.
• It helps you spot trends like skewness, outliers, or data spread by plotting histograms to display the distribution of each numerical attribute.

# Create a figure with a specified size for better visibility of the heatmap
# Plot a heatmap to visualize correlation strengths between numerical features
# Display the heatmap
plt.figure(figsize=(12,8))
sns.heatmap(corr_matrix, cmap="coolwarm")
plt.title("Correlation Heatmap (Numeric Features Only)")
plt.show()

• To graphically display the degree of correlation between the numerical features in the dataset, this code creates a heatmap.
• It aids in finding correlations, trends, and multicollinearity between variables, all of which are helpful for choosing features and enhancing models.

# help to identify class distribution and detect any imbalance in the dataset
full_df['labels'].value_counts()

labels
normal             22490
neptune            13308
satan               1396
smurf               1080
ipsweep              958
guess_passwd         822
portsweep            742
mscan                620
warezmaster          558
apache2              456
back                 454
processtable         422
nmap                 412
warezclient          214
teardrop             208
snmpguess            198
mailbomb             188
saint                186
snmpgetattack         86
httptunnel            82
pod                   70
buffer_overflow       34
multihop              18
ps                    18
sendmail              16
rootkit                8
loadmodule             8
xlock                  8
xterm                  6
named                  4
imap                   4
land                   4
ftp_write              4
xsnoop                 2
phf                    2
perl                   2
Name: count, dtype: int64

• To calculates how many times each class/label appears in the dataset.
• This helps identify whether the dataset is balanced or imbalanced, which is important for selecting the right evaluation methods and models.

# Plot the distribution of attack labels to visualize class imbalance
# Countplot shows how many samples belong to each label/category
# Add title for better readability
# Rotate x-axis labels for better visibility
plt.figure(figsize=(10,5))
sns.countplot(data=full_df, x='labels')
plt.title("Class Distribution - NSL-KDD Train Data")
plt.xticks(rotation=90)
# Display the plot
plt.show()

• Readability is enhanced by rotating the x-axis labels, particularly when there are numerous attack classes with lengthy names.
• To display the number of samples that fall into each assault type in the dataset, the code generates a count plot of the 'labels' column.
• This aids in the visual identification of class imbalance, which is crucial for comprehending model bias and evaluation difficulties.

6) Model Building

# RANDOM FOREST CLASSIFIER
rf = RandomForestClassifier(n_estimators=200, max_depth=15, random_state=42)
rf.fit(X_train, y_train)

RandomForestClassifier(max_depth=15, n_estimators=200, random_state=42)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

• In order to balance performance with overfitting management, this method initializes a Random Forest Classifier with 200 trees and a maximum depth of 15.
• The training dataset (X_train, y_train) is then used to train the model so that it can identify patterns to categorize network attacks.
• The random_state=42 ensures the results remain consistent and reproducible each time the model is run.
• The fit(X_train, y_train) step trains the model, allowing it to learn patterns from the training data for accurate prediction on unseen data.

# Generate predictions on the test set using the trained Random Forest model
# Print the accuracy score to evaluate how well the model performs on unseen data
# Display a detailed classification report (precision, recall, F1-score) for each class
rf_pred = rf.predict(X_test)
print("\nRandom Forest Accuracy:", accuracy_score(y_test, rf_pred))
print("\nClassification Report:\n", classification_report(y_test, rf_pred))

Random Forest Accuracy: 0.9960965223562811

Classification Report:
                  precision    recall  f1-score   support

        apache2       1.00      1.00      1.00       228
           back       1.00      1.00      1.00       227
buffer_overflow       1.00      1.00      1.00        17
      ftp_write       1.00      1.00      1.00         2
   guess_passwd       1.00      1.00      1.00       411
     httptunnel       1.00      0.98      0.99        41
           imap       1.00      1.00      1.00         2
        ipsweep       1.00      1.00      1.00       479
           land       1.00      1.00      1.00         2
     loadmodule       1.00      1.00      1.00         4
       mailbomb       1.00      1.00      1.00        94
          mscan       1.00      1.00      1.00       310
       multihop       1.00      0.78      0.88         9
          named       1.00      1.00      1.00         2
        neptune       1.00      1.00      1.00      6654
           nmap       1.00      0.98      0.99       206
         normal       0.99      1.00      1.00     11245
           perl       1.00      1.00      1.00         1
            phf       1.00      1.00      1.00         1
            pod       1.00      1.00      1.00        35
      portsweep       1.00      1.00      1.00       371
   processtable       1.00      1.00      1.00       211
             ps       1.00      0.78      0.88         9
        rootkit       1.00      0.25      0.40         4
          saint       0.99      0.91      0.95        93
          satan       0.99      1.00      0.99       698
       sendmail       1.00      0.75      0.86         8
          smurf       1.00      1.00      1.00       540
  snmpgetattack       1.00      0.07      0.13        43
      snmpguess       1.00      1.00      1.00        99
       teardrop       0.96      1.00      0.98       104
    warezclient       1.00      0.93      0.97       107
    warezmaster       1.00      0.97      0.98       279
          xlock       1.00      1.00      1.00         4
         xsnoop       1.00      1.00      1.00         1
          xterm       1.00      0.67      0.80         3

       accuracy                           1.00     22544
      macro avg       1.00      0.92      0.94     22544
   weighted avg       1.00      1.00      1.00     22544

• To produce output labels for the unseen test dataset, the model makes use of predict().
• How well the Random Forest accurately identified the test samples is determined by calculating the accuracy score.
• To assess the model's performance for each class, the classification report offers comprehensive metrics like as precision, recall, and F1-score.

# Confusion Matrix
# Plot the confusion matrix as a heatmap for easy visualization
# Add title and axis labels for clarity
# Display the plot
cm_rf = confusion_matrix(y_test, rf_pred)
plt.figure(figsize=(6,5))
sns.heatmap(cm_rf, annot=True, fmt='d', cmap='Blues')
plt.title("Random Forest - Confusion Matrix")
plt.xlabel("Predicted")
plt.ylabel("Actual")
plt.show()

• The confusion matrix is computed to display the number of test samples that the Random Forest model properly or mistakenly categorized.
• These results are shown as a heatmap, which makes it simpler to spot trends like strong forecasts or incorrect classifications.
• By differentiating between the actual and expected class values, the labels and title aid in the interpretation of the matrix.

#Hyperparameter Tuning of RF
print("\nPerforming Hyperparameter Tuning for Random Forest...")
# Define the parameter grid with different values to test for each hyperparameter
param_grid_rf = {
    "n_estimators": [100, 150],
    "max_depth": [10, 20, None],
    "min_samples_split": [2, 5],
}
# Set up GridSearchCV to find the best combination of hyperparameters
grid_rf = GridSearchCV(
    RandomForestClassifier(random_state=42),
    param_grid_rf,
    cv=3,
    n_jobs=-1,
    verbose=1
)
# Train the model using the grid search procedure
grid_rf.fit(X_train, y_train)
# Print the best hyperparameter combination found
print("\nBest RF Parameters:", grid_rf.best_params_)

Performing Hyperparameter Tuning for Random Forest...
Fitting 3 folds for each of 12 candidates, totalling 36 fits

Best RF Parameters: {'max_depth': None, 'min_samples_split': 2, 'n_estimators': 150}

• To test various model settings, a grid of potential Random Forest hyperparameters is generated.
• GridSearchCV uses cross-validation to assess every combination in order to determine which settings yield optimal performance.
• The algorithm helps build a more precise and optimized Random Forest model by printing the optimal hyperparameters after training.
• It provides the optimal parameters after training, enabling the model to be tweaked for better performance.

• To test various model settings, a grid of potential Random Forest hyperparameters is generated.
• GridSearchCV uses cross-validation to assess every combination in order to determine which settings yield optimal performance.
• The algorithm helps build a more precise and optimized Random Forest model by printing the optimal hyperparameters after training.
• It provides the optimal parameters after training, enabling the model to be tweaked for better performance.

# Retrieve the best Random Forest model found during GridSearchCV
# Use the tuned model to make predictions on the test set
rf_best = grid_rf.best_estimator_
rf_best_pred = rf_best.predict(X_test)

# Display the accuracy of the tuned Random Forest model
# Print the confusion matrix to analyze misclassifications of the tuned model

print("\nTuned RF Accuracy:", accuracy_score(y_test, rf_best_pred))
print("\nConfusion Matrix (Tuned RF):")
print(confusion_matrix(y_test, rf_best_pred))

Tuned RF Accuracy: 0.9998225691980128

Confusion Matrix (Tuned RF):
[[228   0   0 ...   0   0   0]
 [  0 227   0 ...   0   0   0]
 [  0   0  17 ...   0   0   0]
 ...
 [  0   0   0 ...   4   0   0]
 [  0   0   0 ...   0   1   0]
 [  0   0   0 ...   0   0   3]]

• The top-performing Predictions on the test data are made using the Random Forest model that was discovered during hyperparameter tuning.
• The accuracy score illustrates how well the optimized model outperforms the original.
• The confusion matrix helps assess how the modified model enhances classification performance by offering a thorough breakdown of accurate and inaccurate predictions.

#GradientBoostingClassifier
# Import necessary libraries for Gradient Boosting and evaluation
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report
import seaborn as sns
import matplotlib.pyplot as plt
# Notify that Gradient Boosting model training is starting

print("\nTraining Gradient Boosting...")

# MUCH FASTER version
# Train the model on the training data
gb = GradientBoostingClassifier(
    n_estimators=60,        # default 100 → reduced for speed
    max_depth=2,           # shallow trees → MUCH faster
    learning_rate=0.1,
    random_state=42
)
# Train the model on the training data
# Make predictions on the test dataset
gb.fit(X_train, y_train)
gb_pred = gb.predict(X_test)
# Display a detailed classification report with precision, recall, and F1-score
# Print accuracy score of the Gradient Boosting model
print("\nGradient Boosting Accuracy:", accuracy_score(y_test, gb_pred))
print("\nClassification Report:\n", classification_report(y_test, gb_pred))

Training Gradient Boosting...

Gradient Boosting Accuracy: 0.9552874378992193

Classification Report:
                  precision    recall  f1-score   support

        apache2       1.00      1.00      1.00       228
           back       1.00      1.00      1.00       227
buffer_overflow       0.19      0.53      0.28        17
      ftp_write       1.00      0.50      0.67         2
   guess_passwd       0.98      0.94      0.96       411
     httptunnel       0.24      0.98      0.38        41
           imap       1.00      0.50      0.67         2
        ipsweep       0.84      0.10      0.18       479
           land       1.00      0.50      0.67         2
     loadmodule       0.01      1.00      0.02         4
       mailbomb       1.00      1.00      1.00        94
          mscan       0.97      0.88      0.92       310
       multihop       0.86      0.67      0.75         9
          named       0.17      0.50      0.25         2
        neptune       1.00      1.00      1.00      6654
           nmap       0.97      0.91      0.94       206
         normal       0.98      0.98      0.98     11245
           perl       0.00      0.00      0.00         1
            phf       0.00      0.00      0.00         1
            pod       1.00      0.06      0.11        35
      portsweep       0.96      0.89      0.92       371
   processtable       1.00      1.00      1.00       211
             ps       0.00      0.00      0.00         9
        rootkit       0.00      0.00      0.00         4
          saint       0.82      0.54      0.65        93
          satan       0.89      0.92      0.91       698
       sendmail       1.00      0.62      0.77         8
          smurf       1.00      1.00      1.00       540
  snmpgetattack       0.62      0.30      0.41        43
      snmpguess       1.00      0.92      0.96        99
       teardrop       0.96      0.99      0.98       104
    warezclient       0.98      0.90      0.94       107
    warezmaster       0.99      0.94      0.96       279
          xlock       0.33      0.25      0.29         4
         xsnoop       0.00      0.00      0.00         1
          xterm       1.00      0.33      0.50         3

       accuracy                           0.96     22544
      macro avg       0.72      0.63      0.61     22544
   weighted avg       0.98      0.96      0.96     22544

• Fewer trees and lower depth are used to produce a Gradient Boosting Classifier, which speeds up training without sacrificing performance.
• Fit() is used to train the model on the training dataset, and predict() is used to provide predictions on the test set.
• To assess Gradient Boosting's performance, measures such as precision, recall, and F1-score for each class are displayed in the accuracy score and classification report.

# Generate the confusion matrix for Gradient Boosting predictions
cm = confusion_matrix(y_test, gb_pred)
# Plot the confusion matrix as a heatmap for visual interpretation
plt.figure(figsize=(6,5))
sns.heatmap(cm, annot=True, fmt="d", cmap="Greens")
# Add title and axis labels for clarity
plt.title("Gradient Boosting - Confusion Matrix ")
plt.xlabel("Predicted")
plt.ylabel("Actual")
# Display the heatmap
plt.show()

• The confusion matrix, which displays accurate and inaccurate classifications, is created by comparing the actual labels with the labels predicted by the Gradient Boosting model.
• These values are presented as a heatmap, which makes it simpler to see where the model performs well and where misclassifications take place.
• By elucidating the significance of predicted versus actual classes, titles and axis labels aid in the interpretation of the figure.

# GRADIENT BOOSTING WITH HYPERPARAMETER TUNING
print("\nRunning Gradient Boosting Hyperparameter Tuning...")

param_grid = {
    'n_estimators': [50, 100],
    'learning_rate': [0.05, 0.1],
    'max_depth': [2, 3],
}
# Initialize the base Gradient Boosting model
gb = GradientBoostingClassifier(random_state=42)
# Perform GridSearchCV to find the best combination of hyperparameters
grid_search = GridSearchCV(gb, param_grid, cv=3, scoring='accuracy', n_jobs=-1)
grid_search.fit(X_train, y_train)
# Train the model with all parameter combinations
print("\nBest Parameters:", grid_search.best_params_)

# Best model
gb_best = grid_search.best_estimator_
gb_pred = gb_best.predict(X_test)
# Display the accuracy of the tuned Gradient Boosting model
# Print the detailed classification report (precision, recall, F1-score)

print("\nGradient Boosting Accuracy:", accuracy_score(y_test, gb_pred))
print("\nClassification Report:\n", classification_report(y_test, gb_pred))

Running Gradient Boosting Hyperparameter Tuning...

Best Parameters: {'learning_rate': 0.05, 'max_depth': 3, 'n_estimators': 100}

Gradient Boosting Accuracy: 0.9699698367636622

Classification Report:
                  precision    recall  f1-score   support

        apache2       1.00      1.00      1.00       228
           back       1.00      1.00      1.00       227
buffer_overflow       1.00      0.76      0.87        17
      ftp_write       1.00      0.50      0.67         2
   guess_passwd       1.00      0.98      0.99       411
     httptunnel       1.00      1.00      1.00        41
           imap       1.00      0.50      0.67         2
        ipsweep       0.63      0.18      0.28       479
           land       0.00      0.00      0.00         2
     loadmodule       0.33      0.25      0.29         4
       mailbomb       1.00      1.00      1.00        94
          mscan       1.00      0.99      1.00       310
       multihop       1.00      0.56      0.71         9
          named       1.00      0.50      0.67         2
        neptune       1.00      1.00      1.00      6654
           nmap       0.31      0.94      0.47       206
         normal       0.99      0.99      0.99     11245
           perl       1.00      1.00      1.00         1
            phf       1.00      1.00      1.00         1
            pod       0.71      0.14      0.24        35
      portsweep       0.98      1.00      0.99       371
   processtable       1.00      1.00      1.00       211
             ps       1.00      0.78      0.88         9
        rootkit       1.00      1.00      1.00         4
          saint       0.92      0.52      0.66        93
          satan       0.95      0.99      0.97       698
       sendmail       0.80      1.00      0.89         8
          smurf       1.00      1.00      1.00       540
  snmpgetattack       0.69      0.51      0.59        43
      snmpguess       1.00      1.00      1.00        99
       teardrop       0.96      1.00      0.98       104
    warezclient       0.64      1.00      0.78       107
    warezmaster       1.00      0.86      0.92       279
          xlock       1.00      0.25      0.40         4
         xsnoop       1.00      1.00      1.00         1
          xterm       1.00      0.67      0.80         3

       accuracy                           0.97     22544
      macro avg       0.89      0.77      0.80     22544
   weighted avg       0.98      0.97      0.97     22544

• GridSearchCV is utilized to test every possible combination of a specific set of Gradient Boosting hyperparameters in order to identify the optimal model.
• Using cross-validation, the grid search trains several models and chooses the most accurate one.
• The accuracy and comprehensive classification report of the best model are then shown to assess performance after it has been used to generate predictions on the test data.

7) Summarising the Model Results

# Predictions from tuned Random Forest model
rf_best_pred = rf_best.predict(X_test)

# Predictions from tuned Gradient Boosting model
gb_best_pred = gb_best.predict(X_test)

• Predictions from tuned Random Forest model
• We use the tuned Random Forest model (rf_best) to generate predictions on the test data,
• so we can evaluate its performance and compare it with other models.

# # Create a dictionary to store model names and their corresponding accuracy scores
summary_data = {
    "Model": [
        "Random Forest",
        "Gradient Boosting",
        "Random Forest (Tuned)",
        "Gradient Boosting (Tuned)"
    ],
    "Accuracy": [
        accuracy_score(y_test, rf_pred),
        accuracy_score(y_test, gb_pred),
        accuracy_score(y_test, rf_best_pred),
        accuracy_score(y_test, gb_best_pred)
    ]
}

• The names of every machine learning model used in the project are stored in a dictionary called summary_data.
• The dictionary contains each model's matching accuracy score, which is determined using accuracy_score.
• Then Later, this dictionary will be transformed into a DataFrame so that model performance may be readily compared.

Model Comparison

# # Create a DataFrame out of the summary dictionary
summary_df = pd.DataFrame(summary_data)
# Plot Visualization
plt.figure(figsize=(9, 5))
sns.barplot(x="Model", y="Accuracy", data=summary_df)
# Add axis labels and the plot title
plt.title("Model Accuracy Comparison")
plt.xlabel("Machine Learning Models")
plt.ylabel("Accuracy Score")
plt.ylim(0, 1)
plt.xticks(rotation=20)
plt.show()
# Display the summary table with all model accuracies
summary_df

	Model	Accuracy
0	Random Forest	0.996097
1	Gradient Boosting	0.969970
2	Random Forest (Tuned)	0.999823
3	Gradient Boosting (Tuned)	0.969970

• For simpler analysis, the dictionary with model names and accuracy scores is transformed into a DataFrame.
• To visually compare each model's accuracy performance, a bar plot is made with Seaborn.
• The chart is made more readable and clear by adding axis labels, a title, and a rotated x-axis.
• Because accuracy scores fall between 0 and 1, the y-axis is restricted to that range.
• Lastly, all model performance statistics are shown in tabular format in the summary DataFrame.