IMPORTING ESSENTIAL LIBRARIES

# Data manipulation and analysis
import pandas as pd           
import numpy as np            

# Visualisation libraries
import matplotlib.pyplot as plt   
import seaborn as sns             

# Splitting data & hyperparameter tuning
from sklearn.model_selection import (
    train_test_split,   
    GridSearchCV,       
    RandomizedSearchCV, 
    cross_val_score     
)

# Data preprocessing
from sklearn.preprocessing import StandardScaler, OneHotEncoder  
from sklearn.compose import ColumnTransformer                    
from sklearn.pipeline import Pipeline                            
from sklearn.impute import SimpleImputer                         

# Model evaluation metrics
from sklearn.metrics import (
    accuracy_score,      
    precision_score,     
    recall_score,        
    f1_score,            
    confusion_matrix,    
    classification_report,
    roc_auc_score,       
    roc_curve,           
    auc                  
)

# ALgorithms to use
from sklearn.tree import DecisionTreeClassifier, plot_tree   
from sklearn.svm import SVC                                  
from sklearn.cluster import KMeans                           
from sklearn.metrics import silhouette_score                 
from sklearn.decomposition import PCA                        

import joblib
import time
import warnings
warnings.filterwarnings('ignore')
sns.set(style='whitegrid')

LOADING THE DATA + QUICK EDA

df = pd.read_csv(r"C:\Users\S2117368\OneDrive - University of Wales Trinity Saint David\Year 3\AI\online_gaming_behavior_dataset.csv")

df.head() # load of the first 5 rows of the dataset  

	PlayerID	Age	Gender	Location	GameGenre	PlayTimeHours	InGamePurchases	GameDifficulty	SessionsPerWeek	AvgSessionDurationMinutes	PlayerLevel	AchievementsUnlocked	EngagementLevel
0	9000	43	Male	Other	Strategy	16.271119	0	Medium	6	108	79	25	Medium
1	9001	29	Female	USA	Strategy	5.525961	0	Medium	5	144	11	10	Medium
2	9002	22	Female	USA	Sports	8.223755	0	Easy	16	142	35	41	High
3	9003	35	Male	USA	Action	5.265351	1	Easy	9	85	57	47	Medium
4	9004	33	Male	Europe	Action	15.531945	0	Medium	2	131	95	37	Medium

df.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 40034 entries, 0 to 40033
Data columns (total 13 columns):
 #   Column                     Non-Null Count  Dtype  
---  ------                     --------------  -----  
 0   PlayerID                   40034 non-null  int64  
 1   Age                        40034 non-null  int64  
 2   Gender                     40034 non-null  object 
 3   Location                   40034 non-null  object 
 4   GameGenre                  40034 non-null  object 
 5   PlayTimeHours              40034 non-null  float64
 6   InGamePurchases            40034 non-null  int64  
 7   GameDifficulty             40034 non-null  object 
 8   SessionsPerWeek            40034 non-null  int64  
 9   AvgSessionDurationMinutes  40034 non-null  int64  
 10  PlayerLevel                40034 non-null  int64  
 11  AchievementsUnlocked       40034 non-null  int64  
 12  EngagementLevel            40034 non-null  object 
dtypes: float64(1), int64(7), object(5)
memory usage: 4.0+ MB

Observations

• There are 40034 observations and 13 columns in the dataset
• All the columns have 40034 non-null values, that means there are no missing values in the data

# checking unique values 
df.nunique()

PlayerID                     40034
Age                             35
Gender                           2
Location                         4
GameGenre                        5
PlayTimeHours                40034
InGamePurchases                  2
GameDifficulty                   3
SessionsPerWeek                 20
AvgSessionDurationMinutes      170
PlayerLevel                     99
AchievementsUnlocked            50
EngagementLevel                  3
dtype: int64

Observations

• PlayerID is a unique identifier will be dropped from modeling.
• PlayTimeHours also has unique values for each player, but it is meaningful as a continuous metric.
• Columns like Gender, Location, GameGenre, InGamePurchases, GameDifficulty are categorical the will be encoded for modeling.
• Columns like Age, SessionsPerWeek, AvgSessionDurationMinutes, PlayerLevel, AchievementsUnlocked are numerical and will be scaled if needed.
• EngagementLevel has 3 unique classes, this is a multiclass classification problem.
• Binary low-cardinality features like InGamePurchases, 2 unique values will be treated as categorical or numerical.

# dropping the columns 
df = df.drop(['PlayerID'], axis=1)

# creating numerical columns
num_cols = ['Age', 'PlayTimeHours', 'SessionsPerWeek', 'AvgSessionDurationMinutes', 
            'PlayerLevel', 'AchievementsUnlocked']

# creating categorical columns
cat_cols = ['Gender', 'Location', 'GameGenre', 'InGamePurchases', 'GameDifficulty']

Checking each numeric variable separately

#  Checking summary statistics
df[num_cols].describe().T

	count	mean	std	min	25%	50%	75%	max
Age	40034.0	31.992531	10.043227	15.000000	23.000000	32.000000	41.000000	49.000000
PlayTimeHours	40034.0	12.024365	6.914638	0.000115	6.067501	12.008002	17.963831	23.999592
SessionsPerWeek	40034.0	9.471774	5.763667	0.000000	4.000000	9.000000	14.000000	19.000000
AvgSessionDurationMinutes	40034.0	94.792252	49.011375	10.000000	52.000000	95.000000	137.000000	179.000000
PlayerLevel	40034.0	49.655568	28.588379	1.000000	25.000000	49.000000	74.000000	99.000000
AchievementsUnlocked	40034.0	24.526477	14.430726	0.000000	12.000000	25.000000	37.000000	49.000000

Observations

• The average player is about 32 years old, and most are between 23 and 41. The youngest is 15 and the oldest is 49, so most players are fairly young.
• Players spend around 12 hours in total, though this is probably total playtime, not daily. Some play very little, while others play a lot more.
• Most players play 9 to 10 times a week, but some play almost every day and some play less often.
• Each session lasts about 95 minutes on average, with most sessions between 52 and 137 minutes.
• Player levels are around 50 on average, ranging from beginners at level 1 to highly advanced players at leve 99.
• Players unlock about 25 achievements on average. Some have none, while the most dedicated players reach upto 50.

Now let’s take a closer look at how these variables are spread out

df["EngagementLevel"].value_counts(normalize=True) * 100

EngagementLevel
Medium    48.393865
High      25.818055
Low       25.788080
Name: proportion, dtype: float64

# Creating histograms 
df [num_cols].hist(figsize=(14, 14))
plt.show()

Observations

• Age is spread fairly evenly from 15 to 49 meaning the game attracts a wide range of players
• PlayTimeHours looks evenly distributed, so players spend very different amounts of total time in the game without any clear pattern.
• SessionsPerWeek also varies widely, showing that some players play very little while others play a lot.
• AvgSessionDurationMinutes is spread across short to long sessions, suggesting players have mixed session habits.
• PlayerLevel ranges smoothly from beginner to high-level players with no strong concentration in one area.
• AchievementsUnlocked is also evenly spread, meaning players earn achievements at many different rates.

Checking each categorical variable separately

for col in cat_cols:
     print(df[col].value_counts(normalize=True))
     print('*' * 40)

Gender
Male      0.598466
Female    0.401534
Name: proportion, dtype: float64
****************************************
Location
USA       0.399660
Europe    0.299845
Asia      0.202203
Other     0.098291
Name: proportion, dtype: float64
****************************************
GameGenre
Sports        0.201029
Action        0.200804
Strategy      0.200130
Simulation    0.199406
RPG           0.198631
Name: proportion, dtype: float64
****************************************
InGamePurchases
0    0.799146
1    0.200854
Name: proportion, dtype: float64
****************************************
GameDifficulty
Easy      0.49995
Medium    0.30002
Hard      0.20003
Name: proportion, dtype: float64
****************************************

Observatiosn

• Gender around 60% male and 40% female, fairly balanced.
• Location mostly USA around 40% and Europe 30% with some players from Asia and other regions.
• GameGenre All genres are evenly distributed 20% each, showing no strong preference.
• InGamePurchases about 80% make no purchases, low microtransaction activity.
• GameDifficulty most players choose easy, around 50%, then medium 30% and hard 20%

Bivariate analysis for categorical variables

Next, we will look at how InGamePurchases is related to other categorical variables. By using crosstabs to check row percentages and create stacked bar charts to visualise the patterns. This will help show how different factors influence InGamePurchases..

for col in cat_cols:
    if col != 'InGamePurchases':  
        (pd.crosstab(df[col], df['InGamePurchases'], normalize='index') * 100).plot(
            kind='bar', figsize=(8,4), stacked=True
        )
        plt.ylabel('Percentage of Players')
        plt.title(f'{col} vs In-Game Purchases')
        plt.show()

Observations

• Across all categories the percentage of players making purchases stays close to 20%, showing that buying behavior doesn’t change much based on gender, location, genre or difficulty.
• Location shows only small differences players in the USA make slightly more purchases than those in Asia or Other, but the gap is small.
• GameGenre differences are minimal purchase rates stay close to 20% for all genres, meaning players behave similarly regardless of what type of game they play.
• GameDifficulty also remains consistent, with Easy, Medium and Hard modes all showing roughly the same purchase rate.

# Checking how numeric behaviour differs between players who purchase in‑game items and those who don’t
df.groupby('InGamePurchases')[num_cols].mean()

	Age	PlayTimeHours	SessionsPerWeek	AvgSessionDurationMinutes	PlayerLevel	AchievementsUnlocked
InGamePurchases
0	31.993467	12.045395	9.456944	94.867408	49.562060	24.525771
1	31.988807	11.940694	9.530780	94.493222	50.027609	24.529287

Observations

Players who make in‑game purchases show only very small differences compared to those who don’t. Their average age, playtime, sessions per week, and session duration differ by less than 1%, meaning both groups behave almost the same in terms of raw activity. Purchasing players have about 1% higher player levels, suggesting they advance slightly faster. They also unlock around 1% more achievements, indicating they may be a little more engaged. Overall, the differences are very small spending money does not strongly change how players behave, but purchasers appear slightly more invested in the game.

Checking correlations between numerical features to understand their relationships and spot any strong positive or negative patterns before modeling. A correlation matrix and heatmap will be created to visualise these patterns.

plt.figure(figsize=(15,8))
sns.heatmap(df[num_cols].corr(), annot=True, fmt='.2f', cmap='YlOrRd')
plt.show()

Observations:

• The numerical features in this dataset show almost no correlation with each other, as all values are close to zero. This means things like age, playtime, session frequency and player level do not strongly relate.
• The variables seem independent and no strong patterns appear in the data.

Model Building

Now that the data has been fully explored and cleaned, we can move on to building predictive models. The goal is to predict EngagementLevel using the prepared dataset

Approach:

• Data Preparation
• Train Test Split
• Build Baseline Models
• Hyperparameter Tuning
• Evaluate on Test Data
• Select the Best Model

Data Preparation

# Creating dummy variables for categorical columns
df = pd.get_dummies(df, columns=cat_cols, drop_first=True)

# Separating independent (X) and dependent (y) variables
X = df.drop('EngagementLevel', axis=1)
y = df['EngagementLevel']

# Quick check
print(X.shape, y.shape)
print(y.value_counts())

(40034, 18) (40034,)
EngagementLevel
Medium    19374
High      10336
Low       10324
Name: count, dtype: int64

Splitting the data into 80% train and 20% test, using stratification to preserve class balance in both sets

80/20 train-test split will be used because it provides a good balance between having enough data to train the model effectively while still keeping a reliable portion for testing. Since the dataset is reasonably sised, this ratio helps the model learn better patterns without reducing test accuracy.

# Train/test split
X_train, X_test, y_train, y_test = train_test_split(
    X, y,
    test_size=0.2,      # 20% test, 80% train
    random_state=1,     # ensures reproducible results
    stratify=y          # keeps class distribution same in train/test
)

# Quick check
print(X_train.shape, X_test.shape)
print(y_train.value_counts(normalize=True))
print(y_test.value_counts(normalize=True))

(32027, 18) (8007, 18)
EngagementLevel
Medium    0.483935
High      0.258188
Low       0.257876
Name: proportion, dtype: float64
EngagementLevel
Medium    0.483952
High      0.258149
Low       0.257899
Name: proportion, dtype: float64

Scaling

• We will use StandardScaler, which standardises features by removing the mean and scaling to unit variance.
• Although Decision Trees do not require scaling, using scaled features does not negatively affect tree‑based models, so a single scaled dataset will be used for consistency.
• Scaling will be performed after the train–test split to prevent data leakage.

# Scaling the data using StandardScaler for SVM
sc = StandardScaler()

# Fit on training data only
X_train_scaled = sc.fit_transform(X_train)
X_train_scaled = pd.DataFrame(X_train_scaled, columns=X.columns)

# Transform test data
X_test_scaled = sc.transform(X_test)
X_test_scaled = pd.DataFrame(X_test_scaled, columns=X.columns)

# Function to print evaluation metrics and confusion matrix
# This will help evaluate how well the model predicts in-game purchases
def metrics_score(actual, predicted):
    print(classification_report(actual, predicted))

    cm = confusion_matrix(actual, predicted)
    plt.figure(figsize=(8,5))
    sns.heatmap(cm, annot=True, fmt='.2f', 
                xticklabels=['No Purchase', 'Purchase'],
                yticklabels=['No Purchase', 'Purchase'])
    plt.ylabel('Actual')
    plt.xlabel('Predicted')
    plt.show()

Model Building

Support Vector Machine

Linear SVM (step 1): train and evaluate on TRAIN set

• we will use a linear kernel SVM to get a simple linear baseline.
• We will train on X_train_scaled (scaled features) and y_train.
• We then predict on X_train_scaled and print metrics to inspect training behaviour.

# Creating the linear SVM model
svm_linear = SVC(kernel='linear', probability=True, class_weight='balanced', random_state=42)

# Fitting on scaled training data (X_train_scaled, y_train must exist)
svm_linear.fit(X_train_scaled, y_train)

# Predict on training set 
y_pred_train_svm = svm_linear.predict(X_train_scaled)

# Evaluate on training set
metrics_score(y_train, y_pred_train_svm)

              precision    recall  f1-score   support

        High       0.81      0.91      0.86      8269
         Low       0.69      0.80      0.74      8259
      Medium       0.86      0.74      0.79     15499

    accuracy                           0.80     32027
   macro avg       0.79      0.81      0.80     32027
weighted avg       0.80      0.80      0.80     32027

# Predicting on the test set using the trained SVM model
y_pred_test_svm = svm_linear.predict(X_test_scaled)

# Evaluating performance on the test set
metrics_score(y_test, y_pred_test_svm)

              precision    recall  f1-score   support

        High       0.81      0.90      0.85      2067
         Low       0.70      0.81      0.75      2065
      Medium       0.86      0.74      0.80      3875

    accuracy                           0.80      8007
   macro avg       0.79      0.82      0.80      8007
weighted avg       0.81      0.80      0.80      8007

Observations:

• The SVM model achieves 80% accuracy on both the training set (0.80) and the test set (0.80).This consistency indicates that the model is not overfitting and generalises well to unseen data.
• The recall for some classes however, is lower around 74% on average,suggesting that the model may miss certain cases and might not correctly identify all instances of those classes.

RBF Kernel

Now that we have trained and evaluated the Linear SVM, the next step is to train an SVM with an RBF kernel (Radial Basis Function).

from sklearn.preprocessing import label_binarize
from sklearn.metrics import precision_recall_curve
import matplotlib.pyplot as plt

# Binarize labels for all three classes
y_train_bin = label_binarize(y_train, classes=['High', 'Medium', 'Low'])

# Column 0 -> High, Column 1 -> Medium, Column 2 -> Low
class_names = ['High', 'Medium', 'Low']
fig, axes = plt.subplots(1, 3, figsize=(18, 5))
for i, class_name in enumerate(class_names):
    precisions, recalls, thresholds = precision_recall_curve(y_train_bin[:, i], y_scores_rbf[:, i])
    ax = axes[i]
    ax.plot(thresholds, precisions[:-1], 'b--', label='Precision')
    ax.plot(thresholds, recalls[:-1], 'g-', label='Recall')
    ax.set_xlabel('Threshold')
    ax.set_ylabel('Score')
    ax.set_title(f'Precision & Recall vs Threshold - {class_name}')
    ax.set_ylim([0, 1])
    ax.legend(loc='upper left')

plt.tight_layout()
plt.show()

Observations

• The plots show how precision and recall change for each class (High, Medium, Low) at different classification thresholds.
• The dashed lines represent precision, and the solid lines represent recall.
• These curves help identify a threshold that balances precision and recall for each class, which can be useful if some classes (like Low) are more important to detect correctly.
• Since this is a multi-class problem, each class is plotted separately (one-vs-rest), making it easier to interpret the model’s behavior for each category.

Hyperparameter tuning for RBF SVM (GridSearchCV)

# Hyperparameter grid
param_grid = {
    'C': [0.1, 1, 10],        
    'gamma': ['scale', 0.01, 0.1, 1],  
    'kernel': ['rbf'],
    'class_weight': [None, 'balanced']
}

# base estimator
svm_rbf = SVC(probability=True, random_state=42)

# GridSearchCV setup
gs = GridSearchCV(
    estimator=svm_rbf,
    param_grid=param_grid,
    scoring='f1_weighted',   
    cv=3,                    
    n_jobs=-1,
    verbose=2
)

# Fit grid on scaled training data
gs.fit(X_train_scaled, y_train)

import os
os.makedirs('models', exist_ok=True)

# Save best model
joblib.dump(gs.best_estimator_, 'models/svm_rbf_grid_best.joblib')

# Show best params and score
print("Best params:", gs.best_params_)
print("Best CV f1_weighted:", gs.best_score_)

# Evaluate on test data
best_svm = gs.best_estimator_
y_test_pred = best_svm.predict(X_test_scaled)

metrics_score(y_test, y_test_pred)

Best params: {'C': 10, 'class_weight': None, 'gamma': 0.01, 'kernel': 'rbf'}
Best CV f1_weighted: 0.8861278256663674
              precision    recall  f1-score   support

        High       0.92      0.85      0.88      2067
         Low       0.89      0.84      0.86      2065
      Medium       0.87      0.93      0.90      3875

    accuracy                           0.89      8007
   macro avg       0.89      0.87      0.88      8007
weighted avg       0.89      0.89      0.89      8007

Tuned SVM (RBF Kernel)

• The tuned RBF SVM achieved a strong cross-validated weighted F1-score of 0.886, indicating excellent generalisation performance.
• The test accuracy improved to 0.89, which is significantly higher than the baseline linear SVM (0.80).
• High and Low classes showed good performance with 92% precision and 85% recall for High, and 89% precision and 84% recall for Low.
• The Medium class achieved the highest recall (0.93), meaning the model is particularly good at identifying Medium-engagement players.
• The overall weighted F1-score of 0.89 suggests balanced performance across all classes.
• Conclusion, The tuned RBF SVM outperforms the linear SVM, successfully capturing non-linear relationships in player behavior and providing the best model for predicting engagement levels in this dataset.

Decision Tree

• First of all, we will train a baseline Decision Tree
• Uses X_train and y_train (unscaled)
• Decision trees don't require feature scaling
• We set random_state for reproducibility

# Create baseline Decision Tree
dt = DecisionTreeClassifier(random_state=42)

# Fit on training data
dt.fit(X_train, y_train)

print("Baseline Decision Tree trained.")

Baseline Decision Tree trained.

Evaluating baseline Decision Tree on train and test

• we will use metrics_score() helper, (prints classification_report + confusion matrix)
• Also print simple accuracy numbers for quick comparison

# Predict on train
y_pred_train_dt = dt.predict(X_train)
print("=== TRAIN SET PERFORMANCE ===")
metrics_score(y_train, y_pred_train_dt)

# Quick train accuracy
train_acc_dt = (y_pred_train_dt == y_train).mean()
print(f"Train accuracy (Decision Tree baseline): {train_acc_dt:.4f}\n")

# Predict on test
y_pred_test_dt = dt.predict(X_test)
print("=== TEST SET PERFORMANCE ===")
metrics_score(y_test, y_pred_test_dt)

# Quick test accuracy
test_acc_dt = (y_pred_test_dt == y_test).mean()
print(f"Test accuracy (Decision Tree baseline): {test_acc_dt:.4f}")

=== TRAIN SET PERFORMANCE ===
              precision    recall  f1-score   support

        High       1.00      1.00      1.00      8269
         Low       1.00      1.00      1.00      8259
      Medium       1.00      1.00      1.00     15499

    accuracy                           1.00     32027
   macro avg       1.00      1.00      1.00     32027
weighted avg       1.00      1.00      1.00     32027

Train accuracy (Decision Tree baseline): 1.0000

=== TEST SET PERFORMANCE ===
              precision    recall  f1-score   support

        High       0.82      0.82      0.82      2067
         Low       0.80      0.81      0.80      2065
      Medium       0.87      0.86      0.86      3875

    accuracy                           0.84      8007
   macro avg       0.83      0.83      0.83      8007
weighted avg       0.84      0.84      0.84      8007

Test accuracy (Decision Tree baseline): 0.8355

Observations on Decision Tree Baseline Model

• Training Performance: achieved 100% accuracy on the training set, with precision, recall and F1 of 1.00 for all classes.This suggests overfitting the model memorised the training data instead of learning general patterns.
• Test Performance: test accuracy dropped to 83.55%, indicating the model struggles to generalise.
• Class-level F1: High: 0.82 Low: 0.80 Medium: 0.86 (best-performing class). This means the model performs decently on the test set but much worse than on training data, confirming overfitting.
• Conclusion: The baseline Decision Tree is overfitted on the training data and needs hyperparameter tuning to improve generalisation and performance on unseen data.

Turning Decision Tree

# We will use smaller grid because full grid is too slow
param_grid_small = {
    'criterion': ['gini', 'entropy'],
    'max_depth': [5, 10, None],
    'min_samples_split': [2, 5]
}

grid_dt_small = GridSearchCV(
    DecisionTreeClassifier(random_state=42),
    param_grid_small,
    scoring='f1_weighted',
    cv=3,
    n_jobs=1,
    verbose=2
)
grid_dt_small.fit(X_train, y_train)
best_dt = grid_dt_small.best_estimator_
joblib.dump(best_dt, 'models/decision_tree_grid_best_small.joblib')
print("Best params (small grid):", grid_dt_small.best_params_)
print("Best CV f1_weighted (small grid):", grid_dt_small.best_score_)

Fitting 3 folds for each of 12 candidates, totalling 36 fits
[CV] END ...criterion=gini, max_depth=5, min_samples_split=2; total time=   0.0s
[CV] END ...criterion=gini, max_depth=5, min_samples_split=2; total time=   0.0s
[CV] END ...criterion=gini, max_depth=5, min_samples_split=2; total time=   0.0s
[CV] END ...criterion=gini, max_depth=5, min_samples_split=5; total time=   0.0s
[CV] END ...criterion=gini, max_depth=5, min_samples_split=5; total time=   0.0s
[CV] END ...criterion=gini, max_depth=5, min_samples_split=5; total time=   0.0s
[CV] END ..criterion=gini, max_depth=10, min_samples_split=2; total time=   0.0s
[CV] END ..criterion=gini, max_depth=10, min_samples_split=2; total time=   0.0s
[CV] END ..criterion=gini, max_depth=10, min_samples_split=2; total time=   0.0s
[CV] END ..criterion=gini, max_depth=10, min_samples_split=5; total time=   0.0s
[CV] END ..criterion=gini, max_depth=10, min_samples_split=5; total time=   0.0s
[CV] END ..criterion=gini, max_depth=10, min_samples_split=5; total time=   0.0s
[CV] END criterion=gini, max_depth=None, min_samples_split=2; total time=   0.1s
[CV] END criterion=gini, max_depth=None, min_samples_split=2; total time=   0.1s
[CV] END criterion=gini, max_depth=None, min_samples_split=2; total time=   0.1s
[CV] END criterion=gini, max_depth=None, min_samples_split=5; total time=   0.1s
[CV] END criterion=gini, max_depth=None, min_samples_split=5; total time=   0.1s
[CV] END criterion=gini, max_depth=None, min_samples_split=5; total time=   0.1s
[CV] END criterion=entropy, max_depth=5, min_samples_split=2; total time=   0.0s
[CV] END criterion=entropy, max_depth=5, min_samples_split=2; total time=   0.0s
[CV] END criterion=entropy, max_depth=5, min_samples_split=2; total time=   0.0s
[CV] END criterion=entropy, max_depth=5, min_samples_split=5; total time=   0.0s
[CV] END criterion=entropy, max_depth=5, min_samples_split=5; total time=   0.0s
[CV] END criterion=entropy, max_depth=5, min_samples_split=5; total time=   0.0s
[CV] END criterion=entropy, max_depth=10, min_samples_split=2; total time=   0.0s
[CV] END criterion=entropy, max_depth=10, min_samples_split=2; total time=   0.0s
[CV] END criterion=entropy, max_depth=10, min_samples_split=2; total time=   0.0s
[CV] END criterion=entropy, max_depth=10, min_samples_split=5; total time=   0.0s
[CV] END criterion=entropy, max_depth=10, min_samples_split=5; total time=   0.0s
[CV] END criterion=entropy, max_depth=10, min_samples_split=5; total time=   0.0s
[CV] END criterion=entropy, max_depth=None, min_samples_split=2; total time=   0.1s
[CV] END criterion=entropy, max_depth=None, min_samples_split=2; total time=   0.1s
[CV] END criterion=entropy, max_depth=None, min_samples_split=2; total time=   0.1s
[CV] END criterion=entropy, max_depth=None, min_samples_split=5; total time=   0.1s
[CV] END criterion=entropy, max_depth=None, min_samples_split=5; total time=   0.1s
[CV] END criterion=entropy, max_depth=None, min_samples_split=5; total time=   0.1s
Best params (small grid): {'criterion': 'entropy', 'max_depth': 10, 'min_samples_split': 5}
Best CV f1_weighted (small grid): 0.8929661417063633

# Evaluating tuned Decision Tree on test
y_pred_test_best_dt = best_dt.predict(X_test)
print("=== TUNED DECISION TREE: TEST PERFORMANCE ===")
metrics_score(y_test, y_pred_test_best_dt)
print("Test accuracy (tuned DT):", (y_pred_test_best_dt == y_test).mean())

=== TUNED DECISION TREE: TEST PERFORMANCE ===
              precision    recall  f1-score   support

        High       0.91      0.87      0.89      2067
         Low       0.89      0.86      0.87      2065
      Medium       0.90      0.93      0.91      3875

    accuracy                           0.90      8007
   macro avg       0.90      0.89      0.89      8007
weighted avg       0.90      0.90      0.90      8007

Test accuracy (tuned DT): 0.8979642812539028

Tuned Decision Tree Performance: key observations

• The tuned Decision Tree achieved approximately 0.90 accuracy, a significant improvement over the baseline (0.84).
• Precision, recall, and F1‑scores are all balanced across High, Medium, and Low engagement classes.
• The model generalises well and no longer overfits (unlike the baseline tree which had 100% train accuracy).
• Performance for the “Medium” class is particularly strong (F1 = 0.91), which is important because it is the largest class.
• Hyperparameter tuning reduced depth and controlled splits, resulting in a more stable and accurate model.

Comparing Decision Tree vs SVM

# Predictions
y_pred_svm = best_svm.predict(X_test_scaled)   
y_pred_dt = best_dt.predict(X_test)            

# Summary function to extract key metrics per model
def summary_metrics(y_true, y_pred):
    cr = classification_report(y_true, y_pred, output_dict=True)
    acc = accuracy_score(y_true, y_pred)
    return {
        'accuracy': acc,
        'precision_weighted': cr['weighted avg']['precision'],
        'recall_weighted': cr['weighted avg']['recall'],
        'f1_weighted': cr['weighted avg']['f1-score']
    }

summary = pd.DataFrame([
    summary_metrics(y_test, y_pred_dt),
    summary_metrics(y_test, y_pred_svm)
], index=['Decision Tree (tuned)', 'SVM (tuned)'])

print(summary.round(3))

                       accuracy  precision_weighted  recall_weighted  \
Decision Tree (tuned)     0.898               0.898            0.898   
SVM (tuned)               0.886               0.888            0.886   

                       f1_weighted  
Decision Tree (tuned)        0.898  
SVM (tuned)                  0.886  

Decision Tree vs SVM (Tuned Models) Comparison Summary

The tuned Decision Tree outperformed the tuned SVM across all major metrics:

• Accuracy: DT 0.898 > SVM 0.886
• Precision (weighted): DT 0.898 > SVM 0.888
• Recall (weighted): DT 0.898 > SVM 0.886
• F1‑score (weighted): DT 0.898 > SVM 0.886
• The Decision Tree gains an advantage in handling the non-linear structure and mixed feature types in this dataset.
• SVM still performs strongly and consistently, but is slightly less effective at separating the three engagement classes.
• Overall, the tuned Decision Tree is the best-performing model for this classification task.

K-Means

Visualising Age vs PlayTimeHours

# Basic inspection of the relationship between Age and PlayTimeHours
plt.figure(figsize=(10,6))
plt.scatter(df["Age"], df["PlayTimeHours"], alpha=0.7)
plt.xlabel("Age")
plt.ylabel("Weekly Play Time (hrs)")
plt.title("Distribution of Player Age vs Weekly Play Time")
plt.show()

Observation

• The scatterplot shows that weekly play time remains high and relatively consistent across all age groups, with most players logging between 20 and 25 hours per week.
• There is no clear trend or separation between younger and older players, suggesting that age alone is not a strong predictor of weekly play time.
• This also explains why clustering on these two features alone may form overlapping groups with limited separation.

Clustering on Age & PlayTimeHours

# Run K-Means with 3 groups on the selected features
kmeans_ap = KMeans(n_clusters=3, random_state=42)
df["Cluster_AgePlay"] = kmeans_ap.fit_predict(df[["Age", "PlayTimeHours"]])

# Plot the resulting clusters
plt.figure(figsize=(10,6))
sns.scatterplot(
    x=df["Age"],
    y=df["PlayTimeHours"],
    hue=df["Cluster_AgePlay"],
    palette="Set1",
    alpha=0.6
)
plt.xlabel("Age")
plt.ylabel("Weekly Play Time (hrs)")
plt.title("K-Means Groups Based on Age and Play Time")
plt.show()

Observations

• The clustering results show that players are grouped mainly based on weekly play time rather than age.
• Although play time decreases slightly with older age groups, the separation between clusters is still weak because age does not vary meaningfully across the dataset.
• As a result, K‑Means mainly divides players into high play-time, moderate play-time, and low play-time groups.
• Age contributes very little to the separation, reinforcing that play behaviour is largely age‑independent in this datase.

SessionsPerWeek vs AchievementsUnlocked

# K-Means using gameplay frequency and achievements
kmeans_sa = KMeans(n_clusters=3, random_state=42)
df["Cluster_SessAch"] = kmeans_sa.fit_predict(df[["SessionsPerWeek", "AchievementsUnlocked"]])

plt.figure(figsize=(10,6))
sns.scatterplot(
    x=df["SessionsPerWeek"],
    y=df["AchievementsUnlocked"],
    hue=df["Cluster_SessAch"],
    palette="Set1",
    alpha=0.7
)
plt.xlabel("Sessions per Week")
plt.ylabel("Achievements Earned")
plt.title("Clusters Based on Sessions vs Achievements")
plt.show()

Observations

• The clustering separates players mainly by activity level (sessions per week) and achievement progression.
• One cluster represents high‑engagement players with many sessions and many achievements unlocked.
• Another cluster includes moderate players with mid‑range sessions and achievements.
• A third cluster contains low‑activity users who play less and unlock fewer achievements.
• This feature pair provides stronger behavioural separation than Age + PlayTimeHours

Elbow Method (Age vs PlayTimeHours

wcss = []
K_range = range(1, 9)

for k in K_range:
    model = KMeans(n_clusters=k, init="k-means++", max_iter=300, n_init=10, random_state=42)
    model.fit(df[["Age", "PlayTimeHours"]])
    wcss.append(model.inertia_)

plt.figure(figsize=(10,6))
plt.plot(K_range, wcss, marker='o')
plt.xlabel("Number of Clusters (k)")
plt.ylabel("WCSS")
plt.title("Elbow Curve for Age vs PlayTimeHours")
plt.show()

Observation:

• The line drops quickly from 1 to 3 clusters, then starts to level off.
• After k = 3, adding more clusters doesn’t improve the result much.
• So 3 clusters is the most sensible choice for this data.

Silhouette Scores

for k in range(2, 9):
    temp = KMeans(n_clusters=k, random_state=42, n_init=10)
    labels = temp.fit_predict(df[["Age", "PlayTimeHours"]])
    score = silhouette_score(df[["Age", "PlayTimeHours"]], labels)
    print(f"k = {k}, silhouette = {score:.3f}")

k = 2, silhouette = 0.431
k = 3, silhouette = 0.394
k = 4, silhouette = 0.383
k = 5, silhouette = 0.388
k = 6, silhouette = 0.382
k = 7, silhouette = 0.367
k = 8, silhouette = 0.382

Observation:

• The silhouette score is highest at k = 2 (0.431), and slightly lower at k = 3 (0.394).
• After k = 3, the silhouette values stay almost flat, showing no clear improvement.
• This means the dataset doesn’t naturally form very strong clusters using only Age and PlayTimeHours.
• Even though silhouette suggests k = 2 is slightly better, k = 3 is still acceptable, especially because it matches the elbow method and gives more meaningful behavioural groups

pd.crosstab(df["Cluster_SessAch"], df["EngagementLevel"])

EngagementLevel	High	Low	Medium
Cluster_SessAch
0	3124	3748	5880
1	3666	2971	6983
2	3546	3605	6511

Observation:

• Cluster 0 contains many Low and Medium engagement players, showing this group represents lower‑activity behaviour.
• Cluster 1 has the highest number of Medium and High engagement players, suggesting they are consistently active and achievement‑driven.
• Cluster 2 includes a mix of High and Low players, but Medium is still strong meaning this group shows mixed behaviour, but still more active than Cluster 0.
• Overall, the crosstab shows that clustering based on gameplay metrics reflects real engagement patterns, which supports the usefulness of the model. model.