{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[],"authorship_tag":"ABX9TyM7HjWnonYzK11yLc6PLWvI"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"code","execution_count":2,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"JpH7FEEITN69","executionInfo":{"status":"ok","timestamp":1770739811392,"user_tz":0,"elapsed":1418,"user":{"displayName":"Vaghani Pushpendra","userId":"09359783643573916449"}},"outputId":"8c5a7a34-8702-4c88-9951-a4f23c3ea240"},"outputs":[{"output_type":"stream","name":"stdout","text":["   PatientId  Pregnancies  Glucose  BloodPressure  SkinThickness  Insulin  \\\n","0          1            6      148             72             35        0   \n","1          2            1       85             66             29        0   \n","2          3            8      183             64              0        0   \n","3          4            1       89             66             23       94   \n","4          5            0      137             40             35      168   \n","\n","    BMI  DiabetesPedigreeFunction  Age  Outcome  \n","0  33.6                     0.627   50        1  \n","1  26.6                     0.351   31        0  \n","2  23.3                     0.672   32        1  \n","3  28.1                     0.167   21        0  \n","4  43.1                     2.288   33        1  \n","\n","Feature Importance:\n","                    Feature  Importance\n","2                   Glucose    0.267089\n","6                       BMI    0.153334\n","8                       Age    0.129185\n","7  DiabetesPedigreeFunction    0.102934\n","0                 PatientId    0.090048\n","3             BloodPressure    0.075748\n","1               Pregnancies    0.065340\n","5                   Insulin    0.060622\n","4             SkinThickness    0.055700\n","\n","Model Comparison Results:\n","                          Model  Accuracy  Precision    Recall  F1 Score\n","0      Random Forest (Original)  0.753247   0.644068  0.690909  0.666667\n","1  Random Forest (Top Features)  0.772727   0.678571  0.690909  0.684685\n","2           Random Forest (PCA)  0.701299   0.591837  0.527273  0.557692\n","3           Logistic Regression  0.733766   0.645833  0.563636  0.601942\n","4                   Naive Bayes  0.772727   0.717391  0.600000  0.653465\n"]}],"source":["# ===============================\n","# Feature Selection & Model Comparison\n","# ===============================\n","\n","import pandas as pd\n","import numpy as np\n","\n","from sklearn.model_selection import train_test_split\n","from sklearn.preprocessing import StandardScaler\n","from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score\n","\n","from sklearn.ensemble import RandomForestClassifier\n","from sklearn.decomposition import PCA\n","from sklearn.linear_model import LogisticRegression\n","from sklearn.naive_bayes import GaussianNB\n","\n","# ===============================\n","# Load Dataset\n","# ===============================\n","df = pd.read_csv('/content/diabetes.csv')\n","print(df.head())\n","\n","# ===============================\n","# Handle Missing Values\n","# ===============================\n","df.fillna(df.mean(), inplace=True)\n","\n","# ===============================\n","# Split Features and Target\n","# ===============================\n","X = df.drop('Outcome', axis=1)\n","y = df['Outcome']\n","\n","# ===============================\n","# Train-Test Split (Original Data)\n","# ===============================\n","X_train, X_test, y_train, y_test = train_test_split(\n","    X, y, test_size=0.2, random_state=42\n",")\n","\n","# ===============================\n","# Random Forest - Original Features\n","# ===============================\n","rf = RandomForestClassifier(random_state=42)\n","rf.fit(X_train, y_train)\n","y_pred_rf = rf.predict(X_test)\n","\n","rf_acc = accuracy_score(y_test, y_pred_rf)\n","rf_prec = precision_score(y_test, y_pred_rf)\n","rf_rec = recall_score(y_test, y_pred_rf)\n","rf_f1 = f1_score(y_test, y_pred_rf)\n","\n","# ===============================\n","# Feature Importance & Selection\n","# ===============================\n","feature_importance = pd.DataFrame({\n","    'Feature': X.columns,\n","    'Importance': rf.feature_importances_\n","}).sort_values(by='Importance', ascending=False)\n","\n","top_features = feature_importance['Feature'][:5]\n","X_top = X[top_features]\n","\n","X_train, X_test, y_train, y_test = train_test_split(\n","    X_top, y, test_size=0.2, random_state=42\n",")\n","\n","rf_top = RandomForestClassifier(random_state=42)\n","rf_top.fit(X_train, y_train)\n","y_pred_top = rf_top.predict(X_test)\n","\n","rf_top_acc = accuracy_score(y_test, y_pred_top)\n","rf_top_prec = precision_score(y_test, y_pred_top)\n","rf_top_rec = recall_score(y_test, y_pred_top)\n","rf_top_f1 = f1_score(y_test, y_pred_top)\n","\n","# ===============================\n","# PCA Feature Reduction\n","# ===============================\n","scaler = StandardScaler()\n","X_scaled = scaler.fit_transform(X)\n","\n","pca = PCA(n_components=5)\n","X_pca = pca.fit_transform(X_scaled)\n","\n","X_train, X_test, y_train, y_test = train_test_split(\n","    X_pca, y, test_size=0.2, random_state=42\n",")\n","\n","rf_pca = RandomForestClassifier(random_state=42)\n","rf_pca.fit(X_train, y_train)\n","y_pred_pca = rf_pca.predict(X_test)\n","\n","rf_pca_acc = accuracy_score(y_test, y_pred_pca)\n","rf_pca_prec = precision_score(y_test, y_pred_pca)\n","rf_pca_rec = recall_score(y_test, y_pred_pca)\n","rf_pca_f1 = f1_score(y_test, y_pred_pca)\n","\n","# ===============================\n","# Logistic Regression\n","# ===============================\n","lr = LogisticRegression(max_iter=1000)\n","lr.fit(X_train, y_train)\n","y_pred_lr = lr.predict(X_test)\n","\n","lr_acc = accuracy_score(y_test, y_pred_lr)\n","lr_prec = precision_score(y_test, y_pred_lr)\n","lr_rec = recall_score(y_test, y_pred_lr)\n","lr_f1 = f1_score(y_test, y_pred_lr)\n","\n","# ===============================\n","# Naive Bayes\n","# ===============================\n","nb = GaussianNB()\n","nb.fit(X_train, y_train)\n","y_pred_nb = nb.predict(X_test)\n","\n","nb_acc = accuracy_score(y_test, y_pred_nb)\n","nb_prec = precision_score(y_test, y_pred_nb)\n","nb_rec = recall_score(y_test, y_pred_nb)\n","nb_f1 = f1_score(y_test, y_pred_nb)\n","\n","# ===============================\n","# Results Comparison\n","# ===============================\n","results = pd.DataFrame({\n","    'Model': [\n","        'Random Forest (Original)',\n","        'Random Forest (Top Features)',\n","        'Random Forest (PCA)',\n","        'Logistic Regression',\n","        'Naive Bayes'\n","    ],\n","    'Accuracy': [\n","        rf_acc,\n","        rf_top_acc,\n","        rf_pca_acc,\n","        lr_acc,\n","        nb_acc\n","    ],\n","    'Precision': [\n","        rf_prec,\n","        rf_top_prec,\n","        rf_pca_prec,\n","        lr_prec,\n","        nb_prec\n","    ],\n","    'Recall': [\n","        rf_rec,\n","        rf_top_rec,\n","        rf_pca_rec,\n","        lr_rec,\n","        nb_rec\n","    ],\n","    'F1 Score': [\n","        rf_f1,\n","        rf_top_f1,\n","        rf_pca_f1,\n","        lr_f1,\n","        nb_f1\n","    ]\n","})\n","\n","print(\"\\nFeature Importance:\")\n","print(feature_importance)\n","\n","print(\"\\nModel Comparison Results:\")\n","print(results)\n"]},{"cell_type":"markdown","source":["The Random Forest model trained on the Diabetes dataset showed that the most important features for predicting diabetes are Glucose, BMI, Age, and DiabetesPedigreeFunction, while features like Insulin and SkinThickness contributed very little. Interestingly, PatientId appeared in the feature importance ranking, but it is not a predictive feature and should be removed in practice. Comparing model performance, the Random Forest trained on all features achieved an accuracy of approximately 75%, while using only the top features slightly improved performance to 77%, indicating that manual feature selection helps reduce noise and enhances model generalization. Applying PCA for dimensionality reduction, however, reduced performance to 70%, showing that PCA may discard useful information in low-dimensional datasets and is more suitable for high-dimensional data. Among other classifiers, Naive Bayes performed surprisingly well, achieving similar accuracy to the top-feature Random Forest and demonstrating that its independence assumptions are reasonably effective for this dataset. Logistic Regression performed moderately, with slightly lower accuracy and F1-score. Overall, this analysis highlights that careful feature selection can improve model performance, PCA may not always be beneficial, and ensemble and probabilistic models are effective choices for tabular medical datasets."],"metadata":{"id":"MEi8Kmd4XWEf"}}]}