{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[],"authorship_tag":"ABX9TyNw/XOKNUbV7eFZsLucJY73"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"JpH7FEEITN69","executionInfo":{"status":"ok","timestamp":1770739811392,"user_tz":0,"elapsed":1418,"user":{"displayName":"Vaghani Pushpendra","userId":"09359783643573916449"}},"outputId":"8c5a7a34-8702-4c88-9951-a4f23c3ea240"},"outputs":[{"output_type":"stream","name":"stdout","text":["   PatientId  Pregnancies  Glucose  BloodPressure  SkinThickness  Insulin  \\\n","0          1            6      148             72             35        0   \n","1          2            1       85             66             29        0   \n","2          3            8      183             64              0        0   \n","3          4            1       89             66             23       94   \n","4          5            0      137             40             35      168   \n","\n","    BMI  DiabetesPedigreeFunction  Age  Outcome  \n","0  33.6                     0.627   50        1  \n","1  26.6                     0.351   31        0  \n","2  23.3                     0.672   32        1  \n","3  28.1                     0.167   21        0  \n","4  43.1                     2.288   33        1  \n","\n","Feature Importance:\n","                    Feature  Importance\n","2                   Glucose    0.267089\n","6                       BMI    0.153334\n","8                       Age    0.129185\n","7  DiabetesPedigreeFunction    0.102934\n","0                 PatientId    0.090048\n","3             BloodPressure    0.075748\n","1               Pregnancies    0.065340\n","5                   Insulin    0.060622\n","4             SkinThickness    0.055700\n","\n","Model Comparison Results:\n","                          Model  Accuracy  Precision    Recall  F1 Score\n","0      Random Forest (Original)  0.753247   0.644068  0.690909  0.666667\n","1  Random Forest (Top Features)  0.772727   0.678571  0.690909  0.684685\n","2           Random Forest (PCA)  0.701299   0.591837  0.527273  0.557692\n","3           Logistic Regression  0.733766   0.645833  0.563636  0.601942\n","4                   Naive Bayes  0.772727   0.717391  0.600000  0.653465\n"]}],"source":["# ===============================\n","# Feature Selection & Model Comparison\n","# ===============================\n","\n","import pandas as pd\n","import numpy as np\n","\n","from sklearn.model_selection import train_test_split\n","from sklearn.preprocessing import StandardScaler\n","from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score\n","\n","from sklearn.ensemble import RandomForestClassifier\n","from sklearn.decomposition import PCA\n","from sklearn.linear_model import LogisticRegression\n","from sklearn.naive_bayes import GaussianNB\n","\n","# ===============================\n","# Load Dataset\n","# ===============================\n","df = pd.read_csv('/content/diabetes.csv')\n","print(df.head())\n","\n","# ===============================\n","# Handle Missing Values\n","# ===============================\n","df.fillna(df.mean(), inplace=True)\n","\n","# ===============================\n","# Split Features and Target\n","# ===============================\n","X = df.drop('Outcome', axis=1)\n","y = df['Outcome']\n","\n","# ===============================\n","# Train-Test Split (Original Data)\n","# ===============================\n","X_train, X_test, y_train, y_test = train_test_split(\n","    X, y, test_size=0.2, random_state=42\n",")\n","\n","# ===============================\n","# Random Forest - Original Features\n","# ===============================\n","rf = RandomForestClassifier(random_state=42)\n","rf.fit(X_train, y_train)\n","y_pred_rf = rf.predict(X_test)\n","\n","rf_acc = accuracy_score(y_test, y_pred_rf)\n","rf_prec = precision_score(y_test, y_pred_rf)\n","rf_rec = recall_score(y_test, y_pred_rf)\n","rf_f1 = f1_score(y_test, y_pred_rf)\n","\n","# ===============================\n","# Feature Importance & Selection\n","# ===============================\n","feature_importance = pd.DataFrame({\n","    'Feature': X.columns,\n","    'Importance': rf.feature_importances_\n","}).sort_values(by='Importance', ascending=False)\n","\n","top_features = feature_importance['Feature'][:5]\n","X_top = X[top_features]\n","\n","X_train, X_test, y_train, y_test = train_test_split(\n","    X_top, y, test_size=0.2, random_state=42\n",")\n","\n","rf_top = RandomForestClassifier(random_state=42)\n","rf_top.fit(X_train, y_train)\n","y_pred_top = rf_top.predict(X_test)\n","\n","rf_top_acc = accuracy_score(y_test, y_pred_top)\n","rf_top_prec = precision_score(y_test, y_pred_top)\n","rf_top_rec = recall_score(y_test, y_pred_top)\n","rf_top_f1 = f1_score(y_test, y_pred_top)\n","\n","# ===============================\n","# PCA Feature Reduction\n","# ===============================\n","scaler = StandardScaler()\n","X_scaled = scaler.fit_transform(X)\n","\n","pca = PCA(n_components=5)\n","X_pca = pca.fit_transform(X_scaled)\n","\n","X_train, X_test, y_train, y_test = train_test_split(\n","    X_pca, y, test_size=0.2, random_state=42\n",")\n","\n","rf_pca = RandomForestClassifier(random_state=42)\n","rf_pca.fit(X_train, y_train)\n","y_pred_pca = rf_pca.predict(X_test)\n","\n","rf_pca_acc = accuracy_score(y_test, y_pred_pca)\n","rf_pca_prec = precision_score(y_test, y_pred_pca)\n","rf_pca_rec = recall_score(y_test, y_pred_pca)\n","rf_pca_f1 = f1_score(y_test, y_pred_pca)\n","\n","# ===============================\n","# Logistic Regression\n","# ===============================\n","lr = LogisticRegression(max_iter=1000)\n","lr.fit(X_train, y_train)\n","y_pred_lr = lr.predict(X_test)\n","\n","lr_acc = accuracy_score(y_test, y_pred_lr)\n","lr_prec = precision_score(y_test, y_pred_lr)\n","lr_rec = recall_score(y_test, y_pred_lr)\n","lr_f1 = f1_score(y_test, y_pred_lr)\n","\n","# ===============================\n","# Naive Bayes\n","# ===============================\n","nb = GaussianNB()\n","nb.fit(X_train, y_train)\n","y_pred_nb = nb.predict(X_test)\n","\n","nb_acc = accuracy_score(y_test, y_pred_nb)\n","nb_prec = precision_score(y_test, y_pred_nb)\n","nb_rec = recall_score(y_test, y_pred_nb)\n","nb_f1 = f1_score(y_test, y_pred_nb)\n","\n","# ===============================\n","# Results Comparison\n","# ===============================\n","results = pd.DataFrame({\n","    'Model': [\n","        'Random Forest (Original)',\n","        'Random Forest (Top Features)',\n","        'Random Forest (PCA)',\n","        'Logistic Regression',\n","        'Naive Bayes'\n","    ],\n","    'Accuracy': [\n","        rf_acc,\n","        rf_top_acc,\n","        rf_pca_acc,\n","        lr_acc,\n","        nb_acc\n","    ],\n","    'Precision': [\n","        rf_prec,\n","        rf_top_prec,\n","        rf_pca_prec,\n","        lr_prec,\n","        nb_prec\n","    ],\n","    'Recall': [\n","        rf_rec,\n","        rf_top_rec,\n","        rf_pca_rec,\n","        lr_rec,\n","        nb_rec\n","    ],\n","    'F1 Score': [\n","        rf_f1,\n","        rf_top_f1,\n","        rf_pca_f1,\n","        lr_f1,\n","        nb_f1\n","    ]\n","})\n","\n","print(\"\\nFeature Importance:\")\n","print(feature_importance)\n","\n","print(\"\\nModel Comparison Results:\")\n","print(results)\n"]},{"cell_type":"markdown","source":["The Random Forest model trained on the Diabetes dataset identified Glucose, BMI, Age, and DiabetesPedigreeFunction as the most influential variables in predicting diabetes. In contrast, features such as Insulin and SkinThickness had minimal impact on the model’s predictions. Although PatientId appeared in the feature importance list, it does not contain meaningful predictive information and should ideally be excluded during preprocessing.\n","\n","When comparing performance, the Random Forest model trained on all available features achieved an accuracy of about 75%. However, restricting the model to only the most important features slightly increased accuracy to approximately 77%, suggesting that removing less relevant variables can reduce noise and improve generalization.\n","\n","Dimensionality reduction using Principal Component Analysis (PCA) resulted in a lower accuracy of around 70%. This indicates that PCA may remove valuable information in datasets with a limited number of features and is generally more effective for high-dimensional data.\n","\n","Among other classification algorithms, Naive Bayes performed competitively, reaching accuracy levels close to the optimized Random Forest model. This suggests that its assumption of feature independence works reasonably well for this dataset. Logistic Regression showed moderate performance, with slightly lower accuracy and F1-score compared to the other models.\n","\n","In summary, this analysis demonstrates that targeted feature selection can enhance model performance, PCA is not always advantageous for smaller datasets, and ensemble as well as probabilistic models are strong choices for structured medical data."],"metadata":{"id":"MEi8Kmd4XWEf"}}]}