{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "be0c0d68",
   "metadata": {},
   "source": [
    "# LineSight — Complete Interactive Tutorial\n",
    "\n",
    "This notebook exercises **every single public function** across all 7 regression models in LineSight.\n",
    "\n",
    "| Model | Type |\n",
    "|---|---|\n",
    "| `LinearRegression` | Single-feature gradient descent |\n",
    "| `MultipleLinearRegression` | Multi-feature gradient descent |\n",
    "| `PolynomialRegression` | Non-linear feature expansion |\n",
    "| `RidgeRegression` | L2 regularization |\n",
    "| `LassoRegression` | L1 regularization + feature selection |\n",
    "| `ElasticNetRegression` | L1 + L2 combined |\n",
    "| `LogisticRegression` | Binary classification |\n",
    "\n",
    "> **After every diagram**, a **Context Panel** is printed to stdout explaining the theory, the formula, the constraints, and how to read the chart."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3adcbcf",
   "metadata": {},
   "source": [
    "## Setup — imports and shared datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6da93bdf",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")      # suppress convergence hints in output\n",
    "from IPython.display import HTML       # needed to display JS animations\n",
    "\n",
    "from linesight import (\n",
    "    LinearRegression,\n",
    "    MultipleLinearRegression,\n",
    "    PolynomialRegression,\n",
    "    RidgeRegression,\n",
    "    LassoRegression,\n",
    "    ElasticNetRegression,\n",
    "    LogisticRegression,\n",
    ")\n",
    "\n",
    "np.random.seed(42)\n",
    "\n",
    "# ── 1-D dataset (for LinearRegression / PolynomialRegression) ──────────────\n",
    "X_lin = np.random.rand(100, 1) * 10\n",
    "y_lin = 2.5 * X_lin.ravel() + 5.0 + np.random.randn(100) * 2.0\n",
    "\n",
    "# ── 3-feature dataset (for Multiple / Ridge / Lasso / ElasticNet) ──────────\n",
    "X_mult = np.random.rand(100, 3) * 10\n",
    "y_mult = (3.0 * X_mult[:, 0]\n",
    "          - 1.5 * X_mult[:, 1]\n",
    "          + 2.0 * X_mult[:, 2]\n",
    "          + 10.0\n",
    "          + np.random.randn(100))\n",
    "\n",
    "# ── 2-feature dataset (for constraint region plots that need exactly 2 cols) ─\n",
    "X_2f   = X_mult[:, :2]          # first 2 columns\n",
    "y_2f   = (3.0 * X_2f[:, 0]\n",
    "          - 1.5 * X_2f[:, 1]\n",
    "          + 10.0\n",
    "          + np.random.randn(100))\n",
    "\n",
    "# ── Curved 1-D dataset (for PolynomialRegression) ──────────────────────────\n",
    "X_poly = np.linspace(-3, 3, 120).reshape(-1, 1)\n",
    "y_poly = (0.5 * X_poly.ravel()**3\n",
    "          - X_poly.ravel()**2\n",
    "          + 2.0\n",
    "          + np.random.randn(120) * 1.5)\n",
    "\n",
    "# ── Binary classification dataset (for LogisticRegression) ─────────────────\n",
    "X_log  = np.random.randn(200, 2) * 2\n",
    "z_log  = 1.5 * X_log[:, 0] - 2.0 * X_log[:, 1] + 0.5\n",
    "y_log  = (np.random.rand(200) < (1 / (1 + np.exp(-z_log)))).astype(float)\n",
    "\n",
    "print(\"Datasets ready.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ff961f1",
   "metadata": {},
   "source": [
    "---\n",
    "## 1  Simple Linear Regression\n",
    "\n",
    "**Concept:** Learns `y = m*x + b` via gradient descent, minimising MSE.\n",
    "\n",
    "**All functions covered:**\n",
    "`fit` · `predict` · `score` · `get_training_history` · `show_equation` · `explain_coefficients` · `explain_fit` · `summary` · `plot_fit` · `plot_residuals` · `plot_loss_curve` · `plot_loss_surface` · `plot_gradient_vectors` · `plot_prediction_intervals` · `plot_sensitivity_analysis` · `plot_actual_vs_predicted` · `plot_learning_curve` · `compare_learning_rates` · `animate_training` · `animate_loss_surface_path`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e66f8c7f",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin = LinearRegression(learning_rate=0.01, epochs=500, store_history=True)\n",
    "model_lin.fit(X_lin, y_lin)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "940e77d3",
   "metadata": {},
   "source": [
    "### 1.1  Textual Explanations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fbccc4d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "84f78b8f",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.show_equation()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c36529b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.explain_coefficients()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11ab7fc7",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.explain_fit(X_lin, y_lin)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2aa5287d",
   "metadata": {},
   "source": [
    "### 1.2  Prediction & Scoring"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b6944df1",
   "metadata": {},
   "outputs": [],
   "source": [
    "preds = model_lin.predict(X_lin[:5])\n",
    "print(\"First 5 predictions:\", preds)\n",
    "print()\n",
    "metrics = model_lin.score(X_lin, y_lin)\n",
    "print(\"Score metrics:\", metrics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "636a1ef9",
   "metadata": {},
   "outputs": [],
   "source": [
    "history = model_lin.get_training_history()\n",
    "print(f\"Epochs run   : {history.epochs_run}\")\n",
    "print(f\"Converged    : {history.converged}\")\n",
    "print(f\"Final loss   : {history.losses[-1]:.6f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5403c944",
   "metadata": {},
   "source": [
    "### 1.3  Visualizations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6058376d",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.plot_fit(X_lin, y_lin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28e4215e",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.plot_residuals(X_lin, y_lin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c853b7db",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.plot_gradient_vectors(X_lin, y_lin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f9f36d70",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.plot_prediction_intervals(X_lin, y_lin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12206f61",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.plot_loss_curve()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9513a357",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.plot_sensitivity_analysis(X_lin, y_lin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "448fd3af",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.plot_loss_surface(X_lin, y_lin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a8c2e3ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.plot_actual_vs_predicted(X_lin, y_lin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "56837303",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.plot_learning_curve(X_lin, y_lin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "165e58bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lin.compare_learning_rates(X_lin, y_lin, learning_rates=[0.001, 0.01, 0.05])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9845031b",
   "metadata": {},
   "source": [
    "### 1.4  Animations  *(run in Jupyter to view JS player)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "89695672",
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(model_lin.animate_training(X_lin, y_lin, display=False).to_jshtml())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "07f45332",
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(model_lin.animate_loss_surface_path(X_lin, y_lin, display=False).to_jshtml())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83434c92",
   "metadata": {},
   "source": [
    "---\n",
    "## 2  Multiple Linear Regression\n",
    "\n",
    "**Concept:** Extends `y = Xw + b` to multiple features using vectorised gradient descent.\n",
    "\n",
    "**All functions covered:**\n",
    "`fit` · `predict` · `score` · `get_training_history` · `show_equation` · `explain_coefficients` · `summary` · `plot_fit` · `plot_feature_importance` · `plot_partial_regression` · `plot_correlation_matrix` · `plot_prediction_plane` · `plot_multicollinearity` · `plot_residual_heatmap` · `plot_3d_loss_slice` · `plot_actual_vs_predicted` · `plot_learning_curve`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f7bd6a84",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult = MultipleLinearRegression(\n",
    "    learning_rate=0.01, epochs=600, normalize=True, store_history=True\n",
    ")\n",
    "model_mult.fit(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "927e3f7b",
   "metadata": {},
   "source": [
    "### 2.1  Textual Explanations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c7fd145",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4fd96779",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.show_equation()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92132c5f",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.explain_coefficients()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32bcec7a",
   "metadata": {},
   "source": [
    "### 2.2  Prediction & Scoring"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5199f172",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Predictions (first 5):\", model_mult.predict(X_mult[:5]))\n",
    "print(\"Metrics:\", model_mult.score(X_mult, y_mult))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "85aee5c3",
   "metadata": {},
   "outputs": [],
   "source": [
    "h = model_mult.get_training_history()\n",
    "print(f\"Final loss: {h.losses[-1]:.4f}  |  Converged: {h.converged}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14b17bdc",
   "metadata": {},
   "source": [
    "### 2.3  Visualizations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bef19b62",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.plot_fit(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "799bc517",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.plot_feature_importance()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c2c27b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.plot_correlation_matrix(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ebf40440",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.plot_partial_regression(X_mult, y_mult, feature_idx=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d01a1656",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.plot_partial_regression(X_mult, y_mult, feature_idx=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3dd8ef2",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.plot_partial_regression(X_mult, y_mult, feature_idx=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d97e0f9b",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.plot_multicollinearity(X_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c7dc7ed2",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.plot_residual_heatmap(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a310e71",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.plot_prediction_plane(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5489748f",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.plot_3d_loss_slice(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "56bd722d",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.plot_actual_vs_predicted(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f72fb095",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_mult.plot_learning_curve(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf7ca61e",
   "metadata": {},
   "source": [
    "---\n",
    "## 3  Polynomial Regression\n",
    "\n",
    "**Concept:** Expands `x → [x, x², ..., xᵈ]` then applies linear regression, capturing non-linear shapes.\n",
    "\n",
    "**All functions covered:**\n",
    "`fit` · `predict` · `score` · `show_equation` · `summary` · `plot_fit` · `plot_residuals` · `plot_loss_curve` · `plot_basis_functions` · `compare_degrees` · `plot_actual_vs_predicted` · `plot_learning_curve` · `animate_degree_increase`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a74887a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_poly = PolynomialRegression(\n",
    "    degree=3, learning_rate=0.01, epochs=4000, normalize=True, store_history=True\n",
    ")\n",
    "model_poly.fit(X_poly, y_poly)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e9cb9dd",
   "metadata": {},
   "source": [
    "### 3.1  Textual Explanations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9947c872",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_poly.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ff758e1e",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_poly.show_equation()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44cd5458",
   "metadata": {},
   "source": [
    "### 3.2  Prediction & Scoring"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1835e760",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Predictions (first 5):\", model_poly.predict(X_poly[:5]))\n",
    "print(\"Metrics:\", model_poly.score(X_poly, y_poly))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2378066b",
   "metadata": {},
   "source": [
    "### 3.3  Visualizations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "76b295ab",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_poly.plot_fit(X_poly, y_poly)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "af0476c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_poly.plot_residuals(X_poly, y_poly)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e44fb896",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_poly.plot_loss_curve()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3ffcf3ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_poly.plot_basis_functions(X_poly, y_poly)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0dc4b9a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_poly.compare_degrees(X_poly, y_poly, degrees=[1, 2, 3, 5, 9])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f56e4691",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_poly.plot_actual_vs_predicted(X_poly, y_poly)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "68057e2e",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_poly.plot_learning_curve(X_poly, y_poly)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0a842ac",
   "metadata": {},
   "source": [
    "### 3.4  Animation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b918438f",
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(model_poly.animate_degree_increase(X_poly, y_poly, max_degree=6, display=False).to_jshtml())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0098378",
   "metadata": {},
   "source": [
    "---\n",
    "## 4  Ridge Regression  (L2 regularization)\n",
    "\n",
    "**Concept:** Adds `alpha * Σwⱼ²` penalty to MSE — shrinks all weights, never zeros.\n",
    "\n",
    "**All functions covered:**\n",
    "`fit` · `predict` · `score` · `get_training_history` · `explain_regularization` · `explain_coefficients` · `summary` · `plot_fit` · `plot_loss_curve` · `plot_coefficient_shrinkage` · `plot_constraint_region` · `plot_bias_variance_tradeoff` · `plot_effective_degrees_of_freedom` · `compare_with_linear` · `plot_actual_vs_predicted` · `plot_learning_curve` · `animate_regularization`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "defadb18",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_ridge = RidgeRegression(\n",
    "    alpha=1.5, learning_rate=0.01, epochs=2000, normalize=True, store_history=True\n",
    ")\n",
    "model_ridge.fit(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1e95aec",
   "metadata": {},
   "source": [
    "### 4.1  Textual Explanations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "608605d0",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_ridge.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a09ff43",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_ridge.explain_regularization()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ddc93717",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_ridge.explain_coefficients()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d677c810",
   "metadata": {},
   "source": [
    "### 4.2  Prediction & Scoring"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad13c8b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Predictions (first 5):\", model_ridge.predict(X_mult[:5]))\n",
    "print(\"Metrics:\", model_ridge.score(X_mult, y_mult))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5393539",
   "metadata": {},
   "outputs": [],
   "source": [
    "h = model_ridge.get_training_history()\n",
    "print(f\"Final loss: {h.losses[-1]:.4f}  |  Converged: {h.converged}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0439eb44",
   "metadata": {},
   "source": [
    "### 4.3  Visualizations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57986a3f",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_ridge.plot_fit(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5e3d8428",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_ridge.plot_loss_curve()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32dddec4",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_ridge.plot_coefficient_shrinkage(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cdd0ccec",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_ridge.plot_bias_variance_tradeoff(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24f75449",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_ridge.plot_effective_degrees_of_freedom(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d54b943",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_ridge.plot_actual_vs_predicted(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "823b5fde",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_ridge.plot_learning_curve(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64402065",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_ridge.compare_with_linear(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "53fa919e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# plot_constraint_region requires exactly 2 features — use X_2f\n",
    "model_ridge_2f = RidgeRegression(alpha=1.5, learning_rate=0.01, epochs=2000, normalize=True)\n",
    "model_ridge_2f.fit(X_2f, y_2f)\n",
    "model_ridge_2f.plot_constraint_region(X_2f, y_2f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a8ef11db",
   "metadata": {},
   "outputs": [],
   "source": [
    "# animate_regularization works best on 1-D data\n",
    "model_ridge_1d = RidgeRegression(alpha=5.0, learning_rate=0.01, epochs=500, normalize=True)\n",
    "model_ridge_1d.fit(X_lin, y_lin)\n",
    "HTML(model_ridge_1d.animate_regularization(X_lin, y_lin, display=False).to_jshtml())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "437d4afd",
   "metadata": {},
   "source": [
    "---\n",
    "## 5  Lasso Regression  (L1 regularization)\n",
    "\n",
    "**Concept:** Adds `alpha * Σ|wⱼ|` penalty — drives unimportant weights to **exactly 0** (automatic feature selection).\n",
    "\n",
    "**All functions covered:**\n",
    "`fit` · `predict` · `score` · `get_training_history` · `explain_regularization` · `explain_sparsity` · `explain_coefficients` · `summary` · `plot_fit` · `plot_loss_curve` · `plot_coefficient_shrinkage` · `plot_constraint_region` · `plot_feature_elimination` · `plot_sparsity_path` · `compare_with_linear` · `plot_actual_vs_predicted` · `plot_learning_curve` · `animate_coordinate_descent`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e6ba4452",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lasso = LassoRegression(\n",
    "    alpha=0.5, epochs=500, normalize=True, store_history=True\n",
    ")\n",
    "model_lasso.fit(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93d9d261",
   "metadata": {},
   "source": [
    "### 5.1  Textual Explanations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da3fb7e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lasso.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fea71668",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lasso.explain_regularization()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6122adc8",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lasso.explain_sparsity()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f30cdfe5",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lasso.explain_coefficients()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "361cfab8",
   "metadata": {},
   "source": [
    "### 5.2  Prediction & Scoring"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "285178fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Predictions (first 5):\", model_lasso.predict(X_mult[:5]))\n",
    "print(\"Metrics:\", model_lasso.score(X_mult, y_mult))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d98eb24",
   "metadata": {},
   "outputs": [],
   "source": [
    "h = model_lasso.get_training_history()\n",
    "print(f\"Final loss: {h.losses[-1]:.4f}  |  Converged: {h.converged}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ee45963",
   "metadata": {},
   "source": [
    "### 5.3  Visualizations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "84dc6e80",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lasso.plot_fit(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eeb9f6fe",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lasso.plot_loss_curve()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "90dd903a",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lasso.plot_coefficient_shrinkage(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28ba1b7e",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lasso.plot_feature_elimination(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ce19b8ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lasso.plot_sparsity_path(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "77cc8cc3",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lasso.plot_actual_vs_predicted(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "576bf60b",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lasso.plot_learning_curve(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cde6bf16",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lasso.compare_with_linear(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13e09a43",
   "metadata": {},
   "outputs": [],
   "source": [
    "# plot_constraint_region requires exactly 2 features — use X_2f\n",
    "model_lasso_2f = LassoRegression(alpha=0.5, epochs=500, normalize=True)\n",
    "model_lasso_2f.fit(X_2f, y_2f)\n",
    "model_lasso_2f.plot_constraint_region(X_2f, y_2f)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07525aa2",
   "metadata": {},
   "source": [
    "### 5.4  Animation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32be9153",
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(model_lasso.animate_coordinate_descent(X_mult, y_mult, display=False).to_jshtml())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "993640ff",
   "metadata": {},
   "source": [
    "---\n",
    "## 6  ElasticNet Regression  (L1 + L2)\n",
    "\n",
    "**Concept:** Blends Lasso and Ridge with `l1_ratio`. `l1_ratio=1` → pure Lasso, `l1_ratio=0` → pure Ridge.\n",
    "\n",
    "**All functions covered:**\n",
    "`fit` · `predict` · `score` · `get_training_history` · `explain_regularization` · `explain_coefficients` · `summary` · `plot_fit` · `plot_loss_curve` · `plot_coefficient_shrinkage` · `plot_l1_l2_balance` · `compare_regularization_methods` · `plot_actual_vs_predicted` · `plot_learning_curve` · `animate_l1_ratio_sweep`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ea01a5d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_elastic = ElasticNetRegression(\n",
    "    alpha=0.5, l1_ratio=0.5, epochs=400, normalize=True, store_history=True\n",
    ")\n",
    "model_elastic.fit(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ffc8096",
   "metadata": {},
   "source": [
    "### 6.1  Textual Explanations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8dd23238",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_elastic.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6704d28e",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_elastic.explain_regularization()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5cd9d092",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_elastic.explain_coefficients()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3eb913c4",
   "metadata": {},
   "source": [
    "### 6.2  Prediction & Scoring"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f14f48c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Predictions (first 5):\", model_elastic.predict(X_mult[:5]))\n",
    "print(\"Metrics:\", model_elastic.score(X_mult, y_mult))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "58857bd1",
   "metadata": {},
   "outputs": [],
   "source": [
    "h = model_elastic.get_training_history()\n",
    "print(f\"Final loss: {h.losses[-1]:.4f}  |  Converged: {h.converged}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e9a65e5",
   "metadata": {},
   "source": [
    "### 6.3  Visualizations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4fcae88e",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_elastic.plot_fit(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed3259f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_elastic.plot_loss_curve()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee5d1a84",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_elastic.plot_coefficient_shrinkage(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "053ef9ea",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_elastic.plot_l1_l2_balance(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3aa79757",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_elastic.compare_regularization_methods(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9805d29c",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_elastic.plot_actual_vs_predicted(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65a4b43d",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_elastic.plot_learning_curve(X_mult, y_mult)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "096a1186",
   "metadata": {},
   "source": [
    "### 6.4  Animation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "875e4421",
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(model_elastic.animate_l1_ratio_sweep(X_mult, y_mult, display=False).to_jshtml())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d87c3e2",
   "metadata": {},
   "source": [
    "---\n",
    "## 7  Logistic Regression  (binary classification)\n",
    "\n",
    "**Concept:** Passes a linear score through the sigmoid function to produce class probabilities. Trains with binary cross-entropy loss.\n",
    "\n",
    "**All functions covered:**\n",
    "`fit` · `predict` · `predict_proba` · `score` · `get_training_history` · `explain_boundary` · `explain_coefficients` · `explain_sigmoid` · `summary` · `plot_decision_boundary` · `plot_loss_curve` · `plot_probability_surface` · `plot_sigmoid` · `plot_confusion_matrix` · `plot_roc_curve` · `plot_threshold_sensitivity` · `plot_log_odds` · `plot_calibration_curve` · `plot_actual_vs_predicted` · `plot_learning_curve` · `animate_boundary`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33d476e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log = LogisticRegression(\n",
    "    learning_rate=0.05, epochs=1500, normalize=True, store_history=True\n",
    ")\n",
    "model_log.fit(X_log, y_log)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a281b245",
   "metadata": {},
   "source": [
    "### 7.1  Textual Explanations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1eef75ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "811e023e",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.explain_sigmoid()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3be8843e",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.explain_boundary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b08421b",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.explain_coefficients()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3db1e34e",
   "metadata": {},
   "source": [
    "### 7.2  Prediction & Scoring"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0bb69787",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Class predictions (first 10):\", model_log.predict(X_log[:10]).astype(int))\n",
    "print(\"Probabilities     (first 10):\", model_log.predict_proba(X_log[:10]).round(3))\n",
    "print()\n",
    "print(\"Score metrics:\", model_log.score(X_log, y_log))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67ad4d73",
   "metadata": {},
   "outputs": [],
   "source": [
    "h = model_log.get_training_history()\n",
    "print(f\"Final loss: {h.losses[-1]:.4f}  |  Converged: {h.converged}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9f34caa",
   "metadata": {},
   "source": [
    "### 7.3  Visualizations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd24bbbc",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.plot_decision_boundary(X_log, y_log)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "86fc5ef7",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.plot_probability_surface(X_log, y_log)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95a4f6b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.plot_sigmoid()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "56536205",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.plot_log_odds(X_log, y_log)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a89f011f",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.plot_loss_curve()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c8fdb515",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.plot_confusion_matrix(X_log, y_log)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "af00b1a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.plot_roc_curve(X_log, y_log)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1b8b77a",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.plot_threshold_sensitivity(X_log, y_log)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8aa69900",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.plot_calibration_curve(X_log, y_log)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf3cf7ab",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.plot_actual_vs_predicted(X_log, y_log)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b20a87d",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_log.plot_learning_curve(X_log, y_log)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c4d940a",
   "metadata": {},
   "source": [
    "### 7.4  Animation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b2271ed1",
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(model_log.animate_boundary(X_log, y_log, display=False).to_jshtml())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "627c91bf",
   "metadata": {},
   "source": [
    "---\n",
    "## All done! 🎉\n",
    "\n",
    "Every function across all 7 LineSight models has been executed in this notebook.\n",
    "\n",
    "| Model | Functions called |\n",
    "|---|---|\n",
    "| LinearRegression | 20 |\n",
    "| MultipleLinearRegression | 17 |\n",
    "| PolynomialRegression | 13 |\n",
    "| RidgeRegression | 17 |\n",
    "| LassoRegression | 18 |\n",
    "| ElasticNetRegression | 14 |\n",
    "| LogisticRegression | 21 |\n",
    "| **Total** | **120** |\n"
   ]
  }
 ],
 "metadata": {},
 "nbformat": 4,
 "nbformat_minor": 5
}