{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "285e6d5e",
   "metadata": {},
   "source": [
    "# Ch 04 — Training Models (notebook)\n",
    "\n",
    "`[← 03 classification]` · **this notebook** · `[05 svms-trees-kernels →]`\n",
    "\n",
    "Runs top-to-bottom in ~3 min on free Colab CPU. Last verified 2026-06-11.\n",
    "\n",
    "**What you'll build**\n",
    "- The normal equation solved four independent ways on a `y = 4 + 3x` problem whose answer you already know, and a machine check that all four agree.\n",
    "- Batch gradient descent and stochastic gradient descent from scratch, each verified to recover that same closed-form answer.\n",
    "- A learning rate you blow up on purpose and then repair by reading the loss curve.\n",
    "- A six-optimizer toolkit (GD, momentum, Nesterov, RMSProp, Adam, AdamW) tested against `torch.optim` on a fixed problem.\n",
    "- Logistic and softmax regression from scratch, with their predictions checked against scikit-learn's on iris.\n",
    "\n",
    "**How this notebook works.** Code cells with a `# TODO` are yours to fill in. Run the cell to grade yourself: `[ ok ]` passed, `[FAIL]` shows what went wrong, `[ -- ]` means not attempted yet. Every exercise has a hint ladder (open only as many as you need) and a folded solution below it. The notebook runs top-to-bottom even if you fill in nothing, because the solution cells redefine the functions the later cells need. See Ch 00 for the full protocol.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29c145b2",
   "metadata": {},
   "source": [
    "## Before you start\n",
    "\n",
    "1. The normal equation $\\hat{\\boldsymbol\\theta} = (\\mathbf{X}^T\\mathbf{X})^{-1}\\mathbf{X}^T\\mathbf{y}$ gives the exact least-squares answer in one line. Why does almost nobody use it to train large models? <details><summary>Answer</summary>It inverts a $(d+1)\\times(d+1)$ matrix, which costs about $O(d^3)$. For $d$ in the millions that is hopeless, and the loss is only quadratic for plain squared error anyway. Gradient descent costs $O(md)$ per step and works for any differentiable loss, so it is what scales.</details>\n",
    "2. You run gradient descent and the loss decreases for a few steps, then shoots up to a huge number. What is the single most likely cause? <details><summary>Answer</summary>The learning rate is too high. Each step overshoots the minimum and lands somewhere with a larger gradient, so the next step overshoots further. Dividing the learning rate by 10 is the first thing to try.</details>\n",
    "3. Predict before you run: you fit `y = 4 + 3x + noise` with 100 points. Roughly what intercept and slope should any correct method recover? <details><summary>Answer</summary>Intercept near 4, slope near 3, both off by a bit because of the noise. With this notebook's seed they come out near 3.99 and 2.93. Every method in Part 1 must agree on those two numbers.</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f58fa14",
   "metadata": {},
   "source": [
    "## Setup\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "3466dc96",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:08.623805Z",
     "iopub.status.busy": "2026-06-10T18:46:08.623701Z",
     "iopub.status.idle": "2026-06-10T18:46:09.920023Z",
     "shell.execute_reply": "2026-06-10T18:46:09.919526Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "numpy 2.2.6 · sklearn 1.7.2 · torch 2.12.0+cpu\n",
      "device cpu (this notebook is CPU-only; any CUDA is unused)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import sklearn\n",
    "import torch\n",
    "print(f\"numpy {np.__version__} · sklearn {sklearn.__version__} · torch {torch.__version__}\")\n",
    "if np.__version__ < \"2.0\":\n",
    "    print(\"WARN: written for NumPy 2.x; older versions may differ in the last digit\")\n",
    "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
    "print(f\"device {device} (this notebook is CPU-only; any CUDA is unused)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a5bd73eb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:09.921392Z",
     "iopub.status.busy": "2026-06-10T18:46:09.921237Z",
     "iopub.status.idle": "2026-06-10T18:46:09.926195Z",
     "shell.execute_reply": "2026-06-10T18:46:09.925848Z"
    }
   },
   "outputs": [],
   "source": [
    "import os, random\n",
    "SEED = 0\n",
    "FAST = bool(os.environ.get('NB_FAST'))  # CI smoke mode: ~10x fewer steps, same code paths\n",
    "\n",
    "# Step budgets. Full run documents the converged numbers; FAST keeps every code path.\n",
    "GD_STEPS = 500 if FAST else 5000      # batch GD epochs on the y=4+3x problem\n",
    "SGD_EPOCHS = 5 if FAST else 50        # passes over the data for stochastic GD\n",
    "SOFTMAX_STEPS = 600 if FAST else 5000 # softmax-regression GD steps on iris\n",
    "OPT_STEPS = 80 if FAST else 500       # optimizer-toolkit training steps\n",
    "\n",
    "rng = np.random.default_rng(SEED)\n",
    "torch.manual_seed(SEED)\n",
    "random.seed(SEED)\n",
    "\n",
    "# ── house self-check harness (identical across all chapter notebooks) ──\n",
    "import numpy as _np\n",
    "\n",
    "def check(label, test_fn, required=False):\n",
    "    \"\"\"Run one self-check. test_fn raises AssertionError (with a teaching\n",
    "    message) on failure, NotImplementedError if the stub is unfilled.\n",
    "    required=True is used only in solution cells; it is what CI grades.\"\"\"\n",
    "    try:\n",
    "        test_fn()\n",
    "    except NotImplementedError:\n",
    "        if required:\n",
    "            raise AssertionError(f\"{label}: reference solution incomplete\")\n",
    "        print(f\"[ -- ] {label}: not attempted yet — fill in the TODO above, then re-run.\")\n",
    "        return False\n",
    "    except AssertionError as e:\n",
    "        if required:\n",
    "            raise\n",
    "        print(f\"[FAIL] {label}: {e}\")\n",
    "        return False\n",
    "    print(f\"[ ok ] {label}\")\n",
    "    return True\n",
    "\n",
    "def attempted(*vals):\n",
    "    \"\"\"Treat None placeholders as 'not attempted'.\"\"\"\n",
    "    if any(v is None for v in vals):\n",
    "        raise NotImplementedError\n",
    "\n",
    "def check_shape(x, want):\n",
    "    assert tuple(x.shape) == tuple(want), \\\n",
    "        f\"shape {tuple(x.shape)}, expected {tuple(want)} — check your reshape/transpose order\"\n",
    "\n",
    "def check_close(got, want, atol=1e-5, rtol=1e-4, msg=\"\"):\n",
    "    g, w = _np.asarray(got, dtype=float), _np.asarray(want, dtype=float)\n",
    "    assert g.shape == w.shape, f\"shape {g.shape} vs expected {w.shape}. {msg}\"\n",
    "    bad = ~_np.isclose(g, w, atol=atol, rtol=rtol)\n",
    "    assert not bad.any(), \\\n",
    "        f\"{bad.mean():.2%} of values wrong (max diff {abs(g - w).max():.3g}). {msg}\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2eee7a6",
   "metadata": {},
   "source": [
    "> **Note:** seeds make this notebook's printed numbers reproduce on CPU. Library versions and BLAS threading can shift the last digit or two; quoted numbers hold for the pinned environment. If your slope is 2.935 and the page says 2.934, you did nothing wrong. The `FAST` flag (set by the `NB_FAST` environment variable) cuts every step count by roughly 10x for continuous-integration smoke runs without changing a single line of the algorithms; the experiment log near the end records the expected loss for both settings.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67e4d251",
   "metadata": {},
   "source": [
    "## The map\n",
    "\n",
    "> **Part 1 — The normal equation, four ways.** Build the `y = 4 + 3x` problem, solve it with `inv`, `lstsq`, `pinv`, and scikit-learn, and machine-check that all four recover the same `(intercept, slope)`.\n",
    "> **Part 2 — Batch gradient descent from scratch.** Derive the MSE gradient, iterate, and assert the result converges to the Part 1 closed form.\n",
    "> **Part 3 — The learning rate is the whole game.** Sweep it, blow it up on purpose, watch the loss go to infinity, then read the curve and fix it.\n",
    "> **Part 4 — Stochastic and mini-batch GD.** Trade smooth-but-slow for noisy-but-cheap; recover the same answer with one example at a time.\n",
    "> **Part 5 — Momentum, Adam, and an optimizer toolkit.** Build six update rules and test them against `torch.optim` on a fixed problem.\n",
    "> **Part 6 — Logistic and softmax regression from scratch.** Same linear core, different loss; check the predictions against scikit-learn on iris.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8788a4f8",
   "metadata": {},
   "source": [
    "## Part 1 — The normal equation, four ways\n",
    "\n",
    "> **Objectives.** Generate a regression problem whose true parameters you know, solve it in closed form four independent ways, and assert they agree. This is the ground-truth oracle the rest of the notebook checks against.\n",
    "\n",
    "A linear regression model predicts a target as a weighted sum of features plus a bias:\n",
    "\n",
    "$$\\hat{y} = \\theta_0 + \\theta_1 x_1 + \\ldots + \\theta_d x_d = \\boldsymbol\\theta^T \\mathbf{x},$$\n",
    "\n",
    "where we prepend a constant $x_0 = 1$ so the bias $\\theta_0$ lives inside $\\boldsymbol\\theta$. Stack $m$ examples into the design matrix $\\mathbf{X} \\in \\mathbb{R}^{m \\times (d+1)}$ and targets $\\mathbf{y}$; the mean-squared-error loss is $\\text{MSE}(\\boldsymbol\\theta) = \\frac{1}{m}\\lVert\\mathbf{X}\\boldsymbol\\theta - \\mathbf{y}\\rVert_2^2$. Setting its gradient to zero gives the **normal equation**:\n",
    "\n",
    "$$\\hat{\\boldsymbol\\theta} = (\\mathbf{X}^T\\mathbf{X})^{-1}\\mathbf{X}^T\\mathbf{y}.$$\n",
    "\n",
    "We start with the simplest possible instance: one feature, a known intercept of 4 and slope of 3, plus Gaussian noise.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "fb08607f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:09.927302Z",
     "iopub.status.busy": "2026-06-10T18:46:09.927225Z",
     "iopub.status.idle": "2026-06-10T18:46:09.929599Z",
     "shell.execute_reply": "2026-06-10T18:46:09.929307Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_b shape (100, 2), y shape (100, 1)\n"
     ]
    }
   ],
   "source": [
    "m = 100\n",
    "X = 2 * rng.random((m, 1))                 # one feature in [0, 2)\n",
    "y = 4 + 3 * X + rng.standard_normal((m, 1))  # true intercept 4, true slope 3, plus noise\n",
    "X_b = np.c_[np.ones((m, 1)), X]            # prepend the bias column -> shape (m, 2)\n",
    "print(f\"X_b shape {X_b.shape}, y shape {y.shape}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "fc8fb3a9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:09.930560Z",
     "iopub.status.busy": "2026-06-10T18:46:09.930492Z",
     "iopub.status.idle": "2026-06-10T18:46:10.008380Z",
     "shell.execute_reply": "2026-06-10T18:46:10.007874Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAGGCAYAAACNCg6xAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAVYJJREFUeJzt3Xl4U2X6PvA7CW1a2nSlK22h7FA2ZZPNghRaRAQVXEYUBB3GwVF0RMFR2UbB5aeo4xeXQRYVmZFFHJCyt4Ds+76VFlpKaYAu6ZaW5Pz+ODaQpkvaJjknyf25Lq+ZnHOSvKcp5OZ9nvMehSAIAoiIiIioTkqpB0BERETkLBiciIiIiKzE4ERERERkJQYnIiIiIisxOBERERFZicGJiIiIyEoMTkRERERWYnAiIiIishKDExEREZGVGJyIiFzQoEGDMGjQIKmHQeRyGJyIyOZ27doFhUIBhUKBGzduOPS9s7OzMW7cOLRv3x4ajQYBAQHo3bs3li5dCt5hiogaq4nUAyAi12I0GvG3v/0NPj4+KC4udvj737hxA1lZWRgzZgxiYmJQUVGBzZs3Y8KECTh37hzef/99h49JCps2bZJ6CEQuScGb/BKRLX311Vd4++23MW7cOHz22WfQarVo1qxZvV9n0KBBaNmyJZYsWWKTcY0cORLbt29HQUEBVCqVTV6zqpYtW2LChAmYNWuWXV6fiKTHUh2Rk9q+fTsUCgXWrFljsW/58uVQKBTYs2ePQ8d069YtvP3225gzZw4CAgIc+t51admyJUpKSlBeXg4AOHPmDLy9vfHss8+aHbdr1y6oVCq8+eabDhvbrFmzoFAocPHiRUyYMAEBAQHw9/fHc889h5KSErNjb9++jblz56J169ZQq9Vo2bIl3nrrLej1erPjqutx+uKLLxAXF4emTZsiMDAQPXv2xPLly82OuXr1KiZOnIiwsDCo1WrExcXhu+++s8t5EzkjBiciJzVo0CBER0fjxx9/tNj3448/onXr1ujbt2+Nzzcajbhx44ZV/1VUVFg1pnfeeQfh4eGYPHlyg8/LVkpLS3Hjxg1kZGRg6dKlWLx4Mfr27Qtvb28AQMeOHTF37lx8//33+PXXXwEAxcXFmDBhAjp06IA5c+Y4fMyPP/44dDod5s2bh8cffxxLlizB7NmzzY55/vnn8e677+Lee+/Fp59+ivj4eMybNw9PPvlkra/97bff4uWXX0anTp2wYMECzJ49G927d8e+fftMx1y/fh333XcftmzZgpdeegmfffYZ2rRpg0mTJmHBggX2OGUi5yMQkdOaMWOGoFarhfz8fNO23NxcoUmTJsLMmTNrfW56eroAwKr/tm/fXudYjh07JqhUKmHjxo2CIAjCzJkzBQCCVqtt0LnFx8cL48ePb9BzBUEQ5s2bZ3YOQ4YMEa5cuWJ2jMFgEAYMGCCEhYUJN27cEKZMmSI0adJEOHDgQIPes0WLFnX+3KtT+bOaOHGi2fZHHnlECA4ONj0+evSoAEB4/vnnzY57/fXXBQDCtm3bTNvi4+OF+Ph40+NRo0YJcXFxtY5j0qRJQkREhHDjxg2z7U8++aTg7+8vlJSU1PfUiFwOm8OJnNizzz6LefPmYeXKlZg0aRIA4D//+Q9u376NcePG1frc8PBwbN682ar36datW53HvPzyyxg+fDiGDRtm1WveraKiAgUFBRbb9Hq9xVV5QUFBUCrrnix/6qmn0LNnT2i1Wqxbtw7Xr19HaWmp2TFKpRJLlixBt27dMHz4cBw8eBBvv/02evbsWefr6/V66HQ6s21GoxElJSUWY7a2x+svf/mL2eOBAwdizZo1KCwshJ+fH3777TcAwGuvvWZ23N///nd8/PHHWL9+PQYPHlztawcEBCArKwsHDhxAr169LPYLgoBVq1bh8ccfhyAIZueQmJiIFStW4PDhw+jfv79V50LksqRObkTUOL169RIGDx5senzfffcJ9913n0PHsGLFCsHDw0M4d+6caVt9Zpy2b99u9exXenp6g8b4wgsvCNHR0dXOmnz00UcCAKFz585CeXm5Va+3ePFiq8dcl8qfVU5OTrXvkZGRIQiCIEyePFlQKpXVjjEgIEAYM2aM6XHVGafTp08LzZs3FwAIbdq0Ef76178Ku3btMu2/fv16neexevVqq342RK6MM05ETu7ZZ5/FK6+8gqysLOj1euzduxf/+te/6nyewWCAVqu16j2CgoLg6elZ4/5p06Zh7Nix8PT0REZGBgAgPz8fAJCZmYny8nJERkbW+Pxu3bpZzH79/e9/R3h4OKZNm2a2PTw83KoxVzVmzBh8++232LFjBxITE832VV66n52djZs3b1r1HomJiRZjHjduHIYNG2bRcG6tmq72E6pc/KxQKOr92h07dsS5c+ewbt06JCcnY9WqVfi///s/vPvuu5g9ezaMRiMA8RzGjx9f7Wt07dq13u9L5GoYnIic3JNPPonXXnsNP/30E0pLS+Hh4YEnnniizudlZmYiNjbWqvfYvn17ratQZ2ZmYvny5RZXaAHAvffei27duuHo0aM1Pj8wMBAJCQkW2yIiIiy2N1Rlma5qSfCrr77C5s2b8d5772HevHmYPHky1q5dW+frRUREICIiwmybl5cXWrVqZbMxV9WiRQsYjUZcuHABHTt2NG2/fv068vPz0aJFi1qf7+PjgyeeeAJPPPEEysvL8eijj+K9997DjBkzEBISAo1GA4PBYLfxE7kCBiciJ9esWTMMHz4cP/zwA8rKypCUlGRVT40te5yqWxJhxYoV+M9//oNly5YhKirKqvexBa1Wi5CQEIvtixYtgkKhwL333mvalp6ejmnTpuGxxx7DW2+9heDgYPzlL3/BsmXLGjxrZE8PPvgg3nrrLSxYsABff/21afsnn3wCABgxYkSNz7158yaCg4NNjz09PdGpUyds2LABFRUV8PLywmOPPYbly5fj5MmT6Ny5s9nza/q5ErkbBiciF/Dss89izJgxAIC5c+da9RwvLy+bzSyMHj3aYlvlDNPw4cMbtABmQ7333nv4/fffkZSUhJiYGNy6dQurVq3CgQMH8Le//Q1t2rQBIJa/Jk6cCG9vbyxcuBAAMHnyZKxatQqvvPIKEhISai0vSqFbt24YP348vvnmG+Tn5yM+Ph779+/H0qVLMXr06BobwwFg2LBhCA8PR//+/REWFoYzZ87gX//6F0aMGAGNRgMAmD9/PrZv344+ffrghRdeQKdOnXDr1i0cPnwYW7Zswa1btxx1qkSyxeBE5AJGjhyJwMBAGI1GPPzww1IPR1IjRoxAWloavvvuO2i1Wnh5eaFr165YvHixWe/OF198gZSUFKxatcpsJmXRokXo3LkzXnjhBaxfv16KU6jVv//9b7Rq1QpLlizBmjVrEB4ejhkzZmDmzJm1Pm/y5Mn48ccf8cknn6CoqAhRUVF4+eWX8fbbb5uOCQsLw/79+zFnzhysXr0a//d//4fg4GDExcXhgw8+sPepETkF3nKFyAXcvn0bkZGRGDlyJBYtWiT1cIiIXBZXDidyAb/88gu0Wq0s+3KIiFwJZ5yInNi+fftw/PhxzJ07F82aNcPhw4elHhIRkUvjjBORE1u4cCFefPFFhIaGYtmyZVIPh4jI5XHGiYiIiMhKnHEiIiIishKDExEREZGVXH4dJ6PRiOzsbGg0mgbd34mIiIhcmyAI0Ol0iIyMhFJZ+5ySywen7OxsREdHSz0MIiIikrnMzMw6bxHl8sGp8lYCmZmZ8PPzk3g0REREJDeFhYWIjo42ZYbauHxwqizP+fn5MTgRERFRjaxp6WFzOBEREZGVGJyIiIiIrMTgRERERGQll+9xspbBYEBFRYXUwyA34+HhAZVKJfUwiIjISm4fnARBQE5ODvLz86UeCrmpgIAAhIeHc50xIiIn4PbBqTI0hYaGomnTpvzyIocRBAElJSXIzc0FAEREREg8IiIiqotbByeDwWAKTcHBwVIPh9yQt7c3ACA3NxehoaEs2xERyZxbN4dX9jQ1bdpU4pGQO6v8/WOPHRGR/Ln1jFMlludISvz9IyJ3odXpkXo+F1l5pYgK9EZ8u1CEaNRSD6teGJyIiIjI7rQ6PT5MPos0bRE8VEpUGIzYd+kW3kjq4FThya1Lda5m0KBBmDp1qtTDICIispB6Phdp2iK0CdWgVYgv2oRqkKYtQur5XKmHVi8MTm4qJSUFCoWCyzAQEZFDZOWVwkOlhEoptieolAp4qJTIyiuVeGT1w+BEREREdhcV6I0KgxEGowAAMBgFVBiMiAr0lnhk9cPg5KSKi4vx7LPPwtfXFxEREfh//+//me3//vvv0bNnT2g0GoSHh+NPf/qTab2gjIwMDB48GAAQGBgIhUKBCRMmAACSk5MxYMAABAQEIDg4GA899BDS0tIcem5EROR64tuFonWILy7m6nBJW4SLuTq0DvFFfLtQqYdWLwxONqDV6bHyUCYWbDmPlYcyodXp7f6e06ZNQ2pqKtauXYtNmzYhJSUFhw8fNu2vqKjA3LlzcezYMfzyyy/IyMgwhaPo6GisWrUKAHDu3Dlcu3YNn332GQAxkL322ms4ePAgtm7dCqVSiUceeQRGo9Hu50RERK4rRKPGG0kd8Kc+MejbOhh/6hPjdI3hAK+qazQprhIoKirCokWL8MMPP2DIkCEAgKVLlyIqKsp0zMSJE03/v1WrVvj888/Rq1cvFBUVwdfXF0FBQQCA0NBQBAQEmI597LHHzN7ru+++Q0hICE6fPo3OnTvb5XyIiMg9hGjUGNMjWuphNApnnBpJiqsE0tLSUF5ejj59+pi2BQUFoX379qbHhw4dwsiRIxETEwONRoP4+HgAwJUrV2p97QsXLuCpp55Cq1at4Ofnh5YtW1r1PCIiInfA4NRIcrxKoLi4GImJifDz88OPP/6IAwcOYM2aNQCA8vLyWp87cuRI3Lp1C99++y327duHffv2WfU8IiIid8Dg1EhSXCXQunVreHh4mEINAOTl5eH8+fMAgLNnz+LmzZuYP38+Bg4ciA4dOpgawyt5enqK4zUYTNtu3ryJc+fO4e2338aQIUPQsWNH5OXl2e08iIiInA17nBopvl0o9l26hYu5OlOPk72vEvD19cWkSZMwbdo0BAcHIzQ0FP/4xz+gVIo5OCYmBp6envjiiy/wl7/8BSdPnsTcuXPNXqNFixZQKBRYt24dHnzwQXh7eyMwMBDBwcH45ptvEBERgStXrmD69Ol2Ow8iIiJnwxmnRpLqKoGPPvoIAwcOxMiRI5GQkIABAwagR48e4phCQrBkyRL8/PPP6NSpE+bPn4+PP/7Y7PnNmzfH7NmzMX36dISFheGll16CUqnEihUrcOjQIXTu3BmvvvoqPvroI7ueBxERkTNRCIIgSD0IeyosLIS/vz8KCgrg5+dntq+srAzp6emIjY2Fl5eXRCMkd8ffQyIiadWWFapiqY6IiIhqpdXpkXo+F1l5pYgK9EZ8u1CnW3/JVhiciIiIqEb2XK/QGQMZgxMRERHV6O71ClVKBQxGARdzdUg9n9uoxSylWEDaFtgcTkRERDWy13qFUiwgbQsMTkRERFQje61XKMcFpK3BUh0RERHVyF7rFd4dyCpLgPZeQNoWGJyIiIioRpXrFdq6iVuKBaRtgcGJiIiIahWiUTeqEbym17RHILM3BiciIiKShD0Cmb2xOZyIiIjISgxOTmjQoEGYOnWq1MOQlRUrVkChUGD06NF2fZ9du3ahf//+CA4Ohre3Nzp06IBPP/3Uru9JRETywVKdixIEAQaDAU2auP5HnJGRgddffx0DBw6s93NnzZqFjIwMLFmyxKrjfXx88NJLL6Fr167w8fHBrl27MHnyZPj4+ODPf/5zvd+fiIicC2ecnMyECROQmpqKzz77DAqFAgqFAhkZGUhJSYFCocCGDRvQo0cPqNVq7Nq1CxMmTLCYhZk6dSoGDRpkemw0GjFv3jzExsbC29sb3bp1w8qVK60ekyAIaNOmDT7++GOz7UePHoVCocDFixcbc8q1MhgMePrppzF79my0atXKbu9T6Z577sFTTz2FuLg4tGzZEuPGjUNiYiJ27twJANBqtQgPD8f7779ves7u3bvh6emJrVu32n18RERkXwxOTuazzz5D37598cILL+DatWu4du0aoqPvNNZNnz4d8+fPx5kzZ9C1a1erXnPevHlYtmwZvvrqK5w6dQqvvvoqxo0bh9TUVNMxLVu2xKxZs6p9vkKhwMSJE7F48WKz7YsXL8b999+PNm3aVPu8H3/8Eb6+vrX+VxlIajJnzhyEhoZi0qRJVp2rrR05cgS7d+9GfHw8ACAkJATfffcdZs2ahYMHD0Kn0+GZZ57BSy+9hCFDhkgyRiIish3Xr+PU00OTAe0tx79vSBCw7uu6j/P394enpyeaNm2K8PBwi/1z5szB0KFDrX5fvV6P999/H1u2bEHfvn0BAK1atcKuXbvw9ddfmwJB69at0axZsxpfZ8KECXj33Xexf/9+9O7dGxUVFVi+fLnFLNTdHn74YfTp06fW8TVv3rzGfbt27cKiRYtw9OjRWl/DHqKioqDVanH79m3MmjULzz//vGnfgw8+iBdeeAFPP/00evbsCR8fH8ybN8/hYyQiIttjcKpCewvIuSH1KBquZ8+e9Tr+4sWLKCkpsQhb5eXluOeee0yP6yozRUZGYsSIEfjuu+/Qu3dv/O9//4Ner8fYsWNrfI5Go4FGo6nXeCtVzuR8++23tQa6qnbu3Inhw4ebHpeXl0MQBLPS5Ndff42nn366ztcpKirC3r17MX36dLRp0wZPPfWUaf/HH3+Mzp074+eff8ahQ4egVst7XRIiIrIOg1MVIUHO/b4+Pj5mj5VKJQRBMNtWUVFh+v9FRUUAgPXr11vM7tT3y/7555/HM888g08//RSLFy/GE088gaZNm9Z4/I8//ojJkyfX+pobNmyotuk7LS0NGRkZGDlypGmb0WgEADRp0gTnzp1D69atLZ7Xs2dPsxmqzz//HFevXsUHH3xg2hYWFlbrmAAgNjYWANClSxdcv34ds2bNMgtOaWlpyM7OhtFoREZGBrp06VLnaxIRyYlWp3e6xSkdgcGpCmvKZVLz9PSEwWCw6tiQkBCcPHnSbNvRo0fh4eEBAOjUqRPUajWuXLliKss11IMPPggfHx8sXLgQycnJ2LFjR63HN6ZU16FDB5w4ccJs29tvvw2dTofPPvvMrO/rbt7e3mY9V0FBQSgsLKyxD8saRqMRer3e9Li8vBzjxo3DE088gfbt2+P555/HiRMnEBoq79sIEBFV0ur0+DD5LNK0Rabboey7dAtvJHVw+/DE4OSEWrZsiX379iEjIwO+vr4ICqp5uuqBBx7ARx99hGXLlqFv37744YcfcPLkSVMZTqPR4PXXX8err74Ko9GIAQMGoKCgAL///jv8/Pwwfvx4AMCQIUPwyCOP4KWXXqrxvVQqFSZMmIAZM2agbdu2pp6pmjSmVOfl5YXOnTubbQsICAAAi+229OWXXyImJgYdOnQAAOzYsQMff/wxXn75ZdMx//jHP1BQUIDPP/8cvr6++O233zBx4kSsW7fObuMiIrKl1PO5SNMWoU2oxnQD3ou5OqSez3W6lb5tjVfVOaHXX38dKpUKnTp1QkhICK5cuVLjsYmJiXjnnXfwxhtvoFevXtDpdHj22WfNjpk7dy7eeecdzJs3Dx07dkRSUhLWr19vKkcBYunpxo26m78mTZqE8vJyPPfccw0/QRkzGo2YMWMGunfvjp49e+LLL7/EBx98gDlz5gAAUlJSsGDBAnz//ffw8/ODUqnE999/j507d2LhwoUSj56I3IFWp8fKQ5lYsOU8Vh7KhFanr/tJVWTllcJDpYRKqQAAqJQKeKiUyMortfVwnY5CqNoA42IKCwvh7++PgoIC+Pn5me0rKytDeno6YmNj4eXlJdEIXcvOnTsxZMgQZGZmWtUrRPw9JCLbqa7E1jrEt94ltpWHMrF83xWLGac/9YlxyRmn2rJCVSzVkU3o9XpotVrMmjULY8eOZWgiIpKArUps8e1Cse/SLVzM1ZkFsPh27NVkcCKb+OmnnzBp0iR0794dy5Ytk3o4RERuyVYlthCNGm8kdeBVddVgcCKbmDBhAiZMmCD1MIiI3FpUoDcqDEYYjIJpxqnCYERUoHe9XytEozbNUnFpgjsYnIiIiFyEPUpsXJrAHIMTERGRi7BHiY1LE5hjcMKdFaeJpMDfPyKypbtLbLbApQnMuXVw8vT0hFKpRHZ2NkJCQuDp6QmFQiH1sMhNCIKA8vJyaLVaKJVKeHp6Sj0kIiIL/t5NcK2gFLqy29B4NUGYn1eD+6ZcgVsHJ6VSidjYWFy7dg3Z2dlSD4fcVNOmTRETEwOlkuvREpG8aHV6HL6cj2K9AbeKywEAF3J16N+6mdsuTSBpcNqxYwc++ugjHDp0CNeuXcOaNWswevRo035BEDBz5kx8++23yM/PR//+/bFw4UK0bdvWZmPw9PRETEwMbt++bfX934hsRaVSoUmTJpzpJCJZSj2fi6v5pejfphlydWUoLKtAsd6Ae1sEumVjOCBxcCouLka3bt0wceJEPProoxb7P/zwQ3z++edYunQpYmNj8c477yAxMRGnT5+26QrLCoUCHh4ephvfEhER0Z3+Jh91E8SqfQEAl7RFKCitkHhk0pE0OA0fPhzDhw+vdp8gCFiwYAHefvttjBo1CgCwbNkyhIWF4ZdffsGTTz7pyKESERG5HVuuC9VQggAcPwes3AhMfAyIjXLYW1dLtj1O6enpyMnJQUJCgmmbv78/+vTpgz179tQYnPR6PfT6Ozc0LCwstPtYiYiIXJGUt165fhNYsxlYmQxcuCxu8/MFpk2y+1vXSrbBKScnBwAs7nkWFhZm2ledefPmYfbs2XYdGxERkaurXC08oKkH2ob5ws/LA+3DNXZdNbysHNjyO/DzRmDHAaDqai0bUoHXJwJStoXKNjg11IwZM/Daa6+ZHhcWFiI62v0W6CIiIucn1a1OqlstvHKmydbvLwjAsbPAz8nAr9uAwiLLY3p1AcYkAiMGSRuaABkHp/DwcADA9evXERERYdp+/fp1dO/evcbnqdVqqNXu2elPRESuQ8pbnThitfAcLbBmixiY0q5Y7g8PEdAuLh/G4Ez4+OlRGhCMMmNzaCDtd7xsg1NsbCzCw8OxdetWU1AqLCzEvn378OKLL0o7OCIiIjuT8lYn9lotvEwPbPpdbPTeedCyFOftBQwfCAy9vxy/XT6JPZduQq81AFrg4OU8HL6cj3dHxkm6FIKkwamoqAgXL140PU5PT8fRo0cRFBSEmJgYTJ06Ff/85z/Rtm1b03IEkZGRZms9ERGR65KqVCUHUt7qpKFX01X3eTXzVePIGbHJ+3/bgMJiy+f16QqMSQIejAd8mwIrD13HiewCAAJCNV4QABSWVuBYVr7k98iTNDgdPHgQgwcPNj2u7E0aP348lixZgjfeeAPFxcX485//jPz8fAwYMADJyck2XcOJiIjkScpSlRzYMrzU9+fVkKvpqn5exbomWPJzExRkheHKVcs7I0SFi31Ljw0DYiLN92XlleK2wQiVUgmFQgEFAKVSAYMRkt8jT9LgNGjQIAiCUON+hUKBOXPmYM6cOQ4cFRERyYGUpSo5sEV4aWjYDNGo8UZSh3oFsNTzubhwrQTKguY4czoABTn+AMw7uZt6ibNKY5LEWaaa7jQVFeiNJiolyioqIAgCBABGowCVByS/R55se5yIiMi9SVmqkoOGhhdbhc0Qjdqq5wgCcPAksPgHDU6f7AXjbcto0aOLEX96UInh8YCPFbknvl0oUs9psTvtJnJ1ZQAAzyZKdIsKkPweeQxOREQkS3JYtVpq1oaXSo4Mm1evA6s3A6s2AulZABBgtr+Jdyk8w68isIUWf3ooql7nEaJR492RcVh3PBt70m4AAPq2DsZDXZtLXqZlcCIiIlmSctVqZ2XvsFlaBmzYCfy03ogDxxUQBPNSnLLJbSiDc6CJyoHKPw/+3h4IaOrRoOAWolHjuf6xeK5/rE3GbisMTkREJEsNKVW5O3uEzcpS3M/JwPoUoKgEAMybk0Kb6zDlcS/oNVex/GA6fNQq+Hn5IlTjhay8EpeaJWRwIiIi2apvqcrd2TJsZuUAqzaJpbjL2Zb7ff30aNUpDzFt83Ct7BZ8m8dgRLsIXLxRgDRtEQQByMorcblZQgYnIiIiF9KYsFlSCmzYId4rbs8Ry/2+TYGWbQtgbJaJbh0F0+1PbmjFPip3mCVkcCIiInJjRiOw/7i4mvdvqUBxlXYkhQLof6+45lLSQGD9qUIs31cAo6CBSmHZR+Xqs4QMTkRELs6dV9+mml25BqzeJAamzGuW+2OjxLD06DAg8q5Km7s37TM4ERG5MHdffZvMFZeKs0ork4G9xyz3a3yAhwaJC1T2iIOpFHc3dyjH1YbBiYjIhbn76tskluL2HhObvH9LBUrKzPcrFMDAHmJYShwAeFmRf1y9HFcbBiciIhfm7qtvu7PLV+9cFZd13XJ/62gxLD0yFIgIse9YXKlczOBEROTCuPq2eykqEddaWrlRbPiuys8HGPmAGJju6Vh9Kc7WXK1czOBEROTC3L2R1x0YjcDuI+LM0oad4ured1MqgYE9gbGJwND+1pXibMnVysUMTkRELszdG3ldWcZVcWZp9SbxvnFVtWkhhqVHhgJhzRw/vkquVi5mcCIicnHu3MjranTFwLoU8aq4gyct9/v5Ag8/AIxNArp1cEwpri6uVi5mcCIiIpIxg0Esxa3cCCTvBMr05vuVSiC+lxiWhvQDvDylGWdNXK1czOBEREQkQ5cy75Tirmkt97drKTZ5j04AwoIdPjyr1VYudsar7RiciIiIZKKgCFi/XbxX3OFTlvsD/IBRQ8QVvbu0k0cpzhrVlYud9Wo7BiciIiIJGQzArkPi7NLGXYC+3Hy/SgkM6iOGpSF9AbXMSnEN5axX2zE4ERERSeDiFXEJgdWbgJwblvvbx4p9S6MTgJAgx4/P3pz1ajsGJyIiIgcp0AH/2y5eFXfkjOX+wMpSXBLQua3zlOIawlmvtmNwIiIisqPbBmDnQXF2adMuQF9hvr+JChjcRwxLD9wHeHrU/ZrO2FRdlbNebcfgREREZAfnM/4oxW0Gcm9a7u/YChg7XJxhahZo/es6a1N1Vc66OCuDExERkY3kFwK/bhMbvY+dtdzv3fQ2evYoxV8eU2NAt4Z1eTtrU3V1nHFxVgYnIiKiRrhtAFL3i2Fpy26gvEopTqUSEB5TCK+IbAQ3L0AJDFh73hftWzVshshZm6pdBYMTERHViyv019jCuXQxLK3ZDGhvWe6PayveK07RLBu/nsr4Y4bIp9EzRM7aVO0qGJyIiMhqrtJf01B5BcDabeJVcSfOW+5vFiguHzAmEejYWty2YEuxTWeInLWp2lUwOBERkdVcqb/GWhW3xVLcz8nA1j3i47t5NBEXphybBMT3Fh/fzdYzRM7aVO0qGJyIiNxcfUpv7tRfcyZNLMX9sgW4kWe5v0s7IGlQBTTNc5FXXox8tTfySy1/dvaYIXLGpmpXweBEROTG6lt6c/X+mlsFwNot4r3iTl2w3B8SCDwyTCzFBTX742d3svafHWeIXAuDExGRG6tv6c0V+2sqbgPb94qzS9v2WpbiPD2AhH5iWIrvLS5YCQArD1n/s+MMketgcCIicmP1Lb250uzJqYviApW/bAFu5lvu79ZBDEsPPwAE+Fnud6eyJd3B4ERE5MYaUnqz9+yJPZc7uJEHrN0qXhV3Os1yf2gw8OhQ4LFEoF3L2l/L1cuWVD0GJyIiNya30ps9ljsorxBLcCuTge37xAUr76b2AIb2F+8VN7DnnVJcXez1s+M6WfLG4ERE5IRs9eUqt9KbrZY7EATg5IU7pbi8Qstj7ukohqWRgwF/Tf3Hao+fnbuvk+UMGJyIiJyMrb9c5dS43Ni+Ie0tMSit3AicvWS5P6wZ8NgwsRTXJqbx47X1z84d18lyNgxOREROxpW/XBvSN6QvFxemXLkRSNkHGIzm+9WeQOIAsdF7QA9AZWUpTgpsOJc/BiciIifjyl+u1vYNCQJw8ry43tLarUB+NaW4e+PEe8WNGAz4+zroBBrJ0Q3n7KeqPwYnIiIn48pXc9XVN3T95h+luGTgfIbl8yNCgEeHieW41jYoxTmaI5v12U/VMAxOREROpq4vV2efRajaN1RWDqxPEe8Vt+NA9aW4pIHiveL63SN9Ka4xP39HNuu7csnXnhiciIicTG1frq4yiyAIwLGzYt/S2q0CCosUFsf07CxeFTciHvCTSSnOFj9/RzXru3LJ154YnIiInFBNX67OPotw/QawZrPYu3TxcuXWO6HJw1uPNp3y8d4LgejR3lOSMdbGmX7+rlzytScGJyIiF+KMswhl5cDmXWJY2nkQMFYpxSlUBsS0LkTrTnloFlmENK0O6UUx6AF5BRHAuX7+clv81FkwOBERuRBnmUUQBODIGXGByl+3AYVFlsf07gr4R+cgz+sK2kV6o7TCgCu3SnG9UI9Np3Jk2bvlLD9/wHb9VM7eU1dfDE5ERC5E7rMIOVpg9Wbxqri0TMv9UWHi4pSPDQNaNAdWHqrA8n0VKNZ74MTVAhSWVkB/24Bz14vwYfJZ2fVuyf3nX1Vj+6lcpaeuPhiciIhcSGNmEew1c1CmBzbuEsPSzkPibNPdvL2AB+8XG73v6wYolXf2VQaRfek3cau4HOomSoRqvNC5uT/StEWy6x2S2y1s7M2ZerpshcGJiMjFNGQWwdYzB4IAHDolhqV1KYCu2PKY+7qJYWn4/YBv05rP5Y2kDnhrzXGczi5EdFBTRPh7w8tDJdveITndwsbenKmny1YYnIiIyGYzB9m5wOpN4jIC6VmW+6MjxFufPDoMiImw7jVDNGokxoXjZlE5YoJ8ZN875E6cqafLVmQdnAwGA2bNmoUffvgBOTk5iIyMxIQJE/D2229DobBc04OIiBqmMTMHpWVA8k4xLP1+2LIU19QLGDFIDEy9u5qX4qzlbL1DUnJks7Y7fi6yDk4ffPABFi5ciKVLlyIuLg4HDx7Ec889B39/f7z88stSD4+IyGXUd+ZAEICDJ8XVvNenAEUllsf0vUe8V1zS/YBPIycg3K13qKEc3aztjp+LrIPT7t27MWrUKIwYMQIA0LJlS/z000/Yv3+/xCMjInIt1s4cZOWIV8Wt2ghkXLV8nZjIO6W46HDbjtGdeocaSopmbXf7XGQdnPr164dvvvkG58+fR7t27XDs2DHs2rULn3zySY3P0ev10Ov1pseFhdXcMpuIiMzUNnNQUiqW4n5OBnYfsXyujzfw0GAxMPXqArCTQjru2KztaLIOTtOnT0dhYSE6dOgAlUoFg8GA9957D08//XSNz5k3bx5mz57twFESkbNwxoX6HDnmu2cOBAHYfxz4IBn4LRUorvK9q1CIN9QdkwQkDQCaukgvsDP+jtzNHZu1HU3Wwem///0vfvzxRyxfvhxxcXE4evQopk6disjISIwfP77a58yYMQOvvfaa6XFhYSGio91nCpGIqueMC/VJMebMHGD1RmDlJuBKtuX+ls3FsPToUKB5mF2GIBk5/o7UN8i5Y7O2o8k6OE2bNg3Tp0/Hk08+CQDo0qULLl++jHnz5tUYnNRqNdRqef4lSETSccaF+hw15uJSYEOqeK+4vUct9/s2vVOK69nZtqU4Oc3wyO13pCFBzh2btR1N1sGppKQEyirXrapUKhir3gGSiKgOztj7Yc8xG43AvuNik/f6FKCkzHy/QgEM6CGGpcQB4urelWwVduQ2wyO335GGBjl3a9Z2NFkHp5EjR+K9995DTEwM4uLicOTIEXzyySeYOHGi1EMjIgex1Ze0M/Z+1GfM1v6crmQDq/5YoDIrx/I9W0WLYemRoUBkNdUdW4Yduc3wyO13RG5BjkSyDk5ffPEF3nnnHfz1r39Fbm4uIiMjMXnyZLz77rtSD42IHMCWX9LO2Pth7Zjr+jkVlYgN3iuTxVmmqvx8/ijFJQH3dqq9FGfLsCO3YCC33xG5BTkSyTo4aTQaLFiwAAsWLJB6KERkB3XNktjyS9oZez+sHXN1P6cL13X49/oC5F4KxYad4ured1MqgYE9xLA0rD/gZeWPwZZhR27BQG6/I3ILciSSdXAiItdlzWySrWcknLH3w5ox3/1z0uV7Iv1sAM6fao/DJZZf+K1jgLFJwCMJQHhI/cdjy7Ajt2Agp0Z1QH5BjkQMTkQkCWtmk+Q2IyFXzbyb4tpFBS6mRuLGNR+L/X6+wMMPiL1L3Ts27qo4W4YdOQUDuTWqV3LGsO/qGJyISBLWzCbJbUZCTgwGYM9RcTXvDTuaQ19unoYUCgF97zXi6REqJPQHvDxt8762DjtyCQZya1Qn+WJwIiJJWDObJKcZCblIzxKviFu9CcjOrdx6JzQFNdOjX59STBnjjU4t7b/CeHXkVvKyhtwa1Um+GJyISBLWzibJZUZCSoVF4lpLPycDh05Z7vfXAKOGiKW4ru3VUCikCylyLXnVhWVhshaDExFJwllmk6SaPTEYgN8Pi2Fp4y5AX26+X6UE4nuLV8Ul9AXUNirFNZazlrxYFiZrMTgRkWTkPpskxexJ2hVxNe9Vm4CcG5b728eKYWl0AhAaZJchNIqzlrycJciT9BiciIhq4KjZk4IiYN028V5xR05b7g/wA0YPEQNT57a2vVecrTlzyUvuQZ7kgcGJiKgG9pw9MRiAnYfE1bw37QL0Feb7VUpg8H1i39ID98mnFFcXlrzI1TE4ERHVwB6zJxcui6W41ZuB69WU4jq0EheoHDUECGlEKU6q3iyWvMjVMTgREdXAVrMnBTrg123i7NLRs5b7g/zFnqXHEoG4No0vxUl9ZRtLXuTKGJyIXJwzrqkjF42ZPbltAHYcEMPS5t1AeZVSXBOVWIIbkwQM7gN4ethu3M56ZZsc8c8PVcXgROTCpJ55cIUvnfrOnpxPF5u812wGtLcs93dqA4xNBEYlAMEBthvn3Zz1yja5kfrPD8kTgxORgzkyTEg58+BOXzp5BX+U4jYCx89Z7g8OEEtxYxLF4GRvznxlm5xw5o6qw+BE5EA1hYlJA2JxMrvA5mFKypkHV//SuW0AUveLC1Ru3WNZivNoAjzQV5xdGtRHfFwbWwZqXtlmG5y5o+owOBE5UHVh4sy1Aryz9iQMRsFiZqbyOQ39MpVy5sFVv3TOXhJnln7ZDGjzLPd3biv2LY0aIjZ9W8PWs3O8ss02OHNH1WFwInKg6sJEsf42bhSVY2DbELOZmXXHr+J0tq5RX6ZSzjy40pfOrQJg7Vax0fvkBcv9zQKBR4aKpbgOrer/+vaYneOVbY3HmTuqDoMTkQNVFyZKyg1oqm5iMTOzJ+0mbhSVN+rL1NEzD3eXm/y9PdA8wNtpv3QqbgMp+8RG7217xMd38/QAhvQV11y6v1fdpbjauOrsnK1wTSqSEwYnIgeq7l+w0YFNUXbbcmZGAGzyZeqomYfqyk3NA7wxslsECkpvO82XzumLf5TitgA38y33d20vziw9/AAQaGUpri6uNDtna1JfZMCZO6qKwYnIgar7F2znSH8s2pVuMTPTKdIP/zuW7TRfpjWVm+Lbh+C5/g2oXznQzXxg7RZxdun0Rcv9IUHAo3+U4trF2v79WRKqmatfZEDOh8GJnJIzrw9U3b9gqysHAMDp7EKn+TJ1tnJTeQWwfZ/Yt7Rtr3iV3N08PYCh/cRG7/t7iQtW2gtLQjVztt8rcn0MTuR0pJ66t4eaygHO9GXqDOUmQQBOXRTD0tqtYtN3Vd07iGFp5GAgwE/c5oigzpJQ9Zzh94rcC4MTOR13mrp3pi/TmspNnSP9sfJQpqTh70ae2LO0Mhk4c8lyf2gw8Ngw8V5xbVuY73PFoO5MWMYkuWFwIqfDqXt5qq1/y9GhQ6vTY+vpXOw8oMKlM4E4d84LBoP5nXPVHkDiQDEsDehRcynOnYK6HLGMSXLD4EROh1P38lV1hmzloUyHhg5BAHYeLcfsxTqknwuHodzyzrn3dBJX837oAcDft+7XZFCXnjPNvJLrY3Aip8Ope+fhqNCRe+tOKe5cuieAZmb7Pbz1uL+vHv8Y74fWMfV7bQZ1IrpbvYPT+PHjMWnSJNx///32GA9RnTh17zzsGTr05cCWPWJYSt0PGIzm+1UqI6JaF6JVxzyUqHPQtW0wWsf41ft9GNSJ6G71Dk4FBQVISEhAixYt8Nxzz2H8+PFo3ry5PcZGVCNO3TsHW4cOQQCOnxNvrPvrNqBAZ3lMy5Z6GIMz0eOeMnh7C3+UBxse1hjUiehuCkEQhPo+SavV4vvvv8fSpUtx+vRpJCQkYNKkSRg1ahQ8PCx7CqRUWFgIf39/FBQUwM+v/v/aJHJ3jb0U3xaX8l+/CazZLM4uXbhsuT8iRGzyfmwYoAmwvAqudYgvJg2IxcnsAoYfIrJQn6zQoOB0t8OHD2Px4sX497//DV9fX4wbNw5//etf0bZt28a8rM0wOBE1XHWX4rcO8bXrVXGVQStDW4b8q4FIOxOIvUdUMFYpxXmpgaSB4r3i+nYHVCrL16jt6j57n4e7cubFacl91ScrNKo5/Nq1a9i8eTM2b94MlUqFBx98ECdOnECnTp3w4Ycf4tVXX23MyxORxBx9KX5uoR7Tl13BsSO+yMuMhaHC8q+oXl3EW5+MGARofKp/Hamv7nNX7rzmFQOj+6h3cKqoqMCvv/6KxYsXY9OmTejatSumTp2KP/3pT6aUtmbNGkycOJHBicjJOeqquBwtsHozsHitErm5lrPVgQG38czIJngsEWjZgJZKLingGHJa88qRQcadA6M7qndwioiIgNFoxFNPPYX9+/eje/fuFscMHjwYAQEBNhgeEUnJnlfFlemBTb+LfUs7D+GPUtydHklVEyNi2hTAKyIbiX098OrQdrI8D7pDLgHV0UFGToGR7K/ewenTTz/F2LFj4eXlVeMxAQEBSE9Pb9TAiEh69rgq7vBpMSyt2w4UFlse49usAB27FqJl20IomxhwMVeH6KB6Lr5k5/O4G0s0d8gloDo6yMglMJJj1Ds4PfPMM/YYB5Fk+MVXM1tdin9NC6zeBKzcCFzKtNwfFS72LQ3qr8fyoxlI0xYhs8B2AcdeSwqwRGNOLmteOTrIyCUwkmNw5XBya/ziq1tD18wqLQM27hLD0q5D4mzT3Zp6iQ3ejyUCfboCSiUAqBEVbp81k+yx9te641exL/0mfNUe8FApERXYFGnaItmXaOz1jwW5rHnl6CAjl8BIjsHgRG7NmXsT5DhTJgjAwZNiWFqfAuiqKcXd1128V9zweMCnmu8xZ1ncVKvT4+eDWbhVXI7SCiNydWXI1ekR2NRD1iUae/9jQQ6fn6ODjFwCIzkGgxO5NWftTZDbTNnV6+JVcSuTgYyrlvujI8Sw9GgiEB3u8OHZRer5XBSXG6BuooRG3QQCgMLSChiM8i7ROPM/FqwlRZCRQ2Akx2BwIrfmrL0JcvjyKykFkneJYWn3EctSnI+3WIobkyiuvSSW4lxHVl4pgn08oVIoUFhWAaVCAf1tA8L81LIu0TjrPxbqi0GG7IXBidyas/YmSPXlJwjAgRPiveJ+SwWKSiyP6XePuJp30kCgqbzzZ6NUhuvOzf2RqytDYVkFivUqjO0ZLesSjbP+Y4FILhicyK05a2+Co7/8snKAVZuAVRuBy9mW+1tEAmOSgEeHilfIuYPK0F1ZLvVqokJchD8e6hop9dBq5az/WCCSi0bfq07ueK86ckUNvYdcfRrKS0qBDTuAnzcCe45Y7vdtCjw0WCzF9ewMKBS2OjvnIccGfWs467iJ7MWhN/mVOwYnclX1/fKzJmwZjcD+4+JVcb+lAsVVKn8KBdD/XjEsJQ0EvGteB5eIyGk47Ca/RCSd+ja/1tZQ3jsy2rRAZeY1y+fGRolh6dFhQCQrOkTkxhiciNxE1YZy420V8i6H4dN9wci6Ynm8xgcYOVhcoLJHnHuW4oiIqmJwInITUYHeKL9tRPaVprh8LghXLvrDcNt8jQCFAhjYQ2z0ThwAeLHthYjIDIMTkRu4fBU4cyACZzeGoUTnabG/dbQYlh4ZCkSESDBAIiInweBE5KKKSsTbnqzcKDZ8V/3j7uVlxIhBAsaNVOGejizFERFZQ/bB6erVq3jzzTexYcMGlJSUoE2bNli8eDF69uwp9dCIZMdoFFfxXrlRXEqgTG++X6kEBvYUb38ydIASXpaTT0REVAtZB6e8vDz0798fgwcPxoYNGxASEoILFy4gMDBQ6qERyUp6lhiWVm8CsnMt97dpIYalR4YCYc0cPz4iIlch6+D0wQcfIDo6GosXLzZti42NlXBERPJRWASsTxXvFXfwpOV+P19g1BDx9idd27MUR0RkC7IOTr/++isSExMxduxYpKamonnz5vjrX/+KF154QeqhEUnCYBBLcT8nA8k7AX25+X6lEojvJYalIf3AUhwRkY3JOjhdunQJCxcuxGuvvYa33noLBw4cwMsvvwxPT0+MHz++2ufo9Xro9XcaOwoLCx01XJfGWzRIK+2KeJ+41ZuBa1rL/e1ailfFjU4AwoIdPjwiIrch61uueHp6omfPnti9e7dp28svv4wDBw5gz5491T5n1qxZmD17tsV23nKl4Rp6XzRqnIIiYP128V5xh09Z7g/wE0txYxKBLu1YiiMiaiiXueVKREQEOnXqZLatY8eOWLVqVY3PmTFjBl577TXT48LCQkRHW39bCrJU26066nPLD6qbwQDsOiQ2em/cCegrzPerlMDg+4DHhgFD+gJqluKIiBxK1sGpf//+OHfunNm28+fPo0WLFjU+R61WQ63mLIgtVb1Vh0qpgIdKiay80jqeSda6cPlOKe76Dcv9HVqJM0ujE4CQIMePj4iIRLIOTq+++ir69euH999/H48//jj279+Pb775Bt98843UQ3MrUYHeqDAYYTAKphmnCoMRUYHeUg/NqRXogP9tF6+KO3LGcn+gnxiUxiQBcW1YiiMikgNZB6devXphzZo1mDFjBubMmYPY2FgsWLAATz/9tNRDcyvx7UKx79ItXMzVmfU4xbcLlXpoTue2Adh5UAxLm3+3LMU1UQEP3CfeWPeB+wBPD2nGSURE1ZN1c7gt1Kfhi2rGq+oa53yGGJbWbAFyb1ru79RanFkaNQRoxvVdiYgcymWaw0k+QjRqp28E1+r0WHf8Kvak3YQAoF/rZnioa6TdAmB+IfDrNrHR+9hZy/1B/uJK3o8liqU4IiKSPwYncgtanR5z/ncKv6fdQPltIwDg8OV8HL6ch3dHxtksPN02AKn7xbC0ZTdQXk0pbkhfsdF7UB+W4oiInA2DE7mF1PO5OJaVDwUUCNF4QQGgoLQcx68W2GRZhXPp4mrev2wGtHmW++PaiveKe3gIEBzQqLciIiIJMTiRW8jKK4XBCCiVCigVlcsqKHHbYGzwsgp5BcDabWLv0onzlvubBf5xVVwi0LF1Y0ZPRERyweBEbiEq0BsqJWCsEGAUBCgAGIxGeHt61GtZhYrbYinu52Rg6x7x8d08moiluLFJQHxv8TEREbkO/rVObiG+XShSz2nxe9oNaHVlAAB1ExW6Nve3almFM2liWFq7FbhRTSmuS7s/rop7AAj0t/XoiYhILhicyC2EaNR4d2Sc6ao6AOhbx1V1N/OBtVuAlZuAUxeqec1A4JFhYimufawdB09ERLLBdZyI7lJxG9i+V7wqbuse8Sq5u3l6AEP7iUsIxPcWr5IjIiLnxnWciOrp1EWxyXvtVnGmqapuHcSZpYcfAAKYv4mI3BaDE7mtG3liUFqZDJxOs9wfGgw8+scCle1aOnx4REQkQwxO5FbKK4Bte8WwtH2fZSlO7QEMGyCGpYE9WYojIiJzDE7k8gQBOHnhTikur9DymHs6ilfFjRwM+GscP0a54j0KiYjMMTiRy8q9BfyyBVi1ETh7yXJ/WDPgsWHi7FKbGMeNy1nCiFanx4fJZ5GmLYKHSokKgxH7Lt3CG0kdZDleIiJHYHAiWbBVmNCXi1fDrdwIpOwDDEbz/WpPIHGA2Og9oAegcnApzpnCSOr5XKRpi9AmVAOVUgGDUcDFXJ3pFjXOEgCJiGyJwYkk19gwIQjiLU9+TgZ+3QbkV1OKuzdOvFfciMGAv68dTsJKdYUROcnKK4WHSgmVsvIWNQp4qJTIyit1qgBIRGRLDE4kuYaGies3gTWbxVLc+QzL/REhwKPDxHJcaweW4mpTWxipL3vP+EQFeqPCYITBKJg+lwqDEVGB3k4VAImIbInBiSRXnzBRVg5s3S3OLqUeAIzVlOKG3y+W4vrd4/hSXF1qCyP14YgZn/h2odh36RYu5upM79E6xBfx7ULx477LNguARETOhMGJJFdXmBAE4NhZMSz9bztQoLN8jZ6dxaviRsQDfhKW4upSWxipD0fM+IRo1HgjqUO1s1q2CoBERM6GwcmO2DxrnZrCRKfgUHz1E/DzRuDiZcvnRYaKV8Q9NgyIjXL8uBuitjBSH7Ys+dU13uqCmK0CoLvi3w1EzovByU7k1jwr57+o7w4TGdoy5F0NxKW9QRjxldKiFOelBh68XwxM/e4BlEppxtwYNYWR+pB6xsdWAdAdye3vBiKqHwYnO6mplLLu+FVovDwc+mUj97+oBQHIzFTjcGo0/rcNKCy2PKZ3V7Fv6cF4QOPj+DHKjRxmfGwRAN0RG+uJnBuDk51UV0oBgJ8PZsHLQ+XQACPXv6ivaYHVm8Sr4tIyLfdHhd0pxbVo7vjxyRlnfJyXo8qsRGQfDE52Ul0p5UaRHgqFAp2bBzg0wMjpL+oyPbBxl3j7k52HxNmmu3l7iQ3ejyUC93VzzlKco3DGxzlJXWYlosZhcLKT6kopvuom8PJQOTzASP0XtSAAh06JYWldCqCrphR3Xzfxqrjh9wO+TR0yLCJJyKHMSkQNx+BkJ9WVUnRlt/G/Y9kODzBS/UV99Tqw+o8FKtOzLPdHR4h9S48OA2Ii7DoUqgc5X0jgClhmJXJuCkGoWixxLYWFhfD390dBQQH8/PwkHUt1TdqtQ3wd0qTtqC/D0jIgead4r7jfD1uW4pp6ASMGAWOTgF5dWIqTGyl/R4mIpFKfrMAZJweS8l+a9uyHEQTg4Elxgcr1KUBRieUxfe8R7xWXdD/gw1YO2ZLrhQRERHLB4ORgrtTQm5UDrPrjqrjL2Zb7YyLvlOKiwx0/Pqo/OV1IQEQkRwxOMuEsfSUlpWIp7udkYPcRy/2+TcVS3JhEsRSnUDh8iNQIUl9IQEQkdwxOMiD3BSqNRuDACTEs/ZYKFFeZfFAogP73iksIJA0AmvI71mnxii8iotoxOMmAXPtKrlwTF6hcuRHIvGa5v2VzcQmBR4cCzcMcPz6yPV7xRURUOwYnGZBTX0lxqTirtHIjsPeo5X6ND/DQIDEw9YhjKc4VuVIfHhGRrTE4yYDUfSVGI7DvuLhA5W+pQEmZ+X6FAhjQQ+xbShwgru5dG2fp1yIiIqovBicZkKqv5PLVP66K2yReIVdVq2gxLD0yFIi0cihy79ciIiJqDAYnGXBkX0lRibjW0qqN4ixTVX4+wMgHxFLcPR3rX4qTa78WERGRLTA4yYQ9+0qMRmDPUbEUt2GnuLr33ZRKYGAPMSwN6w94NSKvyalfi4iIyNYYnFxYxlWxyXv1JvG+cVW1jhFvffJIAhAeYpv3lLpfi4iIyJ4YnJyINU3XumKxFLdyo7j2UlV+vsDDD4i9S90bUIqrC9cBIiIiV8bg1EiOuoKstqbroKZq7DkqLlCZvBMo05s/V6kE4nuJYSmhP+DlafPhmVT2a607no09aTcAAJ0iNfZ7QyIiIgdicGoER15BVl3T9cm0Crz5mR6nj6txTWv5nLYtxL6lRxKAsGY2HU6dTmcX4kZROTxUSvzv2DWcztbxyjoiInJ6DE6N4MgryCqbrg0VKqRf8Mels4G4cc0Hp6oc568BRg0Re5e6tJNmgUpeWUdERK6KwakRGnsFmbVlPoMBKM4NwLnfA3EgOxgGg9Jsv0oJDOoj3isuoS+gtmMpzhq8so6IiFwVg1MjNOYKMmvKfBeviOstrd4E5NywbK72DyrFc6M88PRDTRAaZPPTazBeWUdERK6KwakRGnMFWU3lrA1HtVDmRWHlRuDIacvnNW1qQLtOhRg8QI8/DQ5EqJ/8PkJ3ubKOt5YhInI/CkEQBKkHYU+FhYXw9/dHQUEB/Pz8bP76Df3yXLDlPPak3USrEF8YjUBOpi9OHNUg/2qQRSlOoRDQqVMZxo9sgtGDPexeirNFIHD1UFHdjGHrEF82wBMROaH6ZAX5TVc4mYau+B0V6I3CPDUOnwvD5fOBKC32sDjGP7gUmqgchMbehMJDj4OFvnhA3wEhnvb7YrbVlYL2XAldDtgAT0TknhicHCy/EPh1G7BiQ3OcPm/5BRvgJ+DRoQoExOQgNSvtjy9mNQxGT4d8MTMQWIcN8ERE7onByQFuG4AdB8R7xW3eDZRXAMCdcpxSKaBTp1JMeNgDowZ5wNMDWLClUJIvZgYC67ABnojIPTlVcJo/fz5mzJiBV155BQsWLJB6OHU6ly7e+mTNZkB7y3J/pzbA2ERgVIICwQFNzfZJ9cXMQGAdd2mAJyIic04TnA4cOICvv/4aXbt2lXootcorANZuE5cROH7Ocn9wADA6Qbz9Sac2Nb+OVF/MDATWqby1jCs3wBMRkSWnCE5FRUV4+umn8e233+Kf//yn1MOxUHEbSN0vzi5t2S0+vptHE2BIXzEsDeojPq6LVF/MDATWc/UGeCIisuQUwWnKlCkYMWIEEhISZBWczl4Sb6z7yxbgRp7l/s5txXvFjRoCBPnX//Wl+mJmICAiIqqe7IPTihUrcPjwYRw4cMCq4/V6PfR6velxYWGhXcZVcRt48lUgr8rLhwQCo4eKs0sdWtnlrYmIiEgisg5OmZmZeOWVV7B582Z4eXlZ9Zx58+Zh9uzZdh6ZWG4bNQRYsgbw9AAS+olh6f5e1pXiiIiIyPnIeuXwX375BY888ghUKpVpm8FggEKhgFKphF6vN9sHVD/jFB0dbZeVw8+nA3uOAg8/AAQ2oBRHRERE0nOZlcOHDBmCEydOmG177rnn0KFDB7z55psWoQkA1Go11GrHNDK3ixX/IyIiIvcg6+Ck0WjQuXNns20+Pj4IDg622E5ERERkb7IOTiRvrn4jXyIioqqcLjilpKRIPQSC7W4GTERE5EyUdR9CZOnumwG3CvFFm1AN0rRFSD2fK/XQiIiI7IbBiRqENwMmIiJ3xOBEDXL3zYAB8GbARETkFpyux4nkgTcDJiIid8TgRA3CmwETEZE7YnCiBuPNgImIyN2wx4mIiIjISgxORERERFZicCIiIiKyEoMTERERkZUYnIiIiIisxOBEREREZCUGJyIiIiIrMTgRERERWYnBiYiIiMhKDE5EREREVuItVwhanZ73nCMiIrICg5Ob0+r0+DD5LNK0RfBQKVFhMGLfpVt4I6kDwxMREVEVLNW5udTzuUjTFqFNqAatQnzRJlSDNG0RUs/nSj00IiIi2WFwcnNZeaXwUCmhUioAACqlAh4qJbLySiUeGRERkfwwOLm5qEBvVBiMMBgFAIDBKKDCYERUoLfEIyMiIpIf9ji5ufh2odh36RYu5upMPU6tQ3wR3y5U6qERERHJDoOTmwvRqPFGUgdeVUdERGQFBidCiEaNMT2ipR4GERGR7LHHiYiIiMhKDE5EREREVmJwIiIiIrISe5zI6fAWMUREJBUGJ3IqvEUMERFJiaU6ciq8RQwREUmJwYmcCm8RQ0REUmJwIqfCW8QQEZGU2OPkwlyxiZq3iCEiIikxOLkoV22i5i1iiIhISgxOLuruJmqVUgGDUcDFXB1Sz+c6/e1VeIsYIiKSCnucXBSbqImIiGyPwclFsYmaiIjI9liqc1FsoiYiIrI9BicXxSZqIiIi22NwcmFsoiYiIrIt9jgRERERWYnBiYiIiMhKDE5EREREVmJwIiIiIrISgxMRERGRlRiciIiIiKzE5QjsRKvTcw0lIiIiF8PgZAdanR4fJp9FmrbItGr3vku38EZSB4YnIiIiJ8ZSnR2kns9FmrYIbUI1aBXiizahGqRpi5B6PlfqoREREVEjyDo4zZs3D7169YJGo0FoaChGjx6Nc+fOST2sOmXllcJDpYRKqQAAqJQKeKiUyMorlXhkRERE1BiyDk6pqamYMmUK9u7di82bN6OiogLDhg1DcXGx1EOrVVSgNyoMRhiMAgDAYBRQYTAiKtBb4pERERFRY8i6xyk5Odns8ZIlSxAaGopDhw7h/vvvl2hUdYtvF4p9l27hYq7O1OPUOsQX8e1CpR4aERERNYKsg1NVBQUFAICgoKAaj9Hr9dDr9abHhYWFdh9XVSEaNd5I6sCr6oiIiFyMQhAEQepBWMNoNOLhhx9Gfn4+du3aVeNxs2bNwuzZsy22FxQUwM/Pz55DJCIiIidUWFgIf39/q7KC0wSnF198ERs2bMCuXbsQFRVV43HVzThFR0czOBEREVG16hOcnKJU99JLL2HdunXYsWNHraEJANRqNdRqlsSIiIjI9mQdnARBwN/+9jesWbMGKSkpiI2NlXpIRERE5MZkHZymTJmC5cuXY+3atdBoNMjJyQEA+Pv7w9ubl/YTERGRY8m6x0mhUFS7ffHixZgwYYJVr1GfuiURERG5H5fpcZJxpiMiIiI3JOuVw4mIiIjkhMGJiIiIyEqyLtXZQmW5T4oVxImIiEj+KjOCNS1CLh+cdDodACA6OlrikRAREZGc6XQ6+Pv713qMrK+qswWj0Yjs7GxoNJoar9JrjMqVyTMzM13+qj13OlfAvc6X5+q63Ol83elcAfc6X3ufqyAI0Ol0iIyMhFJZexeTy884KZXKOlcbtwU/Pz+X/8Wt5E7nCrjX+fJcXZc7na87nSvgXudrz3Ota6apEpvDiYiIiKzE4ERERERkJQanRlKr1Zg5c6Zb3FjYnc4VcK/z5bm6Lnc6X3c6V8C9zldO5+ryzeFEREREtsIZJyIiIiIrMTgRERERWYnBiYiIiMhKDE5VfPnll2jZsiW8vLzQp08f7N+/v9bjf/75Z3To0AFeXl7o0qULfvvtN7P9giDg3XffRUREBLy9vZGQkIALFy7Y8xTqpT7n++2332LgwIEIDAxEYGAgEhISLI6fMGECFAqF2X9JSUn2Pg2r1OdclyxZYnEeXl5eZse40mc7aNAgi/NVKBQYMWKE6Ri5frY7duzAyJEjERkZCYVCgV9++aXO56SkpODee++FWq1GmzZtsGTJEotj6vt3gSPU91xXr16NoUOHIiQkBH5+fujbty82btxodsysWbMsPtcOHTrY8SysV9/zTUlJqfb3OCcnx+w4V/hsq/vzqFAoEBcXZzpGrp/tvHnz0KtXL2g0GoSGhmL06NE4d+5cnc+Ty/ctg9Nd/vOf/+C1117DzJkzcfjwYXTr1g2JiYnIzc2t9vjdu3fjqaeewqRJk3DkyBGMHj0ao0ePxsmTJ03HfPjhh/j888/x1VdfYd++ffDx8UFiYiLKysocdVo1qu/5pqSk4KmnnsL27duxZ88eREdHY9iwYbh69arZcUlJSbh27Zrpv59++skRp1Or+p4rIC60dvd5XL582Wy/K322q1evNjvXkydPQqVSYezYsWbHyfGzLS4uRrdu3fDll19adXx6ejpGjBiBwYMH4+jRo5g6dSqef/55s0DRkN8XR6jvue7YsQNDhw7Fb7/9hkOHDmHw4MEYOXIkjhw5YnZcXFyc2ee6a9cuewy/3up7vpXOnTtndj6hoaGmfa7y2X722Wdm55iZmYmgoCCLP7Ny/GxTU1MxZcoU7N27F5s3b0ZFRQWGDRuG4uLiGp8jq+9bgUx69+4tTJkyxfTYYDAIkZGRwrx586o9/vHHHxdGjBhhtq1Pnz7C5MmTBUEQBKPRKISHhwsfffSRaX9+fr6gVquFn376yQ5nUD/1Pd+qbt++LWg0GmHp0qWmbePHjxdGjRpl66E2Wn3PdfHixYK/v3+Nr+fqn+2nn34qaDQaoaioyLRNrp/t3QAIa9asqfWYN954Q4iLizPb9sQTTwiJiYmmx439+TmCNedanU6dOgmzZ882PZ45c6bQrVs32w3MTqw53+3btwsAhLy8vBqPcdXPds2aNYJCoRAyMjJM25zls83NzRUACKmpqTUeI6fvW844/aG8vByHDh1CQkKCaZtSqURCQgL27NlT7XP27NljdjwAJCYmmo5PT09HTk6O2TH+/v7o06dPja/pKA0536pKSkpQUVGBoKAgs+0pKSkIDQ1F+/bt8eKLL+LmzZs2HXt9NfRci4qK0KJFC0RHR2PUqFE4deqUaZ+rf7aLFi3Ck08+CR8fH7PtcvtsG6KuP7e2+PnJldFohE6ns/gze+HCBURGRqJVq1Z4+umnceXKFYlGaBvdu3dHREQEhg4dit9//9203ZU/20WLFiEhIQEtWrQw2+4Mn21BQQEAWPxe3k1O37cMTn+4ceMGDAYDwsLCzLaHhYVZ1Mcr5eTk1Hp85f/W5zUdpSHnW9Wbb76JyMhIs1/UpKQkLFu2DFu3bsUHH3yA1NRUDB8+HAaDwabjr4+GnGv79u3x3XffYe3atfjhhx9gNBrRr18/ZGVlAXDtz3b//v04efIknn/+ebPtcvxsG6KmP7eFhYUoLS21yZ8Nufr4449RVFSExx9/3LStT58+WLJkCZKTk7Fw4UKkp6dj4MCB0Ol0Eo60YSIiIvDVV19h1apVWLVqFaKjozFo0CAcPnwYgG3+3pOj7OxsbNiwweLPrDN8tkajEVOnTkX//v3RuXPnGo+T0/ety9/kl+xj/vz5WLFiBVJSUsyapp988knT/+/SpQu6du2K1q1bIyUlBUOGDJFiqA3St29f9O3b1/S4X79+6NixI77++mvMnTtXwpHZ36JFi9ClSxf07t3bbLurfLbuavny5Zg9ezbWrl1r1vMzfPhw0//v2rUr+vTpgxYtWuC///0vJk2aJMVQG6x9+/Zo37696XG/fv2QlpaGTz/9FN9//72EI7OvpUuXIiAgAKNHjzbb7gyf7ZQpU3Dy5ElZ9F5ZizNOf2jWrBlUKhWuX79utv369esIDw+v9jnh4eG1Hl/5v/V5TUdpyPlW+vjjjzF//nxs2rQJXbt2rfXYVq1aoVmzZrh48WKjx9xQjTnXSh4eHrjnnntM5+Gqn21xcTFWrFhh1V+qcvhsG6KmP7d+fn7w9va2ye+L3KxYsQLPP/88/vvf/1qUO6oKCAhAu3btnO5zrUnv3r1N5+KKn60gCPjuu+/wzDPPwNPTs9Zj5fbZvvTSS1i3bh22b9+OqKioWo+V0/ctg9MfPD090aNHD2zdutW0zWg0YuvWrWYzD3fr27ev2fEAsHnzZtPxsbGxCA8PNzumsLAQ+/btq/E1HaUh5wuIVy3MnTsXycnJ6NmzZ53vk5WVhZs3byIiIsIm426Ihp7r3QwGA06cOGE6D1f8bAHxcl+9Xo9x48bV+T5y+Gwboq4/t7b4fZGTn376Cc899xx++ukns+UlalJUVIS0tDSn+1xrcvToUdO5uNpnC4hXqF28eNGqf+zI5bMVBAEvvfQS1qxZg23btiE2NrbO58jq+9amreZObsWKFYJarRaWLFkinD59Wvjzn/8sBAQECDk5OYIgCMIzzzwjTJ8+3XT877//LjRp0kT4+OOPhTNnzggzZ84UPDw8hBMnTpiOmT9/vhAQECCsXbtWOH78uDBq1CghNjZWKC0tdfj5VVXf850/f77g6ekprFy5Urh27ZrpP51OJwiCIOh0OuH1118X9uzZI6SnpwtbtmwR7r33XqFt27ZCWVmZJOdYqb7nOnv2bGHjxo1CWlqacOjQIeHJJ58UvLy8hFOnTpmOcaXPttKAAQOEJ554wmK7nD9bnU4nHDlyRDhy5IgAQPjkk0+EI0eOCJcvXxYEQRCmT58uPPPMM6bjL126JDRt2lSYNm2acObMGeHLL78UVCqVkJycbDqmrp+fVOp7rj/++KPQpEkT4csvvzT7M5ufn2865u9//7uQkpIipKenC7///ruQkJAgNGvWTMjNzXX4+VVV3/P99NNPhV9++UW4cOGCcOLECeGVV14RlEqlsGXLFtMxrvLZVho3bpzQp0+fal9Trp/tiy++KPj7+wspKSlmv5clJSWmY+T8fcvgVMUXX3whxMTECJ6enkLv3r2FvXv3mvbFx8cL48ePNzv+v//9r9CuXTvB09NTiIuLE9avX2+232g0Cu+8844QFhYmqNVqYciQIcK5c+cccSpWqc/5tmjRQgBg8d/MmTMFQRCEkpISYdiwYUJISIjg4eEhtGjRQnjhhRck/wupUn3OderUqaZjw8LChAcffFA4fPiw2eu50mcrCIJw9uxZAYCwadMmi9eS82dbeQl61f8qz2/8+PFCfHy8xXO6d+8ueHp6Cq1atRIWL15s8bq1/fykUt9zjY+Pr/V4QRCXYoiIiBA8PT2F5s2bC0888YRw8eJFx55YDep7vh988IHQunVrwcvLSwgKChIGDRokbNu2zeJ1XeGzFQTxcntvb2/hm2++qfY15frZVneeAMz+HMr5+1bxx0kQERERUR3Y40RERERkJQYnIiIiIisxOBERERFZicGJiIiIyEoMTkRERERWYnAiIiIishKDExEREZGVGJyIiIiIrMTgRERERGQlBiciIiIiKzE4EREREVmJwYmI3IJWq0V4eDjef/9907bdu3fD09MTW7dulXBkRORMeJNfInIbv/32G0aPHo3du3ejffv26N69O0aNGoVPPvlE6qERkZNgcCIitzJlyhRs2bIFPXv2xIkTJ3DgwAGo1Wqph0VEToLBiYjcSmlpKTp37ozMzEwcOnQIXbp0kXpIRORE2ONERG4lLS0N2dnZMBqNyMjIkHo4RORkOONERG6jvLwcvXv3Rvfu3dG+fXssWLAAJ06cQGhoqNRDIyInweBERG5j2rRpWLlyJY4dOwZfX1/Ex8fD398f69atk3poROQkWKojIreQkpKCBQsW4Pvvv4efnx+USiW+//577Ny5EwsXLpR6eETkJDjjRERERGQlzjgRERERWYnBiYiIiMhKDE5EREREVmJwIiIiIrISgxMRERGRlRiciIiIiKzE4ERERERkJQYnIiIiIisxOBERERFZicGJiIiIyEoMTkRERERWYnAiIiIistL/B/HF2vFPZddIAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# viz: the data and the true line we are trying to recover\n",
    "plt.figure(figsize=(6, 4))\n",
    "plt.scatter(X, y, s=14, alpha=0.6, label=\"data\")\n",
    "xs = np.array([[0.0], [2.0]])\n",
    "plt.plot(xs, 4 + 3 * xs, color=\"#1E40FF\", lw=2, label=\"true: y = 4 + 3x\")\n",
    "plt.xlabel(\"x\"); plt.ylabel(\"y\"); plt.legend(); plt.title(\"y = 4 + 3x + noise\")\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8569dd3",
   "metadata": {},
   "source": [
    "> **Interpretation.** The cloud follows the blue line but does not sit on it; the noise is what makes the recovered slope 2.93 rather than exactly 3. No method can do better than the noise allows, which is why the four solvers below agree with each other but not with the exact `(4, 3)`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e4fc80a",
   "metadata": {},
   "source": [
    "### Exercise 4.1 — The normal equation by hand\n",
    "`Difficulty 1/5 · ~8 min`\n",
    "\n",
    "Fill in `normal_equation(X_b, y)` to compute $(\\mathbf{X}^T\\mathbf{X})^{-1}\\mathbf{X}^T\\mathbf{y}$ directly with `np.linalg.inv`. Return `theta` of shape `(d+1, 1)`. The check compares your `theta` against `lstsq`, the numerically safer route, on this exact problem.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "08faef27",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.009283Z",
     "iopub.status.busy": "2026-06-10T18:46:10.009208Z",
     "iopub.status.idle": "2026-06-10T18:46:10.013288Z",
     "shell.execute_reply": "2026-06-10T18:46:10.012913Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] 4.1 inv == lstsq: not attempted yet — fill in the TODO above, then re-run.\n",
      "[ -- ] 4.1 recovers (4,3): not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def normal_equation(X_b, y):\n",
    "    \"\"\"Closed-form least squares: theta = (X_b^T X_b)^{-1} X_b^T y. Shape (d+1, 1).\"\"\"\n",
    "    # TODO 1: form X_b^T @ X_b, invert it with np.linalg.inv, then multiply by X_b^T @ y\n",
    "    theta = None\n",
    "    attempted(theta)\n",
    "    return theta\n",
    "\n",
    "# self-checks (run this cell): compare your inv-based theta to the lstsq route\n",
    "def _ne_matches_lstsq():\n",
    "    got = normal_equation(X_b, y)\n",
    "    ref = np.linalg.lstsq(X_b, y, rcond=None)[0]\n",
    "    check_close(got, ref, atol=1e-6,\n",
    "                msg=\"inv and lstsq solve the same equations; they must match here\")\n",
    "\n",
    "def _ne_recovers_truth():\n",
    "    got = normal_equation(X_b, y).ravel()\n",
    "    assert abs(got[0] - 4) < 0.3 and abs(got[1] - 3) < 0.3, \\\n",
    "        f\"recovered (intercept, slope) = ({got[0]:.3f}, {got[1]:.3f}); expected near (4, 3)\"\n",
    "\n",
    "check(\"4.1 inv == lstsq\", _ne_matches_lstsq)\n",
    "check(\"4.1 recovers (4,3)\", _ne_recovers_truth)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e2c2e17",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>Read the formula left to right. You need the inverse of `X_b.T @ X_b`, then a matrix product with `X_b.T @ y`. The `@` operator is matrix multiply; `.T` is transpose.</details>\n",
    "\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "theta = np.linalg.inv(X_b.T @ X_b) @ (X_b.T @ y)\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — \"Singular matrix\" from np.linalg.inv</summary>That means `X_b.T @ X_b` has no inverse, which happens when columns are collinear. It will not happen on this problem (one feature plus a bias column are independent), but it is exactly why `pinv` and `lstsq` exist; you meet them two cells down.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "0d605328",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.014048Z",
     "iopub.status.busy": "2026-06-10T18:46:10.013973Z",
     "iopub.status.idle": "2026-06-10T18:46:10.016606Z",
     "shell.execute_reply": "2026-06-10T18:46:10.016326Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 4.1 inv == lstsq\n",
      "[ ok ] 4.1 recovers (4,3)\n",
      "theta (inv): [3.99254002 2.93463751]\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines normal_equation; the checks below re-verify the reference.\n",
    "def normal_equation(X_b, y):\n",
    "    return np.linalg.inv(X_b.T @ X_b) @ (X_b.T @ y)\n",
    "\n",
    "check(\"4.1 inv == lstsq\", _ne_matches_lstsq, required=True)\n",
    "check(\"4.1 recovers (4,3)\", _ne_recovers_truth, required=True)\n",
    "print(\"theta (inv):\", normal_equation(X_b, y).ravel())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7c09400",
   "metadata": {},
   "source": [
    "> **Interpretation.** The recovered intercept is near 3.99 and the slope near 2.93. The noise pushed the slope down from 3; that gap is irreducible, not a bug.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b048c17",
   "metadata": {},
   "source": [
    "scikit-learn's `LinearRegression` does **not** call `np.linalg.inv`. It calls `scipy.linalg.lstsq`, which uses the SVD-based pseudo-inverse. The pseudo-inverse handles the case where $\\mathbf{X}^T\\mathbf{X}$ is singular (collinear features), where `inv` would raise. Here is the same fit computed four ways: the explicit inverse, `lstsq`, `pinv`, and scikit-learn. On a well-conditioned problem they are numerically identical.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7921da66",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.017520Z",
     "iopub.status.busy": "2026-06-10T18:46:10.017445Z",
     "iopub.status.idle": "2026-06-10T18:46:10.045735Z",
     "shell.execute_reply": "2026-06-10T18:46:10.045246Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "inv      intercept 3.992540  slope 2.934638\n",
      "lstsq    intercept 3.992540  slope 2.934638\n",
      "pinv     intercept 3.992540  slope 2.934638\n",
      "sklearn  intercept 3.992540  slope 2.934638\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "theta_inv   = np.linalg.inv(X_b.T @ X_b) @ (X_b.T @ y)\n",
    "theta_lstsq = np.linalg.lstsq(X_b, y, rcond=None)[0]\n",
    "theta_pinv  = np.linalg.pinv(X_b) @ y\n",
    "lin = LinearRegression().fit(X, y)                       # fits the bias itself\n",
    "theta_sklearn = np.r_[lin.intercept_, lin.coef_.ravel()].reshape(-1, 1)\n",
    "\n",
    "for name, t in [(\"inv\", theta_inv), (\"lstsq\", theta_lstsq),\n",
    "                (\"pinv\", theta_pinv), (\"sklearn\", theta_sklearn)]:\n",
    "    print(f\"{name:8s} intercept {t[0,0]:.6f}  slope {t[1,0]:.6f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "21c3c42f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.047039Z",
     "iopub.status.busy": "2026-06-10T18:46:10.046925Z",
     "iopub.status.idle": "2026-06-10T18:46:10.049140Z",
     "shell.execute_reply": "2026-06-10T18:46:10.048687Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] all four routes agree to 1e-6\n"
     ]
    }
   ],
   "source": [
    "# every claim that can be an assert is one: all four routes solve the same problem\n",
    "for name, t in [(\"lstsq\", theta_lstsq), (\"pinv\", theta_pinv), (\"sklearn\", theta_sklearn)]:\n",
    "    assert np.allclose(theta_inv, t, atol=1e-6), f\"{name} disagrees with the explicit inverse\"\n",
    "print(\"[ ok ] all four routes agree to 1e-6\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e65a31b0",
   "metadata": {},
   "source": [
    "> **Common confusion:** \"If they all give the same answer, why have four?\" Because they fail differently. `inv` raises on singular matrices and is the least numerically stable. `lstsq` and `pinv` degrade gracefully through the SVD. For real data, prefer `pinv` or a library `LinearRegression`; reach for the explicit inverse only when you want to see the formula run.\n",
    "\n",
    "> **Key takeaways.** The normal equation is one line of linear algebra. On a small, well-conditioned problem every solver agrees to machine precision. The recovered parameters miss the true `(4, 3)` only by the noise. This fitted line is the oracle: every iterative method in this notebook must converge back to it.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8580326e",
   "metadata": {},
   "source": [
    "## Part 2 — Batch gradient descent from scratch\n",
    "\n",
    "> **Objectives.** Derive the MSE gradient, write the update loop, and prove it converges to the Part 1 closed form. The agreement assert is the point: verify your optimizer on a problem with a known answer before you trust it on one without.\n",
    "\n",
    "The gradient of the MSE with respect to $\\boldsymbol\\theta$ is\n",
    "\n",
    "$$\\nabla_{\\boldsymbol\\theta}\\,\\text{MSE}(\\boldsymbol\\theta) = \\frac{2}{m}\\,\\mathbf{X}^T(\\mathbf{X}\\boldsymbol\\theta - \\mathbf{y}),$$\n",
    "\n",
    "and **batch gradient descent** takes a step against it using the full training set every iteration:\n",
    "\n",
    "$$\\boldsymbol\\theta \\leftarrow \\boldsymbol\\theta - \\eta\\,\\nabla_{\\boldsymbol\\theta}\\,\\text{MSE}(\\boldsymbol\\theta).$$\n",
    "\n",
    "Here $\\eta$ is the **learning rate**. The factor `2 / m` comes straight from differentiating the squared-error average; keep it, because it sets the scale that makes $\\eta = 0.1$ a reasonable step here.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df002554",
   "metadata": {},
   "source": [
    "Before the loop, trace a single step. Start from $\\boldsymbol\\theta = (0, 0)$, compute the gradient, and confirm the first step moves the intercept up toward 4 (the residuals are mostly positive because the line starts at zero and the data sits around `y ≈ 7`).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "f2dc2022",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.049909Z",
     "iopub.status.busy": "2026-06-10T18:46:10.049839Z",
     "iopub.status.idle": "2026-06-10T18:46:10.052132Z",
     "shell.execute_reply": "2026-06-10T18:46:10.051711Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gradient at theta=(0,0): [-14.42122117 -17.97086736]\n",
      "first step theta: [1.44212212 1.79708674] -> intercept moved up toward 4\n"
     ]
    }
   ],
   "source": [
    "theta0 = np.zeros((2, 1))\n",
    "grad0 = 2 / m * X_b.T @ (X_b @ theta0 - y)\n",
    "print(\"gradient at theta=(0,0):\", grad0.ravel())\n",
    "print(\"first step theta:\", (theta0 - 0.1 * grad0).ravel(), \"-> intercept moved up toward 4\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c4a881d",
   "metadata": {},
   "source": [
    "> **Interpretation.** Both gradient components are negative, so subtracting $\\eta$ times the gradient increases both the intercept and the slope on step one. The optimizer is walking downhill on the loss surface toward the closed-form answer.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d5e488e",
   "metadata": {},
   "source": [
    "### Exercise 4.2 — Batch gradient descent\n",
    "`Difficulty 2/5 · ~15 min`\n",
    "\n",
    "Fill in `batch_gd(X_b, y, eta, n_steps)`. Initialize `theta` at zeros, run `n_steps` full-batch updates, and record the MSE after each step in `losses`. Return `(theta, losses)`. The check asserts that with `eta=0.1` and enough steps your `theta` matches the Part 1 normal equation, and that the loss decreased monotonically.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "123163b5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.052931Z",
     "iopub.status.busy": "2026-06-10T18:46:10.052858Z",
     "iopub.status.idle": "2026-06-10T18:46:10.057147Z",
     "shell.execute_reply": "2026-06-10T18:46:10.056800Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] 4.2 GD == normal eq: not attempted yet — fill in the TODO above, then re-run.\n",
      "[ -- ] 4.2 loss decreases: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def batch_gd(X_b, y, eta=0.1, n_steps=5000):\n",
    "    \"\"\"Full-batch GD on MSE. Returns (theta of shape (d+1,1), list of per-step MSE).\"\"\"\n",
    "    theta = np.zeros((X_b.shape[1], 1))\n",
    "    losses = []\n",
    "    for _ in range(n_steps):\n",
    "        # TODO 1: gradient = (2/m) * X_b^T @ (X_b @ theta - y), where m = X_b.shape[0]\n",
    "        gradient = None\n",
    "        attempted(gradient)\n",
    "        # TODO 2: theta <- theta - eta * gradient\n",
    "        theta = theta - eta * gradient\n",
    "        losses.append(float(np.mean((X_b @ theta - y) ** 2)))\n",
    "    return theta, losses\n",
    "\n",
    "def _gd_matches_normal_eq():\n",
    "    theta_gd, _ = batch_gd(X_b, y, eta=0.1, n_steps=GD_STEPS)\n",
    "    ref = np.linalg.lstsq(X_b, y, rcond=None)[0]\n",
    "    check_close(theta_gd, ref, atol=5e-2,\n",
    "                msg=\"batch GD must converge to the closed-form least-squares answer\")\n",
    "\n",
    "def _gd_loss_decreases():\n",
    "    _, losses = batch_gd(X_b, y, eta=0.1, n_steps=GD_STEPS)\n",
    "    assert losses[-1] < losses[0], \"final loss should be far below the initial loss\"\n",
    "    assert all(b <= a + 1e-9 for a, b in zip(losses, losses[1:])), \\\n",
    "        \"with a sane learning rate, full-batch MSE decreases every step\"\n",
    "\n",
    "check(\"4.2 GD == normal eq\", _gd_matches_normal_eq)\n",
    "check(\"4.2 loss decreases\", _gd_loss_decreases)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6d8a7ca",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>The gradient formula is given in the lead-in. `m` is the number of rows, `X_b.shape[0]`. The update subtracts `eta` times that gradient.</details>\n",
    "\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "m = X_b.shape[0]\n",
    "gradient = 2 / m * X_b.T @ (X_b @ theta - y)\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — \"loss decreases\" fails or the loss grows</summary>If the loss increases, your gradient sign is flipped (you wrote `X_b @ theta - y` as `y - X_b @ theta`, or you added instead of subtracted in the update). The residual is `prediction - target`, i.e. `X_b @ theta - y`.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "087f1e9e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.057928Z",
     "iopub.status.busy": "2026-06-10T18:46:10.057815Z",
     "iopub.status.idle": "2026-06-10T18:46:10.170069Z",
     "shell.execute_reply": "2026-06-10T18:46:10.169618Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 4.2 GD == normal eq\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 4.2 loss decreases\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta (batch GD): [3.99254002 2.93463751]   final MSE 0.9401\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines batch_gd; the checks below re-verify the reference.\n",
    "def batch_gd(X_b, y, eta=0.1, n_steps=5000):\n",
    "    m = X_b.shape[0]\n",
    "    theta = np.zeros((X_b.shape[1], 1))\n",
    "    losses = []\n",
    "    for _ in range(n_steps):\n",
    "        gradient = 2 / m * X_b.T @ (X_b @ theta - y)\n",
    "        theta = theta - eta * gradient\n",
    "        losses.append(float(np.mean((X_b @ theta - y) ** 2)))\n",
    "    return theta, losses\n",
    "\n",
    "check(\"4.2 GD == normal eq\", _gd_matches_normal_eq, required=True)\n",
    "check(\"4.2 loss decreases\", _gd_loss_decreases, required=True)\n",
    "theta_gd, losses_gd = batch_gd(X_b, y, eta=0.1, n_steps=GD_STEPS)\n",
    "print(\"theta (batch GD):\", theta_gd.ravel(), f\"  final MSE {losses_gd[-1]:.4f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "4669f242",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.171034Z",
     "iopub.status.busy": "2026-06-10T18:46:10.170961Z",
     "iopub.status.idle": "2026-06-10T18:46:10.321882Z",
     "shell.execute_reply": "2026-06-10T18:46:10.321357Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAGGCAYAAACNCg6xAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAOM5JREFUeJzt3Xd8lFW+x/HvBEhCCEmAhMRAIEAQDELoMQIrLoHIsjQLyGVXBAtSRMXFtS2g6MXVixfUgKsssNhQEBEQkQBSlkWkt7DUKEgPJYUSSs79IzezjqE8gzOZ5Mnn/XqNyTzPyTO/OYB8Oec8ZxzGGCMAAABcl5+vCwAAACgtCE4AAAAWEZwAAAAsIjgBAABYRHACAACwiOAEAABgEcEJAADAIoITAACARQQnAAAAiwhOgIeMHj1aDodDmZmZvi5FkrRs2TI5HA7NmjXL16W4bdq0aXI4HPrhhx98XUoRubm5ql69uj766CNfl1JmPfvss0pMTPR1GSijCE5ACTZx4kRNmzbNpzVkZGRo6NChuvnmmxUUFKSgoCDFx8dryJAh2rJli0vbwvBY+AgKClKtWrXUtWtXTZ06VXl5eT56F54zYcIEVa5cWffff7/zWGHQu9LjyJEjRa4xd+5cNW/eXIGBgapVq5ZGjRqlS5cuFWl3+vRpPfroo4qIiFClSpV05513asOGDVesy+o1b8SCBQs0evRoj1zLE5588klt3rxZc+fO9XUpKIPK+7oAAFc3ceJEhYeH68EHH/TJ68+fP1+9e/dW+fLl1bdvXyUkJMjPz0///ve/NXv2bE2aNEkZGRmqXbu2y89NmjRJwcHBysvL08GDB/XNN99owIABGj9+vObPn6+YmBifvJ9f6+LFi5owYYKeeuoplStXrsj5l19+WXXq1HE5FhYW5vL866+/Vo8ePdS+fXu9/fbb2rp1q1555RUdO3ZMkyZNcrbLz89Xly5dtHnzZo0YMULh4eGaOHGi2rdvr/Xr16t+/fpuX/NGLViwQKmpqSUmPEVFRal79+76n//5H3Xr1s3X5aCsMQA8YtSoUUaSOX78uMeu2ahRI3PHHXfc0M9+++23RpKZOXPmDf38nj17TKVKlcwtt9xiDh06VOT8xYsXzYQJE8z+/fudx67VBx9++KHx8/MziYmJ133tqVOnGkkmIyPjhmr3ltmzZxtJZs+ePS7HC+tdu3btda8RHx9vEhISzMWLF53HXnjhBeNwOMyOHTucxz799NMiv37Hjh0zYWFhpk+fPjd0zRs1ZMgQU9L+upg1a5ZxOBxm7969vi4FZQxTdYCHZWZmqlevXgoJCVG1atX0xBNP6Pz58y5tpk6dqt/+9reqXr26AgICFB8fX2RkIDY2Vtu3b9fy5cud0z7t27d3nj99+rSeeuopxcbGKiAgQDVr1tQDDzxQZI1Vfn6+Xn31VdWsWVOBgYHq0KGD9uzZc9338frrr+vMmTOaOnWqbrrppiLny5cvr2HDhlkePerbt68efvhhrVmzRmlpaZZ+5pcmTpyoRo0aKSAgQNHR0RoyZIhOnz7t0mb37t265557FBUVpcDAQNWsWVP333+/srKynG3S0tLUtm1bhYWFKTg4WA0aNNDzzz9/3defM2eOYmNjVa9evau2ycnJ0eXLl694Lj09Xenp6Xr00UdVvvx/BvwHDx4sY4zLerRZs2YpMjJSd999t/NYRESEevXqpS+//NI57enONa/k4sWLeumll1S/fn0FBgaqWrVqatu2rfPX6MEHH1RqaqokuUxBFsrPz9f48ePVqFEjBQYGKjIyUgMHDtSpU6dcXic2Nla///3vtWjRIjVt2lSBgYGKj4/X7Nmz3aqnUHJysiTpyy+/vOb7AzyNqTrAw3r16qXY2FiNHTtW3333nd566y2dOnVK06dPd7aZNGmSGjVqpG7duql8+fKaN2+eBg8erPz8fA0ZMkSSNH78eD3++OMKDg7WCy+8IEmKjIyUVLBAuV27dtqxY4cGDBig5s2bKzMzU3PnztVPP/2k8PBw52u99tpr8vPz05/+9CdlZWXp9ddfV9++fbVmzZprvo/58+crLi7Oo4tw//jHP+q9997TokWL1LFjR7d+dvTo0XrppZeUnJysQYMGaefOnZo0aZLWrl2rVatWqUKFCrpw4YJSUlKUl5enxx9/XFFRUTp48KDmz5+v06dPKzQ0VNu3b9fvf/97NWnSRC+//LICAgK0Z88erVq16ro1/Otf/1Lz5s2vev7OO+9Ubm6u/P39lZKSonHjxrlMqW3cuFGS1LJlS5efi46OVs2aNZ3nC9s2b95cfn6u/75t3bq13nvvPe3atUuNGzd265pXMnr0aI0dO1YPP/ywWrdurezsbK1bt04bNmxQx44dNXDgQB06dEhpaWn64IMPivz8wIEDNW3aNPXv31/Dhg1TRkaG3nnnHW3cuNH561Jo9+7d6t27tx577DH169dPU6dO1X333aeFCxc6fz9cr55CoaGhqlevnlatWqWnnnrqmu8R8ChfD3kBdlE4TdWtWzeX44MHDzaSzObNm53Hzp49W+TnU1JSTN26dV2OXW2qbuTIkUaSmT17dpFz+fn5xpj/TNXdcsstJi8vz3l+woQJRpLZunXrVd9LVlaWkWR69OhR5NypU6fM8ePHnY+fv5frTVeeOnXKSDI9e/a86msbU3Sq7tixY8bf39906tTJXL582dnunXfeMZLMlClTjDHGbNy48brTk//7v/97Q1OqFy9eNA6Hwzz99NNFzn366afmwQcfNP/4xz/MF198YV588UUTFBRkwsPDXaYy33jjDSPJ5VihVq1amdtuu835vFKlSmbAgAFF2n311VdGklm4cKHb17yShIQE06VLl2u2udpU3cqVK40k89FHH7kcX7hwYZHjtWvXNpLM559/7jyWlZVlbrrpJtOsWTO36inUqVMnc8stt1hqC3gKU3WAhxWOGBV6/PHHJRUssC1UsWJF5/dZWVnKzMzUHXfcoX379rlMKV3N559/roSEBPXs2bPIuZ9Po0hS//795e/v73zerl07SdK+ffuuev3s7GxJUnBwcJFz7du3V0REhPNROI1jReH1cnJyLP+MJC1evFgXLlzQk08+6TIC88gjjygkJERfffWVpIJRCEn65ptvdPbs2Steq3Cx9pdffqn8/HzLNZw8eVLGGFWpUqXIuV69emnq1Kl64IEH1KNHD40ZM0bffPONTpw4oVdffdXZ7ty5c5KkgICAItcIDAx0ni9se7V2P7+WO9e8krCwMG3fvl27d+++ZrsrmTlzpkJDQ9WxY0dlZmY6Hy1atFBwcLC+/fZbl/bR0dEuv2dDQkL0wAMPaOPGjc67D92pp0qVKiVm+w+UHQQnwMN+PjUjSfXq1ZOfn5/LnkSrVq1ScnKyKlWqpLCwMEVERDjX2FgJTnv37tWtt95qqZ5atWq5PC/8i/+Xa1B+rnLlypIKpgR/6W9/+5vS0tL04YcfWnr9nyu8XuH1rfrxxx8lSQ0aNHA57u/vr7p16zrP16lTR8OHD9fkyZMVHh6ulJQUpaamuvRp79691aZNGz388MOKjIzU/fffr88++8xyiDLGWGrXtm1bJSYmavHixc5jhYH5StsynD9/3iVQV6xY8artfn4td655JS+//LJOnz6tm2++WY0bN9aIESOKbDNxNbt371ZWVpaqV6/uEqYjIiKUm5urY8eOubSPi4srEuxvvvlmSXL++XCnHmNMkesB3kZwArzsl/9j37t3rzp06KDMzEy9+eab+uqrr5SWluZcp+HOKIgVV7ptXrp2AAgNDdVNN92kbdu2FTmXmJio5ORktWnTxu1aCq8XFxfn9s9aNW7cOG3ZskXPP/+8zp07p2HDhqlRo0b66aefJBUEjRUrVmjx4sX64x//qC1btqh3797q2LHjVRd1S1LVqlXlcDiuGTh/KSYmRidPnnQ+L1xkf/jw4SJtDx8+rOjoaJe2V2snydnWnWteyW9+8xvt3btXU6ZM0a233qrJkyerefPmmjx58vXenvLz81W9enWlpaVd8fHyyy9f9xq/pp5Tp065rOcDigPBCfCwX04x7NmzR/n5+YqNjZUkzZs3T3l5eZo7d64GDhyo3/3ud0pOTr7iyMDV/jVdr169K4YaT+rSpYv27Nmj77//3mPXLFxcnJKS4tbPFe4TtXPnTpfjFy5cuOI+Uo0bN9aLL76oFStWaOXKlTp48KDeffdd53k/Pz916NBBb775ptLT0/Xqq69q6dKlRaaWfq58+fKqV6+eMjIyLNe9b98+RUREOJ83bdpUkrRu3TqXdocOHdJPP/3kPF/YdsOGDUWC9Jo1axQUFOQcqXHnmldTtWpV9e/fX5988okOHDigJk2auOzZdK3fhydOnFCbNm2UnJxc5JGQkODSfs+ePUUC+65duyTJ+efDSj2FMjIydMstt1z3/QGeRHACPOyXa37efvttSVLnzp0l/WcE6Od/gWRlZWnq1KlFrlWpUqUit9tL0j333KPNmzfriy++KHLO6lTS9TzzzDMKCgrSgAEDdPTo0V/9Oh9//LEmT56spKQkdejQwa2fTU5Olr+/v9566y2X1/373/+urKwsdenSRVLB2qxf7pbduHFj+fn5Oaeyfj4CVKgwXFxvZ/OkpKQiAUWSjh8/XuTYggULtH79et11113OY40aNVLDhg313nvvuYxuTZo0SQ6HQ/fee6/z2L333qujR4+63K6fmZmpmTNnqmvXrs41Te5c80pOnDjh8jw4OFhxcXEufVGpUiVJKvJ7sVevXrp8+bLGjBlT5LqXLl0q0v7QoUMuv2ezs7M1ffp0NW3aVFFRUZbrkQr+zOzdu1e33377Nd8f4GlsRwB4WEZGhrp166a77rpLq1ev1ocffqj/+q//cv7ru1OnTvL391fXrl01cOBA5ebm6v3331f16tWLTLe0aNFCkyZN0iuvvKK4uDhVr15dv/3tbzVixAjNmjVL9913nwYMGKAWLVro5MmTmjt3rt59990i/9K/EfXr19fHH3+sPn36qEGDBs6dw40xysjI0Mcffyw/Pz/VrFmzyM/OmjVLwcHBunDhgnPn8FWrVikhIUEzZ850u5aIiAg999xzeumll3TXXXepW7du2rlzpyZOnKhWrVrpD3/4gyRp6dKlGjp0qO677z7dfPPNunTpkj744AOVK1dO99xzj6SCNTQrVqxQly5dVLt2bR07dkwTJ05UzZo11bZt22vW0b17d33wwQfatWuXc8RHkm6//XY1a9ZMLVu2VGhoqDZs2KApU6YoJiamyP5Qb7zxhrp166ZOnTrp/vvv17Zt2/TOO+/o4Ycfdhk9uffee3Xbbbepf//+Sk9Pd+4cfvnyZb300ks3dM0riY+PV/v27dWiRQtVrVpV69at06xZszR06FBnmxYtWkiShg0bppSUFJUrV07333+/7rjjDg0cOFBjx47Vpk2b1KlTJ1WoUEG7d+/WzJkzNWHCBJfgdvPNN+uhhx7S2rVrFRkZqSlTpujo0aMu/2iwUo9UcMOAMUbdu3e/5vsDPM5Hd/MBtlN4K356erq59957TeXKlU2VKlXM0KFDzblz51zazp071zRp0sQEBgaa2NhY89e//tVMmTKlyG7ZR44cMV26dDGVK1c2kly2Jjhx4oQZOnSoqVGjhvH39zc1a9Y0/fr1M5mZmcaYq+8cnpGRYSSZqVOnWnpfe/bsMYMGDTJxcXEmMDDQVKxY0TRs2NA89thjZtOmTVfsg8JHYGCgqVmzpvn9739vpkyZYs6fP2/pNa+2c/g777xjGjZsaCpUqGAiIyPNoEGDzKlTp5zn9+3bZwYMGGDq1atnAgMDTdWqVc2dd95pFi9e7GyzZMkS0717dxMdHW38/f1NdHS06dOnj9m1a9d168rLyzPh4eFmzJgxLsdfeOEF07RpUxMaGmoqVKhgatWqZQYNGmSOHDlyxet88cUXpmnTpiYgIMDUrFnTvPjii+bChQtF2p08edI89NBDplq1aiYoKMjccccdV92d3Oo1f+mVV14xrVu3NmFhYc5f21dffdXlZy9dumQef/xxExERYRwOR5GtCd577z3TokULU7FiRVO5cmXTuHFj88wzz7jsOF+7dm3TpUsX880335gmTZqYgIAA07BhwyK/P63UY4wxvXv3Nm3btr3u+wM8zWGMh8b1AaAMGDNmjKZOnardu3dfdeE9ioqNjdWtt96q+fPn/+prHTlyRHXq1NGMGTMYcUKxY40TALjhqaeeUm5urmbMmOHrUsqs8ePHq3HjxoQm+AQjTgAAr/PkiBPgS4w4AQAAWMSIEwAAgEWMOAEAAFhEcAIAALCozG+AmZ+fr0OHDqly5cp8WCQAAGWQMUY5OTmKjo6Wn9+1x5TKfHA6dOiQYmJifF0GAADwsQMHDlzx0xB+rswHp8qVK0sq6KyQkBAfVwMAAIpbdna2YmJinJngWsp8cCqcngsJCSE4AQBQhllZssPicAAAAIsITgAAABYRnAAAACwiOAEAAFhEcAIAALCI4AQAAGARwQkAAMAighMAAIBFBCcAAACLCE4AAAAWEZy85Nx5qc39BY9z531dDQAA8IQy/1l13mKM9NPR/3wPAABKP0acAAAALCI4FQMGnAAAsAeCk5c4HL6uAAAAeBrBCQAAwCKCUzFgcTgAAPZAcPISpuoAALAfghMAAIBFBKdiwFQdAAD2QHDyFqbqAACwHYITAACARQQnAAAAiwhOxYA1TgAA2APByUtY4gQAgP0QnAAAACwiOBUDZuoAALAHgpOXsHM4AAD2Q3ACAACwiOBUHJirAwDAFghOXsJUHQAA9kNwAgAAsIjgBAAAYBHByUt+PlXHzuEAANgDwQkAAMAighMAAIBFBKdiwFQdAAD2QHDyEnYjAADAfghOAAAAFhGcigEzdQAA2APByUvYORwAAPshOAEAAFhEcCoG3FUHAIA9EJy8hKk6AADsh+AEAABgEcEJAADAIoKTl/AhvwAA2A/BCQAAwCKCEwAAgEUEp2LAVB0AAPZgi+DUs2dPValSRffee6+vSwEAADZmi+D0xBNPaPr06b4uAwAA2JwtglP79u1VuXJlX5dxVczUAQBgDz4PTitWrFDXrl0VHR0th8OhOXPmFGmTmpqq2NhYBQYGKjExUd9//33xF3oD2D0cAAB78XlwOnPmjBISEpSamnrF859++qmGDx+uUaNGacOGDUpISFBKSoqOHTtWzJUCAICyrryvC+jcubM6d+581fNvvvmmHnnkEfXv31+S9O677+qrr77SlClT9Oyzz7r9enl5ecrLy3M+z87Odr9oAABQJvl8xOlaLly4oPXr1ys5Odl5zM/PT8nJyVq9evUNXXPs2LEKDQ11PmJiYjxVbhGFU3VsRwAAgD2U6OCUmZmpy5cvKzIy0uV4ZGSkjhw54nyenJys++67TwsWLFDNmjWvGaqee+45ZWVlOR8HDhzwWv0AAMBefD5V5wmLFy+23DYgIEABAQFerAYAANhViR5xCg8PV7ly5XT06FGX40ePHlVUVJSPqrLOeVMdU3UAANhCiQ5O/v7+atGihZYsWeI8lp+fryVLligpKcmHlQEAgLLI51N1ubm52rNnj/N5RkaGNm3apKpVq6pWrVoaPny4+vXrp5YtW6p169YaP368zpw547zLDgAAoLj4PDitW7dOd955p/P58OHDJUn9+vXTtGnT1Lt3bx0/flwjR47UkSNH1LRpUy1cuLDIgvGSiLvqAACwF4cxZfuv9ezsbIWGhiorK0shISEevXa9ZOnSZWnNZ1JUhEcvDQAAPMSdLFCi1zgBAACUJASnYlCmh/QAALARgpMX8SG/AADYS5kNTqmpqYqPj1erVq18XQoAACglymxwGjJkiNLT07V27VpflwIAAEqJMhucigPbEQAAYC8EJwAAAIsITgAAABYRnLyo8KY6puoAALAHghMAAIBFBCcAAACLCE7exF11AADYCsEJAADAIoITAACARWU2OBXnR64wUwcAgD2U2eBUHB+5wof8AgBgL2U2OAEAALiL4AQAAGARwcmL+JBfAADsheAEAABgEcEJAADAIoKTFzlvqmOqDgAAWyA4AQAAWERwAgAAsIjg5EXcVQcAgL0QnAAAACwiOAEAAFhUZoNTcXzIL59VBwCAvZTZ4FQcH/JbiCVOAADYQ5kNTgAAAO4iOAEAAFhEcPIitiMAAMBeCE4AAAAWEZwAAAAsIjh5UeFuBEzVAQBgD+Xdabxjxw7NmDFDK1eu1I8//qizZ88qIiJCzZo1U0pKiu655x4FBAR4q1YAAACfsjTitGHDBiUnJ6tZs2b65z//qcTERD355JMaM2aM/vCHP8gYoxdeeEHR0dH661//qry8PG/XDQAAUOwsjTjdc889GjFihGbNmqWwsLCrtlu9erUmTJigcePG6fnnn/dUjaUXd9UBAGArloLTrl27VKFCheu2S0pKUlJSki5evPirCwMAAChpLE3VXS00nT9/3q32AAAApZnbd9Xl5+drzJgxqlGjhoKDg7Vv3z5J0l/+8hf9/e9/93iBpRkf8gsAgL24HZxeeeUVTZs2Ta+//rr8/f2dx2+99VZNnjzZo8XZBUucAACwB7eD0/Tp0/Xee++pb9++KleunPN4QkKC/v3vf3u0OG9KTU1VfHy8WrVq5etSAABAKeF2cDp48KDi4uKKHM/Pzy9Vi8KHDBmi9PR0rV271telAACAUsLt4BQfH6+VK1cWOT5r1iw1a9bMI0XZBR/yCwCAvbi1c7gkjRw5Uv369dPBgweVn5+v2bNna+fOnZo+fbrmz5/vjRoBAABKBLdHnLp376558+Zp8eLFqlSpkkaOHKkdO3Zo3rx56tixozdqBAAAKBHcHnGSpHbt2iktLc3TtdgOH/ILAIC9uD3iBAAAUFZZGnGqUqWKHBZ3czx58uSvKggAAKCkshScxo8f7+Uy7ImdwwEAsBdLwalfv37ersPeWOMEAIAt3NDi8ELnz5/XhQsXXI6FhIT8qoIAAABKKrcXh585c0ZDhw5V9erVValSJVWpUsXlgf9gqg4AAHtxOzg988wzWrp0qSZNmqSAgABNnjxZL730kqKjozV9+nRv1FjqMVMHAIA9uD1VN2/ePE2fPl3t27dX//791a5dO8XFxal27dr66KOP1LdvX2/UCQAA4HNujzidPHlSdevWlVSwnqlw+4G2bdtqxYoVnq0OAACgBHE7ONWtW1cZGRmSpIYNG+qzzz6TVDASFRYW5tHiSjs+5BcAAHtxOzj1799fmzdvliQ9++yzSk1NVWBgoJ566imNGDHC4wUCAACUFG6vcXrqqaec3ycnJ+vf//631q9fr7i4ODVp0sSjxQEAAJQkv2ofJ0mqXbu2ateu7YlailVqaqpSU1N1+fJlr70GH/ILAIC9uD1VN2zYML311ltFjr/zzjt68sknPVFTsRgyZIjS09O1du1aX5cCAABKCbeD0+eff642bdoUOX777bdr1qxZHikKAACgJHI7OJ04cUKhoaFFjoeEhCgzM9MjRdkGO4cDAGArbgenuLg4LVy4sMjxr7/+2rm/E1yxxgkAAHtwe3H48OHDNXToUB0/fly//e1vJUlLlizRuHHjNH78eE/XBwAAUGK4HZwGDBigvLw8vfrqqxozZowkKTY2VpMmTdIDDzzg8QJLMz7kFwAAe7mh7QgGDRqkQYMG6fjx46pYsaKCg4M9XZetMFMHAIA9uL3G6dy5czp79qwkKSIiQidOnND48eO1aNEijxcHAABQkrgdnLp3767p06dLkk6fPq3WrVtr3Lhx6t69uyZNmuTxAgEAAEoKt4PThg0b1K5dO0nSrFmzFBUVpR9//FHTp0+/4saYZRkf8gsAgL24HZzOnj2rypUrS5IWLVqku+++W35+frrtttv0448/erxAAACAkuKG9nGaM2eODhw4oG+++UadOnWSJB07dkwhISEeLxAAAKCkcDs4jRw5Un/6058UGxurxMREJSUlSSoYfWrWrJnHCyzN2I0AAAB7cXs7gnvvvVdt27bV4cOHlZCQ4DzeoUMH9ezZ06PF2QVrnAAAsIcb2scpKipKUVFRLsdat27tkYIAAABKKren6mAdO4cDAGAvBKfiwFQdAAC2QHACAACwiODkRUzVAQBgL24vDp87d+4VjzscDgUGBiouLk516tT51YXZCTN1AADYg9vBqUePHnI4HDK/uMe+8JjD4VDbtm01Z84cValSxWOFAgAA+JrbU3VpaWlq1aqV0tLSlJWVpaysLKWlpSkxMVHz58/XihUrdOLECf3pT3/yRr0ek5qaqvj4eLVq1cprr8FUHQAA9uIwvxw6uo5bb71V7733nm6//XaX46tWrdKjjz6q7du3a/HixRowYID279/v0WK9ITs7W6GhocrKyvL4R8bcfr908Kg0d5KU0NCjlwYAAB7iThZwe8Rp7969V7xoSEiI9u3bJ0mqX7++MjMz3b00AABAieZ2cGrRooVGjBih48ePO48dP35czzzzjHPaa/fu3YqJifFclQAAACWA24vD//73v6t79+6qWbOmMxwdOHBAdevW1ZdffilJys3N1YsvvujZSkshljgBAGAvbgenBg0aKD09XYsWLdKuXbucxzp27Cg/v4IBrB49eni0yNKOD/kFAMAebuhDfv38/HTXXXfprrvu8nQ9AAAAJdYN7Ry+fPlyde3aVXFxcYqLi1O3bt20cuVKT9dW+jFXBwCArbgdnD788EMlJycrKChIw4YN07BhwxQYGKgOHTro448/9kaNpR5TdQAA2IPb+zjdcsstevTRR/XUU0+5HH/zzTf1/vvva8eOHR4t0Nu8uY9Tmz7ST0ekOalSs3iPXhoAAHiIV/dx2rdvn7p27VrkeLdu3ZSRkeHu5WyNncMBALAXt4NTTEyMlixZUuT44sWL2bvpKpipAwDAHty+q+7pp5/WsGHDtGnTJufHrqxatUrTpk3ThAkTPF4gAABASeF2cBo0aJCioqI0btw4ffbZZ5IK1j19+umn6t69u8cLLM2YqgMAwF5uaB+nnj17qmfPnp6uxba4qw4AAHu4oX2cAAAAyiJLI05VqlSRw+K808mTJ39VQQAAACWVpeA0fvx4L5dhTyxxAgDAXiwFp379+nm7DltjjRMAAPZgaY3TmTNn3Lqou+0BAABKA0vBKS4uTq+99poOHz581TbGGKWlpalz58566623PFZgacZ2BAAA2Iulqbply5bp+eef1+jRo5WQkKCWLVsqOjpagYGBOnXqlNLT07V69WqVL19ezz33nAYOHOjtuksVpuoAALAHS8GpQYMG+vzzz7V//37NnDlTK1eu1L/+9S+dO3dO4eHhatasmd5//3117txZ5cqV83bNAAAAPuHWBpi1atXS008/raefftpb9dgKU3UAANgLG2ACAABYRHACAACwiODkRUzVAQBgLwQnAAAAi8pscEpNTVV8fLxatWrl9ddiOwIAAOzBcnB6/fXXde7cOefzVatWKS8vz/k8JydHgwcP9mx1XjRkyBClp6dr7dq1vi4FAACUEpaD03PPPaecnBzn886dO+vgwYPO52fPntXf/vY3z1ZXyrHECQAAe7EcnMwv5pt++RxXR1cBAGAPZXaNEwAAgLsITt7EXB0AALbi1keuTJ48WcHBwZKkS5cuadq0aQoPD5ckl/VPcMVUHQAA9mA5ONWqVUvvv/++83lUVJQ++OCDIm0AAADsynJw+uGHH7xYhj2xczgAAPbCGqdiwEwdAAD2YDk4rV69WvPnz3c5Nn36dNWpU0fVq1fXo48+6rIhJgAAgN1YDk4vv/yytm/f7ny+detWPfTQQ0pOTtazzz6refPmaezYsV4psrRipg4AAHuxHJw2bdqkDh06OJ/PmDFDiYmJev/99zV8+HC99dZb+uyzz7xSJAAAQElgOTidOnVKkZGRzufLly9X586dnc9btWqlAwcOeLY6m2A7AgAA7MFycIqMjFRGRoYk6cKFC9qwYYNuu+025/mcnBxVqFDB8xUCAACUEJaD0+9+9zs9++yzWrlypZ577jkFBQWpXbt2zvNbtmxRvXr1vFJkacV2BAAA2IvlfZzGjBmju+++W3fccYeCg4P1j3/8Q/7+/s7zU6ZMUadOnbxSZGnHVB0AAPZgOTiFh4drxYoVysrKUnBwsMqVK+dyfubMmc6PYwEAALAjtz6rTpJCQ0OveLxq1aq/uhi7YaoOAAB7sRycBgwYYKndlClTbrgYu2KqDgAAe7AcnKZNm6batWurWbNmMiQBAABQBlkOToMGDdInn3yijIwM9e/fX3/4wx+YnrsOpuoAALAXy9sRpKam6vDhw3rmmWc0b948xcTEqFevXvrmm28YgQIAAGWC5eAkSQEBAerTp4/S0tKUnp6uRo0aafDgwYqNjVVubq63agQAACgR3ApOLj/o5yeHwyFjjC5fvuzJmmyDmToAAOzFreCUl5enTz75RB07dtTNN9+srVu36p133tH+/fvZwwkAANie5cXhgwcP1owZMxQTE6MBAwbok08+UXh4uDdrsw2WgAEAYA+Wg9O7776rWrVqqW7dulq+fLmWL19+xXazZ8/2WHGlHXfVAQBgL5aD0wMPPCAHSQAAAJRhbm2AiRvDVB0AAPZww3fVAQAAlDUEJ29iZhMAAFshOBUDpuoAALAHghMAAIBFBCcv4iZEAADsheAEAABgEcGpGLDECQAAeyA4eREzdQAA2AvBCQAAwCKCUzFgOwIAAOyhzAan1NRUxcfHq1WrVl57De6qAwDAXspscBoyZIjS09O1du1aX5cCAABKiTIbnIoTU3UAANgDwQkAAMAigpMXscYJAAB7ITgVA6bqAACwB4ITAACARQQnL2KqDgAAeyE4AQAAWERwAgAAsIjg5EXM1AEAYC8EJwAAAIsITsWA7QgAALAHgpMXcVcdAAD2QnACAACwiOBUDJiqAwDAHghOAAAAFhGcvIk1TgAA2ArBCQAAwCKCUzFgjRMAAPZAcPIitiMAAMBeCE4AAAAWEZyKATN1AADYA8HJi5ipAwDAXghOAAAAFhGcigF31QEAYA8EJy/irjoAAOyF4AQAAGARwakYMFUHAIA9EJwAAAAsIjh5EWucAACwF4ITAACARQQnLyocccpnjRMAALZAcPIiv8KpOoITAAC2QHDyIsf/9+7lfN/WAQAAPIPg5EXl/r938wlOAADYAsHJi/xY4wQAgK0QnLyocKrOMOIEAIAtEJy8qHDEiTVOAADYA8HJi8qVK/jKGicAAOyB4ORFhSNOfFYdAAD2QHDyIgd31QEAYCsEJy9ijRMAAPZCcPIitiMAAMBeCE5eVLg4nO0IAACwB4KTF/EhvwAA2AvByYv8+Kw6AABsheDkRc41TgQnAABsgeDkRYUjTuzjBACAPRCcvMiPfZwAALAVgpMXsR0BAAD2QnDyIueI02Xf1gEAADyD4ORFzuDEiBMAALZAcPIitiMAAMBeCE5eVKliwdez53xbBwAA8AyCkxeFBBd8zc71bR0AAMAzCE5eFFKp4CvBCQAAeyA4eVHhiFMWwQkAAFsgOHlRldCCrydO+bYOAADgGQQnL6oRWfD14FE+dgUAADsgOHlRdPWCr2fPS6eyfVsLAAD49QhOXhTgL0WGF3z/0xHf1gIAAH49WwSn+fPnq0GDBqpfv74mT57s63Jc1Pz/6boDh31bBwAA+PVKfXC6dOmShg8frqVLl2rjxo164403dOLECV+X5RRzU8HXHw76tg4AAPDrlfrg9P3336tRo0aqUaOGgoOD1blzZy1atMjXZTk1qFPw9d/7fFsHAAD49XwenFasWKGuXbsqOjpaDodDc+bMKdImNTVVsbGxCgwMVGJior7//nvnuUOHDqlGjRrO5zVq1NDBgyVneCc+ruDr9j2+rQMAAPx6Pg9OZ86cUUJCglJTU694/tNPP9Xw4cM1atQobdiwQQkJCUpJSdGxY8eKudIbc0u9gq8ZP/GZdQAAlHY+D06dO3fWK6+8op49e17x/JtvvqlHHnlE/fv3V3x8vN59910FBQVpypQpkqTo6GiXEaaDBw8qOjr6qq+Xl5en7Oxsl4c3Va9asC1Bfr60dqtXXwoAAHiZz4PTtVy4cEHr169XcnKy85ifn5+Sk5O1evVqSVLr1q21bds2HTx4ULm5ufr666+VkpJy1WuOHTtWoaGhzkdMTIxX34PDIbVrWfD98rVefSkAAOBlJTo4ZWZm6vLly4qMjHQ5HhkZqSNHCjZGKl++vMaNG6c777xTTZs21dNPP61q1apd9ZrPPfecsrKynI8DBw549T1IUvvWBV+/WiZduuz1lwMAAF5S3tcFeEK3bt3UrVs3S20DAgIUEBDg5YpcdUiSqoVJRzKleUulnh2L9eUBAICHlOgRp/DwcJUrV05Hjx51OX706FFFRUX5qCr3BfhL/e8p+H7MRGn3j76tBwAA3JgSPeLk7++vFi1aaMmSJerRo4ckKT8/X0uWLNHQoUN9W5ybHu0lLVgupe+Rfj9Q6pEsNW1YsHA8qKIUGCCVL1fQ1uEoeEiSw/kf1+MAAJQ1lStJkVdfjVMsfB6ccnNztWfPfzY5ysjI0KZNm1S1alXVqlVLw4cPV79+/dSyZUu1bt1a48eP15kzZ9S/f38fVu2+AH/pwzekwS9J322SZnxV8AAAANb06iy98Yxva/B5cFq3bp3uvPNO5/Phw4dLkvr166dp06apd+/eOn78uEaOHKkjR46oadOmWrhwYZEF46VBtTBpxpvSvzZKS1ZLe/ZLx09I5/Kks+el/MuSkWRMQXtjfvb9z44DAFAWBQX6ugLJYUzZ/us4OztboaGhysrKUkhIiK/LAQAAxcydLFCiF4cDAACUJAQnAAAAi8pscEpNTVV8fLxatWrl61IAAEApwRon1jgBAFCmscYJAADACwhOAAAAFhGcAAAALCI4AQAAWERwAgAAsIjgBAAAYBHBCQAAwCKff8ivrxVuY5Wdne3jSgAAgC8UZgArW1uW+eCUk5MjSYqJifFxJQAAwJdycnIUGhp6zTZlfufw/Px8HTp0SJUrV5bD4fDotbOzsxUTE6MDBw6wK3kxot+LH33uG/S7b9DvvuHNfjfGKCcnR9HR0fLzu/YqpjI/4uTn56eaNWt69TVCQkL4w+UD9Hvxo899g373DfrdN7zV79cbaSrE4nAAAACLCE4AAAAWEZy8KCAgQKNGjVJAQICvSylT6PfiR5/7Bv3uG/S7b5SUfi/zi8MBAACsYsQJAADAIoITAACARQQnAAAAiwhOXpSamqrY2FgFBgYqMTFR33//va9LKjVWrFihrl27Kjo6Wg6HQ3PmzHE5b4zRyJEjddNNN6lixYpKTk7W7t27XdqcPHlSffv2VUhIiMLCwvTQQw8pNzfXpc2WLVvUrl07BQYGKiYmRq+//rq331qJNXbsWLVq1UqVK1dW9erV1aNHD+3cudOlzfnz5zVkyBBVq1ZNwcHBuueee3T06FGXNvv371eXLl0UFBSk6tWra8SIEbp06ZJLm2XLlql58+YKCAhQXFycpk2b5u23V2JNmjRJTZo0ce5Nk5SUpK+//tp5nj73vtdee00Oh0NPPvmk8xj97nmjR4+Ww+FweTRs2NB5vtT0uYFXzJgxw/j7+5spU6aY7du3m0ceecSEhYWZo0eP+rq0UmHBggXmhRdeMLNnzzaSzBdffOFy/rXXXjOhoaFmzpw5ZvPmzaZbt26mTp065ty5c842d911l0lISDDfffedWblypYmLizN9+vRxns/KyjKRkZGmb9++Ztu2beaTTz4xFStWNH/729+K622WKCkpKWbq1Klm27ZtZtOmTeZ3v/udqVWrlsnNzXW2eeyxx0xMTIxZsmSJWbdunbntttvM7bff7jx/6dIlc+utt5rk5GSzceNGs2DBAhMeHm6ee+45Z5t9+/aZoKAgM3z4cJOenm7efvttU65cObNw4cJifb8lxdy5c81XX31ldu3aZXbu3Gmef/55U6FCBbNt2zZjDH3ubd9//72JjY01TZo0MU888YTzOP3ueaNGjTKNGjUyhw8fdj6OHz/uPF9a+pzg5CWtW7c2Q4YMcT6/fPmyiY6ONmPHjvVhVaXTL4NTfn6+iYqKMm+88Ybz2OnTp01AQID55JNPjDHGpKenG0lm7dq1zjZff/21cTgc5uDBg8YYYyZOnGiqVKli8vLynG3+/Oc/mwYNGnj5HZUOx44dM5LM8uXLjTEFfVyhQgUzc+ZMZ5sdO3YYSWb16tXGmILA6+fnZ44cOeJsM2nSJBMSEuLs52eeecY0atTI5bV69+5tUlJSvP2WSo0qVaqYyZMn0+delpOTY+rXr2/S0tLMHXfc4QxO9Lt3jBo1yiQkJFzxXGnqc6bqvODChQtav369kpOTncf8/PyUnJys1atX+7Aye8jIyNCRI0dc+jc0NFSJiYnO/l29erXCwsLUsmVLZ5vk5GT5+flpzZo1zja/+c1v5O/v72yTkpKinTt36tSpU8X0bkqurKwsSVLVqlUlSevXr9fFixdd+r1hw4aqVauWS783btxYkZGRzjYpKSnKzs7W9u3bnW1+fo3CNvzZkC5fvqwZM2bozJkzSkpKos+9bMiQIerSpUuRvqHfvWf37t2Kjo5W3bp11bdvX+3fv19S6epzgpMXZGZm6vLlyy6/uJIUGRmpI0eO+Kgq+yjsw2v175EjR1S9enWX8+XLl1fVqlVd2lzpGj9/jbIqPz9fTz75pNq0aaNbb71VUkGf+Pv7KywszKXtL/v9en16tTbZ2dk6d+6cN95Oibd161YFBwcrICBAjz32mL744gvFx8fT5140Y8YMbdiwQWPHji1yjn73jsTERE2bNk0LFy7UpEmTlJGRoXbt2iknJ6dU9XmZ/5BfAEUNGTJE27Zt0z//+U9fl1ImNGjQQJs2bVJWVpZmzZqlfv36afny5b4uy7YOHDigJ554QmlpaQoMDPR1OWVG586dnd83adJEiYmJql27tj777DNVrFjRh5W5hxEnLwgPD1e5cuWK3A1w9OhRRUVF+agq+yjsw2v1b1RUlI4dO+Zy/tKlSzp58qRLmytd4+evURYNHTpU8+fP17fffquaNWs6j0dFRenChQs6ffq0S/tf9vv1+vRqbUJCQkrV/zw9yd/fX3FxcWrRooXGjh2rhIQETZgwgT73kvXr1+vYsWNq3ry5ypcvr/Lly2v58uV66623VL58eUVGRtLvxSAsLEw333yz9uzZU6p+rxOcvMDf318tWrTQkiVLnMfy8/O1ZMkSJSUl+bAye6hTp46ioqJc+jc7O1tr1qxx9m9SUpJOnz6t9evXO9ssXbpU+fn5SkxMdLZZsWKFLl686GyTlpamBg0aqEqVKsX0bkoOY4yGDh2qL774QkuXLlWdOnVczrdo0UIVKlRw6fedO3dq//79Lv2+detWl9CalpamkJAQxcfHO9v8/BqFbfiz8R/5+fnKy8ujz72kQ4cO2rp1qzZt2uR8tGzZUn379nV+T797X25urvbu3aubbrqpdP1e99gyc7iYMWOGCQgIMNOmTTPp6enm0UcfNWFhYS53A+DqcnJyzMaNG83GjRuNJPPmm2+ajRs3mh9//NEYU7AdQVhYmPnyyy/Nli1bTPfu3a+4HUGzZs3MmjVrzD//+U9Tv359l+0ITp8+bSIjI80f//hHs23bNjNjxgwTFBRUZrcjGDRokAkNDTXLli1zuV347NmzzjaPPfaYqVWrllm6dKlZt26dSUpKMklJSc7zhbcLd+rUyWzatMksXLjQREREXPF24REjRpgdO3aY1NTUMn2L9rPPPmuWL19uMjIyzJYtW8yzzz5rHA6HWbRokTGGPi8uP7+rzhj63Ruefvpps2zZMpORkWFWrVplkpOTTXh4uDl27JgxpvT0OcHJi95++21Tq1Yt4+/vb1q3bm2+++47X5dUanz77bdGUpFHv379jDEFWxL85S9/MZGRkSYgIMB06NDB7Ny50+UaJ06cMH369DHBwcEmJCTE9O/f3+Tk5Li02bx5s2nbtq0JCAgwNWrUMK+99lpxvcUS50r9LclMnTrV2ebcuXNm8ODBpkqVKiYoKMj07NnTHD582OU6P/zwg+ncubOpWLGiCQ8PN08//bS5ePGiS5tvv/3WNG3a1Pj7+5u6deu6vEZZM2DAAFO7dm3j7+9vIiIiTIcOHZyhyRj6vLj8MjjR757Xu3dvc9NNNxl/f39To0YN07t3b7Nnzx7n+dLS5w5jjPHc+BUAAIB9scYJAADAIoITAACARQQnAAAAiwhOAAAAFhGcAAAALCI4AQAAWERwAgAAsIjgBAAAYBHBCQAAwCKCE4Ay4cEHH1SPHj18XQaAUo7gBAAAYBHBCYCtzJo1S40bN1bFihVVrVo1JScna8SIEfrHP/6hL7/8Ug6HQw6HQ8uWLZMkHThwQL169VJYWJiqVq2q7t2764cffnBer3Ck6qWXXlJERIRCQkL02GOP6cKFC755gwB8qryvCwAATzl8+LD69Omj119/XT179lROTo5WrlypBx54QPv371d2dramTp0qSapataouXryolJQUJSUlaeXKlSpfvrxeeeUV3XXXXdqyZYv8/f0lSUuWLFFgYKCWLVumH374Qf3791e1atX06quv+vLtAvABghMA2zh8+LAuXbqku+++W7Vr15YkNW7cWJJUsWJF5eXlKSoqytn+ww8/VH5+viZPniyHwyFJmjp1qsLCwrRs2TJ16tRJkuTv768pU6YoKChIjRo10ssvv6wRI0ZozJgx8vNj4B4oS/gTD8A2EhIS1KFDBzVu3Fj33Xef3n//fZ06deqq7Tdv3qw9e/aocuXKCg4OVnBwsKpWrarz589r7969LtcNCgpyPk9KSlJubq4OHDjg1fcDoORhxAmAbZQrV05paWn617/+pUWLFuntt9/WCy+8oDVr1lyxfW5urlq0aKGPPvqoyLmIiAhvlwugFCI4AbAVh8OhNm3aqE2bNho5cqRq166tL774Qv7+/rp8+bJL2+bNm+vTTz9V9erVFRISctVrbt68WefOnVPFihUlSd99952Cg4MVExPj1fcCoORhqg6AbaxZs0b//d//rXXr1mn//v2aPXu2jh8/rltuuUWxsbHasmWLdu7cqczMTF28eFF9+/ZVeHi4unfvrpUrVyojI0PLli3TsGHD9NNPPzmve+HCBT300ENKT0/XggULNGrUKA0dOpT1TUAZxIgTANsICQnRihUrNH78eGVnZ6t27doaN26cOnfurJYtW2rZsmVq2bKlcnNz9e2336p9+/ZasWKF/vznP+vuu+9WTk6OatSooQ4dOriMQHXo0EH169fXb37zG+Xl5alPnz4aPXq0794oAJ9xGGOMr4sAgJLqwQcf1OnTpzVnzhxflwKgBGCcGQAAwCKCEwAAgEVM1QEAAFjEiBMAAIBFBCcAAACLCE4AAAAWEZwAAAAsIjgBAABYRHACAACwiOAEAABgEcEJAADAIoITAACARf8HY6chXffOmncAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# viz: the loss curve, log scale to see the long flat tail\n",
    "plt.figure(figsize=(6, 4))\n",
    "plt.plot(losses_gd, color=\"#1E40FF\")\n",
    "plt.yscale(\"log\"); plt.xlabel(\"step\"); plt.ylabel(\"MSE (log scale)\")\n",
    "plt.title(f\"batch GD loss ({len(losses_gd)} steps)\")\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99a83622",
   "metadata": {},
   "source": [
    "> **Interpretation.** The loss drops fast for the first ~50 steps, then crawls along a floor set by the noise variance. That floor is the same MSE the normal equation achieves; gradient descent reached the closed-form answer without ever inverting a matrix.\n",
    "\n",
    "> **Key takeaways.** Batch GD is the gradient formula in a loop. With a sane learning rate the full-batch loss decreases every step. Always sanity-check a from-scratch optimizer against a closed form before trusting it where none exists.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49d75f82",
   "metadata": {},
   "source": [
    "## Part 3 — The learning rate is the whole game\n",
    "\n",
    "> **Objectives.** See the three regimes (too small, right, too large) on one plot, then deliberately diverge a run to infinity and repair it by reading the curve. This is the single most common training failure, staged on purpose.\n",
    "\n",
    "For a quadratic loss with Hessian eigenvalues bounded by $\\mu$ (smallest) and $L$ (largest), batch GD converges if and only if $\\eta < 2/L$. You do not compute $L$ in practice; you guess and read the loss curve. Three runs of the same problem at three learning rates tell the whole story.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "31f42b16",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.322916Z",
     "iopub.status.busy": "2026-06-10T18:46:10.322841Z",
     "iopub.status.idle": "2026-06-10T18:46:10.410600Z",
     "shell.execute_reply": "2026-06-10T18:46:10.410261Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAGGCAYAAACNCg6xAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAdF9JREFUeJzt3Xd4U9X/B/B3kjbpSLropoNp2cMCtSBDKBQEpKCCilIo4AAERUH4qQy34EABxa/IUhREaUFEZG9kSdlU0LJpy+hI05E2Ob8/Yq9NFym03I7363nyNPfcc+/95LS0H8499xyFEEKAiIiIiG5LKXcARERERNUFEyciIiIiGzFxIiIiIrIREyciIiIiGzFxIiIiIrIREyciIiIiGzFxIiIiIrIREyciIiIiGzFxIiIiIrIREyeiWqhbt25o0aLFbeudP38eCoUCS5YsqfygKolCocC4cePkDuO2qlNbKxQKzJgxQ+4wiGTBxImIqr29e/dixowZSEtLkzsUquKuXr2KGTNmID4+Xu5QqJpi4kRE1d7evXsxc+ZMJk73SHZ2Nt544w25w7gjV69excyZM5k40R1j4kRUAxgMBrlDqJFqQ7vm5OTAbDaX6xgHBwfY2dlVUkTlcyfxE90NJk5Uren1erz00kuoV68eNBoNvL290bNnT/z5559SnV27duHxxx9HUFAQNBoNAgMD8fLLLyM7O9vqXMOHD4dWq8XFixfRr18/aLVa1K1bF/PnzwcAHD9+HN27d4ezszOCg4Px/fffF4snLS0NL730EgIDA6HRaNCoUSN8+OGHNv1ir1evHvr164eNGzeiTZs2cHBwQLNmzbB69WqrekuWLIFCocCOHTswZswYeHt7IyAgQNr/xRdfoHnz5tBoNPD398fYsWNL7Yk5fPgwOnbsCEdHR9SvXx8LFiy4bZwAcObMGTz22GPw8PCAg4MD2rVrh7Vr15YY5+7duzF+/Hh4eXnBzc0Nzz33HIxGI9LS0jBs2DC4u7vD3d0dkydPhhDCpusXNmPGDEyaNAkAUL9+fSgUCigUCpw/f96qXlxcHFq0aAGNRoPmzZtjw4YNxc6jUChw6tQpPPXUU3B3d8eDDz4o7f/uu+8QGhoKR0dHeHh44IknnsClS5eKxbN//3707t0brq6ucHJyQteuXbFnz55yf64CtrT1rVu38Oqrr6Jly5bQarVwcXFBnz59cPToUat627dvh0KhwIoVK/DGG2+gbt26cHJyQkZGhvTzf+XKFURFRUGr1cLLywuvvvoqTCaT1XmKjnEqaLtz585h+PDhcHNzg6urK0aMGIGsrCyrY7OzszF+/Hh4enpCp9PhkUcewZUrV2waN1VW/La0wfbt29G+fXsAwIgRI6SflcLjymz5/tnye4dqrqrxXwaiO/T888/jp59+wrhx49CsWTPcvHkTu3fvxunTp3H//fcDAFatWoWsrCy88MILqFOnDg4cOIC5c+fi8uXLWLVqldX5TCYT+vTpgy5dumDWrFlYvnw5xo0bB2dnZ7z++usYOnQoBg0ahAULFmDYsGEIDw9H/fr1AQBZWVno2rUrrly5gueeew5BQUHYu3cvpk6dimvXrmHOnDm3/Txnz57FkCFD8PzzzyM6OhqLFy/G448/jg0bNqBnz55WdceMGQMvLy9MmzZN6hmZMWMGZs6ciYiICLzwwgtISEjAl19+iYMHD2LPnj2wt7eXjk9NTcXDDz+MwYMH48knn8SPP/6IF154AWq1GjExMaXGePLkSXTq1Al169bFlClT4OzsjB9//BFRUVH4+eefMXDgQKv6L774Inx9fTFz5kz88ccf+N///gc3Nzfs3bsXQUFBeO+997B+/XrMnj0bLVq0wLBhw27bToUNGjQIf/31F3744Qd8+umn8PT0BAB4eXlJdXbv3o3Vq1djzJgx0Ol0+Pzzz/Hoo4/i4sWLqFOnjtX5Hn/8cTRu3BjvvfeelMi9++67ePPNNzF48GCMGjUK169fx9y5c9GlSxccOXIEbm5uAICtW7eiT58+CA0NxfTp06FUKrF48WJ0794du3btQocOHcr12Wxt63/++QdxcXF4/PHHUb9+fSQnJ+Orr75C165dcerUKfj7+1ud9+2334Zarcarr76K3NxcqNVqAJaf/8jISISFheGjjz7C5s2b8fHHH6Nhw4Z44YUXbhvv4MGDUb9+fbz//vv4888/sXDhQnh7e+PDDz+U6gwfPhw//vgjnnnmGTzwwAPYsWMH+vbtW652KSn+U6dO3bYNmjZtirfeegvTpk3Ds88+i86dOwMAOnbsCMD2758tv3eoBhNE1Zirq6sYO3ZsmXWysrKKlb3//vtCoVCICxcuSGXR0dECgHjvvfekstTUVOHo6CgUCoVYsWKFVH7mzBkBQEyfPl0qe/vtt4Wzs7P466+/rK41ZcoUoVKpxMWLF8uMMzg4WAAQP//8s1SWnp4u/Pz8RNu2baWyxYsXCwDiwQcfFPn5+VJ5SkqKUKvVolevXsJkMknl8+bNEwDEokWLpLKuXbsKAOLjjz+WynJzc0WbNm2Et7e3MBqNQgghEhMTBQCxePFiqV6PHj1Ey5YtRU5OjlRmNptFx44dRePGjYvFGRkZKcxms1QeHh4uFAqFeP7556Wy/Px8ERAQILp27VpmG5Vm9uzZAoBITEwstg+AUKvV4ty5c1LZ0aNHBQAxd+5cqWz69OkCgHjyySetjj9//rxQqVTi3XfftSo/fvy4sLOzk8rNZrNo3Lhxsc+blZUl6tevL3r27FnmZ7ibts7JybH6nhecT6PRiLfeeksq27ZtmwAgGjRoUOzfRcHPf+H6QgjRtm1bERoaalVW9Ge/oO1iYmKs6g0cOFDUqVNH2j58+LAAIF566SWresOHDy92zpKUFb+tbXDw4MFi7SxE+b5/tvzeoZqLt+qoWnNzc8P+/ftx9erVUus4OjpK7w0GA27cuIGOHTtCCIEjR44Uqz9q1Cir84eEhMDZ2RmDBw+WykNCQuDm5oZ//vlHKlu1ahU6d+4Md3d33LhxQ3pFRETAZDJh586dt/08/v7+Vj02Li4uGDZsGI4cOYKkpCSruqNHj4ZKpZK2N2/eDKPRiJdeeglKpdKqnouLC3799Ver4+3s7PDcc89J22q1Gs899xxSUlJw+PDhEuO7desWtm7disGDB0Ov10uf8ebNm4iMjMTZs2dx5coVq2NGjhwJhUIhbYeFhUEIgZEjR0plKpUK7dq1s2rPihQREYGGDRtK261atYKLi0uJ13v++eettlevXg2z2YzBgwdbfV99fX3RuHFjbNu2DQAQHx+Ps2fP4qmnnsLNmzelegaDAT169MDOnTvLNRanPG2t0Wik77nJZMLNmzeh1WoREhJS4u2j6Ohoq38XZX3+zp072/x9KenYmzdvIiMjAwCk26Njxoyxqvfiiy/adP4CJcVf3jYoqjzfP1t+71DNxVt1VK3NmjUL0dHRCAwMRGhoKB5++GEMGzYMDRo0kOpcvHgR06ZNw9q1a5Gammp1fHp6utW2g4OD1S0eAHB1dUVAQIDVH/+C8sLnO3v2LI4dO1bs+AIpKSm3/TyNGjUqdp377rsPgGWeH19fX6m84BZhgQsXLgCwJHWFqdVqNGjQQNpfwN/fH87OzqVe64EHHigW37lz5yCEwJtvvok333yzxM+QkpKCunXrSttBQUFW+11dXQEAgYGBxcqLfn8qStEYAMDd3b3E6xVt17Nnz0IIgcaNG5d47oLbn2fPngVg+aNemvT0dLi7u9sUc3na2mw247PPPsMXX3yBxMREqzFJRW9FAsU/Y4GSfv5La6eSFG3ngs+ampoKFxcXXLhwAUqlstj1GzVqZNP5C5QUf3nboKjyfP9s+b1DNRcTJ6rWBg8ejM6dOyM2NhYbN27E7Nmz8eGHH2L16tXo06cPTCYTevbsiVu3buG1115DkyZN4OzsjCtXrmD48OHFegAK9+DYUi4KDWY2m83o2bMnJk+eXGLdgqSkopTWY1CZCtrr1VdfRWRkZIl1iv4RLE+bijsYHG4LW75/BYq2q9lshkKhwG+//VbiebRarVQPAGbPno02bdqUeL2CurYoT1u/9957ePPNNxETE4O3334bHh4eUCqVeOmll0rs5SrtZ6e0drJVedr5bpQUf3nboKjyfP9u93uHajYmTlTt+fn5YcyYMRgzZgxSUlJw//33491330WfPn1w/Phx/PXXX1i6dKnVoONNmzZVeBwNGzZEZmYmIiIi7vgcBb0MhXud/vrrLwCWp+7KEhwcDABISEiw+p+v0WhEYmJisbiuXr0Kg8Fg1et0u2sVnNfe3v6uPmdFK9pLV5EaNmwIIQTq169fZvJbcCvQxcWlQtqmPG39008/4aGHHsI333xjVZ6WliYNlq8KgoODYTabkZiYaNWDd+7cubs+t61tUNrPSnm/f2X93qGajWOcqNoymUzFbrV5e3vD398fubm5AP77H3Dh//EKIfDZZ59VeDyDBw/Gvn378Pvvvxfbl5aWhvz8/Nue4+rVq4iNjZW2MzIysGzZMrRp08bqNl1JIiIioFar8fnnn1t93m+++Qbp6enFnlzKz8/HV199JW0bjUZ89dVX8PLyQmhoaInX8Pb2Rrdu3fDVV1/h2rVrxfZfv379tp+xMhQkf5UxAeagQYOgUqkwc+bMYj0nQgjcvHkTABAaGoqGDRvio48+QmZmZrHzlLdtytPWKpWqWGyrVq0qNt5MbgU9Z1988YVV+dy5c+/63La2QWk/K7Z+/2z5vUM1G3ucqNrS6/UICAjAY489htatW0Or1WLz5s04ePAgPv74YwBAkyZN0LBhQ7z66qu4cuUKXFxc8PPPP1fKWJpJkyZh7dq16NevH4YPH47Q0FAYDAYcP34cP/30E86fP3/b//3fd999GDlyJA4ePAgfHx8sWrQIycnJWLx48W2v7+XlhalTp2LmzJno3bs3HnnkESQkJOCLL75A+/bt8fTTT1vV9/f3x4cffojz58/jvvvuw8qVKxEfH4///e9/VtMWFDV//nw8+OCDaNmyJUaPHo0GDRogOTkZ+/btw+XLl4vNHXQ3hg8fjqVLlyIxMbHMHreCRO/111/HE088AXt7e/Tv37/YGK470bBhQ7zzzjuYOnUqzp8/j6ioKOh0OiQmJiI2NhbPPvssXn31VSiVSixcuBB9+vRB8+bNMWLECNStWxdXrlzBtm3b4OLigl9++aVc17a1rfv164e33noLI0aMQMeOHXH8+HEsX768yo25CQ0NxaOPPoo5c+bg5s2b0nQEBT2dd9NzaGsbNGzYEG5ubliwYAF0Oh2cnZ0RFhaG+vXr2/T9s+X3DtVw9/oxPqKKkpubKyZNmiRat24tdDqdcHZ2Fq1btxZffPGFVb1Tp06JiIgIodVqhaenpxg9erT0OHrhR5Kjo6OFs7Nzset07dpVNG/evFh5cHCw6Nu3r1WZXq8XU6dOFY0aNRJqtVp4enqKjh07io8++kh6xL80Bef7/fffRatWrYRGoxFNmjQRq1atsqpX8Jj/wYMHSzzPvHnzRJMmTYS9vb3w8fERL7zwgkhNTS3xMx06dEiEh4cLBwcHERwcLObNm2dVr6RH5IUQ4u+//xbDhg0Tvr6+wt7eXtStW1f069dP/PTTT7eNs+DR9evXr1uVl9T+jz76qHB0dCwWf0nefvttUbduXaFUKq2mJgBQ4qPjwcHBIjo6+rZxFfj555/Fgw8+KJydnYWzs7No0qSJGDt2rEhISLCqd+TIETFo0CBRp04dodFoRHBwsBg8eLDYsmVLmfHfTVvn5OSIV155Rfj5+QlHR0fRqVMnsW/fPtG1a1erKR4KHucv+jMlROk//wXtUhhKmY6gaNsV/AwUnibCYDCIsWPHCg8PD6HVakVUVJRISEgQAMQHH3xQZhuVFb+tbSCEEGvWrBHNmjUTdnZ2xdr8dt8/W3/vUM2lEKKSRmMSUbnUq1cPLVq0wLp16+QOpcrw8fHBsGHDMHv2bLlDoUoUHx+Ptm3b4rvvvsPQoUPlDoeoTBzjRERV0smTJ5GdnY3XXntN7lCoAhVd6ggA5syZA6VSiS5dusgQEVH5cIwTEVVJzZs3lyZOpJpj1qxZOHz4MB566CHY2dnht99+w2+//YZnn3222NxeRFUREyciIrpnOnbsiE2bNuHtt99GZmYmgoKCMGPGDLz++utyh0ZkE45xIiIiIrIRxzgRERER2YiJExEREZGNav0YJ7PZjKtXr0Kn01Xqsg1ERERUNQkhoNfr4e/vD6Wy7D6lWp84Xb16lU9yEBERES5duoSAgIAy69T6xEmn0wGwNJaLi4vM0RAREdG9lpGRgcDAQCknKEutT5wKbs+5uLgwcSIiIqrFbBmyw8HhRERERDZi4kRERERkIyZORERERDaq9WOcbGE2m2E0GuUOgwgAYG9vD5VKJXcYRES1EhOn2zAajUhMTITZbJY7FCKJm5sbfH19OfcYEdE9xsSpDEIIXLt2DSqVCoGBgbedFIuosgkhkJWVhZSUFACAn5+fzBEREdUuTJzKkJ+fj6ysLPj7+8PJyUnucIgAAI6OjgCAlJQUeHt787YdEdE9xC6UMphMJgCAWq2WORIiawWJfF5ensyREBHVLkycbMBxJFTV8GeSiEgeTJyIiIioyjPnZEHkyf+EOxMnqpaeeeYZvPfeezbV7datG1566aVKicNoNKJevXo4dOhQpZyfiKi2M2fpoY9diOQXH0bW9ji5w2HiRLY5duwYOnfuDAcHBwQGBmLWrFm3PebixYvo27cvnJyc4O3tjUmTJiE/P1/av3r1avTs2RNeXl5wcXFBeHg4fv/999ue9+jRo1i/fj3Gjx9/V5+pIqjVarz66qt47bXX5A6FiKhGMWWkImPlPCSN64OMlXNh1qcie/9mucPiU3V0exkZGejVqxciIiKwYMECHD9+HDExMXBzc8Ozzz5b4jEmkwl9+/aFr68v9u7di2vXrmHYsGGwt7eXeop27tyJnj174r333oObmxsWL16M/v37Y//+/Wjbtm2p8cydOxePP/44tFptpXze8ho6dCheeeUVnDx5Es2bN5c7HCKias10KwX6dUuRteUniNwcAIBdQAPookbBMTxS5ugAiFouPT1dABDp6enF9mVnZ4tTp06J7OxsGSK7c127dhUvvviimDRpknB3dxc+Pj7izTffvOPzffHFF8Ld3V3k5uZKZa+99poICQkp9Zj169cLpVIpkpKSpLIvv/xSuLi4WJ2nqGbNmomZM2eWuj8/P1+4urqKdevWWZXPnz9fNGrUSGg0GuHt7S0effRRaV/Xrl3FhAkTpO1bt26JZ555Rri5uQlHR0fRu3dv8ddffwkhhDCbzcLT01OsWrVKqt+6dWvh6+srbe/atUuo1WphMBiksoceeki88cYbpcZd0arrzyYRUWnyki+JW1+/JS4PDRWXh7QSl4e0EslThois/ZuE2WSq1GuXlQsUxVt15SCEQF5eniwvIUS5Yl26dCmcnZ2xf/9+zJo1C++88w42btwo7e/Tpw+0Wm2pr8I9J/v27UOXLl2spmWIjIxEQkICUlNTS7z+vn370LJlS/j4+Fgdk5GRgZMnT5Z4jNlshl6vh4eHR6mf69ixY0hPT0e7du2kskOHDmH8+PF46623kJCQgA0bNqBLly6lnmP48OE4dOgQ1q5di3379kEIgYcffhh5eXlQKBTo0qULtm/fDgBITU3F6dOnkZ2djTNnzgAAduzYgfbt21vN7dWhQwfs2rWr1GsSEVHJ8q4k4tYXbyD5pUeQtfknID8P6pC2qDPlC3i99wMcO0RAUYUmoOatunLIz8/H4sWLZbn2iBEjYG9vb3P9Vq1aYfr06QCAxo0bY968edi2bRt69eoFAFi4cCGys7NLPb7wtZKSklC/fn2r/QUJUVJSEtzd3Ysdn5SUZJU0FT2mJB999BEyMzMxePDgUuO6cOECVCoVvL29pbKLFy/C2dkZ/fr1g06nQ3BwcKm3+s6ePYu1a9diz5496NixIwBg+fLlCAwMRFxcHB5//HF069YNX331FQDL7cS2bdvC19cX27dvR5MmTbB9+3Z07drV6rz+/v64cOFCqXETEZE1Y+Jp6OO+Qc6BzcC/nQOalg9AN+hZaJqGyhxd6Zg41VCtWrWy2vbz80NycrK0Xbdu3XsdUpm+//57zJw5E2vWrLFKiorKzs6GRqOxmseoZ8+eCA4ORoMGDdC7d2/07t0bAwcOLHG299OnT8POzg5hYWFSWZ06dRASEoLTp08DALp27YoJEybg+vXr2LFjB7p16yYlTiNHjsTevXsxefJkq/M6OjoiKyvrbpuBiKjGy004An3sQuTG75bKHNo/BF3UKKgbtpAxMtswcSoHOzs7jBgxQrZrl0fR3imFQmG1UHGfPn3KvLUUHBws3VLz9fW1SroASNu+vr4lHu/r64sDBw7YdMyKFSswatQorFq1ChEREWV9LHh6eiIrKwtGo1G6dajT6fDnn39i+/bt2LhxI6ZNm4YZM2bg4MGDcHNzK/N8JWnZsiU8PDywY8cO7NixA++++y58fX3x4Ycf4uDBg8jLy5N6qwrcunULXl5e5b4WEVFtIIRA7vE/oI9dCOPpf6dvUSjh2LE3dFExsA9sLG+A5cDEqRwUCkW5bpdVZeW5VRceHo7XX38deXl5UvmmTZsQEhJS4m26gmPeffddaT21gmNcXFzQrFkzqd4PP/yAmJgYrFixAn379r1t3G3atAEAnDp1SnoPWBLLiIgIREREYPr06XBzc8PWrVsxaNAgq+ObNm2K/Px87N+/X0p+bt68iYSEBCkuhUKBzp07Y82aNTh58iQefPBBODk5ITc3F1999RXatWsHZ2dnq/OeOHGizCcBiYhqI2E2I+fPHdDHLkTe3ycshSo7OHXpD92AGNj5Bskb4B1g4lRLledW3VNPPYWZM2di5MiReO2113DixAl89tln+PTTT6U6sbGxmDp1qjSAulevXmjWrBmeeeYZzJo1C0lJSXjjjTcwduxYaDQaAJbbc9HR0fjss88QFhYmjX1ydHSEq6tribF4eXnh/vvvx+7du6XEad26dfjnn3/QpUsXuLu7Y/369TCbzQgJCSl2fOPGjTFgwACMHj0aX331FXQ6HaZMmYK6detiwIABUr1u3brhlVdeQbt27aRpD7p06YLly5dj0qRJxc67a9cuvP322za3KRFRTSbMJmT/sRH62IXIv3TOUmivgXOPQdD2Gw47z5LvVlQHVWeYOlVZrq6u2LhxIxITExEaGopXXnkF06ZNs5rDKT09HQkJCdK2SqXCunXroFKpEB4ejqeffhrDhg3DW2+9JdX53//+h/z8fIwdOxZ+fn7Sa8KECWXGM2rUKCxfvlzadnNzw+rVq9G9e3c0bdoUCxYswA8//FDqnEqLFy9GaGgo+vXrh/DwcAghsH79eqtetq5du8JkMqFbt25SWbdu3YqVAZYnCNPT0/HYY4+VGTcRUU0n8vNg2BaL5IlRSP18CvIvnYPC0RnaATHwnfcb3IZPqdZJEwAoRHmfc69hMjIy4OrqivT0dLi4uFjty8nJQWJiIurXrw8HBweZIqwYUVFRcHNzw5IlS+QO5a5lZ2cjJCQEK1euRHh4uNzhYMiQIWjdujX+7//+755dsyb9bBJR9SeMOTBsjUXmL0tgumm5e6DQukLbZyi0kU9CqXW5zRnkVVYuUBRv1VG14+joiGXLluHGjRtyhwKj0YiWLVvi5ZdfljsUIqJ7zpxtgGHTj8j89VuY028CAJRuntD2GwbniMehdCj+dHN1x8SJqqWit8vkolar8cYbb8gdBhHRPWXOTEfmb98jc8P3EIYMAIDK0x/aR4bDuVsUFGqNzBFWHiZOtURcXJzcIRARUTVnSruBzF+/hWHTjxA5lrnr7PzrQTtgJJw69YHCrmY8eV4WJk5ERERUpvzrV5H5yxIYtsUCeUYAgH1wCHQDR8GhQw8olCqZI7x3mDgRERFRifKunkfmmm+QtXs9YMoHAKgbt4Zu4Cho2na2WsWhtmDiRERERFbyLvwFfdxCZP+x8b915Jp3gG7gKKibd6iVCVMBJk5EREQEADCePQZ93ELkHN4hlTnc3wW6gaOhbtyqjCNrDyZOREREtZgQAsZTBy0L757YbylUKOAY1hO6gaNgH1x8FYbajIkTERFRLSSEQO6RXZaFd88etRSq7ODUuS+0j8TA3r+erPFVVUycqMrZsmULxo0bhxMnTkClkvdJjeHDhyMtLU2azuGJJ55A+/bt8corr8gaFxHRnRJmE3IObIE+biHyzv+7VJa9Gs4PDYS2/3DYefnLG2AVx7XqyCbHjh1D586d4eDggMDAQMyaNeu2x4wfPx6hoaHQaDTSgry2mDx5Mt544w3Zk6aSvPHGG3j33XeRnp4udyhEROUi8vNg2LEWKa8Owq05k5B3PgEKjSO0/aLh+/l6uMX8H5MmG7DHiW4rIyMDvXr1QkREBBYsWIDjx48jJiYGbm5uVgv9liQmJgb79+/HsWPHbLrW7t278ffff+PRRx+tiNArXIsWLdCwYUN89913GDt2rNzhEBHdljDmwrB9DTJ/WQzT9asAAIWzDtreT8G591NQ6dzkDbCaYY9TDdStWzeMHz8ekydPhoeHB3x9fTFt2rQ7Pt/y5cthNBqxaNEiNG/eHE888QTGjx+PTz75pMzjPv/8c4wdOxYNGjSw+VorVqxAz549iy1c+84778Db2xs6nQ6jRo3ClClTrHqxzGYz3nrrLQQEBEg9XBs2bLA6x/Hjx9G9e3c4OjqiTp06ePbZZ5GZmSntN5lMmDhxItzc3FCnTh1MnjwZJa2B3b9/f6xYscLmz0REJAdzThb065YiaUJfpC96F6brV6F09YDLkxPgO3cDXB4fw6TpDjBxKgchgKxseV4l/P0u09KlS+Hs7Iz9+/dj1qxZeOedd7Bx40Zpf58+faDVakt9NW/eXKq7b98+dOnSBWq1WiqLjIxEQkICUlNT77pdC9u1axfatWtnVbZ8+XK8++67+PDDD3H48GEEBQXhyy+/tKrz2Wef4eOPP8ZHH32EY8eOITIyEo888gjOnj0LADAYDIiMjIS7uzsOHjyIVatWYfPmzRg3bpx0jo8//hhLlizBokWLsHv3bty6dQuxsbHFYuzQoQMOHDiA3NzcCv3sREQVwWzIQMbPXyH5xT7I+O4TmFOvQ+XhA9fhr8Hn8/XQDYiB0kkrd5jVFm/VlUN2DtD0YXmufXo94ORoe/1WrVph+vTpAIDGjRtj3rx52LZtG3r16gUAWLhwIbKzs0s93t7+v/WGkpKSUL9+fav9Pj4+0j53d3fbA7uNCxcuwN/f+h773LlzMXLkSIwYMQIAMG3aNGzcuNGqt+ijjz7Ca6+9hieeeAIA8OGHH2Lbtm2YM2cO5s+fj++//x45OTlYtmwZnJ2dAQDz5s1D//798eGHH8LHxwdz5szB1KlTMWjQIADAggUL8PvvvxeL0d/fH0ajEUlJSQgODq6wz05EdDdM6TeRuf47GDauhMg2AABUvkHQPTICTl3614p15O6FGpE4ffTRR1i8eDEUCgWmTJmCp59+Wu6QZNeqlfVEZX5+fkhOTpa269ate69Dskl2dnax23QJCQkYM2aMVVmHDh2wdetWAJYxWFevXkWnTp2s6nTq1AlHj1oesT19+jRat24tJU0F+81mMxISEuDg4IBr164hLCxM2m9nZ4d27doVu13n6GjJYLOysu7y0xIR3T3TzWTof1mCrK2rIYw5AAC7wEbQRY2C4wM9oVDViD/1VUa1b83jx4/j+++/x+HDhyGEwEMPPYR+/frBzc2twq/l6GDp+ZGDo8Pt6xRWuMcIABQKBcxms7Tdp08f7Nq1q9Tjg4ODcfLkSQCAr6+vVdIFQNr29fUtX2C34enpWeG3/yrarVu3AABeXl4yR0JEtVl+0iXo1y5C1o610jpy9g2bQzdwNBzu7wqFkqNxKkO1T5xOnz6N8PBwqZeidevW2LBhg3TLpiIpFOW7XVaVledWXXh4OF5//XXk5eVJ5Zs2bUJISEiF3qYDgLZt2+LUqVNWZSEhITh48CCGDRsmlR08eFB67+LiAn9/f+zZswddu3aVyvfs2YMOHToAAJo2bYolS5bAYDBIvU579uyBUqlESEgIXF1d4efnh/3796NLly4AgPz8fBw+fBj333+/VTwnTpxAQEAAPD09K/SzExHZIu/SOejjvkH23g2AsPyHWN20nWXh3ZYP1Op15O4F2dPRnTt3on///vD394dCoZAmGixs/vz5qFevHhwcHBAWFoYDBw5I+1q0aIHt27cjLS0Nqamp2L59O65cuXIPP0H1VLduXTRq1KjUV+GxO0899RTUajVGjhyJkydPYuXKlfjss88wceJEqU5sbCyaNGlidY1z584hPj4eSUlJyM7ORnx8POLj42E0GkuNKzIyErt377Yqe/HFF/HNN99g6dKlOHv2LN555x0cO3bM6pfDpEmT8OGHH2LlypVISEjAlClTEB8fjwkTJgAAhg4dCgcHB0RHR+PEiRPYtm0bXnzxRTzzzDPSeK0JEybggw8+QFxcHM6cOYMxY8YgLS2tWIy7du2SxooREd0rxr9P4ubHLyFl0qPI3rMeEGZo2jwIz5lL4DX9Gzi0CmfSdA/I3uNkMBjQunVrxMTESINyC1u5ciUmTpyIBQsWICwsDHPmzJGe6PL29kazZs0wfvx4dO/eHa6urnjggQeq5MSJ1Zmrqys2btyIsWPHIjQ0FJ6enpg2bZrVHE7p6elISEiwOm7UqFHYseO/hSLbtm0LAEhMTES9evVKvNbQoUMxefJkJCQkICQkRCr7559/8OqrryInJweDBw/G8OHDrRLo8ePHIz09Ha+88gpSUlLQrFkzrF27Fo0bNwYAODk54ffff8eECRPQvn17ODk54dFHH7WaUuGVV17BtWvXEB0dDaVSiZiYGAwcONBqssucnBzExcUVm+qAiKiy5J7+E/q4r5F7dK+lQKGAQ/se0A0cBXX9pvIGVwspREkT1chEoVAgNjYWUVFRUllYWBjat2+PefPmAbDM1xMYGIgXX3wRU6ZMKXaOUaNGYeDAgejbt69N18zIyICrqyvS09Ph4uJitS8nJweJiYmoX79+sQHL1U1UVBTc3NywZMkSuUO5rUmTJiEjIwNfffVVqXV69uwJX19ffPvtt/cwMuDLL79EbGys1dQOcqhJP5tEVJwQArlH90If+zWMCUcshUoVHDv2hi5qJOwDGsobYA1TVi5QlOw9TmUxGo04fPgwpk6dKpUplUpERERg3759UllKSgq8vb2RkJCAAwcOYMGCBaWeMzc312r+nYyMjMoJnu7Y66+/ji+++AJmsxlKpRJZWVlYsGABIiMjoVKp8MMPP2Dz5s3YtGnTPY/N3t4ec+fOvefXJaLaQZjNyDm0Ffq4b5D3z7/jPe3s4dRtAHT9R8DOJ0DeAKlqJ043btyAyWSSxqAU8PHxwZkzZ6TtAQMGID09Hc7Ozli8eDHs7Er/WO+//z5mzpxZaTHT3XNzc8P//d//SdsKhQLr16/Hu+++i5ycHISEhODnn39GRETEPY9t1KhR9/yaRFTzCVM+svf9Dn3cQuRf/gcAoFA7wCniMej6DYPKw+c2Z6B7pUonTrYq3Pt0O1OnTrUa1JyRkYHAwMDKCKtKKWnQfXXh6OiIzZs3yx0GEVGFE3lGZO38Bfo1i2BKuQwAUDhq4Rz5BLQPD4XKxUPmCKmoKp04eXp6QqVSlTiH0J3OH6TRaKDRaCoiPCIiojtizs1G1pbV0K9bAvOtFACAUucO7cND4dxrCJTOZY+zIflU6cRJrVYjNDQUW7ZskQaMm81mbNmyxWqNMSIiourAnKWHYeOPyFz/LcwZlsl+le5e0PWLhlOPR6F0cJI5Qrod2ROnzMxMnDt3TtpOTExEfHw8PDw8EBQUhIkTJyI6Ohrt2rVDhw4dMGfOHBgMBmndMiIioqrOlJEKw2/Lkfn7CogsPQBA5V3Xso5c1wFQ2KtvcwaqKmRPnA4dOoSHHnpI2i4YfxQdHY0lS5ZgyJAhuH79OqZNm4akpCS0adMGGzZsKDZgnIiIqKox3UpB5q/LYNi8CiL333Xk6jaAbkAMHDv14Tpy1ZDs37Fu3boVW0S1qHHjxvHWHBERVRv5KVeQuXYxDNvjgPw8AIB9vSaWdeTad+c6ctWY7IkTERFRTZF3JRH6Nd8ge/d6wGwCAKhD2kAXNRqaNp24JEoNwJSXqqVnnnkG7733nrRdr149zJkzR76AKsiNGzfg7e2Ny5cvyx0KEZWDMfE0bn76KlJeHYjsnb8AZhM0LR+A57Rv4DljCRzaPsikqYZg4kQ2OXbsGDp37gwHBwcEBgZi1qxZNh978+ZNBAQEQKFQWC2au337digUimKvpKSkMs939OhRrF+/HuPHj7/Tj1NleXp6YtiwYZg+fbrcoRCRDXIT4nHjw3G4PvUJ5OzfBAgBh3YPwevd5fB8/StomrVjwlTD8FYd3VZGRgZ69eqFiIgILFiwAMePH0dMTAzc3NysFvotzciRI9GqVStcuXKlxP0JCQlWawN5e3uXeb65c+fi8ccfh1arLd8HqSZGjBiB0NBQzJ49Gx4enPyOqKoRQiD3xH7LOnKnDlkKFUo4hkda1pELaixvgFSp2ONUA3Xr1g3jx4/H5MmT4eHhAV9fX0ybNu2Oz7d8+XIYjUYsWrQIzZs3xxNPPIHx48fjk08+ue2xX375JdLS0vDqq6+WWsfb2xu+vr7SS1nGoEmTyYSffvoJ/fv3L7ZPr9fjySefhLOzM+rWrYv58+db7b948SIGDBgArVYLFxcXDB482Gpy1eHDh1stMA0AL730Erp16yZtl9S2M2bMsDomLS0Nzz33HHx8fODg4IAWLVpg3bp1MBgMcHFxwU8//WRVPy4uDs7OztDrLY8oN2/eHP7+/oiNjS21HYjo3hNmM7IPbcf1N57GzXefsyRNKjs4PTQQPp+ugcf4D5g01QJMnMpBCAFzTpYsr9s9eVjU0qVL4ezsjP3792PWrFl45513sHHjRml/nz59oNVqS301b95cqrtv3z506dIFavV/84xERkYiISEBqamppcZw6tQpvPXWW1i2bFmZyVCbNm3g5+eHnj17Ys+ePWV+rmPHjiE9PR3t2rUrtm/27Nlo3bo1jhw5gilTpmDChAnSQsBmsxkDBgzArVu3sGPHDmzatAn//PMPhgwZUub1SlK0bd966y2r6/Tp0wd79uzBd999h1OnTuGDDz6ASqWCs7MznnjiCSxevNjqfIsXL8Zjjz0GnU4nlXXo0AG7du0qd2xEVPGE2YSsvb8hZcpg3PpoAvL+PgHYa+Dc+0n4fPYr3J+bATvfILnDpHuEt+rKQeRm49rwcFmu7bdkHxTlmFG2VatW0jiZxo0bY968edi2bRt69eoFAFi4cCGys7NLPd7e3l56n5SUhPr161vtL5hHKykpCe7u7sWOz83NxZNPPonZs2cjKCgI//zzT/HP5OeHBQsWoF27dsjNzcXChQvRrVs37N+/H/fff3+JcV24cAEqlarE23mdOnXClClTAAD33Xcf9uzZg08//RQ9e/bEli1bcPz4cSQmJkprEy5btgzNmzfHwYMH0b59+1LboqiS2nbLli3o2bMnNm/ejAMHDuD06dO47777AAANGjSQjh01ahQ6duyIa9euwc/PDykpKVi/fn2xtfj8/f1x5MgRm2Miooon8vOQtWudZR25pIsAAIWjM5x7Dob24Wegcqsjc4QkByZONVSrVq2stv38/KxuS9WtW7dSrz916lQ0bdoUTz/9dKl1QkJCEBISIm137NgRf//9Nz799FN8++23JR6TnZ0NjUZT4mDL8PDwYtsFT9qdPn0agYGBVgs6N2vWDG5ubjh9+nS5E6fCChIgAIiPj0dAQICUNBXVoUMHNG/eHEuXLsWUKVPw3XffITg4GF26dLGq5+joiKysLJtjIqKKI4w5MGyNReYvS2C6aXlYRaF1hbbPU9BGPgWlluvI1WZMnMpBoXGE35J9sl27PAr3GAGAQqGA2WyWtvv06VPmraDg4GCcPHkSAODr61viQssF+0qydetWHD9+XBrPU3Cr0dPTE6+//jpmzpxZ4nEdOnTA7t27S43L09MTWVlZMBqNVrcOK4JSqSx2SzQvL69YvbLa1tHx9t+nUaNGYf78+ZgyZQoWL16MESNGFEsEb926BS8vr/J+BCK6C+ZsAwybfkTmr9/CnH4TAKB084S27zA493yc68gRACZO5aJQKMp1u6wqK8+tuvDwcLz++uvIy8uTyjdt2oSQkJASb9MBwM8//2x1/oMHDyImJga7du1Cw4YNS71ufHw8/Pz8St3fpk0bAJbxUwXvC/zxxx/Ftps2bQoAaNq0KS5duoRLly5JvU6nTp1CWloamjVrBgDw8vLCiRMnisVTNFEqS6tWrXD58mX89ddfpfY6Pf3005g8eTI+//xznDp1CtHR0cXqnDhxwmpQOhFVHnNmOjJ/+x6ZG76HMGQAAFSe/tA+MhzO3aKgUGtkjpCqEiZOtVR5btU99dRTmDlzJkaOHInXXnsNJ06cwGeffYZPP/1UqhMbG4upU6fizJkzAFAsObpx4wYASwLj5uYGAJgzZw7q16+P5s2bIycnBwsXLsTWrVutBrEX5eXlhfvvvx+7d+8uljjt2bMHs2bNQlRUFDZt2oRVq1bh119/BQBERESgZcuWGDp0KObMmYP8/HyMGTMGXbt2lQaad+/eHbNnz8ayZcsQHh6O7777DidOnEDbtm1tbquuXbuiS5cuePTRR/HJJ5+gUaNGOHPmDBQKBXr37g0AcHd3x6BBgzBp0iT06tULAQEBVufIysrC4cOHrSb4JKKKZ0q7gcxfv4Vh048QOZZb43Z+wdAOGAmnBx+Gws72/zRR7cGn6ui2XF1dsXHjRiQmJiI0NBSvvPIKpk2bZjWHU3p6OhISEsp1XqPRiFdeeQUtW7ZE165dcfToUWzevBk9evQo87hRo0Zh+fLlxcpfeeUVHDp0CG3btsU777yDTz75BJGRkQAsvYVr1qyBu7s7unTpgoiICDRo0AArV66Ujo+MjMSbb76JyZMno3379tDr9Rg2bFi5PhNg6W1r3749nnzySTRr1gyTJ0+GyWSyqjNy5EgYjUbExMQUO37NmjUICgpC586dy31tIrq9/BvXkLboPSS9+DAyf1kCkZMF++AQuE+YBe+PY+HcbQCTJiqVQpT3OfcaJiMjA66urkhPT7eahBEAcnJykJiYiPr168PBwUGmCCtGVFQU3NzcsGTJErlDuWvZ2dkICQnBypUriw0Iry6+/fZbvPzyy7h69WqxsVoPPPAAxo8fj6eeeqrU42vSzybRvZJ39Twy1y5C1q5fAVM+AMC+cSvookbB4f4unOG7FisrFyiKt+qo2nF0dMSyZcuk23/VSVZWFq5du4YPPvgAzz33XLGk6caNGxg0aBCefPJJmSIkqnnyLvwFfdxCZP+xCRCWBzk0zTtAN3AU1M07MGGicmHiRNVSdR04PWvWLLz77rvo0qULpk6dWmy/p6cnJk+eLENkRDWP8ewx6OMWIufwDqnM4f4u0A0cDXXjVmUcSVQ63qqrJbfqqGbhzyZRyYQQMJ46CH3sQuSe2G8pVCjgGNYTuoGjYB8cUvYJqFbirToiIqpVhBDIjd8NfexCGP+KtxSq7OD04MPQDhgJe/96coZHNQgTJyIiqraE2YScA1ugj1uIvPP/Ptlrr4bzQwOh7RcNO+/KXSWBah8mTjao5XczqQoqPAs8UW0k8vOQtec3ZK5ZhPyriQAsKyw4RzwObb9hULlz5n2qHEycymBvbw+FQoHr16/Dy8uLT16Q7IQQMBqNuH79OpRKZYUvO0NU1QljLrJ2rIF+7WKYrl8FACicddD2fgrOvZ+CSucmb4BU4zFxKoNKpUJAQAAuX76M8+fPyx0OkcTJyQlBQUFQKjmHLdUO5pwsGLb8hMx1y2BOvQ4AULp6QPvwM3DuORhKJ63MEVJtwcTpNrRaLRo3blziYq9EclCpVLCzs2MPKNUKZkMGMn9fAcNvy2HWpwEAVB4+0D4yHE4PDYSynAugE90tJk42UKlUUKlUcodBRFRrmNJvInP9dzBsXAmRbQAAqHwCoRsQA6cu/bkkCsmGiRMREVUZppvJ0K9biqwtP0MYcwAAdgENoRs4Go4P9IRCxT9bJC/+BBIRkezyky5Bv3Yxsnas+W8duQbNoRs4Cg6h3aDgeD6qIpg4ERGRbPIunYN+zTfI3rNBWkdO3TQUuqhR0LQK51g+qnKYOBER0T1n/OcU9LFfI+fgVqlM07oTdANHQdPkfhkjIyobEyciIrpnck//CX3c18g9ulcqc+jQA7qoUVA3aCZjZES2YeJERESVSgiB3GN7LevInfnTUqhUwbFjb+iiRsI+oKG8ARKVAxMnIiKqFMJsRs6hbZZ15P45ZSm0s4dT10egeyQGdj4B8gZIdAeYOBERUYUSpnxk7/sd+rhvkH/5bwCAQu0Apx6PQdd/GFQePjJHSHTnmDgREVGFEHlGZO38xbKOXPIlAIDCUQvnyCegfXgoVC4eMkdIdPeYOBER0V0x52Yja8tq6NctgflWCgBAqXOD9uGn4dxrCJTOLjJHSFRxmDgREdEdMWfpYdj4IzLXfwtzRioAQOnuBV2/aDj1eBRKByeZIySqeEyciIioXEwZqTD8thyZv6+AyNIDAFTedaF7ZAScug6Awl4tc4RElYeJExER2cR0KwWZvy6DYfMqiNx/15Gr2wC6ATFw7NSH68hRrcCfciIiKlN+yhVkrl0Mw/Y4ID8PAGBfrwl0A0fDoX13riNHtQoTJyIiKlHelUTLOnK71wNmEwBAHdIGuqjR0LTpxHXkqFZi4kRERFaMiaehj/sGOQc2A0IAADQtH4Bu4Giom4YyYaJajYkTEREBAHIT4qGPW4jcI7ukMod2D0E3cBTUDVvIGBlR1cHEiYioFhNCIPfEfuhjv4bx1CFLoUIJx/BIyzpyQY3lDZCoiqn2iVNCQgKGDBlitf3DDz8gKipKvqCIiKo4IQRyDu+APvZr5P19wlKosoNTl/7QPTICdn7B8gZIVEVV+8QpJCQE8fHxAIDMzEzUq1cPPXv2lDcoIqIqSphNyP5jo2UduYtnLYX2Gjj3GARtv+Gw8/SVN0CiKq7aJ06FrV27Fj169ICzs7PcoRARVSkiPw9Zu9ZBv2YRTEkXAQAKByc49xoC7cPPQOVWR+YIiaoH2Sff2LlzJ/r37w9/f38oFArExcUVqzN//nzUq1cPDg4OCAsLw4EDB0o8148//mh1246IqLYTxhxkbvgByS/1R9pXM2BKugiF1hW6x1+A77zf4frUS0yaiMpB9h4ng8GA1q1bIyYmBoMGDSq2f+XKlZg4cSIWLFiAsLAwzJkzB5GRkUhISIC3t7dULyMjA3v37sWKFSvuZfhERFWSOdsAw6YfkfnrtzCn3wQAKN08oe07DM4Rj0HpyJ55ojuhEOLfSTqqAIVCgdjYWKuB3WFhYWjfvj3mzZsHADCbzQgMDMSLL76IKVOmSPW+/fZb/P777/juu+/KvEZubi5yc3Ol7YyMDAQGBiI9PR0uLlzBm4iqN3NmOjI3/IDM35ZDGDIAACpPP2gfGQHnblFQqDUyR0hU9WRkZMDV1dWmXED2HqeyGI1GHD58GFOnTpXKlEolIiIisG/fPqu6P/74I5599tnbnvP999/HzJkzKzxWIiI5mdJuWtaR2/QjRE4WAMDOLxjaATFwerAvFHb2MkdIVDNU6cTpxo0bMJlM8PHxsSr38fHBmTNnpO309HQcOHAAP//8823POXXqVEycOFHaLuhxIiKqjvJvXEPmL0tg2BoL5Fl60+2C7oMuaiQcH+gJhVIlc4RENUuVTpxs5erqiuTkZJvqajQaaDTsqiai6i3/2gXo1yxC1q51gCkfAGDfuBV0UaPgcH8XLotCVEmqdOLk6ekJlUpVLClKTk6Gry/nGiGi2ifv4lno4xYie99GQJgBAOrm7eEycDTUzTswYSKqZFU6cVKr1QgNDcWWLVukAeNmsxlbtmzBuHHj5A2OiOgeMp47Dn3sQuQc3i6Vadp2hm7gaGjuay1fYES1jOyJU2ZmJs6dOydtJyYmIj4+Hh4eHggKCsLEiRMRHR2Ndu3aoUOHDpgzZw4MBgNGjBghY9RERJVPCAHjqUOWhXeP/2EpVCjgGNYT2qiRUNdrIm+ARLWQ7InToUOH8NBDD0nbBQO3o6OjsWTJEgwZMgTXr1/HtGnTkJSUhDZt2mDDhg3FBowTEdUUQgjkxu+GPnYhjH/FWwqVKjg92BfaATGwr1tf1viIarMqNY+THMozdwMRUWUSZjNyDmyBPm4h8s7/++SwvRrO3aKg7T8cdt515Q2QqIaqMfM4ERHVBiI/D9l7N1gW3r2aCABQaBzhHPE4tP2GQeXuJXOERFSAiRMRkUyEMRdZO9ZAv3YxTNevAgAUzjpoI5+Ec5+hUOnc5A2QiIph4kREdI+Zc7Jg2PITMtctgzn1OgBA6eIObd9n4NxzCJROWpkjJKLSMHEiIrpHzIYMZP6+AobflsOsTwMAqDx8oO0fDafug6DUOMobIBHdFhMnIqJKZsq4hcxfv4Nh40qI7EwAgMonELoBMXDq0p/ryBFVI0yciIgqielmMvTrliJry88QxhwAgF1AQ+iiRsExvBcUKv4KJqpu+K+WiKiC5Sddgn7tYmTtWPPfOnINmkM3cBQcQrtBoVTKHCER3SkmTkREFSTv0jno1yxC9p7f/ltHrmkodFGjoGkVznXkiGoAJk5ERHfJ+M8p6GO/Rs7BrVKZpnUn6AaOgqbJ/TJGRkQVjYkTEdEdyj39p2UduaN7pDKHDj2gixoFdYNmMkZGRJWFiRMRUTkIIZB7bK9lHbkzf1oKlSo4duwNXdRI2Ac0lDdAIqpUTJyIiGwgzGbkHN4OfexC5P1z0lKosoNTtwHQPRIDO58AeQMkonuCiRMRURmEKR/Z+363rCN3+W8AgELtAKcej0LXLxqqOj4yR0hE91K5Eqe0tDTExsZi165duHDhArKysuDl5YW2bdsiMjISHTt2rKw4iYjuKZFnRNbOXyzryCVfAgAoHLVw7jUE2oeHQuVaR+YIiUgONiVOV69exbRp07B8+XL4+/ujQ4cOaNOmDRwdHXHr1i1s27YNH330EYKDgzF9+nQMGTKksuMmIqoU5txsZG1ZDf26JTDfSgEAKHVu0D78NJx7DYHS2UXmCIlITjYlTm3btkV0dDQOHz6MZs1KflIkOzsbcXFxmDNnDi5duoRXX321QgMlIqpM5qxMGDauROb6b2HOSAUAKN29oOsXDacej0Lp4CRzhERUFSiEEOJ2lW7evIk6dWzvli5vfTllZGTA1dUV6enpcHHh/ySJahtTRioMvy1H5u8rILL0AACVl79lHbmuA6CwV8scIRFVtvLkAjb1OJU3CaouSRMR1V6mWynI/HUZDJt/gsjNBgDY1W0A3YAYOHbqw3XkiKhE5f7NsHbt2hLLFQoFHBwc0KhRI9SvX/+uAyMiqgz5KVeQ+csSGLbFAvl5AAD7ek2gGzgaDu27cx05IipTuROnqKgoKBQKFL3DV1CmUCjw4IMPIi4uDu7u7hUWKBHR3ci7kojMNYuQtftXwGwCAKhD2kAXNRqaNp24jhwR2aTc/7XatGkT2rdvj02bNiE9PR3p6enYtGkTwsLCsG7dOuzcuRM3b97k4HAiqhKM58/g5qevIuXVgcjauRYwm6Bp+QA8p30DzxlL4ND2QSZNRGSzcvc4TZgwAf/73/+s5mzq0aMHHBwc8Oyzz+LkyZOYM2cOYmJiKjRQIqLyyP3rKPSxXyP3yC6pzCG0G3QDR0HdqKWMkRFRdVbuxOnvv/8uccS5i4sL/vnnHwBA48aNcePGjbuPjoioHIQQyD1xAPq4r2E8edBSqFDCMbwXdFGjYB/UWN4AiajaK3fiFBoaikmTJmHZsmXw8vICAFy/fh2TJ09G+/btAQBnz55FYGBgxUZKRFQKIQRy/txhWUfu3HFLocoOTp37QTcgBnZ+wfIGSEQ1RrkTp2+++QYDBgxAQECAlBxdunQJDRo0wJo1awAAmZmZeOONNyo2UiKiIoTZhOw/NlnWkbv4l6XQXgPn7gOh7T8cdp5+8gZIRDWOTRNgFmU2m7Fx40b89ZflF1VISAh69uwJZTV8jJcTYBJVPyI/D1m7f0XmmkXIv3YBAKBwcIJzz8HQ9h0GlRvnkiMi25UnF7ijxKkmYeJEVH0IYw4M2+KQ+csSmG5cAwAonF2g7TMU2t5PQql1lTlCIqqOKnzm8KJ27NiBjz76CKdPnwYANGvWDJMmTULnzp3v5HRERGUyZxtg2LwKmeuWwZx+EwCgdK0Dbb9hcI54HEpHZ5kjJKLaotyJ03fffYcRI0Zg0KBBGD9+PABg9+7d6NGjB5YsWYKnnnqqwoMkotrJnJmOzA0/IPO35RCGDACAytMP2v7D4fxQFBRqB5kjJKLapty36po2bYpnn30WL7/8slX5J598gq+//lrqhaoueKuOqOoxpd20rCO36UeInCwAgMo3CLqokXB6sC8UdvYyR0hENUmljnHSaDQ4efIkGjVqZFV+7tw5tGjRAjk5OeWPWEZMnIiqjvwb1yzryG2NBfJyAQB2QfdBFzUSjg/0hEKpkjlCIqqJKnWMU2BgILZs2VIscdq8eTPnbiKiO5J/7QL0axYha9c6wJQPALBv1BK6gaPgcH9XLolCRFVGuROnV155BePHj0d8fLy07MqePXuwZMkSfPbZZxUeIBHVXHkXz0IftxDZ+zYCwgwAUDdvb1l4t0UHJkxEVOWUO3F64YUX4Ovri48//hg//vgjAMu4p5UrV2LAgAEVHiAR1TzGc8ehj12InMPbpTJN287QDRwNzX2t5QuMiOg2OI8TxzgR3RNCCBhPH7YsvHv8D0uhQgHHsJ7QRo2Eul4TeQMkolqr0udxIiKylRACufG7oY9bCGNCvKVQqYLTg32hHRAD+7r1ZY2PiKg8bEqc3N3dbR5rcOvWrbsKiIhqBmE2I+fAFujjFiLv/BlLob0azt2iLOvIedeVN0AiojtgU+I0Z86cSg6DiGoKYcpH9p7foF+zCPlX/gEAKDSOcI54HNp+w6By95I5QiKiO2dT4hQdHV3ZcRBRNSfyjMjasQb6NYtgun4VAKBw0kHb+0k49xkKlc5N3gCJiCqATYmTwWCAs7Pta0GVtz4RVV/mnCxkbfkZ+nVLYU69DgBQurhD+/AzcO41GEonncwREhFVHJsSp0aNGmHChAmIjo6Gn59fiXWEENi8eTM++eQTdOnSBVOnTq3QQMtSr149uLi4QKlUwt3dHdu2bbtn1yaqrcyGDBg2rkTm+uUw61MBACoPH2j7R8Op+yAoNY4yR0hEVPFsSpy2b9+O//u//8OMGTPQunVrtGvXDv7+/nBwcEBqaipOnTqFffv2wc7ODlOnTsVzzz1X2XEXs3fvXmi12nt+XaLaxpRxC5nrl8Pw+wqI7EwAgMo7wLKOXOd+UNirZY6QiKjy2JQ4hYSE4Oeff8bFixexatUq7Nq1C3v37kV2djY8PT3Rtm1bfP311+jTpw9UKq4lRVQTmW4lQ//LMmRt/Qki17ImpV1AA+iiRsExPBIKFWc3IaKaT/YJMHfu3InZs2fj8OHDuHbtGmJjYxEVFWVVZ/78+Zg9ezaSkpLQunVrzJ07Fx06dJD2169fHx4eHlAqlXjppZcwdOhQm6/PCTCJypaffBn6tYuQtWMtkJ8HALBv0Ay6qFFwaPcQFEqlzBESEd2dajUBpsFgQOvWrRETE4NBgwYV279y5UpMnDgRCxYsQFhYGObMmYPIyEgkJCTA29sbALB7927UrVsX165dQ0REBFq2bIlWrVrd649CVKPkXf4b+rhvkL13A2A2AQDUTe6HbuAoaFp15DpyRFQryd7jVJhCoSjW4xQWFob27dtj3rx5AACz2YzAwEC8+OKLmDJlSrFzTJo0Cc2bN8fw4cNLvEZubi5yc3Ol7YyMDAQGBrLHiehfxn9OQR+3EDkHtkhlmtadoIsaBU3T+2WMjIioclSrHqeyGI1GHD582OoJPaVSiYiICOzbtw+ApcfKbDZDp9MhMzMTW7duxeDBg0s95/vvv4+ZM2dWeuxE1U3u6T+hj1uI3KN7pDKHDj2gixoFdYNmMkZGRFR1VOnE6caNGzCZTPDx8bEq9/HxwZkzliUckpOTMXDgQACAyWTC6NGj0b59+1LPOXXqVEycOFHaLuhxIqqNhBDIPbYX+tiFMJ7501KoVMGxY2/oBsTAPrCRvAESEVUxVTpxskWDBg1w9OhRm+trNBpoNJpKjIio6hNmM3IOb4c+diHy/jlpKbSzh1PXR6B7JAZ2PgHyBkhEVEXZ/DjMrFmzkJ2dLW3v2bPHaqyQXq/HmDFjKjQ4T09PqFQqJCcnW5UnJyfD19e3Qq9FVBsIUz6ydq9HymuP49bHLyPvn5NQqB3g3Odp+H7+K9xHT2PSRERUBpsTp6lTp0Kv10vbffr0wZUrV6TtrKwsfPXVVxUanFqtRmhoKLZs+W+QqtlsxpYtWxAeHl6h1yKqyUR+HgxbVyN5YhRS501F/qVzUDhqoY0aBZ95v8EtehJUHj63PxERUS1n8626og/fVdTDeJmZmTh37py0nZiYiPj4eHh4eCAoKAgTJ05EdHQ02rVrhw4dOmDOnDkwGAwYMWJEhVyfqCYz52Yja+tqZP6yFKZblp5bpc4d2oeHwrnXECid+SQpEVF5yD7G6dChQ3jooYek7YKB29HR0ViyZAmGDBmC69evY9q0aUhKSkKbNm2wYcOGYgPGieg/5qxMGDatROav38KcYVlHTunuBV2/aDj1eBRKByeZIyQiqp5kT5y6det2296rcePGYdy4cfcoIqLqy6RPg+G35cj8/QcIg+XWusq7LnSPjIBT1wFcR46I6C6VK3FauHChtJBufn4+lixZAk9PTwCwGv9ERPeWKfU6Mtctg2HzKohcy0McdnUbQDcgBo6d+nAdOSKiCmLzzOH16tWzaYmFxMTEuw7qXuJadVSd5adcQeYvS2DYHgfkGQEA9vWaQDdwNBzad+c6ckRENqiUmcPPnz9/t3ERUQXJu3oemXHfIGvPesCUDwBQh7SBLmo0NG06cR05IqJKwv57omrEeP4MMuO+Qfb+TcC/ncWalg9AN3A01E1DmTAREVUymxOnffv24ebNm+jXr59UtmzZMkyfPh0GgwFRUVGYO3cuZ+UmqgS5fx2FPvZr5B7ZJZU5tHsIuoGjoG7YQsbIiIhqF5sTp7feegvdunWTEqfjx49j5MiRGD58OJo2bYrZs2fD398fM2bMqKxYiWoVIQRyTxyAPu5rGE8etBQqlHAMj4QuaiTsgxrLGyARUS1kc+IUHx+Pt99+W9pesWIFwsLC8PXXXwMAAgMDMX36dCZORHdJCIGcP3dY1pE7d9xSqLKDU5f+0D0yAnZ+wfIGSERUi9mcOKWmplpNOrljxw706dNH2m7fvj0uXbpUsdER1SLCbEL2H5ugj/sG+Rf/shTaa+DcYxC0/YbDzpPrMxIRyc3mxMnHxweJiYkIDAyE0WjEn3/+iZkzZ0r79Xo97O3tKyVIoppM5Ocha/evyFyzCPnXLgAAFA5OcO41BNqHn4HKrY7MERIRUQGbE6eHH34YU6ZMwYcffoi4uDg4OTmhc+fO0v5jx46hYcOGlRIkUU0kjDkwbItD5i9LYLpxDQCg0LpC2+cpaCOfglLLecWIiKoamxOnt99+G4MGDULXrl2h1WqxdOlSqNX/Ld+waNEi9OrVq1KCJKpJzDlZMGxahcxfl8GcdgMAoHTzhLbvMDhHPAalo7PMERIRUWlsnjm8QHp6OrRaLVQqlVX5rVu3oNVqrZKp6oAzh9O9Ys7MQObv3yPzt+8hMtMBACpPP2gfGQHnblFQqDmVBxGRHCpl5vACrq6uJZZ7eHiU91REtYIp7SYy138Lw6YfIbINAAA7v2BoB8TA6cG+UNhxbCARUXVhc+IUExNjU71FixbdcTBENUn+jWuWdeS2xgJ5uQAAu6D7oBs4Co5hEVAoVbc5AxERVTU2J05LlixBcHAw2rZti3Le3SOqVfKTLkK/ZhGydv4irSNn36ilZeHd+7twWRQiomrM5sTphRdewA8//IDExESMGDECTz/9NG/PERWSd+ks9LHfIHvf74AwAwDUzdtbFt5t0YEJExFRDVCuweG5ublYvXo1Fi1ahL1796Jv374YOXIkevXqVW3/KHBwON0t498noI9diJxD26QyTdvO0A0cDc19rWWMjIiIbFGeXKDcT9UVuHDhApYsWYJly5YhPz8fJ0+ehFarvaOA5cTEie6EEALG04ctC+8e/8NSqFDAMawntFEjoa7XRN4AiYjIZpX6VF0BpVIJhUIBIQRMJtOdnoaoWhFCIDd+j2Xh3YR4S6FSBacH+0I7IAb2devLGh8REVWuciVOhW/V7d69G/369cO8efPQu3dvKJXKyoqRSHbCbEbOgS3Qxy1E3vkzlkJ7NZy7RUHbfzjsvOvKGyAREd0TNidOY8aMwYoVKxAYGIiYmBj88MMP8PT0rMzYiGQn8vOQvXeDZeHdq4kAAIXGEc4Rj0PbbxhU7l4yR0hERPeSzWOclEolgoKC0LZt2zIHgq9evbrCgrsXOMaJSiKMucjasQb6tYthun4VAKBw1kEb+SSc+wyFSucmb4BERFRhKmWM07Bhw6rtk3NEtjLnZMGw5SdkrlsGc+p1AIDSxR3avs/AuecQKJ2q3wMQRERUcco1ASZRTWU2ZCDz9xUw/LYcZn0aAEDl4QNt/2g4dR8EpcZR3gCJiKhKuOOn6ohqAlPGLWT++h0MG1dCZGcCAFQ+gdANiIFTl/5cR46IiKwwcaJayXQzGfp1S5G15WcIYw4AwC6gIXRRo+AY3gsKFf9pEBFRcfzrQLVKftIl6NcuRtaONf+tI9egOXQDR8EhtBsUnFaDiIjKwMSJaoW8S+egX/MNsvds+G8duaah0EWNgqZVOB98ICIimzBxohrN+M8p6GO/Rs7BrVKZpnUn6AaOgqbJ/TJGRkRE1RETJ6qRcs/8aVlH7uheqcyhQw/ookZB3aCZjJEREVF1xsSJagwhBHKP7YU+diGMZ/60FCpVcOzYG7qokbAPaChvgEREVO0xcaJqT5jNyDm0zbKO3D+nLIV29nDq+gh0j8TAzidA3gCJiKjGYOJE1ZYw5SN73++WdeQu/w0AUKgd4NTjMej6D4PKw0fmCImIqKZh4kTVjsgzImvnL5Z15JIvAQAUjlo4Rz4B7cNDoXLxkDlCIiKqqZg4UbVhzs1G1pbV0K9bAvOtFACAUucO7cNPwzlyCJROOpkjJCKimo6JE1V55iw9DBt/ROb6b2HOSAUAKN29oOs/3LKOnIOTzBESEVFtwcSJqixTRioMvy1H5u8rILL0AACVd13oHomBU9dHoLBXyxwhERHVNkycqMox3UpB5q/LYNi8CiL333Xk6jaALmokHDv25jpyREQkG/4FoiojP+WyZR257WuA/DwAgH39ppZ15Np15zpyREQkOyZOJLu8K4mWdeR2rwfMJgCAOqStZR25Np24jhwREVUZ1T5xSktLQ0REBPLz85Gfn48JEyZg9OjRcodFNjAmnoY+7hvkHNgMCAEA0LQKh27gaGiahsocHRERUXHVPnHS6XTYuXMnnJycYDAY0KJFCwwaNAh16tSROzQqRW5CPPRxC5F7ZJdU5tD+Ics6cg1byBgZERFR2ap94qRSqeDkZHkcPTc3F0IIiH97L6jqEEIg9/gflnXkTh+yFCqU/64jFwP7wMbyBkhERGQD2Ufb7ty5E/3794e/vz8UCgXi4uKK1Zk/fz7q1asHBwcHhIWF4cCBA1b709LS0Lp1awQEBGDSpEnw9PS8R9HT7QizGdmHtuH6G0/j5nvPW5ImlR2cug+Cz6dr4PHi+0yaiIio2pA9cTIYDGjdujXmz59f4v6VK1di4sSJmD59Ov7880+0bt0akZGRSElJkeq4ubnh6NGjSExMxPfff4/k5OR7FT6VQphNyNrzG1Jeexy3PnoJeX+fgELtAOfeT8Hns1/h/ux02PkGyR0mERFRuShEFbqvpVAoEBsbi6ioKKksLCwM7du3x7x58wAAZrMZgYGBePHFFzFlypRi5xgzZgy6d++Oxx57rMRr5ObmIjc3V9rOyMhAYGAg0tPT4eLiUrEfqBYS+Xn/rSOXdBEAoHB0hnOvIdA+/DRUrhx7RkREVUtGRgZcXV1tygWq9Bgno9GIw4cPY+rUqVKZUqlEREQE9u3bBwBITk6Gk5MTdDod0tPTsXPnTrzwwgulnvP999/HzJkzKz322sacm42srauR+ctSmG5ZevwUWldo+wyFNvJJKLVMSomIqPqr0onTjRs3YDKZ4OPjY1Xu4+ODM2fOAAAuXLiAZ599VhoU/uKLL6Jly5alnnPq1KmYOHGitF3Q40R3xpyVCcOmlchc/x3M6bcAAEo3T2j7DYNzxONcR46IiGqUKp042aJDhw6Ij4+3ub5Go4FGo6m8gGoJkz7t33XkfoAw/LuOnJc/dI+MgFPXAVCo2cZERFTzVOnEydPTEyqVqthg7+TkZPj6+soUVe1mWUfu23/XkcsGANj51/9vHTk7e5kjJCIiqjxVOnFSq9UIDQ3Fli1bpAHjZrMZW7Zswbhx4+QNrpbJT7mCzLWLYdge9986cvVCoIsaDYcO3aFQquQNkIiI6B6QPXHKzMzEuXPnpO3ExETEx8fDw8MDQUFBmDhxIqKjo9GuXTt06NABc+bMgcFgwIgRI2SMuvYoeR25NtBFjeY6ckREVOvInjgdOnQIDz30kLRdMHA7OjoaS5YswZAhQ3D9+nVMmzYNSUlJaNOmDTZs2FBswDhVrFLXkYsaBXXTUCZMRERUK1WpeZzkUJ65G2qD3IQj0McuRG78bqmM68gREVFNVmPmcaJ7QwiB3GP7oI9bCOPpw5ZChRKOnXpDN4DryBERERVg4lSLCbMZOYe3Qx+3EHl/n7QUquzg1HUAdI+MgJ0v57ciIiIqjIlTLSRM+cjetxH6uIXIv/w3AEChdoBTj0eh6xcNVR2OHyMiIioJE6daROQZ/1tHLvkSAEDhqIVz5BPQPjwUKhcPmSMkIiKq2pg41QLmnCxkbV0N/bqlMN9KAQAode7QPjwUzr2GQOnMQfFERES2YOJUg5kNGTBsXInM9cth1qcCAJQe3tD1i4ZT90FcR46IiKicmDjVQKb0m8j87XsYfl8BkZ0JAFB5B0A3IAZOXfpDYa+WOUIiIqLqiYlTDZJ/IwmZ65Yia+tqCGMOAMAuoKFlHbnwSChU/HYTERHdDf4lrQHyr12Afu1iZO38BTDlAwDsGzaHbuBoONzfFQqlUuYIiYiIagYmTtVY3oW/LOvI7dsICDMAQN2sHXQDR0PTIozLohAREVUwJk7VkPHsMejjFiLn8A6pzOH+LtAOGAlNSBv5AiMiIqrhmDhVE0II5J44AH3c1zCePGgpVCjgGNYTuoGjYB8cIm+AREREtQATpypOmM3I+XMH9LELkff3CUuhyg5OnftB+8gI2PvXkzU+IiKi2oSJUxUlLYuy5hvkXzoH4N9lUboPgrbfMNh5+skcIRERUe3DxKmKEcZcZO1cC/3aJTClXAZQaFmUPk9B5VpH5giJiIhqLyZOVYQ5JwuGzauQ+eu3MKdeB8BlUYiIiKoaJk4yM+nTYNjwAzI3fA9hyAAAqDx8oO3/77IoGkeZIyQiIqICTJxkYrqVgsxfv4Vh8yqI3GwAgJ1fMLSPjIBT535Q2NnLHCEREREVxcTpHstPugj92iXI2rkWyM8DANjXC4EuahQcOvSAQqmSOUIiIiIqDROneyTvQgL0cd8g+49N/83yHdIWuqiR0LR5kLN8ExERVQNMnCpZ7pk/oV+zCLlHdkllmradoRsQA02T+2WMjIiIiMqLiVMlyU+6iNQvp8GYcMRSoFDC8YGe0A6IgbpeE3mDIyIiojvCxKmSKF3ckXfpHGBnD6cuj0D3yHDY+QbJHRYRERHdBSZOlUTppIPHix/APrgxVB4+codDREREFYCJUyVyaPug3CEQERFRBVLKHQARERFRdcHEqRIJIeQOgYiIiCoQb9VVEiEEVq1aBZ1Oh8DAQAQFBcHFhevNERERVWdMnCpJeno60tLSkJaWhkuXLmHv3r1wc3NDUFAQgoKC4OvrC6WSHX5ERETViULU8vtJGRkZcHV1RXp6eoX2CAkhkJqaiosXL+LSpUtISkqyunVnb2+PgIAABAUFITAwEE5OThV2bSIiIrJdeXIBJk6VlDgVlZubi8uXL0uJVE5OjtV+Ly8v6Zael5cXl2AhIiK6R5g4lcO9SpwKE0Lg+vXruHjxIi5evIgbN25Y7XdwcJCSqICAAGg0mnsSFxERUW3ExKkc5EicisrKysKlS5dw8eJFXL58GXl5edI+hUIBHx8fKZHy8PBgbxQREVEFYuJUDlUhcSrMbDYjKSlJuqWXmppqtd/JyUkaF1W3bl2o1WqZIiUiIqoZmDiVQ1VLnIrS6/VSb9TVq1eRn58v7VMoFPD19ZUSKXd3d/ZGERERlRMTp3Ko6olTYfn5+Va9Uenp6Vb7nZ2dERgYyN4oIiKicmDiVA7VKXEqKj09HZcuXcKlS5dw9epVmEwmaZ9CoYCfn5+USLE3ioiIqGRMnMqhOidOheXn5+Pq1atSIpWRkWG139nZGQEBAVJvFJ/UIyIismDiVA41JXEq6na9UT4+PlIi5enpyd4oIiKqtZg4lUNlJk4JicDZ80C/hyr0tOVWMDaqIJFKS0uz2u/g4ICAgADpxVnMiYioNql1idPAgQOxfft29OjRAz/99FO5jq2sxOnoGWDgWECjBrYtA3y9KuzUd63gSb2C3qjC80YBQJ06daTeKB8fH6hUKpkiJSIiqny1LnHavn079Ho9li5dWmUSJ7MZeGw8cPgk8Eh3YO6bFXbqCmU2m5GcnIxLly7h8uXLxWYxt7Ozg7+/v9Qb5erqytt6RERUo9S6xAmwJE/z5s2rMokTAJw4C/R7DhACWPEpEN6mQk9fKbKzs3H58mVcunQJV65cQXZ2ttV+rVYrJVH+/v5wcHCQKVIiIqKKUZ5cQHmPYirVzp070b9/f/j7+0OhUCAuLq5Ynfnz56NevXpwcHBAWFgYDhw4cO8DvQMtGgND+1veT/8cyMsvu35V4OjoiMaNG6N79+54+umnMWjQIHTo0AF169aFUqlEZmYmzpw5g82bN+Pbb79FbGwsDhw4UGwAOhERUU1kJ3cABoMBrVu3RkxMDAYNGlRs/8qVKzFx4kQsWLAAYWFhmDNnDiIjI5GQkABvb28ZIi6fSSOBX7dbBooviwNGPiZ3RLZTKBTw9PSEp6cn2rRpI015cPnyZVy5cgWpqam4fv06rl+/jvj4eNjZ2cHPzw9169ZFQEAA544iIqIap0rdqlMoFIiNjUVUVJRUFhYWhvbt22PevHkALGNyAgMD8eKLL2LKlClSPVtv1eXm5iI3N1fazsjIQGBgYKVOR/DDOmDKx4DOGdi6DPD2qJTL3HOZmZm4cuWK9Cp6W8/R0RF169aVXlqtVqZIiYiISleeW3Wy9ziVxWg04vDhw5g6dapUplQqERERgX379t3ROd9//33MnDmzokK0yeA+wPfrgGMJwAdfAZ9Mvf0x1YFWq0VISAhCQkIghMCtW7ek3qhr164hOzsb586dw7lz5wAALi4uUhLF8VFERFQdVenE6caNGzCZTPDx8bEq9/HxwZkzZ6TtiIgIHD16FAaDAQEBAVi1ahXCw8NLPOfUqVMxceJEabugx6kyqVTA2xOAAWOAnzcCT/YD2res1EvecwqFAnXq1EGdOnXQunVr5OfnIzk5GVeuXMHVq1dx/fp1ZGRkICMjA6dPnwZgmfagIIny9fXl2npERFTlVenEyVabN2+2ua5Go5FluZE2TYEhDwMr1wPTPgfWLbAkVDWVnZ2d1LsEWHoPr169iqtXr0rjo27evImbN2/i2LFjUCgU8PLygr+/v5RI2dnViB9PIiKqQar0XyZPT0+oVCokJydblScnJ8PX11emqO7ca6OB33YCp84By38BhkXJHdG9o1arUa9ePdSrVw8AkJWVJSVSV69eRUZGBlJSUpCSkoL4+HgolUp4e3vDz88Pfn5+8PHxgb29vbwfgoiIar0qnTip1WqEhoZiy5Yt0oBxs9mMLVu2YNy4cfIGdwfquAGvxlh6nGZ/A/TtZimrjZycnNCoUSM0atQIgGWgeeFEKjMzE0lJSUhKSsKRI0egUCiKJVK8tUdERPea7IlTZmamNHgYABITExEfHw8PDw8EBQVh4sSJiI6ORrt27dChQwfMmTMHBoMBI0aMkDHqOzf0EWDFr8Cpv4FZC4EPX5U7oqpBq9Xivvvuw3333QchBPR6Pa5evYpr167h6tWrMBgMSE5ORnJyMuLj46WpEnx9feHn5wdfX18ONiciokon+3QE27dvx0MPFV8FNzo6GkuWLAEAzJs3D7Nnz0ZSUhLatGmDzz//HGFhYRVy/cqcObw0B49blmNRKIC4+ZbxT1S6gkTq2rVr0kuv1xer5+bmBl9fXymZ0mq1nEeKiIhuq1YuuXKn5EicAODl94DVmyyzi6+eZ1kMmGyXmZkpJVHJyclITU0tVsfZ2Rk+Pj7Sy9PTE0ql7JPlExFRFcPEqRzkSpySbwIR0UCGwfK03YevWnqg6M7k5ORIY6KSkpJw/fp1FP3RVqlU8Pb2tkqmeHuPiIiYOJWDXIkTAGzbD4yYalkE+K3xQPTAe3r5Gi0/Px8pKSnSuKjk5GSrGeMLuLi4wNvbG15eXvD29pae5CQiotqDiVM5yJk4AcCCH4D3/wfYqYDvPgLC29zzEGoFIQTS09ORlJQkJVQl3d5TKpWoU6eOlEx5eXnB1dWVt/iIiGowJk7lIHfiJAQw/h1g7VbAwxVYuwAIrH5TVFVLubm5uH79ujR/VEpKCnJycorVs7e3lxY79vLygqenJ1xdXTnwnIiohmDiVA5yJ04AkJ1jecruxFmgWUPg57mAk6MsodRqBU/vFSRR169fl5b9KaogmapTp4701c3Njbf5iIiqISZO5VAVEicAuJIMPPICcCMV6NsVmD+dg8WrArPZjLS0NCmJun79Om7evFliMqVUKuHu7i4lUx4eHvDw8OAAdCKiKo6JUzlUlcQJAA4cA56cCOSbgMmjgLFDZQ2HSmE2m5GamoobN25I6+3dvHkTRqOxxPpOTk7w8PCAu7u7lEy5u7tzLT4ioiqCiVM5VKXECQCWrwX+71NLb9M37wI9wuWOiGxRcJuvIIm6ceMGUlNTS5yos4CLiwvc3Nykl7u7O9zc3GRZhJqIqDZj4lQOVS1xAoDXPwW+WwtonYC4L4DGwXJHRHfKaDQiNTUVt27dkr7eunWrxEHoBRwdHeHm5gZXV1erl06nYy8VEVElYOJUDlUxcTLmAUNftdy68/IA3n0ZiHxQ7qioImVlZSEtLQ1paWlITU2V3hsMhjKP02q1cHV1hYuLC1xcXKDT6aSv7KkiIrozTJzKoSomToBlkPjgl4C/L1q2+3QBZo4HfOrIGhZVMqPRiPT0dKSmpiI9PR3p6enIyMhAeno68vLyyjxWrVZDp9NJyZRWq5Vezs7OcHBw4BQKREQlYOJUDlU1cQKAnFzg82+Br1ZYBoy7OAP/9wLwxMN84q62EUIgOztbSqIKEiq9Xg+9Xl/mrb8CKpVKSqIKvjo5OUlfC16c7JOIahsmTuVQlROnAqfOAa99BBxLsGw/0Bp4/xWgQaC8cVHVkZeXJyVRer0eGRkZyMzMhMFgQGZmJrKzs20+l6Ojo5REOTo6lvpycHBgkkVENQITp3KoDokTAJhMwOLVwEeLLBNmauyBCdHAs0MAe44XptswmUxSElXwysrKQlZWFgwGg/S+vL8O1Go1NBoNHBwcir00Go20v+h7JlxEVJUwcSqH6pI4Fbh4DXj9E2DnIcu2qw5o3xLo0AoIawU0b8xEiu5Mwe3AgiQqKysLOTk5yMrKQnZ2tvQ+JyenXD1YJbG3t4darYZarbZ6X3jb3t6+zJednR3s7Ow4WzsR3TUmTuVQ3RInwLK+Xexm4J0vgJtp1vucHIDQFpZEqkNLoEGQZQ08O/5toQpkNpthNBqRk5NT7JWbmyt9zc3NhdFolN7fboD7nVAoFLCzs7NKpgonVbf7WtpLqVRafS3pvVKphEKh4KB7omqOiVM5VMfEqUC+CTh51jJtwYFjwIHjQFpGyXXdXIA6roCHG1Dn35eHG+DsCDioAQcNoFEDGo1lW/Nvmb0dYGcH2KssX+3s/i1T/bfPTgWoVIBKCfAODJWlIOEqSKjy8vJgNBpLfOXn5yMvL6/UV1X61VWQRBVOthQKhVV54USrpO2CBKxoQlba+9uVFcRV1teixxV9Fa5TtH7Bdml1ynpf0r7SzlfadllfSysjKg0Tp3KozolTUWYzcPYCsP+oJZE6eBxIvmnpobpXlEpLAqVSWRIqpfLfxOrfMlWhsoK60vtCCVhBEqYqVE+lBJRF9xepo1SW/L7gWKXC+jxKhXW5QllyubLwS2H9XqWyPOVYcK2C90X3Fb6GslCZolDdwtsKhfV24WsrCn8toY5CUfOfvDSZTMjPz5deeXl5Vtv5+flSncJ1C5eV9TKbzcXem81mmM1muT863aHbJVZlldlS15by8p7nXu272/1lJap3e63C2/7+/mjWrFmx+nerPLkAR8PUIEolEFLf8hoWZSkzmYA0PXAzFbiZDtxKs9zeu5kG3Eq3DDTPyQVyjZavOcZC73MtvVp5+UD+vy9jwfvia9wCsCRvZrPlGJKfsoQkTHpfQplSCShQJHm7TZkCJZQpLO8LJ3dQWF+7aHnh+gXnLHz+osdCoYICKiiVGqs6Bb9vSzqm6D6ra9m8X/z3EgKi0HtA/NsTVvr+UutI/8MxF+pNK3TeUs+BUq6FEuoBQpihUOC/coFi57TaX3CsEP++E1AAxa5ffBtQFDpeoOhnspQVnOu/a/5HoSiyLb0R1tv/nruk/ywoCq6rKKHM6lqFt4qe/79rWsdR9jn/O3fJ+257/iLxlFbBelNYvS/+uUpp01LPXfb/vEv7bKXXL1pym3iKaJ7jjErIm8qFiVMNp1L9d2uuIglhSZ5MZktylm8q+av0Mv9XV/pqsi4zFzqX2fzfMebC+/+tY7W/6DkKHyMs+4T479ii782FzlPwvqCOtG3+73xm07/n/bfcLKzrFWybC78XRfaVUEcU3mey/DopvP9OmM0A+0cq2r/ZHhHdc/0MBgzoJ28MTJzojigUljFO9nIHUosUTqxMZkj/cS+anBUkfQX7ULROkUStIEErqV5BmRDFjy1aBhQ6r/ivjtWrUB2B/+Itq7zwtVDkXIU/gyj4/Chep3DnStF4Kmp/4c6S8tSR3qPIdqH3Vucs4RqlnRMo4bxS5RKOKS2GQudHCee67TFFrlfS/pLiLuv4EuuU8r5QkfXQhco8Zyn7i3SqFd/GbfaL4hVLirkirlVUsf13Grut5y/hnJ4ezmWf5B5g4kRUTSgU/44BAxNWIiK58BkoIiIiIhsxcSIiIiKyERMnIiIiIhsxcSIiIiKyERMnIiIiIhsxcSIiIiKyERMnIiIiIhsxcSIiIiKyERMnIiIiIhsxcSIiIiKyERMnIiIiIhvV+rXqxL+rCmZkZMgcCREREcmhIAcQt1uJGEycoNfrAQCBgYEyR0JERERy0uv1cHV1LbOOQtiSXtVgZrMZV69ehU6ng0KhqNBzZ2RkIDAwEJcuXYKLi0uFnrumY9vdHbbfnWPb3R22351j292du2k/IQT0ej38/f2hVJY9iqnW9zgplUoEBARU6jVcXFz4j+AOse3uDtvvzrHt7g7b786x7e7Onbbf7XqaCnBwOBEREZGNmDgRERER2YiJUyXSaDSYPn06NBqN3KFUO2y7u8P2u3Nsu7vD9rtzbLu7c6/ar9YPDiciIiKyFXuciIiIiGzExImIiIjIRkyciIiIiGzExKkSzZ8/H/Xq1YODgwPCwsJw4MABuUOqcnbu3In+/fvD398fCoUCcXFxVvuFEJg2bRr8/Pzg6OiIiIgInD17Vp5gq5j3338f7du3h06ng7e3N6KiopCQkGBVJycnB2PHjkWdOnWg1Wrx6KOPIjk5WaaIq5Yvv/wSrVq1kuZ8CQ8Px2+//SbtZ9vZ7oMPPoBCocBLL70klbH9SjZjxgwoFAqrV5MmTaT9bLfbu3LlCp5++mnUqVMHjo6OaNmyJQ4dOiTtr+y/G0ycKsnKlSsxceJETJ8+HX/++Sdat26NyMhIpKSkyB1alWIwGNC6dWvMnz+/xP2zZs3C559/jgULFmD//v1wdnZGZGQkcnJy7nGkVc+OHTswduxY/PHHH9i0aRPy8vLQq1cvGAwGqc7LL7+MX375BatWrcKOHTtw9epVDBo0SMaoq46AgAB88MEHOHz4MA4dOoTu3btjwIABOHnyJAC2na0OHjyIr776Cq1atbIqZ/uVrnnz5rh27Zr02r17t7SP7Va21NRUdOrUCfb29vjtt99w6tQpfPzxx3B3d5fqVPrfDUGVokOHDmLs2LHStslkEv7+/uL999+XMaqqDYCIjY2Vts1ms/D19RWzZ8+WytLS0oRGoxE//PCDDBFWbSkpKQKA2LFjhxDC0lb29vZi1apVUp3Tp08LAGLfvn1yhVmlubu7i4ULF7LtbKTX60Xjxo3Fpk2bRNeuXcWECROEEPzZK8v06dNF69atS9zHdru91157TTz44IOl7r8XfzfY41QJjEYjDh8+jIiICKlMqVQiIiIC+/btkzGy6iUxMRFJSUlW7ejq6oqwsDC2YwnS09MBAB4eHgCAw4cPIy8vz6r9mjRpgqCgILZfESaTCStWrIDBYEB4eDjbzkZjx45F3759rdoJ4M/e7Zw9exb+/v5o0KABhg4diosXLwJgu9li7dq1aNeuHR5//HF4e3ujbdu2+Prrr6X99+LvBhOnSnDjxg2YTCb4+PhYlfv4+CApKUmmqKqfgrZiO96e2WzGSy+9hE6dOqFFixYALO2nVqvh5uZmVZft95/jx49Dq9VCo9Hg+eefR2xsLJo1a8a2s8GKFSvw559/4v333y+2j+1XurCwMCxZsgQbNmzAl19+icTERHTu3Bl6vZ7tZoN//vkHX375JRo3bozff/8dL7zwAsaPH4+lS5cCuDd/N2r9Ir9ENcHYsWNx4sQJq7ESdHshISGIj49Heno6fvrpJ0RHR2PHjh1yh1XlXbp0CRMmTMCmTZvg4OAgdzjVSp8+faT3rVq1QlhYGIKDg/Hjjz/C0dFRxsiqB7PZjHbt2uG9994DALRt2xYnTpzAggULEB0dfU9iYI9TJfD09IRKpSr2JERycjJ8fX1liqr6KWgrtmPZxo0bh3Xr1mHbtm0ICAiQyn19fWE0GpGWlmZVn+33H7VajUaNGiE0NBTvv/8+Wrdujc8++4xtdxuHDx9GSkoK7r//ftjZ2cHOzg47duzA559/Djs7O/j4+LD9bOTm5ob77rsP586d48+dDfz8/NCsWTOrsqZNm0q3O+/F3w0mTpVArVYjNDQUW7ZskcrMZjO2bNmC8PBwGSOrXurXrw9fX1+rdszIyMD+/fvZjrA8cjtu3DjExsZi69atqF+/vtX+0NBQ2NvbW7VfQkICLl68yPYrhdlsRm5uLtvuNnr06IHjx48jPj5eerVr1w5Dhw6V3rP9bJOZmYm///4bfn5+/LmzQadOnYpNu/LXX38hODgYwD36u1EhQ8ypmBUrVgiNRiOWLFkiTp06JZ599lnh5uYmkpKS5A6tStHr9eLIkSPiyJEjAoD45JNPxJEjR8SFCxeEEEJ88MEHws3NTaxZs0YcO3ZMDBgwQNSvX19kZ2fLHLn8XnjhBeHq6iq2b98url27Jr2ysrKkOs8//7wICgoSW7duFYcOHRLh4eEiPDxcxqirjilTpogdO3aIxMREcezYMTFlyhShUCjExo0bhRBsu/Iq/FSdEGy/0rzyyiti+/btIjExUezZs0dEREQIT09PkZKSIoRgu93OgQMHhJ2dnXj33XfF2bNnxfLly4WTk5P47rvvpDqV/XeDiVMlmjt3rggKChJqtVp06NBB/PHHH3KHVOVs27ZNACj2io6OFkJYHi198803hY+Pj9BoNKJHjx4iISFB3qCriJLaDYBYvHixVCc7O1uMGTNGuLu7CycnJzFw4EBx7do1+YKuQmJiYkRwcLBQq9XCy8tL9OjRQ0qahGDblVfRxIntV7IhQ4YIPz8/oVarRd26dcWQIUPEuXPnpP1st9v75ZdfRIsWLYRGoxFNmjQR//vf/6z2V/bfDYUQQlRM3xURERFRzcYxTkREREQ2YuJEREREZCMmTkREREQ2YuJEREREZCMmTkREREQ2YuJEREREZCMmTkREREQ2YuJEREREZCMmTkREREQ2YuJERLXC8OHDERUVJXcYRFTNMXEiIiIishETJyKqUX766Se0bNkSjo6OqFOnDiIiIjBp0iQsXboUa9asgUKhgEKhwPbt2wEAly5dwuDBg+Hm5gYPDw8MGDAA58+fl85X0FM1c+ZMeHl5wcXFBc8//zyMRqM8H5CIZGUndwBERBXl2rVrePLJJzFr1iwMHDgQer0eu3btwrBhw3Dx4kVkZGRg8eLFAAAPDw/k5eUhMjIS4eHh2LVrF+zs7PDOO++gd+/eOHbsGNRqNQBgy5YtcHBwwPbt23H+/HmMGDECderUwbvvvivnxyUiGTBxIqIa49q1a8jPz8egQYMQHBwMAGjZsiUAwNHREbm5ufD19ZXqf/fddzCbzVi4cCEUCgUAYPHixXBzc8P27dvRq1cvAIBarcaiRYvg5OSE5s2b46233sKkSZPw9ttvQ6lkxz1RbcJ/8URUY7Ru3Ro9evRAy5Yt8fjjj+Prr79GampqqfWPHj2Kc+fOQafTQavVQqvVwsPDAzk5Ofj777+tzuvk5CRth4eHIzMzE5cuXarUz0NEVQ97nIioxlCpVNi0aRP27t2LjRs3Yu7cuXj99dexf//+EutnZmYiNDQUy5cvL7bPy8urssMlomqIiRMR1SgKhQKdOnVCp06dMG3aNAQHByM2NhZqtRomk8mq7v3334+VK1fC29sbLi4upZ7z6NGjyM7OhqOjIwDgjz/+gFarRWBgYKV+FiKqenirjohqjP379+O9997DoUOHcPHiRaxevRrXr19H06ZNUa9ePRw7dgwJCQm4ceMG8vLyMHToUHh6emLAgAHYtWsXEhMTsX37dowfPx6XL1+Wzms0GjFy5EicOnUK69evx/Tp0zFu3DiObyKqhdjjREQ1houLC3bu3Ik5c+YgIyMDwcHB+Pjjj9GnTx+0a9cO27dvR7t27ZCZmYlt27ahW7du2LlzJ1577TUMGjQIer0edevWRY8ePax6oHr06IHGjRujS5cuyM3NxZNPPokZM2bI90GJSDYKIYSQOwgioqpq+PDhSEtLQ1xcnNyhEFEVwH5mIiIiIhsxcSIiIiKyEW/VEREREdmIPU5ERERENmLiRERERGQjJk5ERERENmLiRERERGQjJk5ERERENmLiRERERGQjJk5ERERENmLiRERERGQjJk5ERERENvp/466RwwY/jXcAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# viz: three learning rates on the same problem\n",
    "fig, ax = plt.subplots(figsize=(6, 4))\n",
    "for eta, label, c in [(0.02, \"η=0.02 (slow)\", \"#999999\"),\n",
    "                      (0.1,  \"η=0.1 (good)\", \"#1E40FF\"),\n",
    "                      (0.45, \"η=0.45 (bouncy)\", \"#E4572E\")]:\n",
    "    _, ls = batch_gd(X_b, y, eta=eta, n_steps=60)\n",
    "    ax.plot(ls, label=label, color=c)\n",
    "ax.set_yscale(\"log\"); ax.set_xlabel(\"step\"); ax.set_ylabel(\"MSE (log)\")\n",
    "ax.legend(); ax.set_title(\"same problem, three learning rates\")\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30b3659d",
   "metadata": {},
   "source": [
    "> **Interpretation.** At $\\eta = 0.02$ the loss creeps down and is nowhere near the floor after 60 steps. At $\\eta = 0.1$ it lands in about twenty. At $\\eta = 0.45$ it overshoots and oscillates before settling. The Goldilocks band is wide here because the problem is well conditioned; on a badly scaled problem it is narrow, which is why feature standardization matters.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f7be93e",
   "metadata": {},
   "source": [
    "### A deliberate failure: a learning rate that diverges\n",
    "\n",
    "Now push past the convergence threshold on purpose. With $\\eta = 1.5$ each step overshoots the minimum and lands somewhere steeper, so the next step overshoots further. The loss does not just fail to converge; it explodes to infinity. Watch it happen, then fix it.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "4b7825cd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.412021Z",
     "iopub.status.busy": "2026-06-10T18:46:10.411926Z",
     "iopub.status.idle": "2026-06-10T18:46:10.415014Z",
     "shell.execute_reply": "2026-06-10T18:46:10.414747Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "first 6 losses: ['2147.3', '83947.3', '3283329.1', '128418264.2', '5022723738.1', '196449890449.1']\n",
      "final loss: 1.3316090913697503e+129   finite? True\n"
     ]
    }
   ],
   "source": [
    "# this run is SUPPOSED to fail: eta is far above the 2/L convergence threshold\n",
    "import warnings\n",
    "with warnings.catch_warnings():\n",
    "    warnings.simplefilter(\"ignore\")              # overflow is expected here\n",
    "    _, losses_bad = batch_gd(X_b, y, eta=1.5, n_steps=80)\n",
    "print(\"first 6 losses:\", [f\"{l:.1f}\" for l in losses_bad[:6]])\n",
    "print(\"final loss:\", losses_bad[-1], \"  finite?\", np.isfinite(losses_bad[-1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbbda62a",
   "metadata": {},
   "source": [
    "> **Interpretation.** The loss multiplies by roughly 40x every step and overflows to `inf` within a few dozen iterations. This is the canonical \"my loss went to NaN\" failure, and the cause is always the same family: the step size is too large for the curvature.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "aadb99dd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.415914Z",
     "iopub.status.busy": "2026-06-10T18:46:10.415843Z",
     "iopub.status.idle": "2026-06-10T18:46:10.418805Z",
     "shell.execute_reply": "2026-06-10T18:46:10.418415Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fixed final loss: 0.9421   finite? True\n",
      "[ ok ] dropping the learning rate repaired the run\n"
     ]
    }
   ],
   "source": [
    "# the fix: drop the learning rate by ~10x. Same code, smaller eta.\n",
    "_, losses_fixed = batch_gd(X_b, y, eta=0.1, n_steps=80)\n",
    "print(\"fixed final loss:\", f\"{losses_fixed[-1]:.4f}\", \"  finite?\", np.isfinite(losses_fixed[-1]))\n",
    "assert np.isfinite(losses_fixed[-1]) and losses_fixed[-1] < 1.5, \\\n",
    "    \"after dropping eta 15x, the loss should converge near the noise floor (~1.0)\"\n",
    "print(\"[ ok ] dropping the learning rate repaired the run\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba536277",
   "metadata": {},
   "source": [
    "> **Common confusion:** people reach for a fancier optimizer when training diverges. Try the boring fix first: divide the learning rate by 10. If that stops the explosion, the problem was the step size, not the algorithm. The second boring fix is to standardize your features so the curvature is not lopsided.\n",
    "\n",
    "> **Key takeaways.** Convergence needs $\\eta < 2/L$. Too small wastes steps; too large diverges to infinity. The loss curve diagnoses which regime you are in, and the first repair is always to cut the learning rate.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12ec7f06",
   "metadata": {},
   "source": [
    "## Part 4 — Stochastic and mini-batch gradient descent\n",
    "\n",
    "> **Objectives.** Replace the full-batch gradient with a one-example estimate, add a decaying learning-rate schedule, and recover the same `(4, 3)` answer from noisy steps. Then see why the schedule, not the noise, is what makes it converge.\n",
    "\n",
    "Batch GD touches every example per step. **Stochastic gradient descent (SGD)** uses one example per step: the gradient is a noisy estimate of the true gradient, the steps are cheap, and the noise itself helps escape shallow traps. The price is that with a fixed learning rate SGD never settles; it bounces in a ball around the optimum. A decaying schedule\n",
    "\n",
    "$$\\eta_t = \\frac{t_0}{t + t_1}$$\n",
    "\n",
    "shrinks the steps over time so the bouncing dies out. This schedule satisfies the Robbins-Monro conditions $\\sum_t \\eta_t = \\infty$ and $\\sum_t \\eta_t^2 < \\infty$, which is what guarantees convergence for convex problems.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "183e8b70",
   "metadata": {},
   "source": [
    "### Exercise 4.3 — Stochastic gradient descent with a schedule\n",
    "`Difficulty 3/5 · ~20 min`\n",
    "\n",
    "Fill in `sgd(X_b, y, n_epochs, t0, t1)`. Each epoch, shuffle the examples with the provided `gen` (a seeded `np.random.default_rng`) and take one single-example gradient step each, using the schedule $\\eta_t = t_0 / (t + t_1)$ where `t` counts total steps from 0. Return `(theta, losses)` where `losses` records the full-data MSE once per epoch. The check asserts you recover the closed-form answer within a looser tolerance, because SGD is noisy.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "fa2aaabb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.419781Z",
     "iopub.status.busy": "2026-06-10T18:46:10.419718Z",
     "iopub.status.idle": "2026-06-10T18:46:10.424066Z",
     "shell.execute_reply": "2026-06-10T18:46:10.423692Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] 4.3 SGD recovers (4,3): not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def sgd(X_b, y, n_epochs=50, t0=5.0, t1=50.0, seed=SEED):\n",
    "    \"\"\"Single-example SGD with a t0/(t+t1) learning-rate schedule.\n",
    "    Returns (theta of shape (d+1,1), per-epoch full-data MSE list).\"\"\"\n",
    "    gen = np.random.default_rng(seed)\n",
    "    m = X_b.shape[0]\n",
    "    theta = np.zeros((X_b.shape[1], 1))\n",
    "    losses = []\n",
    "    step = 0\n",
    "    for epoch in range(n_epochs):\n",
    "        order = gen.permutation(m)            # reshuffle each epoch\n",
    "        for i in order:\n",
    "            xi = X_b[i:i+1]                    # shape (1, d+1) — keep it 2-D\n",
    "            yi = y[i:i+1]                      # shape (1, 1)\n",
    "            # TODO 1: single-example gradient = 2 * xi^T @ (xi @ theta - yi)\n",
    "            #         (note: no 1/m here — one example, so the batch size is 1)\n",
    "            gradient = None\n",
    "            attempted(gradient)\n",
    "            # TODO 2: eta = t0 / (step + t1)\n",
    "            eta = t0 / (step + t1)\n",
    "            theta = theta - eta * gradient\n",
    "            step += 1\n",
    "        losses.append(float(np.mean((X_b @ theta - y) ** 2)))\n",
    "    return theta, losses\n",
    "\n",
    "def _sgd_recovers_truth():\n",
    "    theta_sgd, _ = sgd(X_b, y, n_epochs=SGD_EPOCHS)\n",
    "    ref = np.linalg.lstsq(X_b, y, rcond=None)[0]\n",
    "    check_close(theta_sgd, ref, atol=0.15,\n",
    "                msg=\"SGD is noisy; it should land near the closed form, not exactly on it\")\n",
    "\n",
    "check(\"4.3 SGD recovers (4,3)\", _sgd_recovers_truth)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b6c2ca5",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>The single-example gradient has the same shape as the full-batch one, but with one row in `xi` and no division by `m`. The residual is still `prediction - target`, i.e. `xi @ theta - yi`.</details>\n",
    "\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "gradient = 2 * xi.T @ (xi @ theta - yi)   # (d+1, 1)\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — the result is way off or the loss is jumpy</summary>If `theta` is wildly wrong, check that `xi` stayed 2-D: `X_b[i]` drops a dimension and breaks the matmul, but `X_b[i:i+1]` keeps the `(1, d+1)` shape. If it is merely jumpy, that is SGD working as intended; the per-epoch loss wobbles even as it trends down.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "60bf9c53",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.424862Z",
     "iopub.status.busy": "2026-06-10T18:46:10.424791Z",
     "iopub.status.idle": "2026-06-10T18:46:10.461594Z",
     "shell.execute_reply": "2026-06-10T18:46:10.461074Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 4.3 SGD recovers (4,3)\n",
      "theta (SGD): [3.9907555  2.93191195]   final MSE 0.9401\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines sgd; the checks below re-verify the reference.\n",
    "def sgd(X_b, y, n_epochs=50, t0=5.0, t1=50.0, seed=SEED):\n",
    "    gen = np.random.default_rng(seed)\n",
    "    m = X_b.shape[0]\n",
    "    theta = np.zeros((X_b.shape[1], 1))\n",
    "    losses = []\n",
    "    step = 0\n",
    "    for epoch in range(n_epochs):\n",
    "        order = gen.permutation(m)\n",
    "        for i in order:\n",
    "            xi = X_b[i:i+1]\n",
    "            yi = y[i:i+1]\n",
    "            gradient = 2 * xi.T @ (xi @ theta - yi)\n",
    "            eta = t0 / (step + t1)\n",
    "            theta = theta - eta * gradient\n",
    "            step += 1\n",
    "        losses.append(float(np.mean((X_b @ theta - y) ** 2)))\n",
    "    return theta, losses\n",
    "\n",
    "check(\"4.3 SGD recovers (4,3)\", _sgd_recovers_truth, required=True)\n",
    "theta_sgd, losses_sgd = sgd(X_b, y, n_epochs=SGD_EPOCHS)\n",
    "print(\"theta (SGD):\", theta_sgd.ravel(), f\"  final MSE {losses_sgd[-1]:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca0b2243",
   "metadata": {},
   "source": [
    "> **Stop and think:** the SGD loss curve below is jagged where batch GD's was smooth. Is that a problem? <details><summary>Answer</summary>No. Each SGD step follows a one-example gradient, which points in a slightly wrong direction, so the loss wobbles step to step even while trending down. The decaying schedule shrinks the wobble over time. Without decay, SGD would converge to a noise ball around the optimum rather than to a point.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "727afd4e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.462645Z",
     "iopub.status.busy": "2026-06-10T18:46:10.462564Z",
     "iopub.status.idle": "2026-06-10T18:46:10.524496Z",
     "shell.execute_reply": "2026-06-10T18:46:10.524116Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAGGCAYAAACNCg6xAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAVcBJREFUeJzt3XlcVGX7BvBrZmCGHUQBQQFxQdASdzM1d9HUXLLMX2/i0qqWS1n6lhlqoVlqLmVZLlmvlpWmmXuKpllumCsu4VIumAvIDjP3749xjowsDjLDwHh9P59pZs56zxyMi+c85zkqEREQERER0V2p7V0AERERUUXB4ERERERkIQYnIiIiIgsxOBERERFZiMGJiIiIyEIMTkREREQWYnAiIiIishCDExEREZGFGJyIiIiILMTgRGQn77zzDlQqlb3LqNBUKhVGjBhh7zKsLi8vD6+//jqCg4OhVqvRu3dve5eEdu3a4YEHHrB3GUR2x+BENnfo0CH069cPoaGhcHFxQbVq1dC5c2fMmTOnwLIGgwFffvklOnfujCpVqsDZ2Rn+/v7o0qULPvvsM2RnZ5str1KplIeTkxN8fX3RpEkTjBw5EkePHrW4xvfeew+rVq0q7UclG9i1axfeeecd3Lhxw96llJmFCxdi+vTp6NevH5YsWYLRo0fbuySHdvToUbzzzjs4c+aMvUuhCoDBiWxq165daNq0KQ4ePIjnnnsOc+fOxbPPPgu1Wo2PPvrIbNnMzEw8+uijiImJQUZGBl577TV89tlneOONN+Di4oJhw4Zh2LBhBfbRuXNnLF26FIsWLcKkSZPQuHFjLFmyBFFRUZgxY4ZFdTI4lV+7du1CbGzsfRWcfvnlF1SrVg0zZ87EM888g7Zt29q7JId29OhRxMbGMjiRRZzsXQA5tnfffRfe3t7Ys2cPfHx8zOYlJyebvR89ejQ2bNiAWbNmYeTIkWbzXn31VZw8eRKbNm0qsI/w8HD85z//MZs2depU9OzZE6+++ioiIiLw6KOPWucDAUhPT4e7u7vVtkd0p+Tk5AL/XkrDYDAgJycHLi4uVtsm0X1LiGyobt260q5du7sud+7cOdFoNNK1a9cSbR+ADB8+vNB5Z8+eFScnJ3n44Yfvuo07HzExMSIiMnHiRAEgR44ckQEDBoiPj480bNhQREQOHjwoMTExEhYWJjqdTgICAmTw4MHy77//FtjHjh07pGnTpqLT6aRmzZoyf/58Zdt3Wrp0qTRu3FhcXFykUqVK0r9/fzl37pzZMidOnJC+fftKQECA6HQ6qVatmvTv319u3LhR7Ge1ZD3Td/rtt99KZGSkuLi4yEMPPSR//vmniIjMnz9fatWqJTqdTtq2bStJSUkF9vPtt98qn6Fy5cry9NNPy99//11guS1btkjr1q3Fzc1NvL295bHHHpOjR48q803f0Z0P0z5Nta5cuVLq168vWq1W6tWrJ+vWrTPbj2k7J0+elJiYGPH29hYvLy8ZNGiQpKen2+wYbNy4UVq1aiXe3t7i7u4u4eHhMn78+CKPT1JSUqGfd+vWrSIikpaWJmPGjJHq1auLVquV8PBwmT59uhgMBrPtmL6Xr776SurVqydOTk6ycuXKIvcrIvLzzz/LI488Ih4eHuLp6SlNmzaVr7/+Wpnftm1bqV+/vhw5ckTatWsnrq6uEhQUJNOmTSuwraysLHn77belVq1aotVqpXr16jJ27FjJysoyW27hwoXSvn178fPzE61WK5GRkfLxxx8X2F5oaKh0795dNmzYIFFRUaLT6SQyMlK+//77Yj+TybJly6Rx48bKZ3vggQdk1qxZIiKyaNGiYr9z03dj+jn18PCQRx99VA4fPmy2j5iYGHF3d5fTp09Lly5dxM3NTQIDAyU2NrbA8SmuHir/GJzIprp06SKenp5y6NChYpf79NNPBYB89dVXJdp+ccFJRKRjx46iVqslJSWlyGWWLl0qOp1O2rRpI0uXLpWlS5fKrl27ROT2L9x69epJr1695OOPP5Z58+aJiMgHH3wgbdq0kUmTJslnn30mI0eOFFdXV2nevLnZ/yj//PNPcXV1lZCQEImLi5PJkydLQECANGjQoEBwmjJliqhUKunfv798/PHHEhsbK1WqVJEaNWrI9evXRUQkOztbwsLCJCgoSKZMmSKff/65xMbGSrNmzeTMmTNFfk5L1wMgDRo0kODgYJk6dapMnTpVvL29JSQkRObOnSv16tWTDz/8UN566y3RarXSvn17s/2YfhE1a9ZMZs6cKePGjRNXV1ezzyAismnTJnFycpLw8HB5//33lc9aqVIlJRgdPHhQBgwYIABk5syZyvFJS0tTao2KipLAwECZPHmyzJo1S2rWrClubm5mAdZ0HBs1aiR9+/aVjz/+WJ599lkBIK+//rpNjsHhw4dFq9VK06ZN5aOPPpL58+fLa6+9Jo888kiRxygtLU2WLl0qERERUr16deXzXrp0SQwGg3To0EFUKpU8++yzMnfuXOnZs6cAkFGjRpltB4BERkaKn5+fxMbGyrx58+TAgQNF7nfRokWiUqnkgQcekHfffVfmzZsnzz77rDzzzDPKMm3btpWgoCAJDg6WkSNHyscffywdOnQQAPLzzz8ry+n1eiU4jBo1Sj799FMZMWKEODk5Sa9evcz226xZMxk0aJDMnDlT5syZI126dBEAMnfuXLPlQkNDJTw8XHx8fGTcuHEyY8YMefDBB0WtVsvGjRuL/FwixvAKQDp27Cjz5s2TefPmyYgRI+SJJ54QEZHTp0/LK6+8IgDkv//9r9l3LiLy5Zdfikqlkq5du8qcOXNk2rRpUqNGDfHx8TH7oyEmJkZcXFykTp068swzz8jcuXOlR48eAkAmTJhgcT1U/jE4kU1t3LhRNBqNaDQaadmypbz++uuyYcMGycnJMVtu9OjRAkASEhLMpmdnZ8uVK1eUx52tOXcLTiNHjhQAcvDgwWLrdHd3V1qZ8jP9wh0wYECBeRkZGQWmLVu2TADI9u3blWm9e/cWFxcXOXv2rDLt6NGjotFozILTmTNnRKPRyLvvvmu2zUOHDomTk5My/cCBAwJAVqxYUexnupOl6wEQnU5n9kvBFGyrVq0qqampyvTx48ebtQDl5OSIv7+/PPDAA5KZmaks99NPPwkAefvtt5VpDRs2FH9/f7l69aoy7eDBg6JWq2XgwIHKtOnTp5vt485atVqtnDp1ymwbAGTOnDnKNNNxHDJkiNn6ffr0kcqVKyvvrXkMZs6cKQDkypUrRS5TFFPrTn6rVq0SADJlyhSz6f369ROVSmX2HQAQtVotR44cueu+bty4IZ6entKiRQuzYyYiZn8AtG3bVgDIl19+qUzLzs6WqlWryuOPP65MW7p0qajVatmxY4fZtubPny8AZOfOncq0wv4NRUdHS82aNc2mhYaGCgCzFqaUlBQJDAyURo0aFfv5Ro4cKV5eXpKXl1fkMitWrCjQyiQicvPmTfHx8ZHnnnvObPqlS5fE29vbbHpMTIwAkJdfflmZZjAYpHv37qLVapWfA0vqofKNncPJpjp37ozffvsNjz32GA4ePIj3338f0dHRqFatGlavXq0sl5qaCgDw8PAwW//nn3+Gn5+f8ggNDS3R/k3bu3nzZqk+x4svvlhgmqurq/I6KysL//77Lx566CEAwP79+wEAer0eGzZsQO/evRESEqIsHxkZiejoaLPt/fDDDzAYDHjyySfx77//Ko+qVauiTp062Lp1KwDA29sbALBhwwZkZGRY/BlKsl7Hjh1Ro0YN5X2LFi0AAI8//jg8PT0LTP/rr78AAHv37kVycjKGDRtm1p+me/fuiIiIwNq1awEAFy9eREJCAgYNGgRfX19luQYNGqBz5874+eefLf5cnTp1Qq1atcy24eXlpdSU353HsU2bNrh69ary82fNY2Dqo/Tjjz/CYDBY/HmK8vPPP0Oj0eCVV14xm/7qq69CRLBu3Tqz6W3btkW9evXuut1Nmzbh5s2bGDduXIE+UHcOl+Hh4WHWn1Cr1aJ58+Zm3/WKFSsQGRmJiIgIs++wQ4cOAKB8h4D5v6GUlBT8+++/aNu2Lf766y+kpKSY7TsoKAh9+vRR3nt5eWHgwIE4cOAALl26VOTn8/HxQXp6eqH9I+9m06ZNuHHjBgYMGGD2WTQaDVq0aGH2WUzyD49hGi4jJycHmzdvLnU9VD4wOJHNNWvWDD/88AOuX7+OP/74A+PHj8fNmzfRr18/ZcgA0y/jtLQ0s3VbtWqFTZs2YdOmTejSpUuJ923aXv5f9vciLCyswLRr165h5MiRCAgIgKurK/z8/JTlTP/Tv3LlCjIzM1GnTp0C69etW9fs/cmTJyEiqFOnjllY9PPzw7Fjx5TO9GFhYRgzZgw+//xzVKlSBdHR0Zg3b16BXzSFfQZL18sf8oDbQSE4OLjQ6devXwcAnD17ttDPBgARERHK/OKWi4yMxL///ov09PRiP09RtQJApUqVlJqKW7ZSpUpm9VvzGPTv3x+tWrXCs88+i4CAADz11FP49ttv7zlEnT17FkFBQQV+liMjI5X5+RX2M1uY06dPA4BFYzRVr169QJi687s+efIkjhw5UuD7Cw8PB2B+UcjOnTvRqVMnuLu7w8fHB35+fvjvf/8LAAV+LmvXrl1g36ZtFnc13LBhwxAeHo5u3bqhevXqGDJkCNavX3/Xz2r6LADQoUOHAp9n48aNBS5wUavVqFmzZrE1lqYeKh94VR2VGa1Wi2bNmqFZs2YIDw/H4MGDsWLFCkycOBEREREAgMOHDyMqKkpZx8/PD506dQIAfPXVVyXe5+HDh6HRaCz+JVKU/H8Zmzz55JPYtWsXxo4di4YNG8LDwwMGgwFdu3a9p1+OBoMBKpUK69atg0ajKTA/f2vchx9+iEGDBuHHH3/Exo0b8corryAuLg67d+9G9erVi9yHpesVtv/ipouIpR/T6kpS092WteYxcHV1xfbt27F161asXbsW69evxzfffIMOHTpg48aNRdZiLYX9zJaWJd+1wWDAgw8+WORQIKbwffr0aXTs2BERERGYMWMGgoODodVq8fPPP2PmzJlWaaUDAH9/fyQkJGDDhg1Yt24d1q1bh0WLFmHgwIFYsmRJseuaali6dCmqVq1aYL6TU8l/hZamHiofGJzILpo2bQrAeMoGALp16waNRoOvv/4aTz/9tFX2ce7cOcTHx6Nly5Z3bXEq6Qje169fx5YtWxAbG4u3335bmW76C9XEz88Prq6uBaYDQGJiotn7WrVqQUQQFham/JVanAcffBAPPvgg3nrrLezatQutWrXC/PnzMWXKFJusZwnTqdTExETl1IxJYmKiMj//cnc6fvw4qlSpogz5UJajq1v7GKjVanTs2BEdO3bEjBkz8N577+HNN9/E1q1blT8ILBUaGorNmzfj5s2bZj/Px48fV+bfC9NpzsOHD6N27dr3tI07t3fw4EF07Nix2GO3Zs0aZGdnY/Xq1WYtgYWd/gKAU6dOQUTMtnnixAkAMDutXBitVouePXuiZ8+eMBgMGDZsGD799FNMmDCh0Jas/J8FMIYdS46XwWDAX3/9ZfazU1iNd6uHyjeeqiOb2rp1a6F/+Zv6sJhO1YSEhGDIkCFYt24d5s6dW+i2StKqce3aNQwYMAB6vR5vvvnmXZd3d3cv0QCLpr+876xp1qxZBZaLjo7GqlWrcO7cOWX6sWPHsGHDBrNl+/btC41Gg9jY2ALbFRFcvXoVgLE/WF5entn8Bx98EGq1usDI6vnd63ol0bRpU/j7+2P+/Plm21y3bh2OHTuG7t27AwACAwPRsGFDLFmyxOx7P3z4MDZu3Gg27pYpQJXFAJjWPAbXrl0rsP2GDRsCwD19348++ij0en2Bfx8zZ86ESqVCt27dSrxNAOjSpQs8PT0RFxeHrKwss3n30pL45JNP4p9//sGCBQsKzMvMzFROwRb2byglJQWLFi0qdLsXLlzAypUrlfepqan48ssv0bBhw0Jbg0xMx8xErVajQYMGAG4fh6J+xqKjo+Hl5YX33nsPubm5BbZ95cqVAtPyHx8Rwdy5c+Hs7IyOHTtaXA+Vb2xxIpt6+eWXkZGRgT59+iAiIgI5OTnYtWsXvvnmG9SoUQODBw9Wlp01axaSkpLw8ssvY/ny5ejZsyf8/f3x77//YufOnVizZk2hfWJOnDiBr776CiKC1NRUHDx4ECtWrEBaWhpmzJiBrl273rXOJk2aYPPmzZgxYwaCgoIQFhamdHwujJeXFx555BG8//77yM3NRbVq1bBx40YkJSUVWDY2Nhbr169HmzZtMGzYMOTl5WHOnDmoX78+/vzzT2W5WrVqYcqUKRg/fjzOnDmD3r17w9PTE0lJSVi5ciWef/55vPbaa/jll18wYsQIPPHEEwgPD0deXh6WLl0KjUaDxx9/vMia73W9knB2dsa0adMwePBgtG3bFgMGDMDly5fx0UcfoUaNGma3Dpk+fTq6deuGli1bYujQocjMzMScOXPg7e2Nd955R1muSZMmAIA333wTTz31FJydndGzZ0+bDEJqzWMwadIkbN++Hd27d0doaCiSk5Px8ccfo3r16mjdunWJa+vZsyfat2+PN998E2fOnEFUVBQ2btyIH3/8EaNGjTLrIF8SXl5emDlzJp599lk0a9YM//d//4dKlSrh4MGDyMjIKPHpo2eeeQbffvstXnzxRWzduhWtWrWCXq/H8ePH8e2332LDhg1o2rQpunTporS8vPDCC0hLS8OCBQvg7++vtETnFx4ejqFDh2LPnj0ICAjAwoULcfny5SKDlsmzzz6La9euoUOHDqhevTrOnj2LOXPmoGHDhkr/sIYNG0Kj0WDatGlISUmBTqdDhw4d4O/vj08++QTPPPMMGjdujKeeegp+fn44d+4c1q5di1atWpkFJRcXF6xfvx4xMTFo0aIF1q1bh7Vr1+K///0v/Pz8LK6HyrkyvYaP7jvr1q2TIUOGSEREhHh4eIhWq5XatWvLyy+/LJcvXy6wfF5enixatEg6dOggvr6+4uTkJFWqVJGOHTvK/PnzC1wujXwD1qnVavHx8ZFGjRrJyJEjLboU2+T48ePyyCOPiKura6EDYBZ2Sfnff/8tffr0ER8fH/H29pYnnnhCLly4IABk4sSJZsvGx8dLkyZNRKvV3nUAzO+//15at24t7u7u4u7uLhERETJ8+HBJTEwUEZG//vpLhgwZIrVq1RIXFxfx9fWV9u3by+bNm4v9jJauh0KGeDANzDh9+nSz6Vu3bi30svxvvvlGGjVqJDqdTnx9fYscAHPz5s3SqlUrcXV1FS8vL+nZs6fZAJgmkydPlmrVqolarS50AMw7hYaGmg0vUdRxNI05dedQB9Y4Blu2bJFevXpJUFCQaLVaCQoKkgEDBsiJEycK1HunwoYjEDFeHj969GgJCgoSZ2dnqVOnTrEDYJbE6tWr5eGHH1aORfPmzWXZsmV3rSkmJkZCQ0PNpuXk5Mi0adOkfv36otPppFKlStKkSROJjY01G1Nt9erV0qBBA3FxcZEaNWrItGnTZOHChQWOSf4BMBs0aCA6nU4iIiIsGpLju+++ky5duoi/v79otVoJCQmRF154QS5evGi23IIFC6RmzZrKMCH5hybYunWrREdHi7e3t7i4uEitWrVk0KBBsnfvXrPv4c4BMAMCAmTixImi1+tLXA+VXyoRO/bqJCIiuosaNWrggQcewE8//WTvUoo0aNAgfPfddwWuDCbHwz5ORERERBZicCIiIiKyEIMTERERkYXYx4mIiIjIQmxxIiIiIrIQgxMRERGRhRx+AEyDwYALFy7A09OzTG/dQERERBWDiODmzZsICgqCWl18m5LDB6cLFy4UuKM7ERER0Z3Onz9f7I3SgfsgOJluhnn+/Hl4eXnZuRoiIiIqb1JTUxEcHHzXG8ID90FwMp2e8/LyYnAiIiKiIlnSpYedw4mIiIgsxOBEREREZCEGJyIiIiILOXwfJyIisi+9Xo/c3Fx7l0H3MWdnZ2g0Gqtsy67Bafv27Zg+fTr27duHixcvYuXKlejdu7fZMseOHcMbb7yB+Ph45OXloV69evj+++8REhJin6KJiMgiIoJLly7hxo0b9i6FCD4+PqhatWqpx3S0a3BKT09HVFQUhgwZgr59+xaYf/r0abRu3RpDhw5FbGwsvLy8cOTIEbi4uNihWiIiKglTaPL394ebmxsHISa7EBFkZGQgOTkZABAYGFiq7dk1OHXr1g3dunUrcv6bb76JRx99FO+//74yrVatWmVRGhERlYJer1dCU+XKle1dDt3nXF1dAQDJycnw9/cv1Wm7cts53GAwYO3atQgPD0d0dDT8/f3RokULrFq1qtj1srOzkZqaavYgIqKyZerT5ObmZudKiIxMP4ul7W9XboNTcnIy0tLSMHXqVHTt2hUbN25Enz590LdvX8THxxe5XlxcHLy9vZUHb7dCRGQ/PD1H5YW1fhbLbXAyGAwAgF69emH06NFo2LAhxo0bhx49emD+/PlFrjd+/HikpKQoj/Pnz5dVyUREROTgym1wqlKlCpycnFCvXj2z6ZGRkTh37lyR6+l0OuX2Kra+zcqeQ0Cbp4H/e9VmuyAiojLUrl07jBo1qsz3e+bMGahUKiQkJJT5vqlkym1w0mq1aNasGRITE82mnzhxAqGhoXaqqqBzF4B/Ltu7CiIiKi+2bdsGlUpVZsMwHDhwAP3790dgYCB0Oh1CQ0PRo0cPrFmzBiIC4HYwMz08PT1Rv359DB8+HCdPniyTOh2FXa+qS0tLw6lTp5T3SUlJSEhIgK+vL0JCQjB27Fj0798fjzzyCNq3b4/169djzZo12LZtm/2Kzsfd2Ekf6Rn2rYOIiO5PP/74I5588kl06tQJS5YsQe3atZGdnY1du3bhrbfeQps2beDj46Msv3nzZtSvXx8ZGRk4dOgQPvroI0RFRWHNmjXo2LGj/T5IRSJ2tHXrVgFQ4BETE6Ms88UXX0jt2rXFxcVFoqKiZNWqVSXaR0pKigCQlJQUK1cvcvYfkZB2IhFdrb5pIqIKLTMzU44ePSqZmZn2LqVE2rZtK8OHD5fhw4eLl5eXVK5cWd566y0xGAzKMl9++aU0adJEPDw8JCAgQAYMGCCXL18WEZGkpKQif6fp9XqZNm2a1KpVS7RarQQHB8uUKVPM1vv++++lXbt24urqKg0aNJBdu3YVWWtaWppUrlxZ+vTpU+QyprpN2z9w4IDZfL1eL+3atZPQ0FDJy8u7l6+swijuZ7IkWcGuLU7t2rVTmhGLMmTIEAwZMqSMKioZt1stThlZgMEAqMvtiU8iIvsSATKz7LNvVxegJBdULVmyBEOHDsUff/yBvXv34vnnn0dISAiee+45AMbL2SdPnoy6desiOTkZY8aMwaBBg/Dzzz8jODgY33//PR5//HEkJibCy8tLGUNo/PjxWLBgAWbOnInWrVvj4sWLOH78uNm+33zzTXzwwQeoU6cO3nzzTQwYMACnTp2Ck1PBX9cbN27E1atX8frrrxf5We52JZlarcbIkSPRp08f7Nu3D82bN7f8i7pP8V51pWA6VQcYw5MHhyshIipUZhYQ+ah99n3s59t/6FoiODgYM2fOhEqlQt26dXHo0CHMnDlTCU75/5ivWbMmZs+ejWbNmiEtLQ0eHh7w9fUFAPj7+yunyW7evImPPvoIc+fORUxMDADjgM6tW7c22/drr72G7t27AwBiY2NRv359nDp1ChEREQXqPHHiBACgbt26yrQ9e/agffv2yvvly5ejR48exX5e07bPnDnD4GQBtpGUgovudisT+zkRETmGhx56yKylpmXLljh58iT0ej0AYN++fejZsydCQkLg6emJtm3bAkCxV3wfO3YM2dnZd+1H1KBBA+W16dYgpluFWKJBgwZISEhAQkIC0tPTkZeXd9d1TGd+OOaWZdjiVAoqlbHV6WY6kJ5p72qIiMovVxdjy4+99m0t6enpiI6ORnR0NL7++mv4+fnh3LlziI6ORk5OTtE1uFrW5OXs7Ky8NgUZ07iGd6pTpw4AIDExEQ899BAA45A8tWvXtmhfJseOHQMAhIWFlWi9+xWDUykpwYktTkRERVKpSna6zJ5+//13s/e7d+9GnTp1oNFocPz4cVy9ehVTp05V7kyxd+9es+W1Wi0AKC1UgDHkuLq6YsuWLXj22WetUmeXLl3g6+uLadOmYeXKlfe0DYPBgNmzZyMsLAyNGjWySl2OjsGplJQhCdjiRETkEM6dO4cxY8bghRdewP79+zFnzhx8+OGHAICQkBBotVrMmTMHL774Ig4fPozJkyebrR8aGgqVSoWffvoJjz76KFxdXeHh4YE33ngDr7/+OrRaLVq1aoUrV67gyJEjGDp06D3V6eHhgc8//xz9+/dH9+7d8corr6BOnTpIS0vD+vXrAaDAzWyvXr2KS5cuISMjA4cPH8asWbPwxx9/YO3ataW68e39hMGplNxvdQhncCIicgwDBw5EZmYmmjdvDo1Gg5EjR+L5558HAPj5+WHx4sX473//i9mzZ6Nx48b44IMP8NhjjynrV6tWDbGxsRg3bhwGDx6MgQMHYvHixZgwYQKcnJzw9ttv48KFCwgMDMSLL75Yqlr79OmDXbt2Ydq0aRg4cCCuXbsGb29vNG3atNCO4Z06dQJgvOFtaGgo2rdvj88++6zEp/fuZyq523gAFVxqaiq8vb2RkpJik9uv9B8N7E4A5kwAHutg9c0TEVVIWVlZSEpKQlhYGFxcrNjJiOgeFfczWZKswKvqSsmDLU5ERET3DQanUnK7FVrZOZyIiMjxMTiVEjuHExER3T8YnEpJ6RzOFiciIiKHx+BUSmxxIiIiun8wOJUShyMgIiK6fzA4lZKpxSmDwYmIiMjhMTiVEk/VERER3T8YnEpJCU7sHE5EROTwGJxKiX2ciIjIVr744gt06dLF3mXY1aBBg9C7d+9il3nooYfw/fffl0k9DE6lxFN1RESO48qVK3jppZcQEhICnU6HqlWrIjo6Gjt37jRb7sCBA+jfvz8CAwOh0+kQGhqKHj16YM2aNTDdyezMmTNQqVTKw9PTE/Xr18fw4cNx8uTJu9aSlZWFCRMmYOLEiTb5rI7krbfewrhx42AwGGy+LwanUnJjcCIichiPP/44Dhw4gCVLluDEiRNYvXo12rVrh6tXryrL/Pjjj3jooYeQlpaGJUuW4NixY1i/fj369OmDt956CykpKWbb3Lx5My5evIiDBw/ivffew7FjxxAVFYUtW7YUW8t3330HLy8vtGrVyiaf1UREkJeXZ9N92Fq3bt1w8+ZNrFu3zub7YnAqJQ8OgElE5BBu3LiBHTt2YNq0aWjfvj1CQ0PRvHlzjB8/Ho899hgAID09HUOHDkX37t2xdu1adOnSBTVr1kRkZCSGDh2KgwcPwtvb22y7lStXRtWqVVGzZk306tULmzdvRosWLTB06FDo9foi61m+fDl69uxpNs102io2NhZ+fn7w8vLCiy++iJycHGUZg8GAuLg4hIWFwdXVFVFRUfjuu++U+du2bYNKpcK6devQpEkT6HQ6/Prrr4XWcP78eTz55JPw8fGBr68vevXqhTNnzpSonuzsbLzyyivw9/eHi4sLWrdujT179pjt58iRI+jRowe8vLzg6emJNm3a4PTp02bLfPDBBwgMDETlypUxfPhw5ObmKvM0Gg0effRRLF++vMjv01oYnErJdKouNw/IyS1+WSKi+5WIwJCVYZeH6dTZ3Xh4eMDDwwOrVq1CdnZ2octs3LgRV69exeuvv17kdlQqVbH7UavVGDlyJM6ePYt9+/YVudyvv/6Kpk2bFpi+ZcsWHDt2DNu2bcOyZcvwww8/IDY2VpkfFxeHL7/8EvPnz8eRI0cwevRo/Oc//0F8fLzZdsaNG4epU6fi2LFjaNCgQYH95ObmIjo6Gp6entixYwd27twJDw8PdO3a1SwY3a2e119/Hd9//z2WLFmC/fv3o3bt2oiOjsa1a9cAAP/88w8eeeQR6HQ6/PLLL9i3bx+GDBli1gq2detWnD59Glu3bsWSJUuwePFiLF682Kze5s2bY8eOHUV+n9biZPM9ODjTqTrA2Oqk9S56WSKi+5VkZ+LioJZ22Xfg4t+gcnG763JOTk5YvHgxnnvuOcyfPx+NGzdG27Zt8dRTTynB4sSJEwCAunXrKuvt2bMH7du3V94vX74cPXr0KHZfERERAIz9oJo3b15g/o0bN5CSkoKgoKAC87RaLRYuXAg3NzfUr18fkyZNwtixYzF58mTk5ubivffew+bNm9GypfH7rlmzJn799Vd8+umnaNu2rbKdSZMmoXPnzkXW+M0338BgMODzzz9XwuCiRYvg4+ODbdu2KZ3Wi6snMzMTn3zyCRYvXoxu3boBABYsWIBNmzbhiy++wNixYzFv3jx4e3tj+fLlcHZ2BgCEh4eb1VKpUiXMnTsXGo0GERER6N69O7Zs2YLnnntOWSYoKAjnz5+HwWCAWm27diG2OJWSsxOgMx5n9nMiIqrgHn/8cVy4cAGrV69G165dsW3bNjRu3LhA60Z+DRo0QEJCAhISEpCenm5RfyFTK1hRrVOZmcZfKC4uLgXmRUVFwc3tdhBs2bIl0tLScP78eZw6dQoZGRno3Lmz0oLm4eGBL7/8ssCpr8Jas/I7ePAgTp06BU9PT2U7vr6+yMrKMttWcfWcPn0aubm5Zv20nJ2d0bx5cxw7dgwAkJCQgDZt2iihqTD169eHRqNR3gcGBiI5OdlsGVdXVxgMhiJbC62FLU5W4O4GZKdw9HAioqKodK4IXPyb3fZdEi4uLujcuTM6d+6MCRMm4Nlnn8XEiRMxaNAg1KlTBwCQmJiIhx56CACg0+lQu3btEu3DFBrCwsIKnV+5cmWoVCpcv369RNtNS0sDAKxduxbVqlUzm6fT6czeu7u733VbTZo0wddff11gnp+fX4nqKo6r692Pz52hSqVSFbiC7tq1a3B3d7doe6XB4GQFbq7AtRQgjcGJiKhQKpXKotNl5VG9evWwatUqAECXLl3g6+uLadOmYeXKlfe0PYPBgNmzZyMsLAyNGjUqdBmtVot69erh6NGjBcZxOnjwIDIzM5WAsHv3bnh4eCA4OBi+vr7Q6XQ4d+6c2Wm5e9G4cWN888038Pf3h5eXV5HLFVdPlSpVoNVqsXPnToSGhgIw9p3as2cPRo0aBcDYYrdkyRLk5uYW2+p0N4cPHy7y+7QmnqqzAtOVdWxxIiKquK5evYoOHTrgq6++wp9//omkpCSsWLEC77//Pnr16gXA2IH8888/x9q1a9G9e3ds2LABf/31F/7880+8//77AGB2Ssm03UuXLuGvv/7C6tWr0alTJ/zxxx/44osvCiybX3R0dKFXu+Xk5GDo0KE4evQofv75Z0ycOBEjRoyAWq2Gp6cnXnvtNYwePRpLlizB6dOnsX//fsyZMwdLliwp0ffx9NNPo0qVKujVqxd27NiBpKQkbNu2Da+88gr+/vtvi+pxd3fHSy+9hLFjx2L9+vU4evQonnvuOWRkZGDo0KEAgBEjRiA1NRVPPfUU9u7di5MnT2Lp0qVITEwsUb07duwom8FCxY7i4+OlR48eEhgYKABk5cqVRS77wgsvCACZOXNmifaRkpIiACQlJaV0xRaj93CRkHYi67bbbBdERBVKZmamHD16VDIzM+1disWysrJk3Lhx0rhxY/H29hY3NzepW7euvPXWW5KRkWG27J49e6Rfv37i7+8vTk5OUrlyZYmOjpbly5eLwWAQEZGkpCQBoDzc3NwkMjJShg0bJidPnrxrPUeOHBFXV1e5ceOGMi0mJkZ69eolb7/9tlSuXFk8PDzkueeek6ysLGUZg8Egs2bNkrp164qzs7P4+flJdHS0xMfHi4jI1q1bBYBcv379rjVcvHhRBg4cKFWqVBGdTic1a9aU5557Tvmdakk9mZmZ8vLLLyvbaNWqlfzxxx9m+zl48KB06dJF3NzcxNPTU9q0aSOnT58220d+I0eOlLZt2yrv//77b3F2dpbz588X+VmK+5ksSVZQiVh4naYNrFu3Djt37kSTJk3Qt29frFy5stBh1VeuXInY2FhcuXIFY8eOVZr3LJGamgpvb2+kpKQU29RYGs+MBbbvBWaOB/re3yPjExEBMI56nZSUhLCwsEI7OJNlnnjiCTRu3Bjjx48HYBw36caNG8qpQ3srL/W88cYbuH79Oj777LMilynuZ7IkWcGufZy6deumXJ5YlH/++Qcvv/wyNmzYgO7du5dRZSVjGpIgjYNgEhGRFU2fPh1r1qyxdxnlnr+/P8aMGVMm+yrXncMNBgOeeeYZjB07FvXr17donezsbLNLEVNTU21VnsI0CCb7OBERkTXVqFEDL7/8sr3LKPdeffXVMttXuQ5O06ZNg5OTE1555RWL14mLizMbsbQsuN/qHM6r6oiIyJaKG0/KHspbPWWh3F5Vt2/fPnz00UdYvHjxXYevz2/8+PFISUlRHufPn7dhlUZscSIiIro/lNvgtGPHDiQnJyMkJAROTk5wcnLC2bNn8eqrr6JGjRpFrqfT6eDl5WX2sDVTixNHDiciInJs5fZU3TPPPINOnTqZTYuOjsYzzzyDwYMH26mqwrnf6pyfzs7hRERm7hzdmcherPWzaNfglJaWhlOnTinvk5KSkJCQAF9fX4SEhKBy5cpmyzs7O6Nq1apmN1csD9jiRERkTqvVQq1W48KFC/Dz84NWqy1RtwsiaxER5OTk4MqVK1Cr1dBqtaXanl2D0969e83uKG26lDAmJqZCdTgz9XFiixMRkZFarUZYWBguXryICxcu2LscIri5uSEkJARqdel6Kdk1OLVr1w4lGX/zzJkztiumFNjiRERUkFarRUhICPLy8qDX6+1dDt3HNBoNnJycrNLqWW77OFUkSosTgxMRkRmVSgVnZ+dS3byVqDwpt1fVVSRuDE5ERET3BQYnK/AwnapjHyciIiKHxuBkBaYWp4wsgFfeEhEROS4GJyvwuBWcRICs7OKXJSIiooqLwckKXF0AU0d93q+OiIjIcTE4WYFKxfvVERER3Q8YnKzE1M8pjR3EiYiIHBaDk5WYrqxjixMREZHjYnCyErdbN/plixMREZHjYnCyEvZxIiIicnwMTlZiul8dW5yIiIgcF4OTlbDFiYiIyPExOFmJqcUpPcu+dRAREZHtMDhZianFiferIyIiclwMTlaiBCeeqiMiInJYDE5WwhYnIiIix8fgZCVKHye2OBERETksBicr4ak6IiIix8fgZCVuDE5EREQOj8HJSkz3qmMfJyIiIsfF4GQlbHEiIiJyfAxOVuLBkcOJiIgcHoOTlbhxOAIiIiKHx+BkJaY+Ttm5QG6efWshIiIi22BwshJTixPA03VERESOyq7Bafv27ejZsyeCgoKgUqmwatUqZV5ubi7eeOMNPPjgg3B3d0dQUBAGDhyICxcu2K/gYmidjQ+AHcSJiIgclV2DU3p6OqKiojBv3rwC8zIyMrB//35MmDAB+/fvxw8//IDExEQ89thjdqjUMqZWpzT2cyIiInJITvbcebdu3dCtW7dC53l7e2PTpk1m0+bOnYvmzZvj3LlzCAkJKYsSS8TdFbiRylN1REREjsquwamkUlJSoFKp4OPjU+Qy2dnZyM7OVt6npqaWQWVGvO0KERGRY6swncOzsrLwxhtvYMCAAfDy8ipyubi4OHh7eyuP4ODgMqvRnUMSEBERObQKEZxyc3Px5JNPQkTwySefFLvs+PHjkZKSojzOnz9fRlUC7qbbrrDFiYiIyCGV+1N1ptB09uxZ/PLLL8W2NgGATqeDTqcro+rM8VQdERGRYyvXwckUmk6ePImtW7eicuXK9i6pWAxOREREjs2uwSktLQ2nTp1S3iclJSEhIQG+vr4IDAxEv379sH//fvz000/Q6/W4dOkSAMDX1xdardZeZReJt10hIiJybHYNTnv37kX79u2V92PGjAEAxMTE4J133sHq1asBAA0bNjRbb+vWrWjXrl1ZlWkxD/ZxIiIicmh2DU7t2rWDiBQ5v7h55RFbnIiIiBxbhbiqrqJgHyciIiLHxuBkRabhCDKy7FsHERER2QaDkxW58151REREDo3ByYpMwYn3qiMiInJMDE5WxFuuEBEROTYGJyviLVeIiIgcG4OTFfGqOiIiIsfG4GRF+VucKtgQVERERGQBBicrMrU4GQxAVrZ9ayEiIiLrY3CyIjeX2695uo6IiMjxMDhZkVp9OzxxSAIiIiLHw+BkZRwEk4iIyHExOFkZhyQgIiJyXAxOVubGIQmIiIgcFoOTlXmwxYmIiMhhMThZmRtvu0JEROSwGJyszIOn6oiIiBwWg5OVscWJiIjIcTE4WZlpOAKO40REROR4GJysjMMREBEROS4GJytzZx8nIiIih8XgZGUMTkRERI6LwcnK3Nk5nIiIyGExOFkZ+zgRERE5LgYnK2OLExERkeNicLIypcUpy751EBERkfXZNTht374dPXv2RFBQEFQqFVatWmU2X0Tw9ttvIzAwEK6urujUqRNOnjxpn2ItxBYnIiIix2XX4JSeno6oqCjMmzev0Pnvv/8+Zs+ejfnz5+P333+Hu7s7oqOjkZVVfptz3HhVHRERkcNysufOu3Xrhm7duhU6T0Qwa9YsvPXWW+jVqxcA4Msvv0RAQABWrVqFp556qixLtZjHrVN1WdmAXg9oNPath4iIiKyn3PZxSkpKwqVLl9CpUydlmre3N1q0aIHffvvNjpUVz9TiBLDViYiIyNHYtcWpOJcuXQIABAQEmE0PCAhQ5hUmOzsb2dnZyvvU1FTbFFgEnTPgpAHy9Mb71Xl5lOnuiYiIyIbKbYvTvYqLi4O3t7fyCA4OLtP9q1S3W53S2OJERETkUMptcKpatSoA4PLly2bTL1++rMwrzPjx45GSkqI8zp8/b9M6C2Pq55TB4ERERORQym1wCgsLQ9WqVbFlyxZlWmpqKn7//Xe0bNmyyPV0Oh28vLzMHmVNaXHikAREREQOxa59nNLS0nDq1CnlfVJSEhISEuDr64uQkBCMGjUKU6ZMQZ06dRAWFoYJEyYgKCgIvXv3tl/RFvC4FZzY4kRERORY7Bqc9u7di/bt2yvvx4wZAwCIiYnB4sWL8frrryM9PR3PP/88bty4gdatW2P9+vVwcXGxV8kWYYsTERGRY7JrcGrXrh1EpMj5KpUKkyZNwqRJk8qwqtJzZ4sTERGRQyq3fZwqMuV+dQxOREREDoXByQbcedsVIiIih8TgZANscSIiInJMDE42oLQ4sXM4ERGRQ2FwsgGeqiMiInJMDE42wBYnIiIix8TgZAPs40REROSYGJxsgKfqiIiIHBODkw24MTgRERE5JAYnG/C4daqOI4cTERE5FgYnG+C96oiIiBwTg5MNeOS7V10xt+IjIiKiCqZEwen9999HZubt8087d+5Edna28v7mzZsYNmyY9aqroEwtTnl6IDvXvrUQERGR9ZQoOI0fPx43b95U3nfr1g3//POP8j4jIwOffvqp9aqroExX1QHs50RERORIShSc5I7zTne+JyONBnDRGV+znxMREZHjYB8nG+GVdURERI6HwclGeGUdERGR43Eq6Qqff/45PDw8AAB5eXlYvHgxqlSpAgBm/Z/ud+4uxme2OBERETmOEgWnkJAQLFiwQHlftWpVLF26tMAyxPvVEREROaISBaczZ87YqAzHo9yvjqfqiIiIHAb7ONkIW5yIiIgcT4mC02+//YaffvrJbNqXX36JsLAw+Pv74/nnnzcbEPN+5s4b/RIRETmcEgWnSZMm4ciRI8r7Q4cOYejQoejUqRPGjRuHNWvWIC4uzupFVkQMTkRERI6nRMEpISEBHTt2VN4vX74cLVq0wIIFCzBmzBjMnj0b3377rdWLrIjYx4mIiMjxlCg4Xb9+HQEBAcr7+Ph4dOvWTXnfrFkznD9/3nrVVWDs40REROR4ShScAgICkJSUBADIycnB/v378dBDDynzb968CWdnZ+tWWEG5scWJiIjI4ZQoOD366KMYN24cduzYgfHjx8PNzQ1t2rRR5v/555+oVauW1YrT6/WYMGECwsLC4Orqilq1amHy5MkV4h55yqm6LPvWQURERNZTonGcJk+ejL59+6Jt27bw8PDA4sWLodVqlfkLFy5Ely5drFbctGnT8Mknn2DJkiWoX78+9u7di8GDB8Pb2xuvvPKK1fZjC+68Vx0REZHDKVFwqlKlCrZv346UlBR4eHhAo9GYzV+xYgU8PT2tVtyuXbvQq1cvdO/eHQBQo0YNLFu2DH/88YfV9mEr7rxXHRERkcMpUXAaMmSIRcstXLjwnoq508MPP4zPPvsMJ06cQHh4OA4ePIhff/0VM2bMsMr2bckUnNjiRERE5DhKFJwWL16M0NBQNGrUqEz6GY0bNw6pqamIiIiARqOBXq/Hu+++i6effrrIdbKzs80G4UxNTbV5nYXhcARERESOp0TB6aWXXsKyZcuQlJSEwYMH4z//+Q98fX1tVRu+/fZbfP311/jf//6H+vXrIyEhAaNGjUJQUBBiYmIKXScuLg6xsbE2q8lSHhyOgIiIyOGopIRNR9nZ2fjhhx+wcOFC7Nq1C927d8fQoUPRpUsXqFQqqxYXHByMcePGYfjw4cq0KVOm4KuvvsLx48eLrO/OFqfg4GCkpKTAy8vLqvUV59/rQJO+xtdJWwA17wpIRERULqWmpsLb29uirFDiX+c6nQ4DBgzApk2bcPToUdSvXx/Dhg1DjRo1kJaWds9FFyYjIwPqOxKHRqOBwWAotj4vLy+zhz2YWpwAIINDEhARETmEEp2qu5NarYZKpYKIQK/XW6smRc+ePfHuu+8iJCQE9evXx4EDBzBjxgyLO6nbk05rbGUyGIz9nPIHKSIiIqqYStzilJ2djWXLlqFz584IDw/HoUOHMHfuXJw7dw4eHh5WLW7OnDno168fhg0bhsjISLz22mt44YUXMHnyZKvuxxZUKt7ol4iIyNGUqMVp2LBhWL58OYKDgzFkyBAsW7YMVapUsVVt8PT0xKxZszBr1iyb7cOW3F2Bm+kMTkRERI6iRMFp/vz5CAkJQc2aNREfH4/4+PhCl/vhhx+sUlxFxyEJiIiIHEuJgtPAgQOtfuWcI3PnkAREREQOpcQDYJLl3NjHiYiIyKFwdCEb4iCYREREjoXByYbcXIzP7ONERETkGBicbIgtTkRERI6FwcmG3HhVHRERkUNhcLIhDoBJRETkWBicbMg0HEEGgxMREZFDYHCyIbY4ERERORYGJxticCIiInIsDE42xFuuEBERORYGJxviLVeIiIgcC4OTDfFUHRERkWNhcLIhtjgRERE5FgYnG2IfJyIiIsfC4GRDpuCUmwfk5Nq3FiIiIio9BicbMt1yBeAgmERERI6AwcmGnJ0AnbPxdRpP1xEREVV4DE42xtuuEBEROQ4GJxszna5LY3AiIiKq8BicbMyDLU5EREQOg8HJxpQWJ/ZxIiIiqvAYnGzM41ZwYosTERFRxcfgZGNscSIiInIcDE425s4WJyIiIodR7oPTP//8g//85z+oXLkyXF1d8eCDD2Lv3r32LstivF8dERGR43CydwHFuX79Olq1aoX27dtj3bp18PPzw8mTJ1GpUiV7l2Yx5X51DE5EREQVXrkOTtOmTUNwcDAWLVqkTAsLC7NjRSXHFiciIiLHUa5P1a1evRpNmzbFE088AX9/fzRq1AgLFiywd1kl4u5ifE5n53AiIqIKr1wHp7/++guffPIJ6tSpgw0bNuCll17CK6+8giVLlhS5TnZ2NlJTU80e9sQWJyIiIsdRrk/VGQwGNG3aFO+99x4AoFGjRjh8+DDmz5+PmJiYQteJi4tDbGxsWZZZLKWPE1uciIiIKrxy3eIUGBiIevXqmU2LjIzEuXPnilxn/PjxSElJUR7nz5+3dZnFYosTERGR4yjXLU6tWrVCYmKi2bQTJ04gNDS0yHV0Oh10Op2tS7MYr6ojIiJyHOW6xWn06NHYvXs33nvvPZw6dQr/+9//8Nlnn2H48OH2Ls1ibgxOREREDqNcB6dmzZph5cqVWLZsGR544AFMnjwZs2bNwtNPP23v0izmcetUHUcOJyIiqvjK9ak6AOjRowd69Ohh7zLuWf4WJxFApbJvPURERHTvynWLkyPwuBWcRIDMLPvWQkRERKXD4GRjri63W5nSeLqOiIioQmNwsjGV6vaVdeznREREVLExOJUBUz+nNA6CSUREVKExOJUBXllHRETkGBicyoDbrRv9ssWJiIioYmNwKgPs40REROQYGJzKAO9XR0RE5BgYnMqAcr86nqojIiKq0BicyoDS4sQBMImIiCo0BqcyYGpxuplm3zqIiIiodBicykBwVePzX+ftWwcRERGVDoNTGYisZXw+9pd96yAiIqLSYXAqA6bg9PclIJWn64iIiCosBqcy4O0JBPkbXycm2bcWIiIiuncMTmUkoqbx+egp+9ZBRERE947BqYyYTtcdZz8nIiKiCovBqYxE3mpxOnbavnUQERHRvWNwKiORtY3Px5MAg8G+tRAREdG9YXAqIzWqATotkJkFnL1g72qIiIjoXjA4lREnDVA3zPiap+uIiIgqJganMhTBfk5EREQVGoNTGarHK+uIiIgqNAanMhRxKzgdZYsTERFRhcTgVIZMQxLw1itEREQVE4NTGfLxAgL9jK956xUiIqKKh8GpjJlGEGcHcSIiooqnQgWnqVOnQqVSYdSoUfYu5Z5xBHEiIqKKq8IEpz179uDTTz9FgwYN7F1KqSgtTryyjoiIqMKpEMEpLS0NTz/9NBYsWIBKlSrZu5xSMV1Zl/gXb71CRERU0VSI4DR8+HB0794dnTp1uuuy2dnZSE1NNXuUJ2HVjbdeycgCzl20dzVERERUEuU+OC1fvhz79+9HXFycRcvHxcXB29tbeQQHB9u4wpJx0gDhNYyvj56yaylERERUQuU6OJ0/fx4jR47E119/DRcXF4vWGT9+PFJSUpTH+fPnbVxlyUVyBHEiIqIKycneBRRn3759SE5ORuPGjZVper0e27dvx9y5c5GdnQ2NRmO2jk6ng06nK+tSS4RX1hEREVVM5To4dezYEYcOHTKbNnjwYEREROCNN94oEJoqisjaxmdeWUdERFSxlOvg5OnpiQceeMBsmru7OypXrlxgekVianE6fxG4mQ54utu3HiIiIrJMue7j5Kjy33qF/ZyIiIgqjnLd4lSYbdu22bsEq4ioCVy8Yuzn1OxBe1dDRERElmCLk53U45V1REREFQ6Dk52YRhA/yivriIiIKgwGJzuJ5K1XiIiIKhwGJzsJqw7onHnrFSIiooqEwclOnDRAeJjxNQfCJCIiqhgYnOwogiOIExERVSgMTnbEK+uIiIgqFgYnO+KVdURERBULg5Md3XnrFSIiIirfGJzsqJI3ULWK8TVP1xEREZV/DE52Fsl+TkRERBUGg5OdmYITr6wjIiIq/xic7EwJTmxxIiIiKvcYnOzMNJbT8dO89QoREVF5x+BkZzWDb9965TxvvUJERFSuMTjZmZMGqFPD+JrjOREREZVvDE7lAK+sIyIiqhgYnMoBXllHRERUMTA4lQO8so6IiKhiYHAqB0y3Xjl3gbdeISIiKs8YnMqB/LdeSUyyby1ERERUNAanciKC/ZyIiIjKPQancqIer6wjIiIq9xicygnTCOIcy4mIiKj8YnAqJ0xX1iX+xVuvEBERlVcMTuWE6dYr6ZnAmX/sXQ0REREVptwHp7i4ODRr1gyenp7w9/dH7969kZiYaO+yrM5JAzSMNL7+cKF9ayEiIqLClfvgFB8fj+HDh2P37t3YtGkTcnNz0aVLF6SnO96ARxOGARo18NM24Ket9q6GiIiI7qQSEbF3ESVx5coV+Pv7Iz4+Ho888shdl09NTYW3tzdSUlLg5eVVBhWWzocLgdlLgUpewKZFgJ+vvSsiIiJybCXJCuW+xelOKSkpAABf38ITRXZ2NlJTU80eFcnLzxhHEr+eCrw5C6hYsZaIiMixVajgZDAYMGrUKLRq1QoPPPBAocvExcXB29tbeQQHB5dxlaWjdQY+HG/s87RhB7D6F3tXRERERCYVKjgNHz4chw8fxvLly4tcZvz48UhJSVEe58+fL8MKraN+beCVgcbXEz4CLl+1bz1ERERkVGGC04gRI/DTTz9h69atqF69epHL6XQ6eHl5mT0qomH/BzxQB0i5Cfx3Bk/ZERERlQflPjiJCEaMGIGVK1fil19+QVhYmL1LKhPOTsCM8cbnzbuAHzbauyIiIiIq98Fp+PDh+Oqrr/C///0Pnp6euHTpEi5duoTMzEx7l2ZzdcOA0YOMr9+ZA1y6YtdyiIiI7nvlfjgClUpV6PRFixZh0KBBd12/og1HcKc8PdB3BHDwONCuBbA4DijiKyEiIqJ74FDDEYhIoQ9LQpMjcNIAH7xhvB3Ltt+Bb9fZuyIiIqL7V7kPTgSE1wDGDDG+nvwx8M9lu5ZDRER032JwqiCeewJoVA+4mQ688QGvsiMiIrIHBqcKQqMBPnwD0GmBHXuBhd8zPBEREZU1BqcKpFYI8PqzxteT5gGdBwNLVhpboYiIiMj2GJwqmCGPA8/3B9xcgJNngbdnA837AW/OBI7/Ze/qiIiIHFu5H46gtCr6cARFSU0DftgELP0ROHX29vTmDYBnegFd2xjve0dERETFK0lWYHCq4ESA3xKMAWrDDkBvME6vUgno/6gxSAVXBapVBVy0di2ViIioXGJwysfRg1N+l/8Flq0F/veT8fWd/CsbQ1RwIFC96u3XQf6Ah5vx4aLjAJtERHR/YXDK534KTia5ecCmncCqLcCZv4HzF4GMLMvWVasBD1fA3Q1wczWGKXfX26FKpzU+tM63HzotoL01TXdrmpOT8T57zk6As7NxIM87pzs5Ac4a47NpupPm9jwnDUMcERHZXkmyglMZ1URlyNkJeLSt8QEYT+ddSwH+vmQMUX9fAs7ne33xyu1gZTAAqenGR3ngpDEOxeDsZP7slP+Rf5rT7XXyL6O512e1+bT879W33jtpjIHT9KzMzz9Nbb6OaZ5GbZyWfznTNOW1+va+GSSJiOyLwakCkpxsQK2Gysmy3t8qFVDZx/iIiih8GYPBGJ7SM4C0DCA989bD9D4DyMwGcnKNj+wc48P0Ov/0vDwgJ8/4nJsL5OpvPefle+Qa78OXl2d8zs0rfFyqPL3xkZ1zz1+XQ1GpCoYrJXBp8s2/I4ip1OZBTH2X16o71lWrzJdRq/K9vvO96nYt6nzrqopZz7Ss5tY0s2WLea9SFdxG/veFTVOrARWKX+fO7Ra63q067pwGVb7a8s2HqpD1VQW3n38aEZU/DE6loL96GRk7fzYGGCdnqG494ORkfNaYT4PKOPpDoTcuvjVNcnNgSLkGfcpVGFJvPadcM5smmcbmIJW7FzTelaH29lWe1V75X1cCoAIMekCfB9HnAXr9Hc95EBGoNBqoNU7w0jjBS6OBSuMMaJygquQEVHGCSuMEQCA5OZDcbEhuDpB7+7Xk5kDycoyJSKOBylkLldYFKq0OKmfdrWctoHUxznPO11NdBIBAn2fs3K7XC/LyBHl6QJ8n0OuBvFvT9XoojzzTa4PxIxqXubWNW68N+abl6QG59aw3W1Z167Xq9nqiQp5BpWzXYADyDCro9SoYBMjVqyAGgUEvEL3AIALRG2AwCAwGMc4zGGDQCwwC6A1q6A0q5OrVMAiQd+t9nl5lfDYYny0+cW4wPlS4+wr6W895hcwTWP+3s6mm/LUVVeed+8//3lq1WfIdFccW35G5orevBLP8QUpVSPACoFKJ+XtTELu1jvrW4DOqAutBCXV3zhOobu/7znVwOygWuT0UPT//Jy+qpqKWMU1Xlsu3v0K3cedzYa/vCKtm9eQ7VHdu687t5z+aZtML2Xb+z1fU8vn3bbbfO7+fourIv14h69y538L2Wej+i9hOgXUL+55QyLRitpv/TYt2AQiq7gZ7YnAqhbzkv5H6v1l227+kpyIvPRW4kGS3GsqCCoDzrUeFpgZHTqPyR249iCqACy7zEdSvpV1rYHAqBbVXJbi1fQySlwfJywXycs2eC7wG8p2PEvP3pslOTtB4+eZrRSqsRckXYjDAkHIVhpSr0Kdcu/2ceuv1jasw3LwBAFA5OQFqjbHVSGPsEKTK/6xS3dESlWtsicrLM2upAgCVVgfcajFSOetutx45aaHSaqFycoYYDEBOtrE1KufW447XMH0fxq3e8adPEX8m3infdFUJWgUk/2+Jwpp4RAo/TsqiYmylM/sz+fY5HZXpz3vTPNP2RCBiMHsPMQAC43QLFdpiaWfK92GmkD8hC13Z4okW7LMo9/qdFayjVNfTFLFqsVsU039Ud1+2kA3eS7Wi7NOCfeQ7BlLgRQn3a/EMy3Zw19ZCCzajfEzIvbU+FvnzXdq20Pybs27ytUUra2l/Nkw8vewfW3hVHREREd3XSpIVeOKAiIiIyEIMTkREREQWYnAiIiIishCDExEREZGFGJyIiIiILMTgRERERGQhBiciIiIiCzE4EREREVmIwYmIiIjIQgxORERERBZicCIiIiKykP3vlmdjplvxpaam2rkSIiIiKo9MGcGS2/c6fHC6efMmACA4ONjOlRAREVF5dvPmTXh7exe7jEosiVcVmMFgwIULF+Dp6QmVSmX17aempiI4OBjnz5+/6x2VyTZ4DOyPx8D+eAzsj8fA/u71GIgIbt68iaCgIKjVxfdicvgWJ7VajerVq9t8P15eXvyHYmc8BvbHY2B/PAb2x2Ngf/dyDO7W0mTCzuFEREREFmJwIiIiIrIQg1Mp6XQ6TJw4ETqdzt6l3Ld4DOyPx8D+eAzsj8fA/sriGDh853AiIiIia2GLExEREZGFGJyIiIiILMTgRERERGQhBqdSmDdvHmrUqAEXFxe0aNECf/zxh71Lcmjbt29Hz549ERQUBJVKhVWrVpnNFxG8/fbbCAwMhKurKzp16oSTJ0/ap1gHFBcXh2bNmsHT0xP+/v7o3bs3EhMTzZbJysrC8OHDUblyZXh4eODxxx/H5cuX7VSx4/nkk0/QoEEDZYyali1bYt26dcp8fv9lb+rUqVCpVBg1apQyjcfBtt555x2oVCqzR0REhDLf1t8/g9M9+uabbzBmzBhMnDgR+/fvR1RUFKKjo5GcnGzv0hxWeno6oqKiMG/evELnv//++5g9ezbmz5+P33//He7u7oiOjkZWVlYZV+qY4uPjMXz4cOzevRubNm1Cbm4uunTpgvT0dGWZ0aNHY82aNVixYgXi4+Nx4cIF9O3b145VO5bq1atj6tSp2LdvH/bu3YsOHTqgV69eOHLkCAB+/2Vtz549+PTTT9GgQQOz6TwOtle/fn1cvHhRefz666/KPJt//0L3pHnz5jJ8+HDlvV6vl6CgIImLi7NjVfcPALJy5UrlvcFgkKpVq8r06dOVaTdu3BCdTifLli2zQ4WOLzk5WQBIfHy8iBi/b2dnZ1mxYoWyzLFjxwSA/Pbbb/Yq0+FVqlRJPv/8c37/ZezmzZtSp04d2bRpk7Rt21ZGjhwpIvx3UBYmTpwoUVFRhc4ri++fLU73ICcnB/v27UOnTp2UaWq1Gp06dcJvv/1mx8ruX0lJSbh06ZLZMfH29kaLFi14TGwkJSUFAODr6wsA2LdvH3Jzc82OQUREBEJCQngMbECv12P58uVIT09Hy5Yt+f2XseHDh6N79+5m3zfAfwdl5eTJkwgKCkLNmjXx9NNP49y5cwDK5vt3+HvV2cK///4LvV6PgIAAs+kBAQE4fvy4naq6v126dAkACj0mpnlkPQaDAaNGjUKrVq3wwAMPADAeA61WCx8fH7NleQys69ChQ2jZsiWysrLg4eGBlStXol69ekhISOD3X0aWL1+O/fv3Y8+ePQXm8d+B7bVo0QKLFy9G3bp1cfHiRcTGxqJNmzY4fPhwmXz/DE5EVGLDhw/H4cOHzfoVUNmoW7cuEhISkJKSgu+++w4xMTGIj4+3d1n3jfPnz2PkyJHYtGkTXFxc7F3Ofalbt27K6wYNGqBFixYIDQ3Ft99+C1dXV5vvn6fq7kGVKlWg0WgK9NK/fPkyqlataqeq7m+m753HxPZGjBiBn376CVu3bkX16tWV6VWrVkVOTg5u3LhhtjyPgXVptVrUrl0bTZo0QVxcHKKiovDRRx/x+y8j+/btQ3JyMho3bgwnJyc4OTkhPj4es2fPhpOTEwICAngcypiPjw/Cw8Nx6tSpMvl3wOB0D7RaLZo0aYItW7Yo0wwGA7Zs2YKWLVvasbL7V1hYGKpWrWp2TFJTU/H777/zmFiJiGDEiBFYuXIlfvnlF4SFhZnNb9KkCZydnc2OQWJiIs6dO8djYEMGgwHZ2dn8/stIx44dcejQISQkJCiPpk2b4umnn1Ze8ziUrbS0NJw+fRqBgYFl8+/AKl3M70PLly8XnU4nixcvlqNHj8rzzz8vPj4+cunSJXuX5rBu3rwpBw4ckAMHDggAmTFjhhw4cEDOnj0rIiJTp04VHx8f+fHHH+XPP/+UXr16SVhYmGRmZtq5csfw0ksvibe3t2zbtk0uXryoPDIyMpRlXnzxRQkJCZFffvlF9u7dKy1btpSWLVvasWrHMm7cOImPj5ekpCT5888/Zdy4caJSqWTjxo0iwu/fXvJfVSfC42Brr776qmzbtk2SkpJk586d0qlTJ6lSpYokJyeLiO2/fwanUpgzZ46EhISIVquV5s2by+7du+1dkkPbunWrACjwiImJERHjkAQTJkyQgIAA0el00rFjR0lMTLRv0Q6ksO8egCxatEhZJjMzU4YNGyaVKlUSNzc36dOnj1y8eNF+RTuYIUOGSGhoqGi1WvHz85OOHTsqoUmE37+93BmceBxsq3///hIYGCharVaqVasm/fv3l1OnTinzbf39q0RErNN2RUREROTY2MeJiIiIyEIMTkREREQWYnAiIiIishCDExEREZGFGJyIiIiILMTgRERERGQhBiciIiIiCzE4EREREVmIwYmIHM62bdugUqkK3OiTiKi0GJyIiPLJzMyEu7s7Tp06VeJ1VSoVVq1aZf2iiKjcYHAiIspn06ZNCA0NRe3ate1dChGVQwxORGRVBoMBcXFxCAsLg6urK6KiovDdd98p802n0dauXYsGDRrAxcUFDz30EA4fPmy2ne+//x7169eHTqdDjRo18OGHH5rNz87OxhtvvIHg4GDodDrUrl0bX3zxhdky+/btQ9OmTeHm5oaHH34YiYmJd63/xx9/xGOPPVbovJycHIwYMQKBgYFwcXFBaGgo4uLiAAA1atQAAPTp0wcqlUp5b9pm48aN4eLigpo1ayI2NhZ5eXnKfJVKhU8++QTdunWDq6sratasafadEVE5YrXbBRMRiciUKVMkIiJC1q9fL6dPn5ZFixaJTqeTbdu2iYjI1q1bBYBERkbKxo0b5c8//5QePXpIjRo1JCcnR0RE9u7dK2q1WiZNmiSJiYmyaNEicXV1lUWLFin7efLJJyU4OFh++OEHOX36tGzevFmWL19uto8WLVrItm3b5MiRI9KmTRt5+OGHi61dr9eLv7+/7Nq1q9D506dPl+DgYNm+fbucOXNGduzYIf/73/9ERCQ5OVkAyKJFi+TixYuSnJwsIiLbt28XLy8vWbx4sZw+fVo2btwoNWrUkHfeeUfZLgCpXLmyLFiwQBITE+Wtt94SjUYjR48evbeDQEQ2w+BERFaTlZUlbm5uBYLH0KFDZcCAASJyO9SYQo6IyNWrV8XV1VW++eYbERH5v//7P+ncubPZNsaOHSv16tUTEZHExEQBIJs2bSq0DtM+Nm/erExbu3atAJDMzMwi69+5c6f4+/uLXq8vdP7LL78sHTp0EIPBUOh8ALJy5UqzaR07dpT33nvPbNrSpUslMDDQbL0XX3zRbJkWLVrISy+9VGStRGQfTvZq6SIix3Pq1ClkZGSgc+fOZtNzcnLQqFEjs2ktW7ZUXvv6+qJu3bo4duwYAODYsWPo1auX2fKtWrXCrFmzoNfrkZCQAI1Gg7Zt2xZbT4MGDZTXgYGBAIDk5GSEhIQUuvyPP/6IHj16QK0uvBfDoEGD0LlzZ9StWxddu3ZFjx490KVLl2JrOHjwIHbu3Il3331XmabX65GVlYWMjAy4ubkBMP8+TO8TEhKK3TYRlT0GJyKymrS0NADA2rVrUa1aNbN5Op3OavtxdXW1aDlnZ2fltUqlAmDsg1WU1atXY+rUqUXOb9y4MZKSkrBu3Tps3rwZTz75JDp16lRsf6S0tDTExsaib9++Bea5uLhY8jGIqBxhcCIiq6lXrx50Oh3OnTt319ag3bt3Ky0/169fx4kTJxAZGQkAiIyMxM6dO82W37lzJ8LDw6HRaPDggw/CYDAgPj4enTp1skrtJ0+exNmzZwu0lt3Jy8sL/fv3R//+/dGvXz907doV165dg6+vL5ydnaHX682Wb9y4MRITE+96ld7u3bsxcOBAs/d3ttIRkf0xOBGR1Xh6euK1117D6NGjYTAY0Lp1a6SkpGDnzp3w8vJCTEyMsuykSZNQuXJlBAQE4M0330SVKlXQu3dvAMCrr76KZs2aYfLkyejfvz9+++03zJ07Fx9//DEA4xVsMTExGDJkCGbPno2oqCicPXsWycnJePLJJ++p9h9//BGdOnVSTp0VZsaMGQgMDESjRo2gVquxYsUKVK1aFT4+PkpdW7ZsQatWraDT6VCpUiW8/fbb6NGjB0JCQtCvXz+o1WocPHgQhw8fxpQpU5Rtr1ixAk2bNkXr1q3x9ddf448//ihwlSARlQP27mRFRI7FYDDIrFmzpG7duuLs7Cx+fn4SHR0t8fHxInK74/aaNWukfv36otVqpXnz5nLw4EGz7Xz33XdSr149cXZ2lpCQEJk+fbrZ/MzMTBk9erQEBgaKVquV2rVry8KFC832cf36dWX5AwcOCABJSkoqtO7WrVvLggULiv1sn332mTRs2FDc3d3Fy8tLOnbsKPv371fmr169WmrXri1OTk4SGhqqTF+/fr08/PDD4urqKl5eXtK8eXP57LPPlPkAZN68edK5c2fR6XRSo0YNpaM8EZUvKhER+0Y3IrqfbNu2De3bt8f169eVlhp7+/fffxEYGIi///4bAQEBZb5/lUqFlStXKi1uRFR+cQBMIrrvXbt2DTNmzLBLaCKiioV9nIjovhceHo7w8HB7l0FEFQBP1RERERFZiKfqiIiIiCzE4ERERERkIQYnIiIiIgsxOBERERFZiMGJiIiIyEIMTkREREQWYnAiIiIishCDExEREZGFGJyIiIiILPT/d7ieH4UrESMAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# viz: SGD's jagged descent vs batch GD's smooth one, on the same axes\n",
    "plt.figure(figsize=(6, 4))\n",
    "plt.plot(losses_gd[:len(losses_sgd)], label=\"batch GD\", color=\"#1E40FF\")\n",
    "plt.plot(losses_sgd, label=\"SGD (per epoch)\", color=\"#E4572E\")\n",
    "plt.xlabel(\"epoch / step\"); plt.ylabel(\"MSE\"); plt.legend()\n",
    "plt.title(\"SGD trades smoothness for cheap steps\")\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ddad8191",
   "metadata": {},
   "source": [
    "> **Note:** mini-batch GD is the middle path: a few dozen examples per step instead of one or all. It cuts the gradient variance versus SGD and vectorizes cleanly on GPUs, which is why essentially every neural network trains with mini-batches. The only change is computing the gradient over a slice of rows rather than one row or all rows.\n",
    "\n",
    "> **Key takeaways.** SGD estimates the gradient from one example, trading a smooth path for cheap steps. A decaying learning-rate schedule is what turns the bouncing into convergence. Mini-batches sit in between and are the practical default.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd16cc2d",
   "metadata": {},
   "source": [
    "## Part 5 — Momentum, Adam, and an optimizer toolkit\n",
    "\n",
    "> **Objectives.** Build six update rules as small classes (GD, momentum, Nesterov, RMSProp, Adam, AdamW), show why momentum helps on ill-conditioned problems, and test one of them against `torch.optim` on a fixed problem so the cross-check is a real reference, not an echo.\n",
    "\n",
    "Plain GD takes the same-sized step in every direction. On an **ill-conditioned** loss (one direction far steeper than another) that forces a tiny global learning rate and a slow zig-zag. **Momentum** fixes this by accumulating a velocity:\n",
    "\n",
    "$$\\mathbf{v} \\leftarrow \\beta\\,\\mathbf{v} - \\eta\\,\\nabla J, \\qquad \\boldsymbol\\theta \\leftarrow \\boldsymbol\\theta + \\mathbf{v}.$$\n",
    "\n",
    "When the gradient points the same way across steps, the velocity builds and the optimizer accelerates; when the gradient flips sign across a valley, successive velocities cancel and the oscillation damps. $\\beta$ is the momentum coefficient, usually 0.9.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a16c2091",
   "metadata": {},
   "source": [
    "### Exercise 4.4 — Momentum on an ill-conditioned quadratic\n",
    "`Difficulty 2/5 · ~12 min`\n",
    "\n",
    "Fill in `momentum_gd(grad_fn, w0, eta, beta, n_steps)`. Keep a velocity `v` initialized to zeros, and each step set `v = beta*v - eta*grad` then `w = w + v`. Return the list of `||w||` after each step. We run it on the quadratic $L = \\tfrac12(a w_0^2 + b w_1^2)$ with $a=50, b=1$ (condition number 50), whose minimum is the origin. The check asserts momentum reaches $\\lVert w\\rVert < 0.05$ in fewer steps than plain GD at the *same* learning rate.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "30013d4e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.525486Z",
     "iopub.status.busy": "2026-06-10T18:46:10.525411Z",
     "iopub.status.idle": "2026-06-10T18:46:10.529864Z",
     "shell.execute_reply": "2026-06-10T18:46:10.529538Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] 4.4 momentum beats plain GD: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the ill-conditioned test problem: gradient of L = 0.5*(a*w0^2 + b*w1^2)\n",
    "A_COND, B_COND = 50.0, 1.0   # eigenvalues 50 and 1 -> condition number 50\n",
    "def quad_grad(w):\n",
    "    return np.array([A_COND * w[0], B_COND * w[1]])\n",
    "\n",
    "def momentum_gd(grad_fn, w0, eta, beta, n_steps):\n",
    "    \"\"\"Heavy-ball momentum. Returns the list of ||w|| after each step.\"\"\"\n",
    "    w = np.array(w0, dtype=float)\n",
    "    v = np.zeros_like(w)\n",
    "    norms = []\n",
    "    for _ in range(n_steps):\n",
    "        g = grad_fn(w)\n",
    "        # TODO 1: v = beta * v - eta * g\n",
    "        v = None\n",
    "        attempted(v)\n",
    "        # TODO 2: w = w + v\n",
    "        w = w + v\n",
    "        norms.append(float(np.linalg.norm(w)))\n",
    "    return norms\n",
    "\n",
    "def _first_below(norms, tol=0.05):\n",
    "    for i, n in enumerate(norms):\n",
    "        if n < tol:\n",
    "            return i\n",
    "    return len(norms)\n",
    "\n",
    "def _momentum_beats_gd():\n",
    "    eta = 0.039                                  # just under 2/A_COND, same for both\n",
    "    n_plain = momentum_gd(quad_grad, [1.0, 1.0], eta, beta=0.0, n_steps=150)\n",
    "    n_mom   = momentum_gd(quad_grad, [1.0, 1.0], eta, beta=0.9, n_steps=150)\n",
    "    s_plain, s_mom = _first_below(n_plain), _first_below(n_mom)\n",
    "    assert s_mom < s_plain, \\\n",
    "        f\"momentum should reach ||w||<0.05 sooner: momentum {s_mom} steps vs plain {s_plain}\"\n",
    "\n",
    "check(\"4.4 momentum beats plain GD\", _momentum_beats_gd)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3f24b33",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>Two lines per step. First update the velocity by blending the old velocity (scaled by `beta`) with the new gradient step. Then move `w` by that velocity.</details>\n",
    "\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "v = beta * v - eta * g\n",
    "w = w + v\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — momentum is slower, not faster</summary>Check the sign: the velocity update is `beta*v - eta*g` (minus the gradient step), and `w` moves by `+ v`. If you wrote `w = w - v` you are walking uphill. If beta=0 and momentum still differs from plain GD, your plain run is using a different learning rate.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "54ca40d0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.530694Z",
     "iopub.status.busy": "2026-06-10T18:46:10.530620Z",
     "iopub.status.idle": "2026-06-10T18:46:10.535418Z",
     "shell.execute_reply": "2026-06-10T18:46:10.535166Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 4.4 momentum beats plain GD\n",
      "steps to ||w||<0.05 -> plain 77, momentum 39\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines momentum_gd; the check below re-verifies the reference.\n",
    "def momentum_gd(grad_fn, w0, eta, beta, n_steps):\n",
    "    w = np.array(w0, dtype=float)\n",
    "    v = np.zeros_like(w)\n",
    "    norms = []\n",
    "    for _ in range(n_steps):\n",
    "        g = grad_fn(w)\n",
    "        v = beta * v - eta * g\n",
    "        w = w + v\n",
    "        norms.append(float(np.linalg.norm(w)))\n",
    "    return norms\n",
    "\n",
    "check(\"4.4 momentum beats plain GD\", _momentum_beats_gd, required=True)\n",
    "eta = 0.039\n",
    "n_plain = momentum_gd(quad_grad, [1.0, 1.0], eta, beta=0.0, n_steps=150)\n",
    "n_mom   = momentum_gd(quad_grad, [1.0, 1.0], eta, beta=0.9, n_steps=150)\n",
    "print(f\"steps to ||w||<0.05 -> plain {_first_below(n_plain)}, momentum {_first_below(n_mom)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "4bc089a8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.536239Z",
     "iopub.status.busy": "2026-06-10T18:46:10.536169Z",
     "iopub.status.idle": "2026-06-10T18:46:10.673218Z",
     "shell.execute_reply": "2026-06-10T18:46:10.672801Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAGGCAYAAACNCg6xAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAuS9JREFUeJzsnXd4FNX3h9/Zll5JgUAgoRO6FEGQXkREQUVApYn4A6kidqUoXxBFQAHBRhELRRRFmoCAUpSO9N5DGiG9bZnfH7M72U2BlA0JcN/nmSczd2bvnNlN+eScc8+RZFmWEQgEAoFAIBDcFk1pGyAQCAQCgUBwtyCEk0AgEAgEAkEBEcJJIBAIBAKBoIAI4SQQCAQCgUBQQIRwEggEAoFAICggQjgJBAKBQCAQFBAhnAQCgUAgEAgKiBBOAoFAIBAIBAVECCeBQCAQCASCAiKEk0AgKBbt2rWjXbt26vHFixeRJInFixerY5MmTUKSpDtvXDFZvHgxkiRx8eJFdSzn896KQYMGERYWViK2OYtt27YhSRLbtm0rbVMKTV6fT0lSEu/VihUr8Pf3JyUlxWlz3mk2bNiAp6cnsbGxpW3KHUEIJ8E9SVpaGpMmTbor/xgI7i4iIyOZNGkShw4dKm1TBE7i888/dxD+JYXZbGbixImMGjUKT0/PEr8fZP9jk9e2bNmyXNefOHGCRx55BE9PT/z9/enfv38ugfTII49QvXp1pk2bdkeeobTRlbYBAkFJkJaWxuTJkwEK7B0QFI0//vijtE24o+R83sjISCZPnkxYWBiNGjVyOPfVV19hsVjuoHUCZ/D5558TEBDAoEGDHMbbtGlDeno6BoPBKfdZs2YNp06d4qWXXnLKfIWhX79+PProow5jLVu2dDi+evUqbdq0wcfHh6lTp5KSksKMGTM4cuQIe/bscXgf/u///o/x48czefJkvLy87sgzlBZCOAkEgmLhrD8idwuFeV69Xl+ClggKgizLZGRk4ObmVuy5NBoNrq6uTrBKYdGiRbRq1YqKFSs6bc6C8sADD/D888/f8pqpU6eSmprK/v37qVy5MgDNmzenc+fOLF682EHwPfXUU4waNYqVK1fywgsvlKjtpY0I1d3D2PJKTp8+zfPPP4+Pjw+BgYG89957yLLMlStXeOKJJ/D29qZ8+fJ88sknueaIiYlhyJAhBAcH4+rqSsOGDVmyZInDNTbX74wZM5g3bx5Vq1bF3d2dLl26cOXKFWRZ5oMPPqBSpUq4ubnxxBNPEB8fn+te69ev5+GHH8bDwwMvLy+6d+/OsWPHHK4ZNGgQnp6eXLt2jZ49e+Lp6UlgYCDjx4/HbDar9gQGBgIwefJk1Q09adIkIP8clZz5KM56rrz4888/1Wf19fXliSee4MSJEw7X2D6/s2fPMmjQIHx9ffHx8WHw4MGkpaXd9h5///03vXv3pnLlyri4uBAaGsorr7xCenp6od/TW1GYnJ+CYLFY+PTTT6lfvz6urq4EBgbyyCOPsG/fPvUak8nEBx98QLVq1XBxcSEsLIy3336bzMxMh7nCwsJ47LHH2LFjB82bN8fV1ZWqVavy7bff5rrvsWPH6NChA25ublSqVIkpU6bk6S2yf95t27bRrFkzAAYPHqx+r9nCPHnlOKWmpvLqq68SGhqKi4sLtWrVYsaMGciy7HCdJEmMHDmS1atXU69ePVxcXKhbty4bNmzIZdO1a9d44YUXCA4OVq9buHBhruuuXr1Kz5498fDwICgoiFdeeSXXe3YrduzYQbNmzXB1daVatWp88cUXufLX8spxs38m288hwKVLl3j55ZepVasWbm5ulCtXjt69e+eZs1TQz8f2mW/cuJGmTZvi5ubGF198AShCpUOHDgQFBeHi4kJERATz58/P9fpjx46xfft29fO0/7zzynH6999/efTRR/Hz88PDw4MGDRrw6aef3vK9zMjIYMOGDXTq1CnP92nkyJHqcWRkJFWrVqVFixYkJyffct7CkJqaSlZWVr7nV61axWOPPaaKJoBOnTpRs2ZNVqxY4XBtUFAQDRo04Ndff3WafWUV4XG6D+jTpw916tThww8/ZO3atUyZMgV/f3+++OILOnTowPTp0/n+++8ZP348zZo1o02bNgCkp6fTrl07zp49y8iRIwkPD2flypUMGjSIhIQExowZ43Cf77//nqysLEaNGkV8fDwfffQRzzzzDB06dGDbtm288cYbnD17ljlz5jB+/HiHX+xLly5l4MCBdO3alenTp5OWlsb8+fNp3bo1Bw8edPjjYzab6dq1Kw8++CAzZsxg8+bNfPLJJ1SrVo3hw4cTGBjI/PnzGT58OL169eLJJ58EoEGDBkV6/4rzXHmxefNmunXrRtWqVZk0aRLp6enMmTOHVq1aceDAgVx/aJ955hnCw8OZNm0aBw4c4OuvvyYoKIjp06ff8j4rV64kLS2N4cOHU65cOfbs2cOcOXO4evUqK1eudLj2du/pnWTIkCEsXryYbt268eKLL2Iymfj777/5559/aNq0KQAvvvgiS5Ys4emnn+bVV1/l33//Zdq0aZw4cYJffvnFYb6zZ8/y9NNPM2TIEAYOHMjChQsZNGgQTZo0oW7dugBERUXRvn17TCYTb775Jh4eHnz55Ze39VLUqVOH999/nwkTJvDSSy/x8MMPA/DQQw/leb0syzz++ONs3bqVIUOG0KhRIzZu3Mhrr73GtWvXmDVrlsP1O3bs4Oeff+bll1/Gy8uLzz77jKeeeorLly9Trlw5AKKjo2nRooX6xzYwMJD169czZMgQkpKSGDt2LKD8PHfs2JHLly8zevRoQkJCWLp0KX/++WeBPpcjR47QpUsXAgMDmTRpEiaTiYkTJxIcHFyg1+fF3r172bVrF3379qVSpUpcvHiR+fPn065dO44fP467uztQ+M/n1KlT9OvXj//7v/9j6NCh1KpVC4D58+dTt25dHn/8cXQ6HWvWrOHll1/GYrEwYsQIAGbPnq3mHL3zzjsAt3zGTZs28dhjj1GhQgXGjBlD+fLlOXHiBL///nuu35H27N+/n6ysLB544IFbvkdxcXF06tQJLy8v1q9f7xAGi4uLu+VrbXh5eeHi4uIwNnnyZF577TUkSaJJkyb873//o0uXLur5a9euERMTo/7M2dO8eXPWrVuXa7xJkyasXr26QDbd1ciCe5aJEyfKgPzSSy+pYyaTSa5UqZIsSZL84YcfquM3b96U3dzc5IEDB6pjs2fPlgH5u+++U8eysrLkli1byp6ennJSUpIsy7J84cIFGZADAwPlhIQE9dq33npLBuSGDRvKRqNRHe/Xr59sMBjkjIwMWZZlOTk5Wfb19ZWHDh3qYH9UVJTs4+PjMD5w4EAZkN9//32Haxs3biw3adJEPY6NjZUBeeLEibnel7Zt28pt27bNNT5w4EC5SpUq6nFxnys/GjVqJAcFBck3btxQxw4fPixrNBp5wIAB6pjt83vhhRccXt+rVy+5XLlyt7yHLMtyWlparrFp06bJkiTJly5dcnjugryn+ZHz/bS9b4sWLcr1LLfjzz//lAF59OjRuc5ZLBZZlmX50KFDMiC/+OKLDufHjx8vA/Kff/6pjlWpUkUG5L/++ksdi4mJkV1cXORXX31VHRs7dqwMyP/++6/DdT4+PjIgX7hwId/n3bt3b67ntZHze2r16tUyIE+ZMsXhuqefflqWJEk+e/asOgbIBoPBYezw4cMyIM+ZM0cdGzJkiFyhQgU5Li7OYc6+ffvKPj4+6veB7ed5xYoV6jWpqaly9erVZUDeunVrLvvt6dmzp+zq6urwvXP8+HFZq9U6fLZ5ff72z2T/M5nX9+ju3btlQP7222/VscJ8PrbPfMOGDbnmzut+Xbt2latWreowVrdu3Tx/R2zdutXhvTKZTHJ4eLhcpUoV+ebNmw7X2r5f8+Prr7+WAfnIkSO5zgHyiBEj5MTERPmBBx6Qa9WqJUdHR+d5XUE2+8/i0qVLcpcuXeT58+fLv/32mzx79my5cuXKskajkX///Xf1Otv3tf3nYOO1116TgVy/66ZOnSoDedp6LyFCdfcBL774orqv1Wpp2rQpsiwzZMgQddzX15datWpx/vx5dWzdunWUL1+efv36qWN6vZ7Ro0eTkpLC9u3bHe7Tu3dvfHx81OMHH3wQgOeffx6dTucwnpWVxbVr1wDlP7aEhAT69etHXFycumm1Wh588EG2bt2a65mGDRvmcPzwww872O5MivpceXH9+nUOHTrEoEGD8Pf3V8cbNGhA586d8/wvLq9nvXHjBklJSbe02/6/8dTUVOLi4njooYeQZZmDBw8W6D4l9Z7mx6pVq5AkiYkTJ+Y6ZwsH2d6jcePGOZx/9dVXAVi7dq3DeEREhOoJAggMDMzze71FixY0b97c4brnnnuumE/kyLp169BqtYwePTqX7bIss379eofxTp06Ua1aNfW4QYMGeHt7q7bLssyqVavo0aMHsiw7/Px07dqVxMREDhw4oN67QoUKPP300+p87u7uBUpMNpvNbNy4kZ49ezqEberUqUPXrl0L/0ZYsf8eNRqN3Lhxg+rVq+Pr66vabbO9MJ9PeHh4nnbZ3y8xMZG4uDjatm3L+fPnSUxMLLT9Bw8e5MKFC4wdOxZfX1+Hc7crv3Hjxg0A/Pz88jyflpZG9+7duXz5Mps3byYoKCjXNZs2bSrQZv9eVK5cmY0bNzJs2DB69OjBmDFjOHjwIIGBgerPEKCG9HN6qgA1zytn2N/2LAX1hN2tiFDdfYD9LzoAHx8fXF1dCQgIyDVu+2EGJf+gRo0aaDSO+rpOnTrq+dvdByA0NDTP8Zs3bwJw5swZADp06JCn/d7e3g7HtrwXe/z8/NT5nE1RnysvbO+ZLXRgT506ddi4cSOpqal4eHjke3/bL6ebN2/mem/suXz5MhMmTOC3337LZVPOPxJ3+j2Nj493yK1wc3PDx8eHc+fOERIS4iAqc3Lp0iU0Gg3Vq1d3GC9fvjy+vr63/b6E3M926dIlVRDbk9fnVBwuXbpESEhIrlVHBf2ZAkfbY2NjSUhI4Msvv+TLL7/M854xMTHq3NWrV8/1B70gzxgbG0t6ejo1atTIda5WrVp5Cv6CkJ6ezrRp01i0aBHXrl1zyPOy/x4t7OcTHh6e5/jOnTuZOHEiu3fvzpUnmJiY6PAPUkE4d+4cAPXq1SvU6+yRc+S22ViyZAlarRaz2czly5epVKlSrmvyyo8qCv7+/gwePJgPP/yQq1evqnmbQJ45cBkZGQC5QqW2Z7kba7YVBiGc7gO0Wm2BxiD/H+Ki3qcg97IleC5dupTy5cvnus7eq3Or+QqKJEl5Pmd+idBFfS5nUZT7mM1mOnfuTHx8PG+88Qa1a9fGw8ODa9euMWjQoFxJtcV9TwvLk08+6eCxHDhwYKHr5hT0l/Od+pxKgoL+7Dz//PMMHDgwz2uLmttXVPL7XPL6+Ro1ahSLFi1i7NixtGzZEh8fHyRJom/fvsUq45BX7tO5c+fo2LEjtWvXZubMmYSGhmIwGFi3bh2zZs2642UjbDlqN2/ezFMUBQcHs2HDBp5++mleeOEFDh06lGtFX1RUVIHu5ePjc9t8Pds/gvHx8VSqVIkKFSoAipc8J9evX8ff3z+XN8om6HP+U36vIYSTIF+qVKnCf//9h8VicfA6nTx5Uj3vDGyhiKCgIKf9B3WrP6p+fn55hqBy/rdfEtjes1OnTuU6d/LkSQICAhy8TUXlyJEjnD59miVLljBgwAB1fNOmTcWe2xl88sknDh6fkJAQQPle2LhxI/Hx8fl6napUqYLFYuHMmTOqpwaUJOmEhIQifV9WqVJF9Xzak9fnlJPC/HddpUoVNm/eTHJysoPXqag/U4GBgXh5eWE2m2/7s1OlShWOHj2KLMsONhfkGQMDA3FzcyvQe2TziCYkJDiM5/Xz9dNPPzFw4ECHFb0ZGRm5Xlucz8fGmjVryMzM5LfffnPw5OWVClDQz9T2u+vo0aOF/t1Vu3ZtAC5cuED9+vVznX/yySdp0KABX3/9Ne3atePdd99lxowZDtfYxM3tWLRoUa6aVDmx/U60eZ4rVqxIYGCgw2pWG3v27MlVs8z2LAEBAbm81/caIsdJkC+PPvooUVFRLF++XB0zmUzMmTMHT09P2rZt65T7dO3aFW9vb6ZOnYrRaMx1vihl/G2rcXL+Agbll93Jkycd5j18+DA7d+4s9H0KS4UKFWjUqBFLlixxsO3o0aP88ccfuQrSFRWbp8LeqyLL8m2XSN8pmjRpQqdOndQtIiICUGrByLKsFi+1x/Ystvdo9uzZDudnzpwJQPfu3Qttz6OPPso///zDnj171LHY2Fi+//77277WJnTz+l7L6z5ms5m5c+c6jM+aNQtJkujWrVuh7NZqtTz11FOsWrWKo0eP5jpv/z3+6KOPEhkZyU8//aSOpaWl5Rviy3mfrl27snr1ai5fvqyOnzhxgo0bNzpc6+3tTUBAAH/99ZfD+Oeff57nvDk9f3PmzMnlnSrO52N/LyBXOHDRokW5rvXw8CjQ5/nAAw8QHh7O7Nmzc11/O49mkyZNMBgMeQoTe9q0acPw4cOZNWsW//zzj8O5ouQ45fX79Nq1ayxcuJAGDRo4iLGnnnqK33//nStXrqhjW7Zs4fTp0/Tu3TvXPPv3789VRPNeRHicBPny0ksv8cUXXzBo0CD2799PWFgYP/30Ezt37mT27NlOqw7r7e3N/Pnz6d+/Pw888AB9+/YlMDCQy5cvs3btWlq1apXrD83tcHNzIyIiguXLl1OzZk38/f2pV68e9erV44UXXmDmzJl07dqVIUOGEBMTw4IFC6hbt+5tE66dwccff0y3bt1o2bIlQ4YMUcsR+Pj4ONS4KQ61a9emWrVqjB8/nmvXruHt7c2qVatKLGfJWbRv357+/fvz2WefcebMGR555BEsFgt///037du3Z+TIkTRs2JCBAwfy5ZdfkpCQQNu2bdmzZw9LliyhZ8+etG/fvtD3ff3111m6dCmPPPIIY8aMUZe727yut6JatWr4+vqyYMECvLy88PDw4MEHH8wzz6ZHjx60b9+ed955h4sXL9KwYUP++OMPfv31V8aOHeuQCF5QPvzwQ7Zu3cqDDz7I0KFDiYiIID4+ngMHDrB582a1ttjQoUOZO3cuAwYMYP/+/VSoUIGlS5eq/2TcjsmTJ7NhwwYefvhhXn75ZfWfqLp16+Z6j1588UU+/PBDXnzxRZo2bcpff/3F6dOnc8352GOPsXTpUnx8fIiIiGD37t1s3rxZDWPZKM7nY6NLly4YDAZ69OjB//3f/5GSksJXX31FUFBQrnBUkyZNmD9/PlOmTKF69eoEBQXlmYOp0WiYP38+PXr0oFGjRgwePJgKFSpw8uRJjh07lktU2uPq6kqXLl3YvHkz77///i1t//DDD/n9998ZPHgwBw8eVEN2RfHQv/7662rYMiQkhIsXL/LFF1+Qmpqa6x+rt99+m5UrV9K+fXvGjBlDSkoKH3/8MfXr12fw4MEO18bExPDff/+pZR3uae7gCj7BHca2BDw2NtZhfODAgbKHh0eu69u2bSvXrVvXYSw6OloePHiwHBAQIBsMBrl+/fq5lhnblh9//PHHDuO2pbsrV650GF+0aJEMyHv37s11fdeuXWUfHx/Z1dVVrlatmjxo0CB53759t7U9r+Xuu3btkps0aSIbDIZcy6C/++47uWrVqrLBYJAbNWokb9y4Md9yBMV9rrzYvHmz3KpVK9nNzU329vaWe/ToIR8/fjzPZ8r5+dnuY78EOy+OHz8ud+rUSfb09JQDAgLkoUOHqsvZ7T/DwryneeHMcgSyrCzx/vjjj+XatWvLBoNBDgwMlLt16ybv379fvcZoNMqTJ0+Ww8PDZb1eL4eGhspvvfVWruXRVapUkbt3735bm2VZlv/77z+5bdu2squrq1yxYkX5gw8+kL/55pvbliOQZVn+9ddf5YiICFmn0zk8e87vKVlWym+88sorckhIiKzX6+UaNWrIH3/8ca7l61iXpOekSpUqDmVDZFn5OR0xYoQcGhoq6/V6uXz58nLHjh3lL7/80uG6S5cuyY8//rjs7u4uBwQEyGPGjJE3bNhQoHIEsizL27dvV3+mqlatKi9YsCDPzzYtLU0eMmSI7OPjI3t5ecnPPPOMHBMTk+vn8ObNm+rvF09PT7lr167yyZMn83zGgn4++X3msizLv/32m9ygQQPZ1dVVDgsLk6dPny4vXLgw1xxRUVFy9+7dZS8vLxlQP++c5Qhs7NixQ+7cubPs5eUle3h4yA0aNHAoGZEfP//8syxJknz58mWH8bw++/Xr18uA/Nprr9123lvxww8/yG3atJEDAwNlnU4nBwQEyL169XL4+bLn6NGjcpcuXWR3d3fZ19dXfu655+SoqKhc182fP192d3dXy9Tcy0iyfBdkSAoEAoGgTDJp0iQmT558VyTblzXMZjMRERE888wzfPDBB6VtTrFo3Lgx7dq1y1XE9V5E5DgJBAKBQFAKaLVa3n//febNm0dKSkppm1NkNmzYwJkzZ3jrrbdK25Q7ghBOAoFAIBCUEn369CE+Ph5PT8/SNqXIPPLII6SkpORZpPNeRAgngUAgEAgEggIicpwEAoFAIBAICojwOAkEAoFAIBAUECGcBAKBQCAQCAqIKIB5GywWC5GRkXh5ed3zjQsFAoFAILgfkWWZ5ORkQkJCcjW2z4kQTrchMjJSbX4oEAgEAoHg3uXKlSt5Nl22Rwin22BrK3LlyhW8vb1L2RqBQCAQCATOJikpidDQ0AK1EhPCKR/mzZvHvHnz1GaT3t7eQjgJBAKBQHAPU5CUHFGO4DYkJSXh4+NDYmKiEE4CgUAgENyDFOZvvVhVJxAIBAKBQFBAhHASCAQCgUAgKCAix0kgEAgEhcJsNmM0GkvbDIGgwOj1erRarVPmEsJJIBAIBAVClmWioqJISEgobVMEgkLj6+tL+fLli12TUQgngUAgEBQIm2gKCgrC3d1dFAUW3BXIskxaWhoxMTEAVKhQoVjzCeEkEAgEgttiNptV0VSuXLnSNkcgKBRubm4AxMTEEBQUVKywnUgOFwgEAsFtseU0ubu7l7IlAkHRsH3vFjc/TwgngUAgEBQYEZ4T3K0463tXCKdSJDHBiDkpobTNEAgEAoFAUECEcColZLOJ3ePe5sCwwaz7LRqTubQtEggEgvuTsLAwZs+eXeDrFy9ejK+vb4nZIyjbCOFUSsRdukFwymFCLOcJ+PYFnn7uKot+hrT00rZMIBAIBLeiT58+nD59utjzyLLMV199RcuWLfH29sbT05O6desyZswYzp49q143adIkJElCkiR0Oh0BAQG0adOG2bNnk5mZWWw7BIVDCKdSIrBqMJWnLSLFvRIVdVeZaB7Mos8v0LIvzFwENxJK20KBQCAQ5IWbmxtBQUHFmkOWZZ599llGjx7No48+yh9//MHx48f55ptvcHV1ZcqUKQ7X161bl+vXr3P58mW2bt1K7969mTZtGg899BDJycnFskVQOIRwKkXKVatI9RmL0IZUJUgbw1cB/XlJfp89y/6mTZ9M3pkFF6+VtpUCgUBw99KuXTtGjhzJyJEj8fHxISAggPfee49b9befOXMm9evXx8PDg9DQUF5++WVSUlLU8zlDdZMmTaJRo0YsXbqUsLAwfHx86Nu37y0FzfLly1m2bBnLly/nvffeo0WLFlSuXJkWLVowffp0Fi1a5HC9TqejfPnyhISEUL9+fUaNGsX27ds5evQo06dPL/obJCg094Vw+v3336lVqxY1atTg66+/Lm1zHND6BxE4aSH6qnXxJJme7qv4xG8ky326smvtGdoPgJcnweGTpW2pQCAQZCPLSmpBaWy30Dx5smTJEnQ6HXv27OHTTz9l5syZt/xboNFo+Oyzzzh27BhLlizhzz//5PXXX7/lPc6dO8fq1av5/fff+f3339m+fTsffvhhvtf/+OOP1KpVi8cffzzP8wVZAVa7dm26devGzz//fNtrBc7jni+AaTKZGDduHFu3bsXHx4cmTZrQq1evMlXATevtR+D7S8g8toeMfdtI37sVv4RY3g39lBcuzmXtdli7HVo0gmF9oV1zECuCBQJBaZKeAXUeLZ17n1gH7m4Fvz40NJRZs2YhSRK1atXiyJEjzJo1i6FDh+Z5/dixY9X9sLAwpkyZwrBhw/j888/zvYfFYmHx4sV4eXkB0L9/f7Zs2cL//ve/PK8/ffo0tWrVynVfm6Dz9fXl6tWrt3222rVr88cff9z2OoHzuOc9Tnv27KFu3bpUrFgRT09PunXrVia/ySSdHteGrfAd8g6BE78BjZba6X+zadIRnuoCOi38cwgGvQldh8CqjZAlemwKBALBbWnRooWDB6dly5acOXMGsznv5cybN2+mY8eOVKxYES8vL/r378+NGzdIS0vL9x5hYWGqaAKlrYetxUdBeeeddzh06BATJkxwCA3eClmWRW2tO0yZ9zj99ddffPzxx+zfv5/r16/zyy+/0LNnT4dr5s2bx8cff0xUVBQNGzZkzpw5NG/eHIDIyEgqVqyoXluxYkWuXSvbiUO6ClVwb92dtL9+w3/3Ama+NY/xQ2DhKvhhDZy6AOM+hI+/gSFPQ7/HwFMU8xUIBHcQN1fF81Na9y4pLl68yGOPPcbw4cP53//+h7+/Pzt27GDIkCFkZWXlWzldr9c7HEuShMViyfc+NWrU4NSpUw5jgYGBBAYGFirx/MSJE4SHhxf4ekHxKfMep9TUVBo2bMi8efPyPL98+XLGjRvHxIkTOXDgAA0bNqRr166FVvplDa8nXwKNlsxDO8g8fZiQIHh3OOxeAW8MhUB/uB4LU+ZDy2fgo68gJr60rRYIBPcLkqSEy0pjK6yD5d9//3U4/ueff6hRo0ae/cr279+PxWLhk08+oUWLFtSsWZPIyMjivFV50q9fP06dOsWvv/5a5DlOnjzJhg0beOqpp5xomeB2lHnh1K1bN6ZMmUKvXr3yPD9z5kyGDh3K4MGDiYiIYMGCBbi7u7Nw4UIAQkJCHDxM165dIyQk5I7YXhx05UNxb9MDgOSf5mPJSMN46RSul/Yw/BkjO3+E6eOhWigkpcK8H6BVX3hjBpy7XMrGCwQCQRni8uXLjBs3jlOnTvHjjz8yZ84cxowZk+e11atXx2g0MmfOHM6fP8/SpUtZsGCB023q27cvTz/9NH379uX999/n33//5eLFi2zfvp3ly5fnEnUmk4moqCgiIyM5cuQIc+bMoW3btjRq1IjXXnvN6fYJ8qfMC6dbkZWVxf79++nUqZM6ptFo6NSpE7t37wagefPmHD16lGvXrpGSksL69evp2rVrvnNmZmaSlJTksJUWXk8OBa2OzP92c31QS2LeeIa4D4aSsmYxLgbo2x02L4avPoAmdZWcp2VroeMgeOk92He01EwXCASCMsOAAQNIT0+nefPmjBgxgjFjxvDSSy/leW3Dhg2ZOXMm06dPp169enz//fdMmzbN6TZJksTy5cuZPXs269ato2PHjtSqVYsXXniB0NBQduzY4XD9sWPHqFChApUrV6Zdu3asWLGCt956i7///htPT0+n2yfIH0m+VTGLMoYkSQ45Trb8pV27dtGyZUv1utdff53t27er7tnffvuN8ePHY7FYeP311/P9gQGlHsfkyZNzjScmJuLt7e3cByoACYumkbpxGQCSiytyZga6yjUI/uinXNfuPQJfLoc/dmaPNa2nrMTr2BI0d7VMFggEpUlGRgYXLlwgPDwcV9cSTDJyMu3ataNRo0aFaqkiuDe51fdwUlISPj4+BfpbX+aTw53B448/nm+tjJy89dZbjBs3Tj1OSkoiNDS0pEy7LT4DXsOj49No/AKRJInrL7XHdPkMppir6IIqOVzbrL6ynbkEX62AXzYpXqcX34VqleGlZ6BXZ3AxlNLDCAQCgUBwl3NX+yACAgLQarVER0c7jEdHR1O+fPkizeni4oK3t7fDVppIWh36yjXQevmi8fTBUKsxABn7t+f7mhpV4KPXYMeP8PKz4O2h5D29MQNaPwvzf4TEgq10FQgEAoFAYMddLZwMBgNNmjRhy5Yt6pjFYmHLli0OobuiMG/ePCIiImjWrFlxzXQqrk3bArcWTjaCyykr8HYth3eGQ/kAiLkBH34JD/WB/y1QVuYJBALBvcq2bdtEmE7gVMq8cEpJSeHQoUMcOnQIgAsXLnDo0CEuX1aWjo0bN46vvvqKJUuWcOLECYYPH05qaiqDBw8u1n1HjBjB8ePH2bt3b3Efwam4PaAIp8wT+7GkKX2QZIuF9P3bMCfE5fkaLw8lTPf3DzDzTagZBilpSj5U637w6odKbSiBQCAQCAS3pswLp3379tG4cWMaN1ZCVOPGjaNx48ZMmDABgD59+jBjxgwmTJhAo0aNOHToEBs2bCA4OLg0zS4xdBWqoAsJB7OJjENKFnjyys+J/3gMCd9MueVrDXp4qiv8sRAWTYMWDcFkhp82QpcXYOCbsPNA4ftACQQCgUBwv1Dmk8PbtWt3yy7WgNr5+n7BtUlbUiIvkLF/O1qfciSvVnobZR7fX6Dy+5IEHVoo26ETsGAZbPgbtv2rbHVrKB6q7u1AX+a/QwQCgUAguHOUeY9TaVFWc5wAXJu2AyDj0N/Ez3tbdRHJqUmYo68Uaq5GdWDBZNi+FAb0BFcXOHYGxvwP2jynrM5LTnXyAwgEAoFAcJcihFM+lNUcJwBDjQZovPyQU5OxxMegq1AFfRWly3bW2eyql7LFQuKyOaT9/XuuOWRZRrbro1SlInwwBnZ+eIQF7ZYQ6GchMsba0qUPTJ1vIfKsyCQXCAQCwf2NEE53IZJGi2vjh5UDrQ6/0dMx1HkAgKxz2cIp89geUlZ/TcLXHyBbsruAy7JM3ISBRI97Ajkrw2Fu83cTaXBiJlte/ZMPxyv1n5JTIe33RcjvduLzNzZz/GzJP6NAIBAIBGURIZzuUjwe6Yc2qBK+Q97BEF4HQ7V6ABjtPE6Z1uRxOTMD0/VL6rg57jpZZw5jjrpM1pkj6rglNQnT1XPKa07toV932LwIFk6Fnn5KG3SPUxvpNhSefw227xGJ5AKBQHC/kZKSwpAhQwgKCsLPz48ePXpw6dKl27/wHkEIp3woyzlOAIaqEZT/bC0eHZ4EQG8VTlkXTyKbjABkHM7uvWK8cNJu/7i6n3Xmv+z9c8fU/czjSohSo4F2dW5QwaS4mVp4HkCrkfl7Hwx4A0YMPszl51uSuG6Zsx9RIBAIBHkgSRKrV68utftPnTqVNWvW8MMPP/Dvv/8iyzLDhg0r1pzbtm3jgQcewMXFherVq7N48eLbvmbFihU0atQId3d3qlSpwscff1wsGwqKEE75UJZznPJCV74ykocXGDMxXjmLKS5K9R4BGC9mC6es8yey988czt4/m+19Ml09jznxBpAtogA8TXFsn32ZIU+Duys0vrkSrSmN/Ut+FxXJBQKB4D5gy5YtDBo0iE6dOlGzZk1efvlldu7cefsX5sOFCxfo3r077du359ChQ4wdO5YXX3yRjRs35vua9evX89xzzzFs2DCOHj3K559/zqxZs5g7d26R7SgoQjjdI0gaDYaqdQEwnjtK5n+7HM7bCycHj9Pp/9RyD0Y74QSQdWI/oORK2eMfu58JI2D3MgsdfZQO3mGcYsaXRlo+A+/Pg6tRTnowgUAgKAbt2rVj1KhRjB07Fj8/P4KDg/nqq6/UQsleXl5Ur16d9evXO7xu+/btNG/eHBcXFypUqMCbb76JyWQq9rxHjx6lW7dueHp6EhwcTP/+/YmLi3OYd/To0bz++uv4+/tTvnx5Jk2apJ4PCwsDoFevXkiSpB4PGjSInj17Otxr7NixtGvXrtg25yQmJoaQkBD1OCoqCp2u6LVrFixYQHh4OJ988gl16tRh5MiRPP3008yaNSvf1yxdupSePXsybNgwqlatSvfu3XnrrbeYPn36bUsYFRchnO4hDNXrA8rKuozDinBybaJUGjdeOqWspJNljHYeJ0vyTczRV5BlWfU46aspAizz+D7l6zHF46QPU1buZZ48AIB77DFcsm4C4CJl0b7KWVLT4ZuflFIGo963cORUiT6yQCAoJWRZxpKRVipbYf8wLlmyhICAAPbs2cOoUaMYPnw4vXv35qGHHuLAgQN06dKF/v37k5aWBsC1a9d49NFHadasGYcPH2b+/Pl88803TJkypVjzJiQk0KFDBxo3bsy+ffvYsGED0dHRPPPMM7nm9fDw4N9//+Wjjz7i/fffZ9OmTQBqFGTRokVcv3690FGRwtqcF7b3X5ZlDhw4wNSpU3nkkUccrqlbty6enp75bt26dVOv3b17N506dXJ4fdeuXdm9e3e+NmRmZuLq6uow5ubmxtWrV0s830qUN7yHsAmerNOH1fYrnt0HkHFoJ5bkBMw3ogEZS/JN0OrQh1bDePEUWacPY9BosCQp456P9ufmnDfJPL4XU9x1zFGXQaPFq+dQ4mePVz1RGQd3ONx/9vPH2O9dhy9XwOmDUQw/+gw79rdlStUPeOkZaP+gkjMlEAjufuTMdK4PKl5P0KJSYfFuJFf3Al/fsGFD3n33XQDeeustPvzwQwICAhg6dCgAEyZMYP78+fz333+0aNGCzz//nNDQUObOnYskSdSuXZvIyEjeeOMNJkyYgMb6i6yw886dO5fGjRszdepU1baFCxcSGhrK6dOnqVmzJgANGjRg4sSJANSoUYO5c+eyZcsWOnfuTGBgIAC+vr5FamZfWJtvRd++fVmxYgUGg4GlS5c6nFu3bh1GozHf17q5uan7UVFRubp9BAcHk5SURHp6usO1Nrp27corr7zCoEGDaN++PWfPnuWTTz4B4Pr166onriQQwikf5s2bx7x58zCbzbe/uIxgW1lnilQaz0mePhhqN0ZXMRzT5TNKuM5alkBfqRqGiGaKcDrzH1jdrPoqtXBpoPwyNF09T/quDcp4tbq4NHwINFrMsZGY4q6TcfBvALQBFTDHXcd4/jhtX4K2zeHskj9wW59IG5et/O+QzD+HJKpXgaG9oWdncDXc0bdGIBDcxzRo0EDd12q1lCtXjvr166tjtj/aMTExAJw4cYKWLVs6dGFo1aoVKSkpXL16lcqVKxdp3sOHD7N161Y8PT1z2Xju3DkH4WRPhQoV1DmKS2FtvhUzZsxg2LBhLF++nPbt27N582Zat24NQJUqVZxib34MHTqUc+fO8dhjj2E0GvH29mbMmDFMmjRJFbYlhRBO+TBixAhGjBhBUlISPj4+pW1OgdD6BaL1D8YcHw2Aa/0WSBot+rDa2cLJrMTo9eF1cKnZkNR13ynCSa8oGUP1emi9fNFVronp8mlS1iwBwKVuczRuHujDa2M8d4z0fzdhPK+swvN8fDCJC6diPJ9dCsH32i4yAS9NMmN63mDhpgDOXoI3ZsCMb2Dgk/B8D/C7O95agUCQA8nFjQqL8w+llPS9C4Ner3d8vSQ5jNkEksWuKHBJzJuSkkKPHj2YPn16rrkqVKhwy3lvZ5tGo8kVwszL4+PM9yI0NJTQ0FDV4/P111+rwqlu3bq3DJk9/PDDai5V+fLliY6OdjgfHR2Nt7d3nt4mm53Tp09n6tSpREVFERgYyJYtWwCoWrXqbW0vDkI43WPoq9fDvEf5BnRp2AoAQ1ht0v9ag9GuVIG+agSGGg0BMF4+oxZksuVJuUQ0xXT5tBLWQxFOAC61m2A8d4yU1d+o87g+0FYRTlfOqQU1M08cUG0a0fkiQ18MYNlaWLgKImMU8TTve3imGwx5SqlcLhAI7h4kSSpUuOxuok6dOqxatcqh9+fOnTvx8vKiUqVKRZ73gQceYNWqVYSFhRUrmVqv1+eKhgQGBnL06FGHsUOHDuUSSs7CPlHeZpN9zlFhQnUtW7Zk3bp1Duc3bdpEy5a3DwVrtVoqVlT+gPz444+0bNlSDWeWFCLj5B7DFq4DcG34EAD6sNqAsrLOtqLOEF4HbblgtP7BYDFjvKRkcevthJOK3oBLLUVkGeo0AcCSnKDco/HDaMsFo/HxB7MJ46XTZJ7YD8ZM9eWmyAt4ecDQZ+Cv7+HTdyCiOqRnwJJfoN0AGDYRDmQv9hMIBIJS4+WXX+bKlSuMGjWKkydP8uuvvzJx4kTGjRtXrDDQiBEjiI+Pp1+/fuzdu5dz586xceNGBg8eXKi0kLCwMLZs2UJUVBQ3byr/3Hbo0IF9+/bx7bffcubMGSZOnJhLSDmTjz/+mF9++YVLly6xdOlSNm3a5JDwXaVKFapXr57vZhM7AMOGDeP8+fO8/vrrnDx5ks8//5wVK1bwyiuvqNfMnTuXjh07qsdxcXEsWLCAkydPcujQIcaMGcPKlSuZPXt2iT2zDSGc7jFc6imeIUONhmj9FNVtWw1njruOJTEeNFr0VZRYuqFmQ/W1koc3ugpKXNomkGzXSAblPwmX2o3BLu7v2vhhxdVrLYWQde6YuqLPhvHaRXVfr4P2cXNY3mUBP3wC7R4EiwXW/wW9RsBTo2DD33AXpZYJBIJ7jIoVK7Ju3Tr27NlDw4YNGTZsGEOGDFGTqotKSEgIO3fuxGw206VLF+rXr8/YsWPx9fUtlCD75JNP2LRpE6GhoTRu3BhQkqXfe+89Xn/9dZo1a0ZycjIDBgwolr23ol69eowfP57atWvzwQcfMHPmTJ544okizRUeHs7atWvZtGkTDRs25JNPPuHrr7+ma9eu6jVxcXGcO3fO4XVLliyhadOmtGrVimPHjrFt2zaaN29erOcqCJJc0gUP7nJsOU6JiYl4e3uXtjkFIvP0YXRBFdH6BqhjUaO7Y465CoCuck2CP1oJQMrapSQunQGAS8OHCHhrvvqa6Nd7Y7p8Gq9nRuD95Eu5xjXefpRf8CeSRkPSyvkkr1qAW5seGM8fw3T1PC71W5B55B9cGrUm4M15AJhirhI9ujsAFRbtQuPmwakL8NUKWL0ZjFbvb+UQeOEpJZTnUbh0BoFAUAJkZGRw4cIFwsPDcy0DF9xfhIWFMXbsWMaOHVvaphSKW30PF+ZvvfA45UNZb7lyK1xqNnQQTZAdrgMwhGfv62tkr7Cw5TfZ8H5mBC6NH8ajw1OO80co3ijXRg8jWf9L0leNACDz8E5MV8+DpMGjSx8ge5UfKDWmbJjjlVUbtcJhxhuwcxmMfB58veFyJEyaAy2egWlfwPXYQr4JAoFAIBCUAEXOTrt8+TKXLl0iLS2NwMBA6tati4uLizNtK1XuxlV1t8IQXpuMPZsB0IdH2I3XAZ0eTMZcwsmtaTvcmrbLNZfXky8hGVzx7PZc9jzWGlKWxHjlHtXrqcnn5thI5KxMJIMLxnPZwslyMwYqhqvHweXgtSEw4llY9YdSSPPCVViwDL5eCY+1hxd7Q/2axXwzBAKBQCAoIoUSThcvXmT+/PksW7aMq1evOix9NBgMPPzww7z00ks89dRTJV5HQVA47D1ONu8QgKQ34PXkSxjPH1dXzt0Orbc/Ps+OdRzzDXAshdDgITQ+/kjuXshpyZiiLqOvXMOhkbD5Zt5uJHc36P8EPNcD/vwHvloJ/xxSQnmrN0OLhoqA6thSFNQUCASCO83FixdL24RSpcB/dkaPHk3Dhg25cOECU6ZM4fjx4yQmJpKVlUVUVBTr1q2jdevWTJgwgQYNGtw1zXHvF/ThEaDVIRlc1cRwG95PvkS58bORDMXzGOqrZQsyl4YPIUkSupAwAEyRF5HNJoc+ebZQnQ3ZZMSSkqQeazTQ6SFYPgt+XwA9O4FOC/8chhffhQ4D4dvVkJZeLLMFAoFAICgwBfY4eXh4cP78ecqVK5frXFBQEB06dKBDhw5MnDiRDRs2cOXKlbsyP+heRetbjnLjP0UyuKApZPG4gmKoWo+MvVuRPLzU0J0uJAzj2SMYIy+gu1oFOTNDvT6nx+nGx6PJOnGA4M/W5srRql9LKWPw5ktKCYPv1yhhvPc+hU8WKd6pgT0h2PFlAoFAIBA4lQJ7nKZNm5anaMqLRx55hCeffLLIRglKBtfGrXGpW3Ji1rV5B9C74NHhKSSttYVLiJLDZIq8SNY5x5oi9sJJlmWyThxAzsrAdO0C+VEhUBFP/6yAyaOV1XcJSUoxzVb9YNw0OHa2BB5OIBAAha+uLRCUFZz1vSsqhwuchr5iVUK+/ddhTBei1IUyRV5Ua0Fpy1fGHHUZi12ozpIYr1Ydt6Ql3/ZeHm4wqBf0fxw27VKSx/ceUZLKV/0BDzVW8qBEY2GBwDkYDAY0Gg2RkZEEBgZiMBgcerkJBGUVWZbJysoiNjYWjUaDwVC8ZqlFEk6NGzfO8wdGkiRcXV2pXr262rH4buVubPJbFsj5faGzeZyuX1T75Lk160DKmsUOHidz7DV135KWkmte2WxSvVj2aLXwyMPKduiEshJv7TbYdVDZqlWGF5+GJ7uA672z6FMguONoNBrCw8O5fv06kZGRpW2OQFBo3N3dqVy5crEXrxWpAOZbb73F/PnzqV+/vlqlc+/evfz3338MGjSI48ePs2XLFn7++eciVxItK9yNBTDLErIxi8gBD4Kc7SINmLyEuIkDQasj5Lt9SJJE2q4N3PzsDQB8Br7uUOogec0Skn+aT8DEhRjsVgTmx7VoWPQzLFsLyanKmL8PPP849O8JQf5OfUSB4L5ClmVMJpP4p1JwV6HVatHpdPl6SQvzt75IHqe4uDheffVV3nvvPYfxKVOmcOnSJf744w8mTpzIBx98cNcLJ0HxkPQGtMGVMEddBkDjF6gmjmM2YUlOQOvthzk2+z/YnB6nzP92IWemk3XqoINwkmUZU+QFdBWqIGm06njFYHh3OIwZACvWw8Kf4Go0fLZUqQn1REelb16tcAQCQSGRJAm9Xl9izWMFgrJOkfxVK1asoF+/frnG+/bty4oVKwDo168fp06dKp51gnsCW/87UJoQSzo9Gm8/wFoEEzDZherkHMLJlvOUU1ClbVtNzKu9SFmzxGFcNptIXvstrnGnGPI0bP8ePp8EjSMgywgrN0CXF6D/a7B9D4imQwKBQCAoKEUSTq6uruzatSvX+K5du9T+LxaLRfQzEgDZK+tAEU4AWr8gAMzxSp6To8fJMTnckqoc5xRUpqtKw0fjhRMO4xmHdpK09BMSFk4DlNpP3dvC6nnw27un6NfyGhoN/LUPBryhiKhlayEjq9iPKhAIBIJ7nCKF6kaNGsWwYcPYv3+/Wqtp7969fP3117z99tsAbNy4kUaNGjnNUMHdi60IJiitWAA0/kFw6RRmq8fpVqE62SqcLOk5PVHKsW0OG7ZmxsarZ5FlWY1pmxNvELDoOcb6BvDy0vUs/kVi2Vo4fRHemAEff6NULH/+cQjwK+ZDCwQCgeCepEjC6d133yU8PJy5c+eydOlSAGrVqsVXX33Fs88+C8CwYcMYPny48ywV3LXo7D1O1hwlrV8goIge2WLBZCecZDuPkyzLWFKTrOM5hJNVUOUspGmrSC6nJqs5VADGS6fBZMQcd52KrrFMGBHE2IHw41pY/DNExsCsxfD599CrCwx5CmqKPCiBQCAQ2FHkOk7PPfcczz33XL7n3dxKpjq14O7DUC0CfdW66ELC0HgoqxWyhVMslsQbYMyOk9l7nOTMdLCYreOOITw5PUWdw8GzZFcfynT9kiqcTJHZhTWNV86i9Q/C2xP+rw88W3EDx/dGM+XUQP47pYTulq2FNk3hhaehbTNRD0ogEAgExSyAuX//fk6cUPJL6tatS+PGjZ1ilODeQjK4EjT1B4cxm3Cy3Ix18DZBtiACHHrX5Qzh2TxOGLOwpCSi9fIFcginqMu41Gqk7NtVJDdePYtrw4eU+5mMJH3xHpWMWaz6qBWHEqrzzU/wx04lD+qvfUo9qBeegqe6gJtI3RMIBIL7liIJp5iYGPr27cu2bdvw9fUFICEhgfbt27Ns2TICAwOdaWOpIApgliwaNTk8Rs1vkty9kNOSHT1O9mG7fFbbgSLAVOF009HjZMNo53EyXcnuy2K8clb1eJmunqX5Q9Vp3gAuX1dCeMvXwbnL8M4s+OhrpS/egJ5K+xeBQCAQ3F8UKfgwatQokpOTOXbsGPHx8cTHx3P06FGSkpIYPXq0s20sFUaMGMHx48fZu3dvaZtyT6L1twqnm7GYY5RSBPoqNQE7T1LO/RzJ4faiyiaWZFl2aOViL5xMkRfVfeOVc9n7549nX3PtvLpfuQJMGKH0xZs4QumLl5gMn/8ArfvBqA+UauUCgUAguH8oknDasGEDn3/+OXXq1FHHIiIimDdvHuvXr3eacYJ7FzVUl3gDU7RSHFNfpZZy0piJbDIq51OzQ3W5ksbtPFD2CeG2nneQLZwsaclY7JLITVfPIVsbPmZdyBZOxqvZwsn2OpfYk7zwNGz7Fr78AFo0BJMZfvsTnngZnhyptHkxCeekQCAQ3PMUSThZLJY8q8bq9XrROVtQIDTe/qDRgiyTdeYIAPrKNdTztjCcg1jKzFAFFcZMsO2TvbLOfDPa4T7mqMvKqj2rt0nj4w86PXJmOuY4JUTo6HE65/D6m19MJvbNPmSePoxWC11bw/LZ8PsXSr6TXgf7j8HLk6HNc/DFckjM3WpPIBAIBPcIRRJOHTp0YMyYMQ6NHq9du8Yrr7xCx44dnWac4N5F0mjQ+gYA2eExXfnKSK7uQHY+k31yOIAlXWk+Zx/CA1Rvks3zpKtYFbQ65KwMzPExqnDSVaqGvmJVQAnXySYjxstn1HlM1y+r4ky2mMn8Tyn0ajzzn8P96teEmW/BzmUwur/SC+9aNExdAC16w4TP4MLVor47AoFAICirFEk4zZ07l6SkJMLCwqhWrRrVqlUjPDycpKQk5syZ42wbBfcoGj/H7GptYAiSuyeQvYIuVwkCddzRrWMTTLav2oAK6IIqAkq4zmhdUacPCUdXqZoyfvWskhhuMiJ5eCuizWzCFK0oHlPkRWSrUDNFX8nzGYLLwasvwO7lMH280v8uLQOW/ALtB8CQd2DnAdHWRSAQCO4VirSqLjQ0lAMHDrB582ZOnjwJQJ06dejUqZNTjRPc22j9gzDaImMaLVr/IDTunljiY9QQnX2OE+QdwgO7UJ1NOPkHIWm1mK5fwhx1CVOk1atVMRw5I510lNV0Gk9fQCnMaUlLxnjuGKZr59BXDCfr7BF1fpuYyg9XF+jbHfo8qgilb36CP/+BzbuUrU5VpR7U4x3B1VDot0ogEAgEZYQi13GSJInOnTvTuXNnZ9ojuI/Q2nmctAHlkbQ6NO5KgUxVIKXm53GyjksakC3ZrVtuZgsnjdV7Zbx+CdO1i4BSxVzOygSUBHFbaFAfXgdLwg2M545hvHoet+ZgPHtUvW9+HqecSBK0bqJs5y7Dop/hp41w4jy89hFM/0pp6fL84xDoX7D3SSAQCARlhwILp88++6zAk94rJQkEJYut0S8oYTogd6gup8cp3TauCCdd+VBM1y9hSbiBbDappQi0fkEgKwsVTFfPqcJHFxIOZhOAGr4DxeNkspZFMFlX1tl7nMyxkcgWM5JGqxzfjCXtrzW4t++lVibPSbXKMGUsjLc2EV78C1yPhdlLlJIGT3RUimpGVC/4eyYQCASC0qXAwmnWrFkFuk6SJCGcBAXCPsdJZxVONi9RLs+SFdu47au2fGUljGYxY0mMdwzVGVwAyDxxAMwmJBc3tX6U5OKKnJmB8eIpAPThEaBXrjddO48lMz07aVySwGzCHBel5k0lr/6a1I3LkE1GvJ/6v1s+p683DOsHQ3rDhr+UMN7BE7Byg7K1bKz0xevYUrR1EQgEgrJOgYXThQsXbn+RQFAIbCIGQBuoCJL8PE6Sh5fStNeW+2T9qvHyReNbDkt8jFJM0xqq0/gHofWyeoKMSmhOFxKGZFUmuorVMJ4/Zp3bG61VEIFSYdx4/jhYzGh8A9C4eSi5UjFXVeGUZS1hYM7RLuZW6HXQo4OyHTgOC3+Cddth90FlC6sIg56EZ7qBh2j1KBAIBGUS8f+toNTQ5uVxcrMJJ8ccJ11AiHXccVWdxt0zu2FwbCSWxHhlbv8gNH6BSC7ZjeV0FcPVfb11ZR2AIbwOkiShDayAZHAFYxbpuzYo56rVQxscCmQniMsWCyarN8psV1SzMDwQAXMnwN8/wLC+4O0JF6/BpDlKOYMp8+FqVJGmFggEAkEJIoRTPsybN4+IiAiaNWtW2qbcszjmOCmeHI2Hkhwu5/A42XKgskN1Vo+Tu5fqucqyepDQ6dF4+SFpNOjKV1HvoQvJFk660OzEIn24UgFf0mjRhYQBkLZLqYBvqF4fXXAlIDtB3BxzFTkzXdkvonCyUTEY3vo/pa3LB2MgvBIkpcJXK+Dh52D4JNh3VJQzEAgEgrKCEE75IHrVlTySh5dSyVurQxeiCJzsUF0ystmEnJEGKHWZlHHH3CfJ3VMVYMZzinDS+gUiSRIAugrZwklvJ5z0dsLJUK2uuq+rpBTHtHm69NXrocvhcTJeOq1ebymmcLLh4aY0Dv5zCSycCq0eAItFCeU9NUpp7bJ6M2QZbzuVQCAQCEoQIZwEpYYkSQS8vYCAd75A61MOcEwOt68Org0obx231Xey8zhZQ3W2vCP73Cl74aSrGJa9bxeq04dHZO9XzB5HkjBUq4vW6nEyWz1Oxkun1EssyTez28CghPEyj/6bK6m9oGg0SpL4D5/Axm+UulAuejh8Esb8D1o/C58thRsJRZpeIBAIBMVECCdBqaKvUguXiKbqseSWnRxuE0mSm4daqNJWjkBWc5y80Fg9TrL1nEMIsHxl68SOYTttuWA8HumHR5c+DonhNo8TKKE9jbsXuiBbqO4qsiw7tGgBMCfEqfsZ+7cRN+UlEr/9uLBvRS5qV4WPXoNdy2HcIKXuU3QcfLIQWj4D46fD8bPFvo1AIBAICoHThdPly5cxm0WbeEHR0Lh7AUoozpbfpHH3Uj1RuUJ1Hl4OSeagrKizYQirBYAutBqSPrtktyRJ+A56E98X3lbDemDtcWd7bbV6ypjV4ySnp2BJSXTwOIFjuM5W3sB0/XLhHvwWBPjBmIGwaxnMehsa1IJMo1LKoNtQ6DMW1v8F4sdOIBAISh6nC6ewsDAiIiL4+eefnT214D5ADdWlZ4fqJA9vh3Gw9zh5OoTmwDFUp69SC//xn+I/dkaB7q8LrgQ6vfLa6vWV+xtc1ZpTxgsn1BIEtkRyWwkEyC5PYElJLND9CoNBD092ht/mw6o50KM9aDXwz2EYNhHaPAdfLIfEokUJBQKBQFAAnC6ctm7dyptvvsny5cudPbXgPkDysHqcUu08Th5eSKonytHjpOQ45S+cANyatkNvFTm3vb9Wp4QO9QZc6z+ojtsSxDP2bVXuUa68midlvpkdqjOpwimhQPcrCpIETesp5Qx2LoMRz4KfN1yNhqkL4MFn4J1ZcNZ5Ti+BQCAQWHG6cGrbti2DBw8WwklQJGyeJUxGLNbcIY29xyktBdliRk5PBZRVdZKHl1r1G0DrF1wsG/zHziB41m+OieXWcF363j8B0FepidY3AHAsSZDtcUpCzlFDIG3nejWB3VlUCITXhyrlDD4cD7XCIT0DvvsNOg6E/q8pzYYtFqfeViAQCO5bRHK4oEwhuXooLhXAHHcdULxKtjIFckYalpTs/nUady+leKV9FfIcHqfConH3RGctf6DOafU42fKZ9FVqqZ4uizVUJ5tNmOOjlReYTWpYEcB49Rw357zJzblvFcu2/HB1gX7dlZV4P86ELq2Ut/GvfTD4Leg4SOmVl5JWIrcXCASC+4YiCacbN24wYsQIIiIiCAgIwN/f32ETCIqKpNEo4onssJeS4+SlXmO+YS2prXdRE77tE8RzJos7A9vKOvW4cg0178m2qs58Iwos2RnaluTsPCebJ8oUcy2XJ0q2mJEtzsnsliR4qDF8NQX++g6GPA1eHnD+Ckz8DFo8A+/Pg8sF7xQjEAgEAjsK3KvOnv79+3P27FmGDBlCcHCww6okgaC4aNw9MaenZHucPLyQdHokgytyVobDuA2bWNJ4+arNfZ2JLVRnQ1+lliqGbKG6nH3rLCkJYKsBZW0Fg8mInJqM5GmtkC7LxL77PHJGOkEfrUSyJqY7g8ohMGEEvPoC/LRB8Tidv6I0GV64Cjq1hMFPKUJL/AgLBAJBwSiScPr777/ZsWMHDRs2dLY9AoGSs3QjShUiNoEkuXs6Cic7L5QtPKfxK16YLj9soTpQVtnpyoeCtfClLXxnyimc7DxOluSb6r458QYaq3CyJCcoDYVRWrro7cohJCycSvqeLQRN/bFY4UcPNxjYC/o/Adv3wqJVytdNu5StVjgMfhJ6dVZCfgKBQCDInyKF6mrXrk16erqzbREIALtGv8kJyrG1f50tQdwmnGx5T4AaNituflO+Nnn5qsU5daHVkDRa9Z6W5ARkY1beHifbflK2cLIk3sjetyueaYq8qO7LskzaznVYEuLIOLDdYV5j5EUSf/xUXWFY4GfQQPsH4duPYMsSRUi5u8KpC/DmJ8pqvOlfwXXndJERCASCe5IiCafPP/+cd955h+3bt3Pjxg2SkpIcNoGgONgLIsguUWATLqY4JcfJPlTn1rwjhjpN8OjUu2RskiQ1XKevohTV1Hj6qDWfzAlxuT1OdrWcLLZQHYrHSd23W5FnL5wsiTfUfnmZpw45zJv03Sek/LqQlLXfFvl5qleGKWPhn5XwznCoVB4SkuDzH6BVXxgxWTQXFggEgrwoknDy9fUlKSmJDh06EBQUhJ+fH35+fvj6+uLn5+dsGwX3GfYhOOU4p8fJGsJzyxZYuqBKBE5ciFvTdiVml61uk6Gq0hRYkiQ1t8p8M1b1OKm1qOyTw5Pz9jjZt2sxXb+YvX/tgrqfZSecZJORzOP7AMg4uKNYzwPg4wkvPaMkkn/xPrRoBGYL/L5NaS7cYxj8/AdkZhX7VgKBQHBPUKQcp+eeew69Xs8PP/wgksMFTkeTw+Ok8bTlOClfzVaPk+ThKLBKGp9nx+IS0RT3hx9Tx7S+gZhjI7HYCSdDWB0yj+1xDNXZeZwc9hPsPU6X1H2jnXAyx1zFnBCH1jcA44UTyBlKTQHj+WOYE2+oDZItKYkk/7oQjw5POtSgKghaLTzysLIdPwuLfoZfN8OR0/DKNKWw5vNPwHM9lJ55AoFAcL9SJOF09OhRDh48SK1atZxtT4nQq1cvtm3bRseOHfnpp59K2xzBbcgZqsvpcbJ5bHJ6pkoarX8QHh2edLTN6nEyxUVivqHUcNJXjVCEk31yeJJjcri6fzMfj1PkeYf7ZJ06hNuDncg8usdhPPPwLtzb9AAgacU8Uv9Yjunaecq9PqcojwhARHX4+HV48yX44XdY+qvSXHjWYpj3vdLqZfCTUP/u+PEXCAQCp1KkUF3Tpk25cuWKs20pMcaMGcO33xY9H0RwZ8kpiGyepVyeqDssnPJC62/tYXf2KMgW0LuojYIdk8PtvUxxee8nJ2C2JsTbQnW2Z888dVD5ekwRTjbBlnFICddZMtNJ27FOGTvyL5bM4i/eKOcLo56HnT/CnPegcQRkGWHVH/DYMCWUt3YbmERzYYFAcB9RJOE0atQoxowZw+LFi9m/fz///fefw1bWaNeuHV5epf9HVlAwHASSVofk4gbkkTSe47g00PoqAibr9GEAdIEV0Hj5AtnJ4ZaMNOSsDPU15nxynABM15VwnfGa4nFyb91dmf/UIeSsTDVR3Lv3ywBk/Lcb2WIm/Z9NyNb+fRgzyTq211mPiF4Hj3eA1fPg18/hiY6g0yrJ4y9Phoefhfk/KsnlAoFAcK9TJOHUp08fTpw4wQsvvECzZs1o1KgRjRs3Vr8Whr/++osePXoQEhKCJEmsXr061zXz5s0jLCwMV1dXHnzwQfbs2ZN7IsE9g2TnSdJ4eKk5dBq3HEnjdzjHKS/U6uHWaubawIrZwsnqPbIP00GOFXZW4WQTh6bIi1jSUrDEK21c3Nv3AsB48aTibTJmovELxL3t40geXsgpiWSdPUran6uUeawJ8xkH/nL2owLQqA589i7sWgaj+yteqcgY+PBLpZzBGzPgxLkSubVAIBCUCYoknC5cuJBrO3/+vPq1MKSmptKwYUPmzZuX5/nly5czbtw4Jk6cyIEDB2jYsCFdu3YlJiZGvaZRo0bUq1cv1xYZKfpK3I3Ye5zsRVQuj5NbGfA45WjvogsMUcoUYOdxsoXpNFpA8TjZ2q7YimcaajUClDwnU6QSptP4lENfpZYizswmkn9bBIBL3WZIWh2u9VsCkLJ2qbLyTqPFp/+rgCKcbPeQszJJWj6XzKP/Ou25gwOUiuS7lsOMN5S8qIxMWLYWHnkR+oyFddtFGE8gENx7FCk5vEqVwq3YuRXdunWjW7du+Z6fOXMmQ4cOZfDgwQAsWLCAtWvXsnDhQt58800ADh065DR7MjMzyczMVI9FXao7j0NhS2vxS8gjx6kMeJxyCidtYIjqcZLTU5FNRtXjpKtQBdO182DMQk5PQZY0yNZcJJeIZmT+txtT5EV1RZ2uYjiSJOFSqzHp//xB1on9yrV1mytfG7Um/Z8/yPh3EwCujR/GvXV3EpdMxxwfjfHSKQxhtUn+dSHJv3xF2l9rCJ67wamrYF0N0PsReLor7D0Ci3+GDX/DP4eVLSRIKbTZtzv4+zjttgKBQFBqFEk43S7ResCAAUUyJidZWVns37+ft97K7iiv0Wjo1KkTu3fvdso9cjJt2jQmT55cInMLCoYmR6gue7wMJofnJZw8vJTmb7KMJSURs9XjpA2ogPlGFHJGGuaE7DwnydUdfXhtwOpxsuY32dqvGGo2Iv2fP9TrbcLJtVErh3u7d3gSyeCCS/2WZOzbSsaBv9B4+pKyZjGghBNN186jt9ajciaSBM0bKNv1WGUl3o+/K2G86V/B7CVKbtSgJ6FudaffXiAQCO4YRRJOY8aMcTg2Go2kpaVhMBhwd3d3mnCKi4vDbDYTHBzsMB4cHMzJkycLPE+nTp04fPgwqampVKpUiZUrV9KyZcs8r33rrbcYN26cepyUlERoaGie1wpKBkfhlO1xknKutisDwkny8Aa9AYxKhUhdYAiSRovk4Y2ckoglOUH1OGl8/NH4BmCOuqyUVLCG0jS+AegqhAFgirqC8cpZZa6K4UB2GA9AG1QRXVBFZd83AH1YbYwXT6L1D1aFlOsDbVThZLp2wSExPePwzhIRTvZUCITXX4TRA+D3P5XmwkdOw4r1ytasPgzqBV0fVhLPBQKB4G6iSL+2bt68mWvszJkzDB8+nNdee63YRjmbzZs3F/haFxcXXFxEp9PSxD5UJ93S41T6OU626uHmmGuAkhwOSjsWc0oilpRENRlc6+WH1sdfFU6yRUkA0voFog2oAHoXZUWctTK4rayBvkpNJBdX5MwM1dtkw61VN4wXT+LxSD8krfLj7Nr4YQCMZ49gPHsEJAm31t1J//t3Mg/vwqu7c/6xuR2uBnj6EXiqKxw4roTx1m1XQnp7j0D5AKWoZr/uECAaDggEgruEIiWH50WNGjX48MMPc3mjikNAQABarZbo6GiH8ejoaMqXL++0+wjKFpKrO0jKt6Ymv+RwSaNcVwZQw3V6FzQ+Sllt+5V1tnYrGh9/NNYq3+aEG1is4TqtbwCSRoOuQmUANe9Jb/U4STo9hohmALg2bu1wb8/uAwj66Cc8ewxysEdvbQsD4N6uJ149hwCQeWK/U2o8FQZJgiZ1lVpQttV4AX4QFQczvoGWfWDcNPjv1B01SyAQCIqE04QTgE6nc+pKNoPBQJMmTdiyZYs6ZrFY2LJlS76hNmcxb948IiIiaNasWYneR5AbSZKQ3D2AnMnh9iLKA0nj1G/fIqOx1nLSBYZkl06wW1mnhuq8/bLboyTeUBv8qq+3husAJDcPNH5B6rHfSxPxf+1TXJt1dLi3pNGgr1wjV8K36wNt1Hm8+4xEFxKueLWMWWQd3++U5y4K6mq8ZTDrbWhUO7uoZo9h0Gsk/LpFGRMIBIKySJFCdb/99pvDsSzLXL9+nblz59KqVat8XpU3KSkpnD17Vj2+cOEChw4dwt/fn8qVKzNu3DgGDhxI06ZNad68ObNnzyY1NVVdZVdSjBgxghEjRpCUlISPj1gOdKfRuHlhTk12SA6XdHo1nFUWEsNt2DxO2sAQdcze42QrR6Dx8sv2OCXGIxszra8PAEAfEoYtG0kXEu4ghrR+gbg1aVdgmzw6PU3W6UO4t++F1leZ36XhQ6RtWUXG4Z24Nm6NbDaR+N1MtN7+ePV6sdDPXRxcDPBkZ2U7eFzJg1q7DQ4cU7agcvB8D+jXA4JEbzyBQFCGKJJw6tmzp8OxJEkEBgbSoUMHPvnkk0LNtW/fPtq3b68e2xKzBw4cyOLFi+nTpw+xsbFMmDCBqKgoGjVqxIYNG3IljAvuLTQeXpjjrMnX9uPunlgSy5ZwsiVb68Oym7fl5XHS+vhjsYbyLIk31KRtm7Cxb8xrW1FXVLS+AQS8vcBhzLVhK9K2rCLz8E4AklbOJ3X99wC4t3si1wrBO0XjCGV7Zzj8uAaW/gYxN2DmYpjzHXRvpySTN44oFfMEAoHAgSIJJ4vF4jQD2rVrpxbqy4+RI0cycuRIp92zIMybN4958+ZhNosKfqWB6wNtMN+MwVCjgcO4IpxulIkVdTbc2z2BrkIV9NXrqWNa+xwna3K4xtsPjVUkmRPi1FwmTR7Cybaizpm41GsOWh2m65dI3bSClNVfq+cyj+1R27uUFkH+MGYgDH8W1v+leKEOHIPVm5WtUW0Y+CR0b6t4rAQCgaA0KBtJImWQESNGcPz4cfbudV7PL0HB8e4zkvJfbEUXUMFh3JYgXhZW1NmQdHpc6jZDY22bAtkeJ/ONKLCG5DTe2cnhlsR4td2KLVSnCwlTX68rpscpLzTuXhhqNgQg4Zv/KbZbPXqZR/5x+v2KikGv1Hz6ZS6sWQBPdVHGDp2EV6bCQ31h5iKIjrv9XAKBQOBsiuRxkmWZn376ia1btxITE5PLA/Xzzz87xTjB/U1eFa5tITqpDFQNvxW2HCeTtQq4ZHBF4+quJoebE2LBpGRA25LDNR7eaMtXxhxzDUN4nRKxy7VhK7UCuT6sFl69Xyb+4zFkHvkXWZadWlXcGTSoBTPfgreHwY9rlcKa0XHw6bcw73t4tC0M7KWs2itjpgsEgnuUInmcxo4dS//+/blw4QKenp74+Pg4bAJBSWHrT1eWPE55oXqc4pVSGhpva5kCa46TTTSh06vXAgS8vYDAD5aiLVcyOXz2q+38x3yMa/0WoDdgjo/GdP1SidzTGQT4wajnYeePMG+CUkTTZIbf/oSnRikr8n7aABlZpW2pQCC41ymSx2np0qX8/PPPPProo862RyC4JbZVdmUpOTwvbB4n9dhbqfCocXVHcnFT85u0vgEOXh5dUEWwVgYvCfSVa1Durflo/YPUnCqXmo3IPLaHzKP/orcLF5ZF9Dp4rL2yHT2jFNX8bYtSmfzV6fC/BdDvMaU/XoXSyXUXCAT3OEXyOPn4+FC1qvNzMMoSoo5T2cS93RO41G+B20OPlLYpt0Tj6et47J1dGtuW5wTZieF3EteGD6EPzW4Y51LvQaBs5TkVhHo1YMYb8M9KeGOoIpTiE5UQXqu+8PIk2POf2tlGIBAInEKRhNOkSZOYPHky6el3tgLxnUQkh5dNXGo1JuCdL0q831px0Xg5hqy13tnFiLR2YsmWGF6auNS3Cqdje5EtZmSLmZtfTCJ20mAsGWmlbN3t8feBl5+FHT/C/EnQoiGYLbB2O/QeA4++BMvXQUZmaVsqEAjuBYoUqnvmmWf48ccfCQoKIiwsDL1e73D+wIEDTjFOILhbkQyuSAZXtVaTo8fJXkSVfjxJXzUCyd0LOS0Z4/njZOzfTtrWXwDIOLAd94e6lbKFBUOnVZLFH20LJ84p5QxWb4bjZ+H1j2HaF9C3uxLGqyjKwAkEgiJSJOE0cOBA9u/fz/PPP09wcHCZW4kjEJQFNJ4+mOOtwsleLJVyqC4nkkaLS91mZOz9k8TvZ6mr7gAy9t09wsmeOtVg+nh46yVYtg6Wroar0TD/R/hiOXRpBYOeVLxT4teXQCAoDEUSTmvXrmXjxo20bt369hffpYgCmILiovHyzV5V55V3jlNpVevOiUu9B8nY+6cqmlwatCTzv91kHNqBbDIq7W7uQny9YVhfGNobNu9Wksl3HYQNfytbzTClnEGvzuDhdtvpBAKBoGg5TqGhoXh7e9/+wrsYkeMkKC72ZQa0dh4nB+FUBjxOAC71Wzjsl3t9Dhoff+S0ZLJO3v2hd60WuraGH2fCHwvhuR7g5gqnL8I7s+DB3jB5Lpy/UtqWCgSCsk6RhNMnn3zC66+/zsWLF51sjkBw72BfksA+x0nrayeiyojHSVehCq5N26OvXh+/0dORdHpcGys1n9L3by9l65xLrXCYOg7+XQkTR0B4JUhOhYWroP0AGPA6bNkNwtksEAjyokihuueff560tDSqVauGu7t7ruTw+Ph4pxgnENzN2HucNN72Hie7VXVlxOMkSRLlxs92GHNt2o60bavJ2L8NecBrSJJE1tkjpO/dilfPIWjcPErHWCfh4wkvPK3kOv29D5ashj//ge17lS20gpJI3qebEvITCAQCKKJwmj17tpPNEAjuPSQH4WTncbKVINBoHZLGyxou9R4EvQvmmGuYrp5F0rsQN204cmoycmoSvi++W6j5zPHRpG5aib56PdyatHM8l5yAJEkOYvNOodFA2+bKdjkSlv6mlC+4ch2mLoBPFkLPTjCgp1I7SiAQ3N8UeVWdQCC4NbZaTpKLq0MDYG1gRTy6PY/WPxBJoy0t826LxtUd1/oPknHgL9J2rFM8T6nJAKRu+Qn3to9jqNFAvd6SkgQ6HZKLm8NKW0tKEsm/LSRl/Q9qw2PP7gPwfnYMAClrl5K08nM0Ht4EfbTSoebVnaZyCLwzDMYNgl+3wJJf4Pg5RUgtXwdN6ynJ5I88rDQeFggE9x+SLBesrm5SUpKaEJ6UlHTLa++FxHH7VXWnT58mMTHxnnguwZ0jdftvJMx/D21gCOXnrC9tc4pE6pZVJHz1vnqs8QvEUL0eGXu3og+rReD/fgBZJmHRNNK2rFIu0rug9fJFli3Ixizk9FQwmwDQVaqK6ep5AAx1miJnZWA8d1Sd373t4/gN/+DOPeBtkGXYdxS+XQ3rtiv98QAC/ZUE82cfg+CyEW0VCATFICkpCR8fnwL9rS+wcNJqtVy/fp2goCA0Gk2etZts3dXvpSX8hXkzBQJ7Mo/tJe6DFzFENCVwwjelbU6RMN+MJWp4J+VApydw4kK0wZWIfuUJ5NQkvHoNJfP4PrJOHbzlPLrQ6nj3HY3rA23I2LuFm5+/h2ytSi65e+HRpQ8pq78GIGDiIlzqPABA5qmDWBJu4PZgp5J7yAISfQN+/B2++w1irWmcOi10a6N4oZrWEzWhBIK7lRIRTtu3b6dVq1bodDq2b7/1Kpu2bdsW3NoyjhBOgqIiWyyk/bUGQ82GZb557q2I/eBFso7txff/JuHRvheQ2xMluXniP3o6hjoPYEmKx5KcCFotkt4FyeCCtlx5JE32Il7jtfMkfPk+Gt8AfAe+htY/mJtfvk/an6vQVapG0NQfSPppASm/LQKg3Dtf4mptDVPaZBlh499KMvneI9njEdVgQC/o2VEpdSAQCO4eSkQ42XP58mVCQ0NzeZ1kWebKlStUrly5sFOWWYRwEtzvWFKSMN+IQl+lpjomWyzETRxE1pnD6CpUwX/8p+grhhfzPolEv/IEluSbaPwCsdyMVc+5tXoU/1HTijV/SXDsrBLGW705uxeetyf0eRT6Pw5VKpaqeQKBoICUuHCyD9vZc+PGDYKCgkSoTiC4DzAnJ5CxbytuzTui8XDOz0bqtl9JWDABULxYno8+T/KqBaB3ocKCzU67j7NJSIIVG2Dpr8rKPFDCdu0fVMJ4bZoqq/cEAkHZpDB/64v0o2zLZcpJSkoKrq7CRy0Q3A9ovXzxaN/LqWLGve3juHd8CpfGDxM0bRleTw9DV6kaGDNJ3/2H0+7jbHy94aVnYNu3sHCqUtpAlpW6UAPfUAprfvMTJKaUtqUCgaC4FMrjNG7cOAA+/fRThg4diru7u3rObDbz77//otVq2blzp/MtLSWEx0kgKF2S1ywh6fuZ6Gs0IOiDpQCk7dpA2l9r8HtpIlr/oNvMUDpcuKp4oFauh6RUZczNFZ7srNSEql21VM0TCAR2lFiorn379oCSKN6yZUsMBoN6zmAwEBYWxvjx46lR4+6vEifKEQgEZQNzQhxRL3cBi5mgT37BFHWZ+BmvgGzB65kReD/5UmmbeEtS02H1JiWZ/NSF7PEWDZVk8i6tQF+kinoCgcBZlHiO0+DBg/n000/vCyEhPE4CQelz4+PRZOzfjkvjh8k6thc5KwNQGhIHvPNFKVtXMGQZ/j2sCKiNf4PZooyXD4Bne0C/xyCo7BaSFwjuaUpcONlz5YrSTjw0NLQ405RZhHASCEqf9D1biJ85Tj3Wh9XGePEkkosrFb7ZgaS7u8p4X4+F79codaHibipjeh082lZJJn8gQtSEEgjuJCWeHG4ymXjvvffw8fEhLCyMsLAwfHx8ePfddzEajUUyWiAQCPLD9YE2aLyUfn/6qnUJmLgQydMHOTMD44UTpWxd4akQCONfgF3LYPbb0DgCjCalzcuTI+Gx/4MV67NLHAgEgrJDkYTTqFGj+PLLL/noo484ePAgBw8e5KOPPuKbb75h9OjRzrZRIBDc50g6Pb5D3sbt4cco98YcNG4euNS2Vhc/sb+UrSs6Lgbo1RlWz4M1C6D3I+Cih6Nn4LWP4MFnYNoXcCWqtC0VCAQ2ihSq8/HxYdmyZXTr1s1hfN26dfTr14/ExESnGVjaiFCdQFA2SVm7lMSlM3Bp/DABb8wFIOPwLm4umIDvkHdxa9qudA0sIvGJsGKdsiLvarQyJknQqaUSxmvdRITxBAJnU+KhOhcXF8LCwnKNh4eHO6y0EwgEgpLCENEUgKyTB5EtZmRZJvH7mVhuxpK+Y20pW1d0/H1gWD/463v4ego83FRJLN+0C55/DToOhEWrIEnUhBIISoUiCaeRI0fywQcfkJmZHYDPzMzkf//7HyNHjnSacQKBQJAf+io1kdw8kdNTMF46TeZ/uzFdPgMovfDudrRa6NwKvvsYtiyBQb3A0x3OXYFJc6F5b3jrEzhxrrQtFQjuL4oUquvVqxdbtmzBxcWFhg0bAnD48GGysrLo2LGjw7U///yzcywtJUSoTiAou8RNH0nmwb/x6T+ejEM7yDzyj3JCpydkyT9I2nurQFJKGvz8h9If78yl7PFm9ZWimo88DIa7a4GhQFAmKMzf+iL9VvH19eWpp55yGLvXyhHYF8AUCARlE5c6Tcg8+Depf/6M6dp5kDSKq8ZkxBR9FX1IWGmb6FQ83RWB1P8JpSbUt6thw9+w94iyBfpB38fguR7Kyj2BQOB8il3H6V5HeJwEgrJL1pn/iH2vv3rs1rIrpuuXMF48if/42bg1VbodmONjuDFjDO7teuLZpU9pmVsiRMfBD78rW8wNZUyrUcJ8A3rCQ41FMrlAcDtKPDncRmxsLDt27GDHjh3ExsYWZyqBQCAoNPrwOkgubuqx52MD0VVUmsCZrmbnOaXv3ojx/HFSN6+84zaWNMEB8MogpSbU5xOVVi5mi+KJevZV6DQYFv8CyamlbalAcG9QJOGUmprKCy+8QIUKFWjTpg1t2rQhJCSEIUOGkJaW5mwbBQKBIE8knR5DTSXP0hDRFEO1uugrKcLJPkE868x/AJhjr5PTyZ7+zx9knjiQa25LahLm+Ohc47IsI1vKXghfr4Pu7WD5bPhjITz/OHi4wdlLMPEzaP40vDPLsV+eQCAoPEUSTuPGjWP79u2sWbOGhIQEEhIS+PXXX9m+fTuvvvqqs20UCASCfPHsPgB9WC18nnsFAF3FcABM17IVgk04yekpyKnJ6rgpNpL42a8RP/OVXIIq9r0BRI/riSXFsS5d4rczuD74IYyRF0vicZxCrXD43yvw70p4fzRUrwJpGfDdb9DlBXhmDPy+ValWLhAICkeRhNOqVav45ptv6NatG97e3nh7e/Poo4/y1Vdf8dNPPznbRoFAIMgX10atCPpwBYZq9QCyQ3XXziNbLJjjozHfyC69bYq9lr1vDedZkhOwJMar45bUJEyRF5Az0sg6d8zhfun//oGcmUHG/u0O48ZrF4if9w7mG7m9VKWFl4dSNHPzIvhxJnRro+Q//fsfjHgfWvaBmYsgSmRaCAQFpkjCKS0tjeDg4FzjQUFBIlQnEAhKFV1wKGh1yJnpmOOjyTp7xOG8Oe66um+KvpLP/lV133jljLpvSUnEEh+jjJ93FFTJq74g/e/fSf79W+c8iBORJCVJfMFk2LkMxgyAQH+IjYdPv4WH+sLwSbD7kFJsUyAQ5E+RhFPLli2ZOHEiGRkZ6lh6ejqTJ0+mZcuWTjNOIBAICouk06MrXxlQvE5ZZ3IIp9hIdd9eIJnthVPU5ez9y2fVfePlbBGVlaO5cNaZw8o1F8t20+EKgTBusJJMPvc9aN5ASSZftx36vgKdB8OSX5SaUQKBIDdFquP06aef0rVrVypVquRQANPV1ZWNGzc61UCBQCAoLLqKVTFdO4/p6nk1v0njUw5L4g1MdsLJHG0nkPIRUcbLp+32s4WTOeoyltQkNB7emBPiVEFmvHgK2WJB0ij/l8pZmSSvWYxbiy7orflXZQGDHnp0ULaT55XeeD//oRTWnPAZTP8KnuwCA56AmmXHbIGg1CmSx6levXqcOXOGadOm0ahRIxo1asSHH37ImTNnqFu3rrNtFAgEgkJhEyjGy6cxnj8OgFvLLgCY4/L2OJkcRJSdcLp2Htlsss6XLZwAjFavk02cgZKAbo7JzqNK3byS5JWfk/TDrOI9VAlSu2p2MvmkUVCtMqSmK2Kq8wvQ5xVYu00kkwsEUESPE4C7uztDhw51pi0CgUDgFHSVqgGQvm8rclYGkoc3LvVbkLrhR9UzJFssmGLshZPdflS2cMKYhSnqCvqK4Wq+k+TiipyZQdaFE7jUe5Cs04cd7m+8eAJdeaWbQubxfcrYJUfRVRbx9oTBTyp98XYeUITTpp3wzyFlCw6AZx+Dfo9BcLnStlYgKB2KVQBTIBAIyiK2kgS20gOG6vXRBVUCUEN1lpuxYMxSX2POI1FcMrgqx1fOKELL6nFyfbAzgOrNsnmcJHcv5fjCSeX+FgtZpw4q88dFYsm4OxKHJAlaN4Ev3leSyUf1V9q5RMfBrMXwUB8YMVkRUyKZXHC/IYSTQCC459CHhDn0GTHUaIA2MARQxJQlLVkVRxoffwAsSTexpKdiyUxXRBXg0vAhQAnRmWMjkTPTQW/AveUjAGSdP45sMqoCyr1ND+V6a4K4KfICluQE1Q7TdbvOvHcJFQJh/AuwaznMeU9pKGwyw+/blBBe1yGKZ0okkwvuF4Rwyod58+YRERFBs2bNStsUgUBQSCSDK9rAiuqxoUZ9NK7uaLx8AaWCuE046avUUsdN0VcwW8N3koc3LnWaAIpwsiWJ6ytWxVCjvjJP9BUyTxywhgO9cHtIEVTGiyeRZZmsk44VyU121czlrAzipo8k+bdFTn76ksGgh8c7wE+fwfqvlJCdm6tSifzd2fBgbyWp/Mzdpw0FgkIhhFM+jBgxguPHj7N3797SNkUgEBQBnd0KNkN1RejYvE6m2Eg1p0kXHIrWGsYzR19V85t05UPRV64B2ISTEqbTV66JxtMHbZAizFI3LbfeowH6KjVB0mBJjMdyMza7lYuk/Kq1r2ae8d8/ZB78m5Q1S5z/8CVMRHWY9qo1mXwkVAtVPE5LfoFOg6DvOKW8gansdaYRCIpNsYTT/v37+e677/juu+84cCB3ryeBQCAoLfTWBHFdxapoPJRu59oARTiZY69httZq0gZXyq77FH1F9UTpgkPRWYWTOeaqmsekq1wdAENVZQVxxt6tynGNBmhc3FTBlnXhhOpxcm3cGnDsn2c8dxQAS/JN5Kzsmnh3Ez6eMPgp2LIEvp8BXVuDRgO7DyoFNVv1VQpsxsTfdiqB4K6hSMIpJiaGDh060KxZM0aPHs3o0aNp2rQpHTt2JDZW1O4XCASlj6s1P8mteUd1TBdYAVCqh9tW1OmCQ9EFWxPHo69gjsoWTlpvfzQ+yvKxzCP/AIrHCUAfXkeZVLYAqOE7fVhtADL2bVVavWh1uLd7Qpk/0q5/nl1Fc1NcdkuYuxFbMvmXH8COH2Dk8xDgB1FxSkuXls/AyPdhz38imVxw91Mk4TRq1CiSk5M5duwY8fHxxMfHc/ToUZKSkhg9erSzbRQIBIJC41LvQcp/8SdevYerY46huuyQnDZYKR1gjsr2ONnGbOE6rLWcbMf6qo4162zhQH24IpzSdqyzHtdBHx6h3Pf6ZWSzSVltZ9cDz76X3t1OxWB4bYhSmfzTd6BJXSVkt2Yr9B4Dj7wI3/+m1IkSCO5GiiScNmzYwOeff06dOnXUsYiICObNm8f69eudZpxAIBAUB61POSSNNvvYKpyMF0+qpQq0QZXsPE5XHQQV2AknQOPtp3qgDOHZv//sw4GGMOu4MRMAl9oPoC1XHsnFFcwm5R5Rl5HTktXX5xROssmInJVZzKcvXVwM0LMT/DwX1n0F/boryeQnz8Pbs5Rk8omfwdnLt59LIChLFEk4WSwW9Hp9rnG9Xo/FYim2UQKBQFAS6Kwr7WyVvTV+gUpeks3jdCMKc6zSBNg2pgvNFk760BpI1jIHGk9v1StlqNEg+5qwWg73NNR+AEmjQRei5D6Zrp3HmKvxcLZwki1mYl7vTfT4J9WK5Xc7davDh+OVZPIJIyC8EiSnwuJfoONAePZV2PC3SCYX3B0USTh16NCBMWPGEBmZ3brg2rVrvPLKK3Ts2PEWrxQIBILSQxtQweHYVhRT4xugeIRkC8gWJBdXNL4BgKPHSWfNb7LhUv9B5as1nwpA4+GtrtIDcKndWHltRZtwupCd36RVmjeY466r15vjojBFXsAccxVL4o2iP2wZxMcThjwNfy6B7z6GLq2UZPKdB+D/JkDrfvDZUpFMLijbFEk4zZ07l6SkJMLCwqhWrRrVqlUjPDycpKQk5syZ42wbBQKBwClo3D2RrCE1yPYqSZKkeo8AtMGVVc+SvlJVtZyA3rqizobPc+MImKw08LXHliCuC62OxtNHGatYFVBW1mVZV9S51G8BOIbq7ItkWlKSivqoZRqNBh5uCl9Ngb+/hxHPQjlfuB4LnyzMTib/97BIJheUPYrUqy40NJQDBw6wefNmTp5UWgvUqVOHTp06OdU4gUAgcDa6wBCMqYog0QZne4Z0wZXUlio6u3HJ4IqhRgOyzh3FUKuxw1waNw9ccowBuNRtRsaezbg2ap09v63x8KVTaj0ntwc7kXloh6NwispO+rGkJBb5Oe8WKpWH14fCmIGwdjt89yvsP6Ykk6/ZCjXDoP8T0KszeHmUtrUCQRGF07fffkufPn3o3LkznTt3VsezsrJYtmwZAwYMcJqBAoFA4Ey0gSEYLyr/8OnsvEy6ILt9a10nG/7jZmJJildauRQAj8690ZWvrFYeByWBHFDFmcbLTz1vjotClmUkScrhcbr3hZMNFwM82VnZjp1VBNQvm+H0RXjvU/jwS0U89X8CalctbWsF9zNFCtUNHjyYxMTcP9DJyckMHjy42EYJBAJBSWErggmOniX7UJ39OIDWt5xDrtPtkDRaXBs+hGRwsZszVM1pAtBXr4e2XHlAab9i62l3K+GU/NsiYicOvGuaBReVutbK5HtWwuTRUL2KUr7gu9+U3nhPj4Zft0Bm1u3nEgicTZGEk+0/o5xcvXoVHx+fYhslEAgEJYUuMFs4ae08S44iytHj5Awknd7Bw2WoXg9Jb1DLG9jCdQ7CKdUxxyn1jxVknTqkVh2/1/H2hEG9YPMiWDYLurcFnRb2HoHRU+ChvvDR13D13imDJbgLKFSornHjxkiShCRJdOzYEZ0u++Vms5kLFy7wyCOPON1IgUAgcBbaIEU4Se5eauI25Ajb5fA4OQtdxXC1erihmrV/XkB5LIk3MN+IQg6tjjk2e7VyTo+TJfkmALLRWCL2lVUkCVo2UrboOFi2Fr7/Xdmf9z3M/xE6tFDCeG2aKsnnAkFJUSjh1LNnTwAOHTpE165d8fT0VM8ZDAbCwsJ46qmnnGqgQCAQOBND9QZIHl64NnrYwXOuDQpBV6kqIKENKF8i99ZVrAp7/7TaUU+5b7nyGM8dU9rARF9VW7iAo3CSszKQM5Vy27LJMUaVdeEEaVt/wevpYWi9/UvE9rJCcICSSD7iedi8C5b+Cjv2K/ubd0HlEHj+cXjmEfATARBBCVAo4TRx4kQAwsLC6NOnD66uriVilEAgEJQUWt9yVPhiq0O+ESh5SUHTV6r7JYG+kpLVrC1fWfV2actZ++fdiHYI04FjOQJzUkL2CZOjxyll7VLSd6xFV6EKnt2eU8dlYxYZB//GpW4ztbL5vYJOC488rGznLsP3a2DlergcCVMXwCffwGPtFS9UozqK10ogcAZFcmgOHDjwrhFNV65coV27dkRERNCgQQNWrlxZ2iYJBIJSRtLp88zTlLQ6JG2RFhsXCNdmHXBr0wOf515Rx3RW75Y57nq2cLLaINt5nGxhOlAEkT2yNVncnOhYOTJt+6/EzxxH0k9fOF4vy6Ru/QXj1XPFfKKyQbXKSkXyPT/BR69BvRqQaYRVf0DPEfDY/ynhvfSM0rZUcC9wz0eCdTods2fP5vjx4/zxxx+MHTuW1NTU0jZLIBDch2hc3PB/eQpuzTqoY7aVdea4KFU42dq2WFLthFPSLYST9diSkuAwboq+qny9csZhPOvYHhK+mMTN+ROK8zhlDjdX6PMo/P4F/Po5PN0VXPRw9Ay8MQOaPw2T5yoeKoGgqNzzwqlChQo0atQIgPLlyxMQEEB8vKjnLxAIyga2fCrTjeuYohThZOt9Zx+qc/A4mRx72NlynizJeSeTm6y9+WwYr55Xvl4+jWxxbBCX/u9mMg7vKtrDlBEkSQnPffKm0h/v7WFK7lNSKixcBR2s/fHW/yX64wkKT6kLp7/++osePXoQEhKCJEmsXr061zXz5s0jLCwMV1dXHnzwQfbs2VOke+3fvx+z2UxoaOjtLxYIBII7gK1/nuVmHKZriqAxVLcJJ3uPU0L2i3Ikh9tynnJ6nGy1ocw3ohwEkjnWKqSMWZiirmRPExdF/OzxxH/yCnJWZlEfqUzh5wP/1we2L4Vvp0Pnh7L74w2bCK36wqdLlBV6AkFBKJZwysrK4tSpU5hMRe/gnZqaSsOGDZk3b16e55cvX864ceOYOHEiBw4coGHDhnTt2pWYmBj1mkaNGlGvXr1cm30T4vj4eAYMGMCXX35ZZFsFAoHA2Wi8/ZWcJtmCxZqjZKiulCqQM9KQbaLIwePkmBxuK0+Qy+NkE1tmE+b4WHXcFHM1e98uz8l44TjIMnJWBkariLtX0GigbXP4+n/W/njPQYAfRMXBzMVKTajhk2DXQdEfT3BriiSc0tLSGDJkCO7u7tStW5fLl5WA8ahRo/jwww8LNVe3bt2YMmUKvXr1yvP8zJkzGTp0KIMHDyYiIoIFCxbg7u7OwoUL1WsOHTrE0aNHc20hIUq9lszMTHr27Mmbb77JQw89lOd9bGRmZpKUlOSwCQQCQUkhaTRqnhMorVi0QSHqMjBbuM58q+RwU945TvbHqpcJMEVn79sniBsvnsrev3S6sI9y11CpPLz+IuxeDnPeg+YNlJDduu3Qbxx0HASLfoaklNK2VFAWKZJweuuttzh8+DDbtm1zWF3XqVMnli9f7jTjsrKy2L9/v0PzYI1GQ6dOndi9e3eB5pBlmUGDBtGhQwf69+9/2+unTZuGj4+PuomwnkAgKGns60bpKlRB0miR3L2A7ARxyy3KEWBLDk9OQLZzl5iTs19jK6wpy7KjiLIXTpfyF043v3yfmLefvafavRj08HgHWPkpbPxGqf/k4aYkj0+aA817w5szlN55AoGNIgmn1atXM3fuXFq3bu2wpLdu3bqcO+e85a1xcXGYzWaCg4MdxoODg4mKKliN/Z07d7J8+XJWr15No0aNaNSoEUeOHMn3+rfeeovExER1u3LlSr7XCgQCgTOw5TkB6Coo7V5sdZ5seU4FCdVhzMoukmkxI9u1bLEliFuSE5DTs1cW5+dxMl3OFk6WtGTStv6M8fwxss78V4QnLPvUrgr/e0VJJv9gDNQMU8oX/LgWHh0KvUbCz5sgQ/THu+8pUsGS2NhYgoKCco2npqbmWRulNGndujUWi+X2F1pxcXHBxcXl9hcKBAKBk9CVs/c4hQGg8fTGHJ2PcMonVAfK9RpXdyXEZ+99snqZzLb8Jp0eTEZMkReRTUbkjHTMcdl5ocbLp9W+pFln/lPnMl05B/VbOOGpyyZeHjCgp1I4c+8RpTL5+r/gwDFl+2CeUvLg2R5QucJtpxPcgxTJ49S0aVPWrl2rHtvE0tdff03Lli2dYxkQEBCAVqslOjraYTw6Opry5UumJYKNefPmERERQbNmzUr0PgKBQKAt5xiqA3uPk+I1cqjjZMopnLI9ULaVdBa7MB2AKSbS+lURUIaqdZFc3BTxFH0F46WTyn39AkHSYElOwHJTWYSTdeqQOs+9UjTzdkiSkvs05z3YtRzGD4GQIIhPVHrjtXkOBr0JW3aDWZQ0uK8oksdp6tSpdOvWjePHj2Mymfj00085fvw4u3btYvv27U4zzmAw0KRJE7Zs2aL2ybNYLGzZsoWRI0c67T55MWLECEaMGEFSUhI+PqLhkUAgKDkcQnXlraE6D+X3jpySiGyxOK6YyyfHCfL2UEF2jpPZKpy05UORzUaM545hunIO8w0l/cFQvT6myIuYrp3HeOkMWv9gB+Fkuk+Ekz1B/jDqeRjeD/7cDd/9Btv3wtZ/la1SeWt/vG5Qzre0rRWUNEXyOLVu3ZpDhw5hMpmoX78+f/zxB0FBQezevZsmTZoUaq6UlBQOHTrEoUOHALhw4QKHDh1SV+qNGzeOr776iiVLlnDixAmGDx9OamoqgwcPLorpAoFAUObQlsvO49SWVxakaDyV3nKWlEQsqUkOzX/VnCbbcV4eJ6uAUiuTx0cjm01qKQJdYEX0laoBihcpy5oYrg+rhb5KTWX88ilks4mss9l5ocZr5xwS0DNP7Ofml+/fU0nj+aHTQpfW8O1HsG0pDH0GfLzgahR8+CW0eAbG/A/2HRUlDe5lityUqVq1anz11VfFNmDfvn20b99ePR43bhyg9MNbvHgxffr0ITY2lgkTJhAVFUWjRo3YsGFDroRxgUAguFvRVQzHtUlbtIEhaFzcALtQXWpSrrCbfY6TbLGAObuWnupxsob2dKHVMSfFgzEL840o1eOkC66E5OoOKF4kY+RFAPRVaiPp9KTv2oDx0mmMl04jZ6YjuXkqdaVSk7HcjEXrr+S5Jn77McYLJzBUr4dHhyed/M6UXcIrwbvDYfwLsGarkgt1+CSs3qxsEdWUPKknOikr9QT3DkUSTuvWrUOr1dK1a1eH8Y0bN2KxWOjWrVuB52rXrp3Dfy95MXLkyBIPzeVk3rx5zJs3D7MIXgsEghJG0mgp99pnDmP2q+osSTnaRJntPE458p1y5jhpvP3QBVTAdP0S5thI1eOkDaqIxssPgKwLJzDHXQdAH1YbSasFlATxrFMHATDUaoQ5+gqm65cwXj2H1j8IS3qquhLPFHV/NoBzdYHejyjbf6cUAfXrFjh+Dt6aCVO/gCe7KKG8mmGlba3AGRQpVPfmm2/mKShkWebNN98stlFlgREjRnD8+HH27t1b2qYIBIL7EAfhdCuPU44Vdtk5TsprtF6+aAOVYsCm65cxxym5TLqgSuhClVCdOfoKmE1Inj5oywWroTpT5CUyjyotrlxqNUJnDe3Z8pyU1XZKCNHWUPh+pkEt+Ph12LMS3ntZ8Uolp8KSX6DzYOjzCqzdBsaiN9sQlAGKJJzOnDlDRERErvHatWtz9qyoFCYQCATFRfKw5TjlFaoz5rkP9jlOylelEnlFADJPHgCLGfQGNL4BaMuVR3LzUF9rqFILSZLQ+AWh8fIFi5mMg38r52o2csiJAlRvFFjFlwAAX294sTf8uQS++xi6tlZavvxzCF6erLR3mbkIrsfedipBGaRIwsnHx4fz53P3MTp79iweHh55vEIgEAgEhcExVKfkK9nElP2qupzFMHN6nDSePuisHqfMY4r3SBdYEUmjQZIkdBWrqq/Vh9VS7iNJ6CrXsE5oBq0OffV6qofKdMUqnE5mCydT9NXbpl3cb2g08HBT+PID2PkjjO4PgX4QcwM+/VZpMPzSBNixHwpRblBQyhRJOD3xxBOMHTvWoUr42bNnefXVV3n88cedZlxpIuo4CQSC0iQ7OTxRLS1gS8h2qOOUX46TtUWLxtsXbaDicbLcVFwc2uBK6vU2LxIo+U3qfuVaduO10Li4oa9UHVA8TrLJSNbZ7CricnqKg2dMNhnJPH04l7C7XwkJgldfUGpCzZsALRqC2QIb/4bnxiv98b75CRKTS9tSwe0oknD66KOP8PDwoHbt2oSHhxMeHk6dOnUoV64cM2bMcLaNpYLIcRIIBKWJrRyBnJqMOeEGYC+c7EN1+eQ42YXqbB4nG/bH+tDq2fthdmLJmucEYKjZWHldhSqg0SKnp5Bx8G/kzAwkD2+laCaO4bqU9d8TN2EAKet/KMxj3/MY9PBYe1g+GzYthIG9wNMdzl+B9+cp/fFe+0hJNBeUTYq0qs7Hx4ddu3axadMmDh8+jJubGw0aNKBNmzbOtk8gEAjuSzS2sBzZK9a0VoHikByeK1SXAGQ3+NV4+qir52zYe5xsCd/oDWq7F3AUTi61GgEg6Q3oyodiirxI6uaf1HOW9FSybsZiir6KoUYDADKP7wOUlXmCvKkZDu+PhjeGwupNyoq8E+dhxXpla1BLWY33eAdwcy1tawU2ilzHSZIkunTpQpcuXZxpj0AgEAgASadHcvNATk/FdP0SAFp/a/26PDxOtmvl1GSl95y1wa/Gyw+Njz+SwRU5KwNQVtTZcKnzAC71W6CvVg9Jp1fH9RWrIrm6IxuzMNRurI7rKlXDFHmRzP92AWCo1RjT9YtkndiPKTq7JIHxotLCxRIf47T35F7Fww2ee1zpf7fvqFKZfN12xev0+scwZT483VW5pnrl0rZWUGThtGXLFrZs2UJMTEyuJroLFy4stmECgUBwv6Px8MGcnoqcpiS+5BWqs7Vb0fiUw5yeCoAp+opaulrj6Y0kSWgDQzBdUxb16Kyr7AAkgysB73yR696SwYVyby8AYxZa3wB1XF+pGhl7tqjzG2o3VhLIyS5JYE64oeZTmeOjERQMSYJm9ZVtwghYuR6+WwNXrsPCVcrWsjH0f1ypYK4v8l9wQXEo0ts+efJk3n//fZo2bUqFChXUJr/3EqIApkAgKG00nt6Y4yKzj/2swimPUJ1kcEXy8EJOTcZkrQIueXipXiRtUEVVONmSxW+HS82GucZ0dsnk6A0YqtZV+9zZcpxs3iYAc3wMsizfk38nSpJyvjCsH7zUB/7aB9/9Clv+gd0HlS3QH/p1h36PKYnngjtHkYTTggULWLx4Mf3793e2PWUG0eRXIBCUNraVdTZsHif7lXS2FXaSXo/G0xeznXDSePqq1+kCQ8jEGrpz9yyyTfrQbOFkqFpXyXsKVvrr2TxO9sJJzkxHTkvOLqWAksAueXgLMVUANBpo11zZrkXDD7/D8rUQGw+fLYW530Onlkou1MNNlesFJUuR3uKsrCweeughZ9siEAgEAjschJNWh9bbHwDZZFd62loAU9IZ1OtN1y8qr/fyzX651cukDSqYtyk/dBXCQKO0ZLHlPtmEkyUhDktGmoNwAsXrZCPr3FGuD21L4pKPimXH/UjFYHhtiF1Jg0ZK/ac/dsKAN6Bdf1iwDOITS9vSe5siCacXX3yRH34QS0wFAoGgJJE8s700Gm8/0FuTt80mpbkv9h4ngyqUTJFKMrm9cHKp1xzJ4IrrA8Vb/Szp9OgrKyUMXCKaKvfx9FE9SuaYa2TlEk7ZeU6Zx/eDLJN1+lCx7LifUUsazILNi2Hwk+DtAZciYdoX0KI3jJ2qJJqLmqTOp0ihuoyMDL788ks2b95MgwYN0Ov1DudnzpzpFOMEAoHgfkbjke1x0nj5IukM2SdNWWBwzc530hvUEgZ5eZwM4XWosHCHw8q5ouI3/AOyLpzApUF25EEXHIrx/DGMF09itpZP0Feri/HcMQePk9naZNgsVts5hRpVYNIoeP1F+O1PZUXekdPwyyZli6imrMbr2UmpFyUoPkUSTv/99x+NGjUC4OjRow7nRMxaIBAInIN9qE7r5YekzxZOssmolBhQQ3V6tV6TfYNfe5whmgD0VWqhr1LLYUwXXAnj+WOk792i3LtcefSVa1qFU7bHyWRNILck3lCewc4mc1I8Gld3JIMoWlRY3N2gb3dlO3xSqQn1259w/By8M0vxRPXqDP2fgFrhpW3t3U2RhNPWrVudbUeZQ6yqEwgEpY3GPlTn5Qva7F/ZqqfJZC+cHJPJcxa+LEm01jynjEM7AaV9i63ulL13yZZAjixjTriBLqC8ck3STaJHdUMXWoOgKd/dMbvvRRrWVrZ3h8NPG+H7NUpl8qW/Kluz+koyebc24GK4/XwCR0T+fT6IlisCgaC0sfc4abz9kDSabPFkTRBX+9bpDQ6r6HK+vqTR2aqRWwWdIpyUVYA24SSbTZjjrquvsdy0E1SXTyNnZmA8ewRLZrrD3Gl//07K+u9L0vx7El9veLE3/LkEvp+hCCWtBvYegTH/g5Z94MMv4fL1288lyKbI5bP27dvHihUruHz5MllZjr2Sfv7552IbJhAIBPc7DsLJ6j2S9AZks0n1OKmVw3WG3B4nb987YyjZK+ts6MNrI2mVMJyterg5LgrM2SsCzfbCyU5Qma5fwmBtOCxnZXBzwUQwm3Cp1xx9aA31uvQ9W8g4tAOfAa+hcRUJPPkhSdC6ibJFx8GytUpZg6g4mP+jshKvbXOlsGb7B0GrLW2LyzZF8jgtW7aMhx56iBMnTvDLL79gNBo5duwYf/75p6h5JBAIBE4iV6gOwJoTpHqabKE6ax0nh9ffwVBdLuGUh8fJZNcE2H4cwBxrJ5wiL6j7xmsXVLGVeWSPOi7LMgmLppH258+k/Sn+WS8owQEwZiDsXAZffgBtmior77b9C0PegYefgznfKXWiBHlTJOE0depUZs2axZo1azAYDHz66aecPHmSZ555hsqVRSMdgUAgcAYOq+q8sz1OkF0x/JYepxzJ4SWJxi8QrLZJHt5oy5VHYxVOluSbyFmZamVxGw7Cya5Cuunaxez9K2fV/cxj2cLJdPm02tYlbftvDvMaL53m5oIJmG+Idi/5odNC19aw9GPYthSGPqOE9q5Fw4xvlDDeyPfhn0OipEFOiiSczp07R/fu3QEwGAykpqYiSRKvvPIKX375pVMNFAgEgvsVR4+TVTjZVqHZQnW3zHFyPC5JJI1GbR5sCKuNJElKqFHvAoD5Zmx2Yrg1T8shaTwuH4/T1XPqfubxfchW75MtCR3AeOmUWjtKlmVuzn+XtG2/kvjjp858xHuW8EpKIvm/K2Dmm9A4AowmWLMV+rwCnQfDop8hKaW0LS0bFEk4+fn5kZysNJ2sWLGiWpIgISGBtLQ051knEAgE9zGSwVVdmq/1dhROqsfJPlSX0+NkJ7zuBLYEcb01P0mSJLtwXbQaqtOH1wFQPUaQM1R3Ud032nmc5PQUjOePA5BxWBFOkovy/qRtX6OM79uK8eIpANJ3bxT1ogqBqws81RVWz4N1X8Gzj4G7K5y5BJPmQPPe8MYMpU7U/UyRhFObNm3YtGkTAL1792bMmDEMHTqUfv360bFjR6caWFrMmzePiIgImjVrVtqmCASC+xiPbs/h+kAbdJWqKgM5QnWooTo9Ghc31cNj3+D3TuHe9gl0Favi1vpRdcy+JIHN4+RS+wFlzJocLlssaqNgAGPkRbUyusnqcdL4BgCQeXQPlrQUsk4dAsC772gA0neuRTYZSV61QJlEowWzidRNK0riUe956laHaa/Cvyvh/dFQMwzSM5TE8sf+D54YDis3QEZmaVt65ymScJo7dy59+/YF4J133mHcuHFER0fz1FNP8c033zjVwNJClCMQCARlAZ9+oyn3+hwka3hLrR6urqozOozbvE53Mkxnw+3BTgR/8gsGq0cJcPA42aqGG+pYhZPVG2RJiFMSwDVaJfndmIk57jqWjDTMsUruk0en3gBkHv1XyXUym9CWr4xHlz5ofMphSbpJwtdTMF48heTqju8LbwOQuvkn5KwMAEwx17j55ftknfnvDrwb9wbenjCwF/yxEFbMhsc7gF4Hh07C+OmKF+r9eUqdqPuFIpUj8Pf3V/c1Gg1vvvmm0wwSCAQCQf5IOVbV2ec4gSKYLPExdzQx/FbYhJPx4inkjDSQJFxqKc2B5Yw0LGkpan6T1j8IydUd09VzmCIvoklSlnZpfMrh1rIryT/NJ/P0ITTlFC+Wa6NWSFod7q27k7L2W9K2rQbAo2s/3Nv3JHn115jjrpO2Yx0udZsT9/4QzDeiyDr7H0HTV4pOF4VAkuDBhsoWGw8r1sMPa+BqNHzzk7K1ekAprNm5lSKu7lWK5HHSarXExOSOG9+4cQOtKAAhEAgEJYc+R46TXagO7DxOZUw4ZZ7YZz0OVpoCu3kCStK4bUWdNqACuhClH4gp8oKaGK6vVA1dSJiycs+YRfqOdQC4NmwFgHvbx9X7SS5ueHbvj6TV4dG1HwApaxYTN/kFNRxounyGzCP/lOhz38sE+sOI5+Cv72HhVOjQQhFWOw/A8EnQqh/MWgxRsbeb6e6kSMJJzmdtYmZmJgaDqN8uEAgEJYXqcbKG6LLrOGV7nKAMCSerd8hWBFNrTSC3r/FkSwzXBlRAX1ERTsbIC5iuKMJJF1odSZJwqdtcmdRiBr0BQ0RTAPSVa6CvGgGAxyP91ER6j/a9kFzcMF2/hDk+Gl1IOG6tlPyrlLVLS/S57we0WujYEhZNg79/gBHPQoCfUmRz9hJ4qC+8NAH+3gfWlLV7gkI50z777DNAWSnx9ddf4+npqZ4zm8389ddf1K5d27kWCgQCgUBFzXHKtapOGbc19s3Z4Le00PoFORzbCmVq/YMwXTuP5WaM2oZFFxiCLiQMUGo5SdZ/xPWVqgHgUu9B0nesVfbrNFGS4a34DX+f9L1b8ezeXx3TeHrj3u4JUjcuQ1epKgHvfo2cmU76rg1k/n97dx4fVXX+D/xzZ8ueTPYQSMKqCISoBCggoiUKuGIqtohKEbXYUED6xQ0BK1UUfoDVRqhal1Za0AoiuIFIABUQwqpBFgVCQvZtsmdm7vn9cWfuzBCQYUgyk+Tzfr14debemzvnBEyenue5zzn4DcxnTkCf0Lt1Jt7JJMQBjz8MzPo98PkO4L31wO5DwBc7lD/duwKT7gAmjAHC23mf7EsKnJYvXw5AWXFauXKlS1rOYDCge/fuWLlyZcuOkIiIVGoDzHO2XLF3FA+47hY0nT6KgOFjvTK+c9lXnOzUwCncseKk1jhFxrmk6uwF8boEe+A0RL2PPU1np0/o47Idi13oxJnQd++LgMG/Vrew8R/8azR89yVqPluF8EcWXPYcycGgVwrI7/g1cOwk8N4G4MMvgFP5wPMrgP/3JnDbjcB9dwLXXKWk+NqbSwqcTp5UmpLdeOONWLt2LcLD266dPxERAdApP7YdW644OocDyqP+MQt9Jw2lCYsEJA0glFyNNqarcjw8GoDSksC+4qSN7qKuOMlVZeo97CtOuqgu0PdOhiX3OPxTb3Tv8/0DEXTjXS7Hgm+9Hw3ffYm6HRsRevejsJrKYT71IwxXXA297fPp8l3RQ2ll8MTDwPovgX9/DOScAD7cpPzp30fZH+/O0UBgwMXv5ys8qnvfunWry3ur1YrDhw8jKSmJwRQRUStypOqUDtpqOwJ92/Zscpek1UFjjFSbXTqn6gBbjZM9cIqKhyYgCJqIGLUmShMRA02Qo5Fn1FOvQa6vgy4qzuMxGa5Igb53MswnDqMwY4wjqIvqgtiXN7R5/6uOLigAuPd2YOJtwP4jShpv41bgh+PAk0uB51cC6TcD992uBFu+zqPi8FmzZqn9mqxWK66//npce+21SEhIQFZWVkuOz2vYAJOIfNGFU3W++2COvQkm0DxwsuT9BFFfqxyzBUP6eMdvz3NrkDRBoZcVNAFKnW7InQ8qb4QMKSAYkl8ArKUFqN+16bLuTRcmScC1/YBlTymNNec+qtQ+VdcC764DbnoQuGcm8PFXQJPZ26O9MI8Cpw8++AApKSkAgA0bNuDUqVP48ccf8dhjj2Hu3LktOkBvYQNMIvJJF+jjZA+ofJE9SJKCQtVtYNTAqeA0AGUTY3uxt84pXWZP07W0gMG/RvTCfyNmyYfo8s8dCBn/EACgZuO7F3xynFpOeBjwyD3A1n8B7y0BxowEtBqloPxPC5VNhhe/AZwpvPi92ppHgVNZWRni4pSI/9NPP8WECRNwxRVX4MEHH8Thw4dbdIBEROSgppHULVd8O1UHOArE7atNQPOn7bRR8eprndOKk66VAicAMPQZCH1Cb0gaDYJumgDJzx/mU0fR+P13rfaZ5EqjAUamAq8/B3yzGpj5ABATCZRWAJn/AUbeC0x5CtiyE7BavT1ahUeBU2xsLHJycmC1WvH555/jpptuAgDU1dWxASYRUStqlqqzB1Ba3w2cdNFKQbguPkk9pgmLUIrGbbRRXRzXd71wqq61aILDEGgrIq/Z+G6bfCa56hINzJ4CfLsaWPGs0olcCOCrXcCDTwPX3wdkrgKqarw7To8CpylTpuCee+7BgAEDIEkS0tLSAAC7d+9mHyciotakO6dzeDtI1QWOugMhdz2MkPQ/qMfsReN2Wqe6JXsTTEiaVl1xOlfwuPsASWPr8XS8zT6XXOl1wC2jgP8sVVJ5U+8GwkKAvEKlI7nZy/VPHj1V9+yzz2LAgAE4c+YMJkyYAD8/ZTdurVbLfeuIiFqRuuJkMSu1OGbfD5w0wWEI/e30Zse1ETGOp+2cUnXaiFiEPTAHkl8ANP6BbTZOXWw3BAxNQ/2uTTCt/jvCH12o1mSRd/RMAOZnAI8/BGzYCpwtUrqTe5PH2/DdfffdzY5Nnjz5sgZDRES/TK1xMjcBVovTcd8NnC5EGxED808/KK+ju7icC77lPm8MCcG3PYD6XZvQkJ2FwoybEDjydgTfNhm6uISLfzG1Gn8/YIJv9HR1P3B65ZVX8Mgjj8Df31/deuVCZsyYcdkDIyKi5iSnVJ3aigBQN/9tT5wLxJ1rnLzJ0DsZEY8thenDf8CSewy1X36Aup2fI3bpR9Aao7w9PPIBbgdOy5cvx6RJk+Dv769uvXI+kiQxcCIiai3OqTqnwMmXU3UX4hw46aLjf+HKthUwNA3+Q0ajKWcvKt9eBEveTzCtfgXh057z9tDIB7gdONm3Wzn3NRERtR01JWducrQk0GghadrfE80ae38n/0BIQb5VSyRJEvz6D0b4H55Fybz7UZe1HkE33QNDrwHeHhp5mUdP1RERkXe4pOrUferaX5oOUIqxAUDXJQmSj+72augzEAEjbwMAVL2zmM0xyf0Vp9mzZ7t902XLlnk0GCIiughbLZMwN6n71LXH+iYAMFx5DYwPzYO+t2+v4oRNnIGG77ag6fhB1H/zKQKvu9XbQyIvcjtw2r9/v8v7ffv2wWKx4MorrwQAHDt2DFqtFoMGDWrZERIRkcqlc7i64tT+6psAJR0WlNb8CW1fo42IRcj4h2Ba8yqqVi2HX79Ul/33qHNxO3DaunWr+nrZsmUICQnBu+++i/BwpaFCRUUFpkyZgpEjR7b8KL0gMzMTmZmZsPpKj3ciIjiCJOfi8PZYGN7eBN96P2q3rYe1MBclzz2E6HlvqlvJUOfiUY3T0qVLsWjRIjVoAoDw8HD89a9/xdKlS1tscN7ETX6JyCc5bbmipuraaY1TeyIZ/BD19Epoo+NtwdNUWEo924FWNDWgesM7sBTmuhyXa0wofWEaTB+ubIkhUyvxKHAymUwoKSlpdrykpATV1dWXPSgiIjo/R3F4k7rtClec2oYupiui5v8T2phusBadQcm8+1D137+h8dhBCPn82Qkhy82OmT5YAdOq5ShbNtvl66o/fguNh3ai+n//gKW0wOVr5LoayPW1LTsh8ohHgdNdd92FKVOmYO3atcjLy0NeXh4+/PBDTJ06Fenp6S09RiIispH056tx4opTW9FFxyvBU1wi5IoS1Kx/C6XzH0DRjNtgzvvJ5drqDe/g7P2DUff1p+oxa1UZajetBgBYco+jbvsG5XhlKWo//69ykZBR++X/1K+Ra0wo/r90FP9fOkRTQyvPkC7Go8Bp5cqVGDduHO69914kJSUhKSkJ9957L8aOHYvXXnutpcdIREQ2ao2T2anGqZ0Wh7dXuqg4xLy4BuF/ehEBw8dCCgyBtfQsypf/H+SGOgBAY85emP7zN8BqQeU/n1fTejUfvwPR2ADJ4A8AML2fCbmxHtUf/ROiqUHtZ1X31Yfq32/1+n/CWl4Ea1khGg7v8sKMyZlHgVNgYCBee+01lJWVYf/+/di/fz/Ky8vx2muvISgoqKXHSEREdufp49Re2xG0Zxr/QASOGIeIGS8hdvl6aMKjYcn/GZVvLITVVI7yV58EhAzo9BD1Nah84zllVWnz+wCA8D8tgjYqHnJ5MUyrlqP2yw8AABF/ehHaiFjIpgrU79oES8lZ1Hz+H/VzG/ZsPe94qO1cVgPMoKAgDBw4EAMHDmTARETUBtR6JoujOJw1Tt6lDYtExMzFgEaL+m8+Rckz90GuKIEuvgein/sXoDeg8eA3KH1hGkRTA/S9k+GfeiNCfzcdAFC7aQ1gMcNwVSr8UoYj6KYJAICaL1bD9H4mYG6CJjwaANCQvc2lLsp89hTqs7PafM6dmduB07Rp05CXl+fWtWvWrMGqVas8HhQREZ2fmpYTAqKx3vUYeY1f32sROnEmAMBanA/o/RAxczEMPfsh9J4MAEpNEwCETngUkiQhYPg46Lv3Ve8R+tsMSJKEwF+nAzo9zCcOo37HRgBA5OxlkIJCIVdXoOnoQQDK03mlCx9C+ZKZaDp+qC2n26m5HThFR0ejf//+uOWWW7BixQrs2bMH+fn5KCsrw4kTJ/Dxxx/j8ccfR2JiIpYvX47k5OTWHDcRUeekc7TfE7anrLji5BuCb3sAAb+6GZA0ME55EvqkK5Tjt94PfW/ld6K+z0D4DRwOAJA0GoQ9MAfQ6uA/JA1+fa8FoKxgBfzqZvW+AcPHwdBnIPyvUfokNuxV0nW1Wz6EXKE84d54JLttJknuN8BcuHAhpk+fjjfffBOvvfYacnJyXM6HhIQgLS0Nr7/+OsaOHdviAyUiItcgSdgKkZ2DKfIeSZIQPnMxwqoroA2NcBzXaBExawlq1r+FoLH3uuzL59cvFXErvoQmMNjlXsFjfof6rz8BdHo1pRcw+Neo//oT1O/ditDfZqD647fV65uOH2zl2ZHdJf3XFhsbi7lz52Lu3LmoqKhAbm4u6uvrERUVhV69evnsJo1ERB2FpNUBkgYQMuT6GuUYU3U+Q5Ikl6DJThfVBcapc8/7NdrQ8GbHDH0GImLWEmiCw6CLUTZD9ksZDugNsBadQeW7i5XVJr0fYG5E0/HDEELw93Ab8Pj/poSHh7t0Diciojai0wPmRog6puo6Mud0HaA8yec3YCga9+9A3ZYPASgbEFetWg65shTW0gLoouMBAEIIiNpqaIJD23zcHd1lPVVHRERtzx4oyQ22TtJcceo0Agb/Wn2tjYxD0M2/hb77lQBc03XV72ei4KGRaPyB24a1NAZORETtjL17uKM4nH2cOgv/a0cBtnRcyPipkHR6GPoMBAA0HT8MQOnxVbNpDQCg4dBO7wy0A2PgRETUzkhaJVCS+VRdp6M1RiL0ngwEjroDgTeMBwAYetsCp2PKilPDgW8gak0AAEvBaa+MsyPr8I9iVFZWIi0tDRaLBRaLBTNnzsTDDz/s7WEREXnOFigJW6qOxeGdS8hdrr/DDFcogZP51I8QTY2o/8axN56lkIFTS+vwgVNISAi2b9+OwMBA1NbWYsCAAUhPT0dkZKS3h0ZE5BE1VVdnr3Fiqq4z00Z3hSYsAnJVORqPZKMhe5t6zlKQCyHLkDRMMLUUtwOnf/3rXx59wNVXX42BAwd69LUtQavVIjAwEADQ2NioPGkghNfGQ0R0uSTduak6Bk6dmSRJMPRJQcPerTCtfgWiqQHauERYS84C5kZYy4ugi+oCAJDrqlG75UMEDh8HbWSsl0fePrkdOL399tsXv+g8pkyZ8ouB0/bt27FkyRJkZ2ejoKAA69atw/jx412uyczMxJIlS1BYWIiUlBS8+uqrGDJkiNtjqKysxKhRo3D8+HEsWbIEUVFRHs2FiMgn6JiqI1eGPslo2LsV5pNHAACB192K+m8/g+XsKVgKctXAqfaL1TCt+TusJWdhfPBpbw653XI7cHr33Xc9+gCj0fiL52tra5GSkoIHH3wQ6enpzc6vWbMGs2fPxsqVKzF06FC8/PLLGDNmDI4ePYqYmBgAyqqWxWJp9rWbNm1CfHw8jEYjDh48iKKiIqSnp+Puu+9GbCwjbSJqn+wrTrCvnrM4vNOzF4jbBY4YB/PJHFvgdBpIHgoA6p52luL8Nh9jR+F24NS9e3ePOpIuWLAA8+fPv+D5cePGYdy4cRc8v2zZMjz88MOYMmUKAGDlypX45JNP8NZbb+HJJ58EABw4cMCtscTGxiIlJQU7duzA3Xff7f4kiIh8yLlP0TFVR/pe/dWO8vpe/aHrkgRdXBIAx5N1Qgg0/axslyZXlnhtrO2d24GTLMutOY7zampqQnZ2Np566in1mEajQVpaGnbudK83RVFREQIDAxESEoKqqips374djz766AWvb2xsRGNjo/reZDJ5PgEiotZwTjE4U3Wk8Q+EvvuVMJ88gsARtwAAdF1sgZPtyTq5ohhyZSkAwFpR6vL1QgjU79gIfe9k6OO7t93A2yGfLrMvLS2F1WptllaLjY1FYWGhW/c4ffo0Ro4ciZSUFIwcORJ/+tOfkJycfMHrFy1ahLCwMPVPQkLCZc2BiKilSc0CJ644ERA25SkE3zkVQWkTAAC6LokAHCtOTT8fUa+VTeUQVkeJS1POHlS89gwq31jYhiNunzxqRzB//nzceOONGDZsGPz9/Vt6TC1qyJAhbqfyAOCpp57C7Nmz1fcmk4nBExH5lGYNL1njRAD8rkiB3xUp6nv7ipO1OB/CYob55x8cFwsBuaoc2gilVthy9pRybYlr7ZOQrSh7MQPaiBiET3vO5Vz93ixYCk8j+NYHOtXmwh6tOO3cuRO33347jEYjRo4ciWeeeQZffvkl6uvrW3RwUVFR0Gq1KCoqcjleVFSEuLi4Fv0sOz8/P4SGhrr8ISLyJeem5piqo/PRhMdA8vMHZCssxflqfZOdtdKRrrOUKb9nrVVlLi17LIVn0HhoJ+qy1kNudP0dX/mPBTC9twyW3OMuxxsOfovyvz+ttsvoaDwKnDZv3ozKykps2bIFt9xyC/bu3Yv09HQYjUZcd911LTY4g8GAQYMGYcuWLeoxWZaxZcsWDBs2rMU+53wyMzPRr18/DB48uFU/h4jokp2bqmNxOJ2HJEmOOqeCUzCftAVOthVKa4WjQNxaZit/MTdB1Neox+WqMsc1pY4SGbnWBLm6UvmSMydcPtf0/t9R//UnqNvxSYvNxZd43Dlcp9NhxIgRiI6ORkREBEJCQvDRRx/hxx9/vKT71NTU4MQJxzf95MmTOHDgACIiIpCYmIjZs2dj8uTJSE1NxZAhQ/Dyyy+jtrZWfcqutWRkZCAjIwMmkwlhYWGt+llERJfi3ECJe9XRhejikmA+dRSN3++BXFUOaHXwu/IaNH6/Wy0UB5wCJwByVTk0gSHKcedrSs9C37WH7XWBetyS/7P6WsgyLHk/KcfPuK5EiaYG1O/dioDUGyEZfLvM55d4FDi9/vrryMrKwrZt29DY2IiRI0fihhtuwDPPPHPJXcL37t2LG2+8UX1vry+aPHky3nnnHfz2t79FSUkJ5s+fj8LCQlx99dX4/PPP2YeJiDqtZqk5puroAuwrTvW7vgAA6BN6QRvTFcA5K07lxY7XVWXq17msOJWcVV9bnF6bnQIna+lZiMYG5XjuMZexVK97E9Xr3oBlwh8R+ps/XN7EvMijwGnatGmIjo7Gn//8Z/zxj39EcHCwxwO44YYbLroFyvTp0zF9+nSPP4OIqEPhU3XkJjUAsgVJ+h79oA2PBgBYbb2chBCwljlqiWVTufraWukInCwljlUma7FTEJXnCJyc03bm3OMQQqiF4405ewEATScOX+asvMujGqe1a9di0qRJWL16NaKjozF8+HA8/fTT2LRpE+rq6lp6jF7BGici8lVsgEnusgdOdoae/aA1KtuOybZeTnJ1BWB29C+0VjoCJ5d0nnN6rtQpcCo6A2ExK69taToAEPW16iqVsFpgPqW0Q3BO7bVHHgVO48ePx7Jly7Bv3z4UFhbi6aefRn5+Pm677TZERES09Bi9IiMjAzk5OdizZ4+3h0JE5KJZoMRUHV2ANi7R5b2+Z39ozllxcl5tAs5Jz7kUhzuCJee0HawWWIrOAADMZxyBk/JeqXOy5J9UU3jWkrOQGxyLLEIINBz4xuWzfJnHxeFlZWXYtm0bsrKykJWVhR9++AHh4eEYOXJkS46PiIjO1eypOgZOdH7aECOk4DCImipAq4M+sQ8glJ1A7CtOzQInp1Sda42TU6rOOXCCkq7Td+0JS56SqrN/piX3ODDoBjSd00PKcvYUDD37AQAa9nyF8mWzETB8LCJmvNQCs25dHq04JScnIzY2Fn/4wx+Qn5+Phx9+GPv370dpaSnWrVvX0mMkIiInzfo4MXCiX2BP1+kT+0DSG6Ax2lacqsogZNnliTr7cfW1c6quosSRkrOl7fQ9rgKgFIgL2Qpz/ikAQMDQNOX4aaVA3PyTU+AE15Re4w/fKdecurSn8r3F4+LwUaNGYcCAAS09Hp+RmZmJzMxMWK1Wbw+FiMhFs0BJ63HygDoBXZckmI8fgr6HssKjNUYqJ6wWyDWVauCkCY+GXFGirjIJWVZaGNgJGdbyImhCwpUVLAB+KSNgPnkElvyTsBblAeZGSAZ/BAy6AXVbPoTZ1hyzyRY4acIiIVeVwewUODWd+B4AYCnOh5BlSBqf3g3OsxWnjIyMDh00AaxxIiLf5fIUnd7Qqba7oEsXPOZ3MPRLRfC4ewEo/340IeEAlHSdPVVnT51ZbcGSXGsCbPvZaaPilXMlZ9VaJyk4DIbeyt6vlvyf1CfqdF17QN+9r3K84DTkumqYTx8FAARed4vteqVAXJib1HOwmNWn/3yZ2/83JTc316MPMBqN3LaEiKglOQVObEVAF2PoNQDR8//pckwTHgW5ugLWyhJ1xUnfsx8asrepq0z2J+qk4DDouiTCWnoWlpICaBqUrVd0UV3Uhpjm/FNqIbiuWy9owqOhCTFCrq5E/bdfAFYLNCFG+F97PWo++bdaRG7OPQbY0n8AYCnOgzbSt/s0uh04de/e/ZJvLkkSFixYgPnz51/y1xIR0fk5p+q4Tx15QmuMgiX3OKwVJbCW21acbKk8UV8D0dSo9nDShkVCG21bcSotgLA9EaeNjleaaeoNgLkRDft2AAD0Cb2V7V4SeqMpZy9qtyq1z/qe/aDr1ku5T0k+5MZ6NB137elkKcqD31WDAAByQx1K5k+GoU8ywh/2nTjC7cBJluXWHAcREbnJNVXHFSe6dFpbgbhcUaKm6nQJvZXVTIsZVlO5WuukMUZCG9UFgK0zuD1wioqHpNVB1yUJltzjMP+k1CrpbcGRPvEKNOXsVY8bevaHJjRCXYmynD2FJts5SBIgBKzFeeoYm44dhCX3GKzFee0zcOrRo4dHefRZs2ZhxowZl/x13sbicCLyWTquONHlsfdyMp8+ptQxSRK04dHQhkbAWl4EuaoM1iolVacNi4LOHjiVFECuVwInnW0VSt+1p9J2wEanBk59XD5T32uAshLVrReajmTDkveTI6i6ahCacvbC4hQ42VN/oqEOckMdNP6BLf598ITbgdM777zj0Qd4kuLzBdzkl4h8lfOKE1sRkCe04Ur3cPsTbRpjlFI0boyEtbwI1soyyLZUncYpVWcpLYCmoVa5R4xyTNe1p3pfyT9QXZ3SJ13h8pmGXv2V4117oulItrKidPYUACBw+FglcCpyBE4Wp+1b5Kry9hc4jRo1qjXHQUREbnLuHM7tVsgT9lSd/Qk5bWQcACVIApQmmPZ+TlrjOTVOddXKcXuA1M0ROOm69lTbCei69VJTcJqIGHWPPPuKVN23nyv3iekGfS/lSX2ry4qTI3CyVpZCF9utZSZ/mXy7WQIRETXjssqkZeBEl05jW3Gy00YoT7JpQ21tCqrK1afqNMYoJejRaJXeT9WVABypOp3tyTpAKQxXP8MvANrYBABKfZN6jS3QErUm5VzvAdDFdHV8bkMdhCy7NMmUfWg7FgZORETtDVN1dJnsK07q+3NWnKxV5Y6n6oyRkLQ6NbgCACkgGJogpdWQLi4JkGyrTAm9XO5rsHUWN/RJVo/puvV2vabXAGiCQiHZ7mctzoe1OF/d2w5g4NQuZGZmol+/fhg8eLC3h0JE5MK5IJyBE3lCe+6KU5RtxcmeqqsqczxVZzumje7iuN622gQo/wZ1XbsDAPSJV7rcN/R3MxBy96MIGjNRPaYJi4AU7Kgd1vdW0nT2VJylOA/mvBMu97EHcb6AgdMFsHM4Efkql7omNsAkD0gGf0iBIep7nbriFAFA2ZdONlUAUHo+AY6aJsCRprMzPvIsQu+bDb/+rosNuthuCL17mkthtyRJ0NsLyjVadVVKF6METtaiPHWrFjuuOBERkeeYqqMWYA+IgOapOkveT4CQAUkDja3uyTlYcl59AgC/K1IQcttkt/eZs6f09Il9IBn8lXva6pwsxXnqE3VqUXpV6Xnu4h0MnIiI2hmJfZyoBTgXiKvF4fZUXbWy2qQJNULSaJVzUedP1XnCf8CvlP+9ZqR6zL7iZCnOV5+o80tWrpMry+ErGDgREbUzrluuMFVHnlELxDVaNYjShEa4XKMJcwqunIIlXdRlBk5D0xCzdB1CfvMHx/3tNU75J2EpOA0A8BswFADU1gi+wO0+TkRE5CO0Tj+62ceJPKTWLoVHq6tKmlCj8oSckG3XRKrX61pwxcmlzsl+f3uNk62XkxQYAkNPZf88mak6IiLylKTRqMETa5zIU5oIZcXJXt8EAJJGqwRP9mvCHIGTNqoLoPcDNFo1yGlJ2shYpVeUjT6hFzS24E40NkC27ZHnbVxxIiJqhyS9AcJqYY0Tecyv7yBAb4BfyjCX45rQCMhVSk2RcwG5pDcg8s/LIcyN0ASHtvh4JJ0e2qg4WIvzASibDmv8AyH5BUA01kOuLIUmLrHFP/dSMXC6AG7yS0Q+zVbbxBUn8pShV3/Ev/VNs39D2rBI9ak2jVOqDgD8rx7RqmPSxXRTAye9rVGmJiwS1uI8WKvKoPOBwImpugtgHyci8mXqLzsWh9NlOF/g7ZKeC4tsdr41aZ1SgPrEPsox26qXr/RyYuBERNQOSVxxolbi/GSd81N1bUEX29Xx2rYZsNqU00e6hzNwIiJqh9TAiStO1MK0xgin195ZcdIYo9QNh523gfEFrHEiImqP1FQdV5yoZbmuOLVt4OTXfwh0Cb0R8KubHGOwpeqslb7RkoCBExFRO2R/mk5iHydqYWpdk1YHjdNmvG3y2aHhiF3y4TnjUQI5+5N+3sZUHRFRO8QaJ2ot9n3otJFxbu8915rUFSem6oiIyFO6uEQ0HTsAXWyCt4dCHYw+oQ+MD82DrlvPi1/cBuzpQpmpOiIi8pTx4XkIHj8V+vju3h4KdUBBaXd7ewgqR3F4OYQQkCTJq+Px/hqcj8rMzES/fv0wePBgbw+FiKgZSW9g0ESdgr0dgWhqgKiv9fJoGDhdEBtgEhEReZ/GPxCSfyAA32hJwMCJiIiIfJq9zskXCsQZOBEREZFPszfilH2gezgDJyIiIvJpXHEiIiIicpO60a8PtCRg4EREREQ+Td3o1we6hzNwIiIiIp/m6OXEFSciIiKiX+TY6Jc1TkRERES/yLl7uLcxcCIiIiKf5niqrhRCCO+OxaufTkRERHQRWmMkoNNDGxoB0Vjv1bFwk98LyMzMRGZmJqxWq7eHQkRE1KlJBn/E/3uP1zf4BQBJeHvNy8eZTCaEhYWhqqoKoaGh3h4OERERtbBL+V3PVB0RERGRmxg4EREREbmJgRMRERGRmxg4EREREbmJgRMRERGRmxg4EREREbmJgRMRERGRmxg4EREREbmJgRMRERGRmxg4EREREbmJe9VdhH1HGpPJ5OWREBERUWuw/453Zxc6Bk4XUV1dDQBISEjw8kiIiIioNVVXVyMsLOwXr+EmvxchyzLOnj2LkJCQFt+V2WQyISEhAWfOnOlUGwhz3px3Z9AZ590Z5wxw3h1h3kIIVFdXIz4+HhrNL1cxccXpIjQaDbp169aqnxEaGtru/9F5gvPuXDjvzqMzzhngvNu7i6002bE4nIiIiMhNDJyIiIiI3MTAyYv8/PywYMEC+Pn5eXsobYrz5rw7g8447844Z4Dz7mzzZnE4ERERkZu44kRERETkJgZORERERG5i4ERERETkJgZOXpSZmYnu3bvD398fQ4cOxXfffeftIbWYRYsWYfDgwQgJCUFMTAzGjx+Po0ePulzT0NCAjIwMREZGIjg4GL/5zW9QVFTkpRG3jhdffBGSJGHWrFnqsY467/z8fNx3332IjIxEQEAAkpOTsXfvXvW8EALz589Hly5dEBAQgLS0NBw/ftyLI758VqsV8+bNQ48ePRAQEIBevXph4cKFLts2dIR5b9++Hbfffjvi4+MhSRI++ugjl/PuzLG8vByTJk1CaGgojEYjpk6dipqamjacxaX7pXmbzWY88cQTSE5ORlBQEOLj4/HAAw/g7NmzLvdob/O+2N+1s2nTpkGSJLz88ssux9vbnC8VAycvWbNmDWbPno0FCxZg3759SElJwZgxY1BcXOztobWIbdu2ISMjA7t27cLmzZthNptx8803o7a2Vr3msccew4YNG/DBBx9g27ZtOHv2LNLT07046pa1Z88e/OMf/8DAgQNdjnfEeVdUVGDEiBHQ6/X47LPPkJOTg6VLlyI8PFy9ZvHixXjllVewcuVK7N69G0FBQRgzZgwaGhq8OPLL89JLL2HFihX4+9//jiNHjuCll17C4sWL8eqrr6rXdIR519bWIiUlBZmZmec9784cJ02ahB9++AGbN2/Gxo0bsX37djzyyCNtNQWP/NK86+rqsG/fPsybNw/79u3D2rVrcfToUdxxxx0u17W3eV/s79pu3bp12LVrF+Lj45uda29zvmSCvGLIkCEiIyNDfW+1WkV8fLxYtGiRF0fVeoqLiwUAsW3bNiGEEJWVlUKv14sPPvhAvebIkSMCgNi5c6e3htliqqurRZ8+fcTmzZvFqFGjxMyZM4UQHXfeTzzxhLjuuusueF6WZREXFyeWLFmiHqusrBR+fn7iv//9b1sMsVXceuut4sEHH3Q5lp6eLiZNmiSE6JjzBiDWrVunvndnjjk5OQKA2LNnj3rNZ599JiRJEvn5+W029stx7rzP57vvvhMAxOnTp4UQ7X/eF5pzXl6e6Nq1q/j+++9FUlKSWL58uXquvc/ZHVxx8oKmpiZkZ2cjLS1NPabRaJCWloadO3d6cWStp6qqCgAQEREBAMjOzobZbHb5HvTt2xeJiYkd4nuQkZGBW2+91WV+QMed98cff4zU1FRMmDABMTExuOaaa/DGG2+o50+ePInCwkKXeYeFhWHo0KHtet7Dhw/Hli1bcOzYMQDAwYMH8fXXX2PcuHEAOu68nbkzx507d8JoNCI1NVW9Ji0tDRqNBrt3727zMbeWqqoqSJIEo9EIoGPOW5Zl3H///ZgzZw769+/f7HxHnPO5uFedF5SWlsJqtSI2NtbleGxsLH788Ucvjar1yLKMWbNmYcSIERgwYAAAoLCwEAaDQf0BYxcbG4vCwkIvjLLlrF69Gvv27cOePXuaneuo8/7555+xYsUKzJ49G08//TT27NmDGTNmwGAwYPLkyerczvdvvj3P+8knn4TJZELfvn2h1WphtVrx/PPPY9KkSQDQYeftzJ05FhYWIiYmxuW8TqdDREREh/k+NDQ04IknnsDEiRPVfds64rxfeukl6HQ6zJgx47znO+Kcz8XAiVpdRkYGvv/+e3z99dfeHkqrO3PmDGbOnInNmzfD39/f28NpM7IsIzU1FS+88AIA4JprrsH333+PlStXYvLkyV4eXet5//33sWrVKvznP/9B//79ceDAAcyaNQvx8fEdet7kymw245577oEQAitWrPD2cFpNdnY2/va3v2Hfvn2QJMnbw/Eapuq8ICoqClqtttmTVEVFRYiLi/PSqFrH9OnTsXHjRmzduhXdunVTj8fFxaGpqQmVlZUu17f370F2djaKi4tx7bXXQqfTQafTYdu2bXjllVeg0+kQGxvbIefdpUsX9OvXz+XYVVddhdzcXABQ59bR/s3PmTMHTz75JH73u98hOTkZ999/Px577DEsWrQIQMedtzN35hgXF9fswReLxYLy8vJ2/32wB02nT5/G5s2b1dUmoOPNe8eOHSguLkZiYqL68+306dP485//jO7duwPoeHM+HwZOXmAwGDBo0CBs2bJFPSbLMrZs2YJhw4Z5cWQtRwiB6dOnY926dfjqq6/Qo0cPl/ODBg2CXq93+R4cPXoUubm57fp7MHr0aBw+fBgHDhxQ/6SmpmLSpEnq64447xEjRjRrN3Hs2DEkJSUBAHr06IG4uDiXeZtMJuzevbtdz7uurg4ajeuPUa1WC1mWAXTceTtzZ47Dhg1DZWUlsrOz1Wu++uoryLKMoUOHtvmYW4o9aDp+/Di+/PJLREZGupzvaPO+//77cejQIZefb/Hx8ZgzZw6++OILAB1vzufl7er0zmr16tXCz89PvPPOOyInJ0c88sgjwmg0isLCQm8PrUU8+uijIiwsTGRlZYmCggL1T11dnXrNtGnTRGJiovjqq6/E3r17xbBhw8SwYcO8OOrW4fxUnRAdc97fffed0Ol04vnnnxfHjx8Xq1atEoGBgeK9995Tr3nxxReF0WgU69evF4cOHRJ33nmn6NGjh6ivr/fiyC/P5MmTRdeuXcXGjRvFyZMnxdq1a0VUVJR4/PHH1Ws6wryrq6vF/v37xf79+wUAsWzZMrF//3716TF35jh27FhxzTXXiN27d4uvv/5a9OnTR0ycONFbU3LLL827qalJ3HHHHaJbt27iwIEDLj/nGhsb1Xu0t3lf7O/6XOc+VSdE+5vzpWLg5EWvvvqqSExMFAaDQQwZMkTs2rXL20NqMQDO++ftt99Wr6mvrxd//OMfRXh4uAgMDBR33XWXKCgo8N6gW8m5gVNHnfeGDRvEgAEDhJ+fn+jbt694/fXXXc7LsizmzZsnYmNjhZ+fnxg9erQ4evSol0bbMkwmk5g5c6ZITEwU/v7+omfPnmLu3Lkuvzg7wry3bt163v+eJ0+eLIRwb45lZWVi4sSJIjg4WISGhoopU6aI6upqL8zGfb8075MnT17w59zWrVvVe7S3eV/s7/pc5wuc2tucL5UkhFOLWyIiIiK6INY4EREREbmJgRMRERGRmxg4EREREbmJgRMRERGRmxg4EREREbmJgRMRERGRmxg4EREREbmJgRMRERGRmxg4EREREbmJgRMRdUq///3vMX78eG8Pg4jaGQZORERERG5i4EREHdr//vc/JCcnIyAgAJGRkUhLS8OcOXPw7rvvYv369ZAkCZIkISsrCwBw5swZ3HPPPTAajYiIiMCdd96JU6dOqfezr1T95S9/QXR0NEJDQzFt2jQ0NTV5Z4JE1KZ03h4AEVFrKSgowMSJE7F48WLcddddqK6uxo4dO/DAAw8gNzcXJpMJb7/9NgAgIiICZrMZY8aMwbBhw7Bjxw7odDr89a9/xdixY3Ho0CEYDAYAwJYtW+Dv74+srCycOnUKU6ZMQWRkJJ5//nlvTpeI2gADJyLqsAoKCmCxWJCeno6kpCQAQHJyMgAgICAAjY2NiIuLU69/7733IMsy3nzzTUiSBAB4++23YTQakZWVhZtvvhkAYDAY8NZbbyEwMBD9+/fHc889hzlz5mDhwoXQaLiQT9SR8b9wIuqwUlJSMHr0aCQnJ2PChAl44403UFFRccHrDx48iBMnTiAkJATBwcEIDg5GREQEGhoa8NNPP7ncNzAwUH0/bNgw1NTU4MyZM606HyLyPq44EVGHpdVqsXnzZnz77bfYtGkTXn31VcydOxe7d+8+7/U1NTUYNGgQVq1a1excdHR0aw+XiNoBBk5E1KFJkoQRI0ZgxIgRmD9/PpKSkrBu3ToYDAZYrVaXa6+99lqsWbMGMTExCA0NveA9Dx48iPr6egQEBAAAdu3aheDgYCQkJLTqXIjI+5iqI6IOa/fu3XjhhRewd+9e5ObmYu3atSgpKcFVV12F7t2749ChQzh69ChKS0thNpsxadIkREVF4c4778SOHTtw8uRJZGVlYcaMGcjLy1Pv29TUhKlTpyInJweffvopFixYgOnTp7O+iagT4IoTEXVYoaGh2L59O15++WWYTCYkJSVh6dKlGDduHFJTU5GVlYXU1FTU1NRg69atuOGGG7B9+3Y88cQTSE9PR3V1Nbp27YrRo0e7rECNHj0affr0wfXXX4/GxkZMnDgRzz77rPcmSkRtRhJCCG8Pgoiovfj973+PyspKfPTRR94eChF5AdeViYiIiNzEwImIiIjITUzVEREREbmJK05EREREbmLgREREROQmBk5EREREbmLgREREROQmBk5EREREbmLgREREROQmBk5EREREbmLgREREROQmBk5EREREbvr/+w1bVc0rxLMAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# viz: distance to the optimum, plain GD vs momentum, same learning rate\n",
    "plt.figure(figsize=(6, 4))\n",
    "plt.plot(n_plain, label=\"plain GD\", color=\"#1E40FF\")\n",
    "plt.plot(n_mom, label=\"momentum β=0.9\", color=\"#E4572E\")\n",
    "plt.yscale(\"log\"); plt.xlabel(\"step\"); plt.ylabel(\"‖w‖ (distance to optimum, log)\")\n",
    "plt.legend(); plt.title(\"momentum on an ill-conditioned quadratic (κ=50)\")\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87b87b8d",
   "metadata": {},
   "source": [
    "> **Interpretation.** At the same learning rate momentum reaches the optimum roughly twice as fast and ends an order of magnitude closer. On a well-conditioned problem the gap would be small; the worse the conditioning, the more momentum is worth. The Distill explorable \"Why Momentum Really Works\" (Goh, 2017) is the canonical visual on this.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fb4db59",
   "metadata": {},
   "source": [
    "Now assemble the full toolkit in one cell. Each optimizer is a small class with a `step(theta, grad)` method; building them as standalone tested classes (rather than editing one class in place) avoids the stale-definition trap. **Adam** combines momentum with a per-parameter adaptive step size, tracking the first moment (mean of gradients) and second moment (mean of squared gradients):\n",
    "\n",
    "$$\\mathbf{m} \\leftarrow \\beta_1\\mathbf{m} + (1-\\beta_1)\\nabla J, \\quad \\mathbf{v} \\leftarrow \\beta_2\\mathbf{v} + (1-\\beta_2)(\\nabla J)^2, \\quad \\boldsymbol\\theta \\leftarrow \\boldsymbol\\theta - \\eta\\,\\frac{\\hat{\\mathbf{m}}}{\\sqrt{\\hat{\\mathbf{v}}}+\\epsilon},$$\n",
    "\n",
    "with bias-corrected $\\hat{\\mathbf{m}} = \\mathbf{m}/(1-\\beta_1^t)$ and $\\hat{\\mathbf{v}} = \\mathbf{v}/(1-\\beta_2^t)$. The bias correction matters most on the first few steps, when `m` and `v` are still near their zero initialization. **AdamW** differs from Adam in one line: it applies weight decay directly to the parameters rather than folding it into the gradient, so the decay does not get rescaled by the adaptive denominator.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "c9e1f084",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.674017Z",
     "iopub.status.busy": "2026-06-10T18:46:10.673942Z",
     "iopub.status.idle": "2026-06-10T18:46:10.679899Z",
     "shell.execute_reply": "2026-06-10T18:46:10.679645Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "toolkit: ['GD', 'Momentum', 'Nesterov', 'RMSProp', 'Adam', 'AdamW']\n"
     ]
    }
   ],
   "source": [
    "class GD:\n",
    "    def __init__(self, lr=0.1):\n",
    "        self.lr = lr  # the only knob: step size\n",
    "    def step(self, theta, grad):\n",
    "        return theta - self.lr * grad\n",
    "\n",
    "class Momentum:\n",
    "    def __init__(self, lr=0.01, beta=0.9):\n",
    "        self.lr, self.beta, self.v = lr, beta, None\n",
    "    def step(self, theta, grad):\n",
    "        if self.v is None:\n",
    "            self.v = np.zeros_like(theta)\n",
    "        self.v = self.beta * self.v - self.lr * grad\n",
    "        return theta + self.v\n",
    "\n",
    "class Nesterov:\n",
    "    def __init__(self, lr=0.01, beta=0.9):\n",
    "        self.lr, self.beta, self.v = lr, beta, None\n",
    "    def step(self, theta, grad):\n",
    "        if self.v is None:\n",
    "            self.v = np.zeros_like(theta)\n",
    "        v_prev = self.v\n",
    "        self.v = self.beta * self.v - self.lr * grad      # look-ahead correction form\n",
    "        return theta - self.beta * v_prev + (1 + self.beta) * self.v\n",
    "\n",
    "class RMSProp:\n",
    "    def __init__(self, lr=0.01, rho=0.9, eps=1e-8):\n",
    "        self.lr, self.rho, self.eps, self.s = lr, rho, eps, None\n",
    "    def step(self, theta, grad):\n",
    "        if self.s is None:\n",
    "            self.s = np.zeros_like(theta)\n",
    "        self.s = self.rho * self.s + (1 - self.rho) * grad ** 2   # running mean square\n",
    "        return theta - self.lr * grad / (np.sqrt(self.s) + self.eps)\n",
    "\n",
    "class Adam:\n",
    "    def __init__(self, lr=0.05, beta1=0.9, beta2=0.999, eps=1e-8):\n",
    "        self.lr, self.beta1, self.beta2, self.eps = lr, beta1, beta2, eps\n",
    "        self.m = self.v = None\n",
    "        self.t = 0\n",
    "    def step(self, theta, grad):\n",
    "        if self.m is None:\n",
    "            self.m = np.zeros_like(theta); self.v = np.zeros_like(theta)\n",
    "        self.t += 1\n",
    "        self.m = self.beta1 * self.m + (1 - self.beta1) * grad\n",
    "        self.v = self.beta2 * self.v + (1 - self.beta2) * grad ** 2\n",
    "        m_hat = self.m / (1 - self.beta1 ** self.t)       # bias correction\n",
    "        v_hat = self.v / (1 - self.beta2 ** self.t)\n",
    "        return theta - self.lr * m_hat / (np.sqrt(v_hat) + self.eps)\n",
    "\n",
    "class AdamW(Adam):\n",
    "    def __init__(self, lr=0.05, weight_decay=0.01, **kw):\n",
    "        super().__init__(lr=lr, **kw)\n",
    "        self.weight_decay = weight_decay\n",
    "    def step(self, theta, grad):\n",
    "        theta = theta - self.lr * self.weight_decay * theta  # decoupled weight decay\n",
    "        return super().step(theta, grad)\n",
    "\n",
    "print(\"toolkit:\", [c.__name__ for c in (GD, Momentum, Nesterov, RMSProp, Adam, AdamW)])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4b1ead5",
   "metadata": {},
   "source": [
    "The strongest possible check on a hand-written optimizer is that it reproduces a real one. Run our `Adam` and PyTorch's `torch.optim.Adam` on the *same* quadratic from the *same* start with the *same* hyperparameters, and assert the parameter trajectories match step for step. If they agree, our update rule is the same computation.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "1f72d21a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:10.680666Z",
     "iopub.status.busy": "2026-06-10T18:46:10.680598Z",
     "iopub.status.idle": "2026-06-10T18:46:11.346728Z",
     "shell.execute_reply": "2026-06-10T18:46:11.346174Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] our Adam matches torch.optim.Adam over 60 steps (max diff 5.06e-07)\n"
     ]
    }
   ],
   "source": [
    "# reference-equivalence check: our Adam vs torch.optim.Adam, identical setup\n",
    "def torch_adam_path(n_steps):\n",
    "    w = torch.tensor([[3.0], [3.0]], requires_grad=True)\n",
    "    opt = torch.optim.Adam([w], lr=0.05, betas=(0.9, 0.999), eps=1e-8)\n",
    "    A = torch.tensor([[A_COND], [B_COND]])\n",
    "    path = []\n",
    "    for _ in range(n_steps):\n",
    "        opt.zero_grad()\n",
    "        loss = 0.5 * (A * w ** 2).sum()      # same L = 0.5*(a w0^2 + b w1^2)\n",
    "        loss.backward()\n",
    "        opt.step()\n",
    "        path.append(w.detach().numpy().copy())\n",
    "    return np.array(path)\n",
    "\n",
    "def ours_adam_path(n_steps):\n",
    "    w = np.array([[3.0], [3.0]])\n",
    "    opt = Adam(lr=0.05)\n",
    "    path = []\n",
    "    for _ in range(n_steps):\n",
    "        grad = np.array([[A_COND], [B_COND]]) * w   # d/dw of 0.5*a*w^2 = a*w\n",
    "        w = opt.step(w, grad)\n",
    "        path.append(w.copy())\n",
    "    return np.array(path)\n",
    "\n",
    "n = 60\n",
    "ours, ref = ours_adam_path(n), torch_adam_path(n)\n",
    "check_close(ours, ref, atol=1e-5, msg=\"our Adam should match torch.optim.Adam step for step\")\n",
    "print(f\"[ ok ] our Adam matches torch.optim.Adam over {n} steps (max diff {abs(ours-ref).max():.2e})\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66ae9653",
   "metadata": {},
   "source": [
    "> **Interpretation.** The two trajectories agree to about `1e-6`. The small residual is float ordering, not a logic difference. Having a real optimizer as the oracle is the difference between \"I think my Adam is right\" and \"my Adam is right.\"\n",
    "\n",
    "> **Key takeaways.** Momentum accumulates a velocity to power through ill-conditioned valleys. Adam adds a per-parameter adaptive step with bias correction; AdamW decouples weight decay in one line. The honest test of a from-scratch optimizer is element-wise agreement with a library reference, not a passing loss.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "331082d7",
   "metadata": {},
   "source": [
    "## Part 6 — Logistic and softmax regression from scratch\n",
    "\n",
    "> **Objectives.** Reuse the exact linear core from Parts 1-5 with two new losses. Build a numerically stable softmax, derive the cross-entropy gradient, train softmax regression on iris by gradient descent, and check the class predictions against scikit-learn. This is the consolidation build: the micro-pieces assemble into the chapter's trainable classifier.\n",
    "\n",
    "Logistic regression keeps the linear function and changes the loss. The model outputs a probability through the **sigmoid** $\\hat{p} = \\sigma(\\boldsymbol\\theta^T\\mathbf{x}) = 1/(1+e^{-\\boldsymbol\\theta^T\\mathbf{x}})$, trained against **binary cross-entropy**. The gradient comes out to the same clean form as MSE: $\\frac{1}{m}\\mathbf{X}^T(\\hat{p} - y)$. For $K$ classes the sigmoid generalizes to the **softmax**\n",
    "\n",
    "$$\\hat{p}_k = \\frac{e^{s_k}}{\\sum_j e^{s_j}}, \\qquad s_k = \\boldsymbol\\theta_k^T\\mathbf{x},$$\n",
    "\n",
    "which turns a vector of scores into a probability distribution over classes. We start with the piece that bites first: a softmax that does not overflow.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3356b468",
   "metadata": {},
   "source": [
    "### Exercise 4.5 — A numerically stable softmax\n",
    "`Difficulty 2/5 · ~10 min`\n",
    "\n",
    "Fill in `softmax(logits)` over the last axis. The trick is to subtract the per-row max before exponentiating: softmax is shift-invariant, so this changes nothing mathematically but stops `np.exp` from overflowing on large logits. The checks assert rows sum to 1, that shifting the input by a huge constant leaves the output unchanged, and that there is no overflow on logits of size 1000.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "c04a0856",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:11.347719Z",
     "iopub.status.busy": "2026-06-10T18:46:11.347566Z",
     "iopub.status.idle": "2026-06-10T18:46:11.352340Z",
     "shell.execute_reply": "2026-06-10T18:46:11.352011Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] 4.5 rows sum to 1: not attempted yet — fill in the TODO above, then re-run.\n",
      "[ -- ] 4.5 shift-invariant: not attempted yet — fill in the TODO above, then re-run.\n",
      "[ -- ] 4.5 no overflow: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def softmax(logits):\n",
    "    \"\"\"Numerically stable softmax along the last axis. logits: (..., K) -> (..., K).\"\"\"\n",
    "    # TODO 1: subtract the per-row max: logits - logits.max(axis=-1, keepdims=True)\n",
    "    shifted = None\n",
    "    attempted(shifted)\n",
    "    # TODO 2: exponentiate the shifted logits, then divide by the per-row sum\n",
    "    exp = np.exp(shifted)\n",
    "    return exp / exp.sum(axis=-1, keepdims=True)\n",
    "\n",
    "def _softmax_sums_to_one():\n",
    "    P = softmax(rng.standard_normal((5, 3)))\n",
    "    check_close(P.sum(axis=-1), np.ones(5), msg=\"each row of softmax is a distribution\")\n",
    "\n",
    "def _softmax_shift_invariant():\n",
    "    z = np.array([[1.0, 2.0, 3.0]])\n",
    "    check_close(softmax(z), softmax(z + 1000.0),\n",
    "                msg=\"softmax is shift-invariant; subtracting the max must not change it\")\n",
    "\n",
    "def _softmax_no_overflow():\n",
    "    P = softmax(np.array([[1000.0, 1000.0, 1000.0]]))   # naive exp(1000) overflows\n",
    "    assert np.isfinite(P).all(), \"stable softmax must not overflow on large logits\"\n",
    "\n",
    "check(\"4.5 rows sum to 1\", _softmax_sums_to_one)\n",
    "check(\"4.5 shift-invariant\", _softmax_shift_invariant)\n",
    "check(\"4.5 no overflow\", _softmax_no_overflow)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a20af61a",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>Compute the maximum along the last axis with `keepdims=True` so it broadcasts, subtract it from `logits`, then exponentiate and normalize.</details>\n",
    "\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "shifted = logits - logits.max(axis=-1, keepdims=True)\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — \"overflow encountered in exp\" or NaN rows</summary>You skipped the max-subtraction, so `np.exp` saw a number like 1000 and returned `inf`, and `inf / inf` is `NaN`. Subtract the per-row max first; the result is identical mathematically because softmax ignores additive shifts.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "a1745bd6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:11.353448Z",
     "iopub.status.busy": "2026-06-10T18:46:11.353368Z",
     "iopub.status.idle": "2026-06-10T18:46:11.356375Z",
     "shell.execute_reply": "2026-06-10T18:46:11.356073Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 4.5 rows sum to 1\n",
      "[ ok ] 4.5 shift-invariant\n",
      "[ ok ] 4.5 no overflow\n",
      "softmax([2,1,0]): [0.66524096 0.24472847 0.09003057]\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines softmax; the checks below re-verify the reference.\n",
    "def softmax(logits):\n",
    "    shifted = logits - logits.max(axis=-1, keepdims=True)\n",
    "    exp = np.exp(shifted)\n",
    "    return exp / exp.sum(axis=-1, keepdims=True)\n",
    "\n",
    "check(\"4.5 rows sum to 1\", _softmax_sums_to_one, required=True)\n",
    "check(\"4.5 shift-invariant\", _softmax_shift_invariant, required=True)\n",
    "check(\"4.5 no overflow\", _softmax_no_overflow, required=True)\n",
    "print(\"softmax([2,1,0]):\", softmax(np.array([[2.0, 1.0, 0.0]])).ravel())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2b41eec",
   "metadata": {},
   "source": [
    "> **Interpretation.** The largest logit gets the largest probability, all three are positive, and they sum to 1. The max-subtraction is mathematically a no-op (it cancels in the ratio) but numerically essential; every production softmax does it.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7be7210",
   "metadata": {},
   "source": [
    "Now train softmax regression on iris: three classes, four features, standardized. The gradient for class $k$ is the same structure as everything in this notebook: $\\frac{1}{m}\\mathbf{X}^T(\\hat{\\mathbf{P}} - \\mathbf{Y})$, where $\\mathbf{Y}$ is the one-hot label matrix. We standardize the features first because, exactly as in Part 3, unscaled features make the loss surface lopsided and slow the descent.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "a190c684",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:11.357203Z",
     "iopub.status.busy": "2026-06-10T18:46:11.357128Z",
     "iopub.status.idle": "2026-06-10T18:46:11.374955Z",
     "shell.execute_reply": "2026-06-10T18:46:11.374642Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Xb_iris (150, 5), Y_onehot (150, 3), classes [0 1 2]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "iris = load_iris()\n",
    "X_iris = StandardScaler().fit_transform(iris.data)   # 4 features, standardized\n",
    "y_iris = iris.target                                 # labels in {0, 1, 2}\n",
    "Xb_iris = np.c_[np.ones((len(X_iris), 1)), X_iris]   # prepend bias -> (150, 5)\n",
    "K = 3\n",
    "Y_onehot = np.eye(K)[y_iris]                          # (150, 3) one-hot targets\n",
    "print(f\"Xb_iris {Xb_iris.shape}, Y_onehot {Y_onehot.shape}, classes {np.unique(y_iris)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aff8b3b0",
   "metadata": {},
   "source": [
    "### Exercise 4.6 — Softmax regression by gradient descent\n",
    "`Difficulty 3/5 · ~20 min`\n",
    "\n",
    "Fill in `softmax_regression(Xb, Y, n_steps, eta)`. Initialize `Theta` of shape `(d+1, K)` at zeros, and each step: compute logits `Xb @ Theta`, turn them into probabilities `P` with your `softmax`, form the gradient `Xb.T @ (P - Y) / m`, and step `Theta`. Return `(Theta, losses)` where `losses` is the per-step cross-entropy. The check trains it and asserts the resulting class predictions agree with scikit-learn's softmax fit on at least 95% of iris.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "b9d97606",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:11.375936Z",
     "iopub.status.busy": "2026-06-10T18:46:11.375857Z",
     "iopub.status.idle": "2026-06-10T18:46:11.380385Z",
     "shell.execute_reply": "2026-06-10T18:46:11.380039Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] 4.6 softmax reg == sklearn: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def cross_entropy(P, Y):\n",
    "    \"\"\"Mean categorical cross-entropy. P, Y: (m, K). Clipped for log stability.\"\"\"\n",
    "    return float(-(Y * np.log(np.clip(P, 1e-12, 1.0))).sum(axis=1).mean())\n",
    "\n",
    "def softmax_regression(Xb, Y, n_steps=5000, eta=0.5):\n",
    "    \"\"\"Softmax regression via full-batch GD. Returns (Theta of shape (d+1,K), losses).\"\"\"\n",
    "    m, n_features = Xb.shape\n",
    "    K = Y.shape[1]\n",
    "    Theta = np.zeros((n_features, K))\n",
    "    losses = []\n",
    "    for _ in range(n_steps):\n",
    "        # TODO 1: logits = Xb @ Theta            # (m, K)\n",
    "        logits = None\n",
    "        attempted(logits)\n",
    "        P = softmax(logits)                       # (m, K)\n",
    "        # TODO 2: gradient = Xb.T @ (P - Y) / m   # (d+1, K)\n",
    "        gradient = Xb.T @ (P - Y) / m\n",
    "        Theta = Theta - eta * gradient\n",
    "        losses.append(cross_entropy(P, Y))\n",
    "    return Theta, losses\n",
    "\n",
    "def _softmax_reg_matches_sklearn():\n",
    "    from sklearn.linear_model import LogisticRegression\n",
    "    Theta, _ = softmax_regression(Xb_iris, Y_onehot, n_steps=SOFTMAX_STEPS, eta=0.5)\n",
    "    ours = np.argmax(Xb_iris @ Theta, axis=1)\n",
    "    # unregularized softmax reference: large C disables sklearn's default penalty\n",
    "    ref = LogisticRegression(C=1e4, max_iter=5000).fit(X_iris, y_iris).predict(X_iris)\n",
    "    agree = (ours == ref).mean()\n",
    "    assert agree >= 0.95, \\\n",
    "        f\"only {agree:.1%} of predictions match sklearn; expected >=95% on iris\"\n",
    "\n",
    "check(\"4.6 softmax reg == sklearn\", _softmax_reg_matches_sklearn)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "737ed3c0",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>Logits are the raw linear scores `Xb @ Theta`. Pass them through your `softmax` to get `P`. The gradient is the same residual structure as everywhere else: design matrix transposed, times `(predicted - target)`, averaged over examples.</details>\n",
    "\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "logits = Xb @ Theta             # (m, K)\n",
    "P = softmax(logits)             # (m, K)\n",
    "gradient = Xb.T @ (P - Y) / m   # (d+1, K)\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — predictions disagree with sklearn or accuracy is low</summary>Two usual causes. First, you forgot to standardize and ran on raw features (here `Xb_iris` is already standardized, so check you used it). Second, too few steps; softmax regression on iris needs a few thousand GD steps at `eta=0.5` to converge. Increase `n_steps` if the agreement is just under threshold.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "d6d3e9a3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:11.381182Z",
     "iopub.status.busy": "2026-06-10T18:46:11.381100Z",
     "iopub.status.idle": "2026-06-10T18:46:11.766090Z",
     "shell.execute_reply": "2026-06-10T18:46:11.765627Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 4.6 softmax reg == sklearn\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train accuracy 0.980  final cross-entropy 0.0454"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines softmax_regression; the check below re-verifies the reference.\n",
    "def softmax_regression(Xb, Y, n_steps=5000, eta=0.5):\n",
    "    m, n_features = Xb.shape\n",
    "    K = Y.shape[1]\n",
    "    Theta = np.zeros((n_features, K))\n",
    "    losses = []\n",
    "    for _ in range(n_steps):\n",
    "        logits = Xb @ Theta\n",
    "        P = softmax(logits)\n",
    "        gradient = Xb.T @ (P - Y) / m\n",
    "        Theta = Theta - eta * gradient\n",
    "        losses.append(cross_entropy(P, Y))\n",
    "    return Theta, losses\n",
    "\n",
    "check(\"4.6 softmax reg == sklearn\", _softmax_reg_matches_sklearn, required=True)\n",
    "Theta_iris, losses_iris = softmax_regression(Xb_iris, Y_onehot, n_steps=SOFTMAX_STEPS, eta=0.5)\n",
    "pred_iris = np.argmax(Xb_iris @ Theta_iris, axis=1)\n",
    "print(f\"train accuracy {(pred_iris == y_iris).mean():.3f}  final cross-entropy {losses_iris[-1]:.4f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "b9c412ef",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:11.767061Z",
     "iopub.status.busy": "2026-06-10T18:46:11.766943Z",
     "iopub.status.idle": "2026-06-10T18:46:11.840218Z",
     "shell.execute_reply": "2026-06-10T18:46:11.838878Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAGGCAYAAACNCg6xAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAATP5JREFUeJzt3XlYlOX+BvB7WGYAWZVdEdwVF1BUQjNPSZJ5XCqPZp40cjlupVLmlvsxtXI7qZlZ5mnTMrXzUyOVNE1NcyF3yy3QBCQExIVl+P7+GGdkYMBhmHGY8f5c11wz88zzvu93XlFun+eZdxQiIiAiIiKi+3KwdgFEREREtoLBiYiIiMhIDE5ERERERmJwIiIiIjISgxMRERGRkRiciIiIiIzE4ERERERkJAYnIiIiIiMxOBEREREZicGJ7EZiYiIiIyPh4uIChUKB7Oxsa5dEJvjkk0+gUChw6dIla5diMQqFAjNmzDCq74wZM6BQKMx27Ly8PPj7++Pzzz832z6pciZOnIjo6Ghrl0EmYnAiu/DXX3+hb9++cHV1xbJly/Dpp5+iRo0aeOutt7Bp0yZrl0dUbSxZsgQeHh54/vnndW3asGrolpaWVmYf//vf/9CmTRu4uLigbt26mD59OoqKisr0y87OxrBhw+Dn54caNWrg8ccfx5EjRwzWZew+TbF161ajg+qDMHbsWPz666/43//+Z+1SyBRCZAe+++47ASDbt2/Xa69Ro4YMGjTIOkWRSYqKiuT27dtSXFxs7VIs5vbt21JYWGhU38LCQrl9+7ZZjltQUCB+fn7y1ltv6bWvXr1aAMisWbPk008/1buVPvbWrVtFoVDI448/LitXrpRXXnlFHBwcZPjw4Xr91Gq1dOjQQWrUqCEzZsyQpUuXSnh4uHh4eMhvv/1m0j5NNWrUKKluv+769u0rnTp1snYZZILq9ZNEZKI1a9YIAPnll1/02u0tOKnVarP9EjXGzZs3H9ixSF9eXp7Z97lhwwYBIOfOndNr1wan0n9/DAkPD5eIiAi94DdlyhRRKBRy+vRpXdu6desEgHz99de6toyMDPH29pb+/fubtE9TVcfgtH79elEoFHL+/Hlrl0KVVL1+kuihkpubK2PGjJHQ0FBRKpXi5+cnsbGxcvjwYb1+X331lbRp00ZcXFykVq1aMmDAALl8+bLu9c6dOwsAvdugQYPKtGnbRUSmT58uAOTs2bMyYMAA8fT0FF9fX3nzzTeluLhYUlJSpGfPnuLh4SEBAQHy7rvv6tWUn58vU6dOlTZt2oinp6e4ubnJo48+Kj/88INev2nTpolCoZAdO3botQ8dOlScnZ0lOTm5wnMEQEaNGiWfffaZhIeHi5OTk2zcuFFERC5fvizx8fHi7+8vSqVSwsPD5aOPPiqzj0uXLkmPHj3Ezc1N/Pz8ZOzYsZKYmCgAZOfOnXrnsXnz5nLo0CHp1KmTuLq6ypgxY0RE5M6dOzJt2jRp0KCBKJVKqVOnjowfP17u3Lmjd6xt27ZJx44dxcvLS2rUqCGNGzeWSZMm6fX5z3/+I+Hh4eLq6ire3t4SFRUln3/+ue517S/xixcv6m23bNkyCQ8PF6VSKUFBQTJy5Ei5fv26Xh/tezh58qT87W9/E1dXVwkODpb58+dXeJ61CgsLZdasWVK/fn1RKpUSGhoqkyZNKvM+Q0NDpXv37rJnzx5p166dqFQqqVevnqxZs8ao4wCQ6dOn655rfx5Pnjwp/fv3F29vb4mMjNR7rSRjzrMhAwcOlLCwsDLtJYNTbm6uFBUVGdz+5MmTAkCWLVum137lyhUBILNnz9a1/eMf/5CAgABRq9V6fYcNGyZubm66c1qZfRpSUFAgM2bMkIYNG4pKpZKaNWtKx44dZdu2bSIi5f5boKVWq2XRokUSHh4uKpVK/P39ZdiwYZKVlaV3HO2f+ffffy8RERGiUqmkWbNm8s0331SqHq3s7GxRKBSycOHCCt8fVT8MTmQ1L7zwgiiVSklISJBVq1bJ/PnzpUePHvLZZ5/p+mj/QW/Xrp0sWrRIJk6cKK6urhIWFqb7pblt2zYZNmyY3lTDvn375NNPPxWVSiWdOnXSTTvs27dPRO79MoqMjJT+/fvL8uXLpXv37gJAFi5cKE2aNJERI0bI8uXLpWPHjgJAfvzxR11d165dk6CgIElISJD3339f3n77bWnSpIk4OzvL0aNHdf0KCgqkdevWEhoaKrm5uSIiutByv18IIppfsM2aNRM/Pz+ZOXOmLFu2TI4ePSppaWlSp04dCQkJkVmzZsn7778vPXv2FACyaNEi3fZ5eXlSv359cXV1lYkTJ8rixYulffv2EhERYTA4BQYGip+fn7zyyivywQcfyKZNm0StVkvXrl3Fzc1Nxo4dKx988IGMHj1anJycpFevXrrtT5w4IUqlUtq2bStLliyRFStWyOuvvy6PPfaYrs/KlSsFgPTp00c++OADWbJkiQwePFheffXVMn/mJYOT9s8rNjZW3nvvPRk9erQ4OjpKu3btpKCgQO89BAcHS0hIiIwZM0aWL18uTzzxhACQrVu33vd8a3/J9unTR5YtWyYDBw4UANK7d2+9fqGhodKkSRMJCAiQyZMny9KlS6VNmzaiUCjkxIkTRv25GgpO4eHh0qtXL1m+fLkuSJQOTsac5/I0bNhQnn322TLt2nPu7u4uAESpVEqPHj3KTKl99tlnAkAOHDhQZh916tTR23fDhg2lW7duZfqtWrVKAMixY8cqvU9DJk+eLAqFQoYOHSoffvihLFiwQPr37y/z5s0TEZF9+/bJk08+KQD0piC1hgwZIk5OTjJ06FBZsWKFTJgwQWrUqFHmZys0NFQaN24s3t7eMnHiRFm4cKG0bNlSHBwc9ELR/eopqWHDhvLcc89V+P6o+mFwIqvx8vKSUaNGlft6QUGB+Pv7S4sWLfSmpzZv3iwAZNq0abq28qYaypuq0/4yGjZsmK6tqKhI6tSpIwqFQu8fuevXr4urq6vefoqKiiQ/P19vn9evX5eAgAB5+eWX9dqPHz8uSqVShgwZItevX5fatWtL27ZtjVrjAkAcHBzk5MmTeu2DBw+WoKAgyczM1Gt//vnnxcvLS27duiUiIgsWLBAAsmnTJl2f27dvS9OmTQ0GJwCyYsUKvX1++umn4uDgIHv27NFrX7FihQCQvXv3iojIokWLBIBcu3at3PfTq1cvad68eYXvuXRwysjIEKVSKV27dtUbvVi6dKkAkI8//rjMe/jvf/+ra8vPz5fAwMD7/oJKTk4WADJkyBC99tdff10A6I0mhoaGCgDZvXu3ri0jI0NUKpW89tprFR5HpPzgVHoKq+RrWsacZ0MKCwtFoVAYrG/dunXy0ksvyZo1a2Tjxo3y5ptvipubm/j6+kpKSoqu3zvvvCMA9Nq02rVrJ4888ojueY0aNcr8XRAR2bJliwCQxMTESu/TkIiICOnevXuFfcqbqtuzZ48A0BvxFLn3n5uS7do/85IjTDk5ORIUFCStW7euVD1aXbt2lWbNmhnVl6oPfqqOrMbb2xsHDhzAn3/+afD1Q4cOISMjAyNHjoSLi4uuvXv37mjatCm2bNlS5RqGDBmie+zo6Ii2bdtCRDB48GC9Ops0aYILFy7o9VUqlQCA4uJiZGVloaioCG3bti3zqaEWLVpg5syZWLVqFeLi4pCZmYk1a9bAycnJqBo7d+6M8PBw3XMRwTfffIMePXpARJCZmam7xcXFIScnR1dDYmIiateujZ49e+q2d3FxwdChQw0eS6VSIT4+Xq/t66+/RrNmzdC0aVO9Yz3xxBMAgJ07d+rOEwB8++23KC4uNrh/b29vXL58Gb/88otR7x0AduzYgYKCAowdOxYODvf+yRo6dCg8PT3L/By4u7vjn//8p+65UqlE+/bt9f78DNm6dSsAICEhQa/9tddeA4AyxwkPD0enTp10z/38/Mr8nFTW8OHD79vHmPNsSFZWFkQEPj4+ZV7r27cvVq9ejYEDB6J3796YPXs2vv/+e/z111+YM2eOrt/t27cBaH5OSnNxcdG9ru1bXr+S+6rMPg3x9vbGyZMn8fvvv1fYz5Cvv/4aXl5eePLJJ/V+tqOiouDu7q772dYKDg7GM888o3vu6emJgQMH4ujRo7pPH1amHh8fH2RmZla6brIuBieymrfffhsnTpxASEgI2rdvjxkzZuj90vnjjz8AAE2aNCmzbdOmTXWvV0XdunX1nnt5ecHFxQW+vr5l2q9fv67XtmbNGrRq1QouLi6oVasW/Pz8sGXLFuTk5JQ5zvjx4xEREYGDBw9i+vTpekHofurVq6f3/Nq1a8jOzsbKlSvh5+end9OGnoyMDACac9igQYMy1wFq2LChwWPVrl1bFwi1fv/9d5w8ebLMsRo3bqx3rH79+qFjx44YMmQIAgIC8Pzzz+Orr77S++U+YcIEuLu7o3379mjUqBFGjRqFvXv3Vvj+y/s5UCqVqF+/fpmfgzp16pR5vz4+PmX+/Awdx8HBocy5CQwMhLe3d5njlP7ZMfY4FSn9Z22IMee5IiJiVL9HH30U0dHR2LFjh67N1dUVAJCfn1+m/507d3Sva/uW16/kviqzT0NmzZqF7OxsNG7cGC1btsT48eNx7Nix+709AJqf7ZycHPj7+5f5+c7Ly9P9bGs1bNiwzM+W9u+B9rpjlalHRMx6jS56MIz7Ly+RBfTt2xedOnXCxo0bsW3bNrzzzjuYP38+NmzYgG7duj2QGhwdHY1qA/R/4Xz22Wd46aWX0Lt3b4wfPx7+/v5wdHTE3Llzcf78+TLbXrhwQfc/0OPHj1eqxtK/OLS/IP/5z39i0KBBBrdp1apVpY5R3rG0x2vZsiUWLlxocJuQkBDdtrt378bOnTuxZcsWJCYmYt26dXjiiSewbds2ODo6olmzZjh79iw2b96MxMREfPPNN1i+fDmmTZuGmTNnmlRzacb8+VXE2F9kVT2OIfcLCdo+9zvPhtSsWRMKhaJSwS4kJARnz57VPQ8KCgIAXL16VffnrnX16lW0b99er+/Vq1fL7FPbFhwcXOl9GvLYY4/h/Pnz+Pbbb7Ft2zasWrUKixYtwooVK/RGlA0pLi6u8GKgfn5+FW5f1XquX79e5j9pVP1xxImsKigoCCNHjsSmTZtw8eJF1KpVSzc1EBoaCgB6/3BrnT17Vvd6RSz1v7n169ejfv362LBhA1588UXExcUhNjZW97/pkoqLi/HSSy/B09MTkydPxpdffokNGzaYfGw/Pz94eHhArVYjNjbW4M3f3x+A5hyeP3++zC/zc+fOGX28Bg0aICsrC126dDF4rJIjQQ4ODujSpQsWLlyIU6dOYc6cOfjhhx/0pjxq1KiBfv36YfXq1UhJSUH37t0xZ84cg+dO+x6Asj8HBQUFuHjxolE/B8YIDQ1FcXFxmSmW9PR0ZGdnm+045mDMeS7NyckJDRo0wMWLF40+zoULF/TCQ2RkJADNNHpJf/75Jy5fvqx7Xdv3yJEjZUbCDhw4ADc3N91ITWX2WZ6aNWsiPj4eX375JVJTU9GqVSu9C16W9+9AgwYN8Ndff6Fjx44Gf7YjIiL0+p87d67M36XffvsNABAWFmZ0PVoXL15Es2bN7vv+qHphcCKrUKvVZaa0/P39ERwcrBuyb9u2Lfz9/bFixQq9YfzvvvsOp0+fRvfu3e97nBo1aljkq1e0/6sv+Y/ogQMHsH///jJ9Fy5ciH379mHlypWYPXs2OnTogBEjRpi8tsHR0RHPPfccvvnmG5w4caLM69euXdM9jouLw5UrV/SuUHznzh18+OGHRh+vb9++uHLlisFtbt++jZs3bwLQrKEpTftLT/vn99dff+m9rlQqER4eDhFBYWGhwePHxsZCqVTiP//5j975/uijj5CTk2PUz4Exnn76aQDA4sWL9dq1I23mOk5VGXOeyxMTE1MmoAD6PzNaW7duxeHDh/HUU0/p2po3b46mTZti5cqVUKvVuvb3338fCoUCffr00bX16dMH6enpev9JyMzMxNdff40ePXro1jRVZp+GlP6Zcnd3R8OGDfXORY0aNQCgzL8Fffv2hVqtxuzZs8vst6ioqEz/P//8Exs3btQ9z83NxX//+19ERkYiMDDQ6HoAICcnB+fPn0eHDh0qfH9U/XCqjqzixo0bqFOnDvr06YOIiAi4u7tjx44d+OWXX7BgwQIAgLOzM+bPn4/4+Hh07twZ/fv3R3p6OpYsWYKwsDCMGzfuvseJiorCjh07sHDhQgQHB6NevXpm+Y6ov//979iwYQOeeeYZdO/eHRcvXsSKFSsQHh6OvLw8Xb/Tp09j6tSpeOmll9CjRw8Amq+3iIyMxMiRI/HVV1+ZdPx58+Zh586diI6OxtChQxEeHo6srCwcOXIEO3bs0P1y/de//oWlS5eif//+GDNmDIKCgvD555/rFugaMyL34osv4quvvsLw4cOxc+dOdOzYEWq1GmfOnMFXX32F77//Hm3btsWsWbOwe/dudO/eHaGhocjIyMDy5ctRp04dPProowCArl27IjAwEB07dkRAQABOnz6NpUuXonv37vDw8DB4fD8/P0yaNAkzZ87EU089hZ49e+Ls2bNYvnw52rVrp7cQvCoiIiIwaNAgrFy5EtnZ2ejcuTMOHjyINWvWoHfv3nj88cfNcpyqMuY8l6dXr1749NNP8dtvv+lGfACgQ4cOaN26Ndq2bQsvLy8cOXIEH3/8MUJCQjB58mS9fbzzzjvo2bMnunbtiueffx4nTpzA0qVLMWTIEL3Rkz59+uCRRx5BfHw8Tp06BV9fXyxfvhxqtbrMtKyx+zQkPDwcf/vb3xAVFYWaNWvi0KFDWL9+PUaPHq3rExUVBQB49dVXERcXB0dHRzz//PPo3Lkz/vWvf2Hu3LlITk5G165d4ezsjN9//x1ff/01lixZohfcGjdujMGDB+OXX35BQEAAPv74Y6Snp2P16tWVqgfQfOhBRNCrV68K3x9VQ1b4JB+R5Ofny/jx4yUiIkI8PDykRo0aEhERIcuXLy/Td926ddK6dWvdxeRKXwBTpPzLEZw5c0Yee+wxcXV1NXgBzNIf6R40aJDUqFGjTA3aCytqFRcXy1tvvSWhoaGiUqmkdevWsnnzZhk0aJCEhoaKiOaSBe3atZM6depIdna23v6WLFkiAGTdunUVnifcvQCmIenp6TJq1CgJCQkRZ2dnCQwMlC5dusjKlSv1+l24cEG6d+8urq6u4ufnJ6+99pp88803AkB+/vnnct9jSQUFBTJ//nxp3ry5qFQq8fHxkaioKJk5c6bk5OSIiEhSUpL06tVLgoODRalUSnBwsPTv31/vWkAffPCBPPbYY1KrVi1RqVTSoEEDGT9+vG4fIuVfAHPp0qXStGlTcXZ2loCAABkxYkS5F8AsreSfS0UKCwtl5syZUq9ePXF2dpaQkJAKL4BZWufOnaVz5873PQ7KuRyBoUsMlL4cgTHnuTz5+fni6+tb5hpiU6ZMkcjISPHy8hJnZ2epW7eujBgxQtLS0gzuZ+PGjRIZGSkqlUrq1Kkjb775pt41j7SysrJk8ODBUqtWLXFzc5POnTuXe3VyY/dZ2r///W9p3769eHt7i6urqzRt2lTmzJmjt21RUZG88sor4ufnJwqFosylCVauXClRUVHi6uoqHh4e0rJlS3njjTfkzz//1PUpeQHMVq1aiUqlkqZNm+pdGd3YekRE+vXrJ48++uh93x9VPwqRKqxkJCKbtHjxYowbNw6XL19G7dq1rV0OPUCzZ8/G6tWr8fvvv5e7kJzKCgsLQ4sWLbB58+Yq7ystLQ316tXD2rVrOeJkg7jGicjOlb4Ozp07d/DBBx+gUaNGDE0PoXHjxiEvLw9r1661dikPrcWLF6Nly5YMTTaKa5yI7Nyzzz6LunXrIjIyEjk5Ofjss89w5syZcj+CTfbN3d29zPWJ6MGaN2+etUugKmBwIrJzcXFxWLVqFT7//HOo1WqEh4dj7dq16Nevn7VLIyKyOVzjRERERGQkrnEiIiIiMhKDExEREZGRHro1TsXFxfjzzz/h4eHBL1ckIiIiiAhu3LiB4OBgODhUPKb00AWnP//8s8wXSRIRERGlpqaiTp06FfZ56IKT9msdUlNT4enpaeVqiIiIyNpyc3MREhJS7lc/lfTQBSft9JynpyeDExEREekYs4SHi8OJiIiIjMTgRERERGQkBiciIiIiIzE4ERERERmJwYmIiIjISAxOREREREZicCIiIiIyEoMTERERkZEYnIiIiIiMxOBEREREZCQGJzP67RLQ8Xmg1whrV0JERESW8NB9V50lFRYBl9OBQrW1KyEiIiJL4IiTBYhYuwIiIiKyBAYnM7r/dyoTERGRLWNwIiIiIjISg5MFcKqOiIjIPjE4mZGCc3VERER2jcGJiIiIyEgMTpbAqToiIiK7xOBkRpyqIyIism8MThbAASciIiL7xOBkRhxxIiIism8MTkRERERGYnCyAF7HiYiIyD4xOJkRZ+qIiIjsG4MTERERkZEYnCyAU3VERET2icHJnDhXR0REZNcYnCyAI05ERET2icHJjHgdJyIiIvvG4ERERERkJAYnC+BMHRERkX1icDIjTtURERHZN6sGp927d6NHjx4IDg6GQqHApk2b7rvNrl270KZNG6hUKjRs2BCffPKJxeskIiIiAqwcnG7evImIiAgsW7bMqP4XL15E9+7d8fjjjyM5ORljx47FkCFD8P3331u40srhp+qIiIjsk5M1D96tWzd069bN6P4rVqxAvXr1sGDBAgBAs2bN8NNPP2HRokWIi4uzVJlG40wdERGRfbOpNU779+9HbGysXltcXBz2799f7jb5+fnIzc3Vu1kaR5yIiIjsk00Fp7S0NAQEBOi1BQQEIDc3F7dv3za4zdy5c+Hl5aW7hYSEWKw+Lg4nIiKybzYVnEwxadIk5OTk6G6pqanWLomIiIhslFXXOFVWYGAg0tPT9drS09Ph6ekJV1dXg9uoVCqoVKoHUd49nKojIiKySzY14hQTE4OkpCS9tu3btyMmJsZKFenjVB0REZF9s2pwysvLQ3JyMpKTkwFoLjeQnJyMlJQUAJpptoEDB+r6Dx8+HBcuXMAbb7yBM2fOYPny5fjqq68wbtw4a5RPREREDxmrBqdDhw6hdevWaN26NQAgISEBrVu3xrRp0wAAV69e1YUoAKhXrx62bNmC7du3IyIiAgsWLMCqVauqxaUISuJMHRERkX1SiDxcH57Pzc2Fl5cXcnJy4OnpadZ9X7oCdP4n4FEDOLHZrLsmIiIiC6lMNrCpNU624uGKokRERA8PBicz4tpwIiIi+8bgRERERGQkBicL4FQdERGRfWJwMiNex4mIiMi+MTgRERERGYnByQI4VUdERGSfGJzMiVN1REREdo3BiYiIiMhIDE4WwJk6IiIi+8TgZEb8VB0REZF9Y3CyAC4OJyIisk8MTmbEASciIiL7xuBEREREZCQGJwvgVB0REZF9YnAyIy4OJyIism8MTkRERERGYnCyBE7VERER2SUGJzPiVB0REZF9Y3CyAA44ERER2ScGJzPiiBMREZF9Y3AiIiIiMhKDkwXwOk5ERET2icHJjDhTR0REZN8YnIiIiIiMxOBkAZyqIyIisk8MTmbET9URERHZNwYnC+CIExERkX1icDInjjgRERHZNQYnIiIiIiMxOFkAZ+qIiIjsE4OTGXFxOBERkX1jcCIiIiIyEoOTBfBTdURERPaJwcmMOFNHRERk3xicLIAjTkRERPaJwcmMuDiciIjIvjE4ERERERmJwYmIiIjISAxOZsSpOiIiIvvG4ERERERkJKsHp2XLliEsLAwuLi6Ijo7GwYMHK+y/ePFiNGnSBK6urggJCcG4ceNw586dB1St8fjJOiIiIvtj1eC0bt06JCQkYPr06Thy5AgiIiIQFxeHjIwMg/2/+OILTJw4EdOnT8fp06fx0UcfYd26dZg8efIDrtwwTtURERHZN6sGp4ULF2Lo0KGIj49HeHg4VqxYATc3N3z88ccG++/btw8dO3bECy+8gLCwMHTt2hX9+/e/7yiVNXDEiYiIyP5YLTgVFBTg8OHDiI2NvVeMgwNiY2Oxf/9+g9t06NABhw8f1gWlCxcuYOvWrXj66acfSM33wwEnIiIi++ZkrQNnZmZCrVYjICBArz0gIABnzpwxuM0LL7yAzMxMPProoxARFBUVYfjw4RVO1eXn5yM/P1/3PDc31zxvgIiIiB46Vl8cXhm7du3CW2+9heXLl+PIkSPYsGEDtmzZgtmzZ5e7zdy5c+Hl5aW7hYSEPJBaOVVHRERkf6w24uTr6wtHR0ekp6frtaenpyMwMNDgNlOnTsWLL76IIUOGAABatmyJmzdvYtiwYZgyZQocHMrmwEmTJiEhIUH3PDc312LhiYvDiYiI7JvVRpyUSiWioqKQlJSkaysuLkZSUhJiYmIMbnPr1q0y4cjR0REAIOUM8ahUKnh6eurdiIiIiExhtREnAEhISMCgQYPQtm1btG/fHosXL8bNmzcRHx8PABg4cCBq166NuXPnAgB69OiBhQsXonXr1oiOjsa5c+cwdepU9OjRQxegqgvO1BEREdkfqwanfv364dq1a5g2bRrS0tIQGRmJxMRE3YLxlJQUvRGmN998EwqFAm+++SauXLkCPz8/9OjRA3PmzLHWW9DDqToiIiL7ppDy5rjsVG5uLry8vJCTk2P2abucG0CrnprH57YDzlaNpURERGSMymQDm/pUXbXHESciIiK7xuBEREREZCQGJwt5uCZAiYiIHg4MTmbEmToiIiL7xuBEREREZCQGJwvhVB0REZH9YXAyI17HiYiIyL4xOFkIB5yIiIjsj0nB6ebNm+auwy5wxImIiMi+mRScAgIC8PLLL+Onn34ydz1ERERE1ZZJwemzzz5DVlYWnnjiCTRu3Bjz5s3Dn3/+ae7abBvn6oiIiOyOScGpd+/e2LRpE65cuYLhw4fjiy++QGhoKP7+979jw4YNKCoqMnedNoFTdURERPatSovD/fz8kJCQgGPHjmHhwoXYsWMH+vTpg+DgYEybNg23bt0yV51EREREVudUlY3T09OxZs0afPLJJ/jjjz/Qp08fDB48GJcvX8b8+fPx888/Y9u2beaq1abwOk5ERET2x6TgtGHDBqxevRrff/89wsPDMXLkSPzzn/+Et7e3rk+HDh3QrFkzc9VpEzhTR0REZN9MCk7x8fF4/vnnsXfvXrRr185gn+DgYEyZMqVKxdkyjjgRERHZH5OC09WrV+Hm5lZhH1dXV0yfPt2komwVF4cTERHZN5OCk5ubG9RqNTZu3IjTp08DAJo1a4bevXvDyalKy6aIiIiIqi2TUs7JkyfRo0cPpKeno0mTJgCA+fPnw8/PD//3f/+HFi1amLVIW8SZOiIiIvtj0uUIhgwZghYtWuDy5cs4cuQIjhw5gtTUVLRq1QrDhg0zd402g1N1RERE9s2kEafk5GQcOnQIPj4+ujYfHx/MmTOn3MXiRERERLbOpBGnxo0bIz09vUx7RkYGGjZsWOWi7AE/VUdERGR/TApOc+fOxauvvor169fj8uXLuHz5MtavX4+xY8di/vz5yM3N1d0eKpyqIyIismsKkcqPjTg43MtbirsLe7S7KflcoVBArVabo06zyc3NhZeXF3JycuDp6WnWfd8pAJrEaR6f3AK4V3zFBiIiIqoGKpMNTFrjtHPnTpMKe5hwqo6IiMj+mBScOnfubO467AJn6oiIiOybyVerzM7OxkcffaS7AGbz5s3x8ssvw8vLy2zF2TKOOBEREdkfkxaHHzp0CA0aNMCiRYuQlZWFrKwsLFy4EA0aNMCRI0fMXaPN4HWciIiI7JtJI07jxo1Dz5498eGHH+q+YqWoqAhDhgzB2LFjsXv3brMWSURERFQdmBScDh06pBeaAMDJyQlvvPEG2rZta7bibBln6oiIiOyPSVN1np6eSElJKdOempoKDw+PKhdlqzhVR0REZN9MCk79+vXD4MGDsW7dOqSmpiI1NRVr167FkCFD0L9/f3PXSERERFQtmDRV9+6770KhUGDgwIEoKioCADg7O2PEiBGYN2+eWQu0WZyrIyIisjuVDk5qtRo///wzZsyYgblz5+L8+fMAgAYNGsDN7eG+VDan6oiIiOxbpYOTo6MjunbtitOnT6NevXpo2bKlJeqyebyOExERkf0xaY1TixYtcOHCBXPXYvM44ERERGTfTApO//73v/H6669j8+bNuHr1KnJzc/VuRERERPbIpMXhTz/9NACgZ8+eUJRY2CMiUCgUUKvV5qnOhnGqjoiIyP6YFJx27txp7jrsAheHExER2TeTglO9evUQEhKiN9oEaEacUlNTzVIYERERUXVj0hqnevXq4dq1a2Xas7KyUK9evSoXZQ84U0dERGR/TApO2rVMpeXl5cHFxaXKRdkqTtURERHZt0pN1SUkJAAAFAoFpk6dqnfBS7VajQMHDiAyMrJSBSxbtgzvvPMO0tLSEBERgffeew/t27cvt392djamTJmCDRs2ICsrC6GhoVi8eLFuwXp1wcXhRERE9qdSweno0aMANCNOx48fh1Kp1L2mVCoRERGB119/3ej9rVu3DgkJCVixYgWio6OxePFixMXF4ezZs/D39y/Tv6CgAE8++ST8/f2xfv161K5dG3/88Qe8vb0r8zYshiNORERE9k0hUvmxkfj4eCxZsgSenp5VOnh0dDTatWuHpUuXAgCKi4sREhKCV155BRMnTizTf8WKFXjnnXdw5swZODs7m3TM3NxceHl5IScnp8r1GxL6uOb+yEaglrfZd09ERERmVplsYNIap9WrV1c5dBQUFODw4cOIjY29V4yDA2JjY7F//36D2/zvf/9DTEwMRo0ahYCAALRo0QJvvfVWhdeNys/Pt8oFOouLH8hhiIiI6AEy6XIEN2/exLx585CUlISMjAwUl0oJxnwdS2ZmJtRqNQICAvTaAwICcObMGYPbXLhwAT/88AMGDBiArVu34ty5cxg5ciQKCwsxffp0g9vMnTsXM2fONPKdVZ2DgyY0cYkTERGR/TEpOA0ZMgQ//vgjXnzxRQQFBRn8hJ0lFBcXw9/fHytXroSjoyOioqJw5coVvPPOO+UGp0mTJukWtQOa4biQkBCL1eigAIoBCEeciIiI7I5Jwem7777Dli1b0LFjR5MP7OvrC0dHR6Snp+u1p6enIzAw0OA2QUFBcHZ2hqOjo66tWbNmSEtLQ0FBgd5idS2VSgWVSmVynZXl4ABADRRzyImIiMjumLTGycfHBzVr1qzSgZVKJaKiopCUlKRrKy4uRlJSEmJiYgxu07FjR5w7d05vavC3335DUFCQwdBkDdrBN65xIiIisj8mBafZs2dj2rRpuHXrVpUOnpCQgA8//BBr1qzB6dOnMWLECNy8eRPx8fEAgIEDB2LSpEm6/iNGjEBWVhbGjBmD3377DVu2bMFbb72FUaNGVakOc3K4e0Y54kRERGR/TJqqW7BgAc6fP4+AgACEhYWVuTTAkSNHjNpPv379cO3aNUybNg1paWmIjIxEYmKibsF4SkoKHBzuZbuQkBB8//33GDduHFq1aoXatWtjzJgxmDBhgilvwyIc7o44cY0TERGR/TEpOPXu3dtsBYwePRqjR482+NquXbvKtMXExODnn3822/HNjSNORERE9suk4FTeJ9iIa5yIiIjsmUlrnADNd8atWrUKkyZNQlZWFgDNFN2VK1fMVpwt0o44qRmciIiI7I5JI07Hjh1DbGwsvLy8cOnSJQwdOhQ1a9bEhg0bkJKSgv/+97/mrtNmcI0TERGR/TJpxCkhIQEvvfQSfv/9d7i4uOjan376aezevdtsxdkirnEiIiKyXyYFp19++QX/+te/yrTXrl0baWlpVS7KlnGNExERkf0yKTipVCqDX5b722+/wc/Pr8pF2TJHjjgRERHZLZOCU8+ePTFr1iwUFhYCABQKBVJSUjBhwgQ899xzZi3Q1min6rjGiYiIyP6YFJwWLFiAvLw8+Pv74/bt2+jcuTMaNmwIDw8PzJkzx9w12hTt4nCOOBEREdkfkz5V5+Xlhe3bt2Pv3r349ddfkZeXhzZt2iA2Ntbc9dkchXaqjiNOREREdsek4KTVsWNHhIaGIigoCI6OjuaqyaZxxImIiMh+mXwBTK3w8HD88ccf5qjFLjhwxImIiMhuVTk4iXBopSQHXo6AiIjIblU5OJE+BS9HQEREZLeqHJwmT56MmjVrmqMWu8ARJyIiIvtV5eA0adIkeHh4IDk5GdevXzdHTTZNdx0njjgRERHZHZOC09ixY/HRRx8BANRqNTp37ow2bdogJCQEu3btMmd9NocjTkRERPbLpOC0fv16REREAAD+7//+DxcuXMCZM2cwbtw4TJkyxawF2hqucSIiIrJfJgWnzMxMBAYGAgC2bt2Kvn37onHjxnj55Zdx/PhxsxZoazjiREREZL9MCk4BAQE4deoU1Go1EhMT8eSTTwIAbt269dBfCJNrnIiIiOyXSVcOj4+PR9++fREUFASFQqH7qpUDBw6gadOmZi3Q1vACmERERPbLpOA0Y8YMtGjRAqmpqfjHP/4BlUoFAHB0dMTEiRPNWqCt0U7VqRmciIiI7I7J31XXp08fvefZ2dkYNGhQlQuydRxxIiIisl8mrXGaP38+1q1bp3vet29f1KpVC3Xq1MGxY8fMVpwt0o44cY0TERGR/TEpOK1YsQIhISEAgO3bt2P79u347rvv8NRTT+H11183a4G2RsERJyIiIrtl0lRdWlqaLjht3rwZffv2RdeuXREWFobo6GizFmhrdJcj4IgTERGR3TFpxMnHxwepqakAgMTERN2n6kQEarXafNXZIK5xIiIisl8mjTg9++yzeOGFF9CoUSP89ddf6NatGwDg6NGjaNiwoVkLtDVc40RERGS/TApOixYtQlhYGFJTU/H222/D3d0dAHD16lWMHDnSrAXaGoe71/8sergH3oiIiOySScHJ2dnZ4CLwcePGVbkgW6e8e0aLiqxbBxEREZmfyddxOn/+PBYvXozTp08DAMLDwzF27FjUr1/fbMXZIqe7Z7SAwYmIiMjumLQ4/Pvvv0d4eDgOHjyIVq1aoVWrVjhw4ADCw8Oxfft2c9doU5y0U3UMTkRERHbHpBGniRMnYty4cZg3b16Z9gkTJui+9PdhpHTW3DM4ERER2R+TRpxOnz6NwYMHl2l/+eWXcerUqSoXZcs4VUdERGS/TApOfn5+SE5OLtOenJwMf3//qtZk05y5OJyIiMhumTRVN3ToUAwbNgwXLlxAhw4dAAB79+7F/PnzkZCQYNYCbY02OBUyOBEREdkdk4LT1KlT4eHhgQULFmDSpEkAgODgYMyYMQOvvvqqWQu0NQxORERE9qvSwamoqAhffPEFXnjhBYwbNw43btwAAHh4eJi9OFukXeNUWGjdOoiIiMj8Kr3GycnJCcOHD8edO3cAaAITQ9M9uk/V8crhREREdsekxeHt27fH0aNHzV2LXdBex4lTdURERPbHpDVOI0eOxGuvvYbLly8jKioKNWrU0Hu9VatWZinOFjnfHXHiVB0REZH9MSk4Pf/88wCgtxBcoVBARKBQKKBWP7zzVM78kl8iIiK7ZdJU3cWLF8vcLly4oLuvrGXLliEsLAwuLi6Ijo7GwYMHjdpu7dq1UCgU6N27d6WPaSnaEacCjjgRERHZHZOC0xdffIGkpCSEhobq3ZKSkrB27dpK7WvdunVISEjA9OnTceTIEURERCAuLg4ZGRkVbnfp0iW8/vrr6NSpkylvwWK0n6rjiBMREZH9MSk4ffDBB2jatGmZ9ubNm2PFihWV2tfChQsxdOhQxMfHIzw8HCtWrICbmxs+/vjjcrdRq9UYMGAAZs6cifr161e6fktS8jpOREREdsuk4JSWloagoKAy7X5+frh69arR+ykoKMDhw4cRGxt7ryAHB8TGxmL//v3lbjdr1iz4+/sb/L48a+N1nIiIiOyXSYvDQ0JCsHfvXtSrV0+vfe/evQgODjZ6P5mZmVCr1QgICNBrDwgIwJkzZwxu89NPP+Gjjz4y+F15huTn5yM/P1/3PDc31+j6TKFS3j1ugUUPQ0RERFZg8nfVjR07FoWFhXjiiScAAElJSXjjjTfw2muvmbXAkm7cuIEXX3wRH374IXx9fY3aZu7cuZg5c6bFairNzUVzf/vOAzskERERPSAmBafx48fjr7/+wsiRI1FQoBlacXFxwYQJE3TfXWcMX19fODo6Ij09Xa89PT0dgYGBZfqfP38ely5dQo8ePXRtxcXFmjfi5ISzZ8+iQYMGettMmjRJ74uHc3NzERISYnSNleWqDU75FfcjIiIi22NScFIoFJg/fz6mTp2K06dPw9XVFY0aNYJKparUfpRKJaKiopCUlKS7pEBxcTGSkpIwevToMv2bNm2K48eP67W9+eabuHHjBpYsWWIwEKlUqkrXVRUudw91iyNOREREdsek4KTl7u6Odu3aVamAhIQEDBo0CG3btkX79u2xePFi3Lx5E/Hx8QCAgQMHonbt2pg7dy5cXFzQokULve29vb0BoEy7tXCqjoiIyH5VKTiZQ79+/XDt2jVMmzYNaWlpiIyMRGJiom7BeEpKChwcTPrwn1Vop+rucKqOiIjI7ihERKxdxIOUm5sLLy8v5OTkwNPT0+z7z84FInppHp/fce9Lf4mIiKh6qkw2sJ2hHBvh4nLvMafriIiI7AuDk5mpnAGFQvOYC8SJiIjsC4OTmSkU9xaI32FwIiIisisMThbAazkRERHZJwYnC9AGp5u3rVsHERERmReDkwV41NDc37hp3TqIiIjIvBicLMDTXXOfm2fdOoiIiMi8GJwswMtDc59zw7p1EBERkXkxOFmA592pOo44ERER2RcGJwvgVB0REZF9YnCyAC8GJyIiIrvE4GQBnlzjREREZJcYnCxAN+LEyxEQERHZFQYnC9B+qi4717p1EBERkXkxOFlALW/NfeZ1q5ZBREREZsbgZAG+Ppr7v7IBEauWQkRERGbE4GQBvjU19wWF/GQdERGRPWFwsgAX5b2LYF7jdB0REZHdYHCyEO2oU2aWdesgIiIi82FwshDtOqdrDE5ERER2g8HJQrTBiZ+sIyIish8MThbCESciIiL7w+BkIUF+mvur16xbBxEREZkPg5OF1A7Q3F9Jt24dREREZD4MThYS7K+5Z3AiIiKyHwxOFlInUHN/9RpQpLZuLURERGQeDE4W4l8TcHIE1MVAeqa1qyEiIiJzYHCyEEdHIOjudN2fGdathYiIiMyDwcmCtAvEU9OsWwcRERGZB4OTBYUGa+4vXbZuHURERGQeDE4W1KCu5v5CqnXrICIiIvNgcLKgBiGa+/Mp1q2DiIiIzIPByYLq3w1OFy4DxcXWrYWIiIiqjsHJgkKCAGcn4E4+P1lHRERkDxicLMjZCah7d4E4p+uIiIhsH4OThTWpp7k/fcG6dRAREVHVMThZWItGmvuTv1u3DiIiIqo6BicLa87gREREZDcYnCyseUPN/YXLwM3b1q2FiIiIqobBycL8agKBvoAIcOqctashIiKiqmBwegBaNtbcHz1t3TqIiIioahicHoB2rTT3vxyzbh1ERERUNdUiOC1btgxhYWFwcXFBdHQ0Dh48WG7fDz/8EJ06dYKPjw98fHwQGxtbYf/qIPpucDp4nFcQJyIismVWD07r1q1DQkICpk+fjiNHjiAiIgJxcXHIyDB8qe1du3ahf//+2LlzJ/bv34+QkBB07doVV65cecCVG695I8DNBcjOBX67ZO1qiIiIyFQKERFrFhAdHY127dph6dKlAIDi4mKEhITglVdewcSJE++7vVqtho+PD5YuXYqBAwfet39ubi68vLyQk5MDT0/PKtdvrH+OB/YcAma+Crz0zAM7LBEREd1HZbKBVUecCgoKcPjwYcTGxuraHBwcEBsbi/379xu1j1u3bqGwsBA1a9Y0+Hp+fj5yc3P1btbQobXmftcBqxyeiIiIzMCqwSkzMxNqtRoBAQF67QEBAUhLSzNqHxMmTEBwcLBe+Cpp7ty58PLy0t1CQkKqXLcpusRo7vcdAW7xek5EREQ2yeprnKpi3rx5WLt2LTZu3AgXFxeDfSZNmoScnBzdLTU19QFXqdE4DKgTCOQXAj8dtkoJREREVEVWDU6+vr5wdHREenq6Xnt6ejoCAwMr3Pbdd9/FvHnzsG3bNrRq1arcfiqVCp6enno3a1AogNi7o07b91mlBCIiIqoiqwYnpVKJqKgoJCUl6dqKi4uRlJSEmJiYcrd7++23MXv2bCQmJqJt27YPolSz6Pqo5j5xD3CnwLq1EBERUeVZfaouISEBH374IdasWYPTp09jxIgRuHnzJuLj4wEAAwcOxKRJk3T958+fj6lTp+Ljjz9GWFgY0tLSkJaWhry8PGu9BaM9EgEE+QG5eUASR52IiIhsjtWDU79+/fDuu+9i2rRpiIyMRHJyMhITE3ULxlNSUnD16lVd//fffx8FBQXo06cPgoKCdLd3333XWm/BaI6OwLNPah6v/966tRAREVHlWf06Tg+ata7jpHUuBegyCHBwAHZ/DoRUvJSLiIiILMxmruP0MGpYF3g0SvPVK6u/sXY1REREVBkMTlYw5B+a+3VbNeudiIiIyDYwOFnB39oDjUKBvFvAR+utXQ0REREZi8HJChQKYJzmQ4NYuQ64lmXdeoiIiMg4DE5W8vRjQERT4NYdYNEn1q6GiIiIjMHgZCUKBTD5X5rHn/8f8Mtx69ZDRERE98fgZEWPRAJ9u2keT3gXuJNv1XKIiIjoPhicrOzNEYBfTeB8CjD9PWtXQ0RERBVhcLIyLw9g8WTN1N3aLZpLFBAREVH1xOBUDTwaBSTc/ZTd5IXAroPWrYeIiIgMY3CqJkYPAHo+ARSpgeHTuViciIioOmJwqiYcHIAFE4HO7YDbd4AX3+DIExERUXXD4FSNKJ2BFTOBzu014WnwZODLzcDD9TXMRERE1ReDUzXj5gqs+ve9abuJC4Dxb2uCFBEREVkXg1M1pHQGlkwBJgzVTOF9nQg8NQQ48Ku1KyMiInq4MThVUw4OwMgXgM/eAQJ9gUtXgL5jgYnvAhn8bjsiIiKrYHCq5jq2AbavBvp31zz/cgvQeYDm++1y86xaGhER0UNHIfJwLT3Ozc2Fl5cXcnJy4Onpae1yKuXgMWDO+0DyGc1zdzfghR7A4OeAQD/r1kZERGSrKpMNGJxsjAiwZReweA3w+x+aNidHoEsHoF83zSfynBytWiIREZFNYXCqgK0HJ63iYmDnAeCDtcCBY/faA3yBv/8NeKoTENUccGSIIiIiqhCDUwXsJTiVdOYC8NV3wMbtQFbOvXZfHyC2A/BYO6BDJODjZbUSiYiIqi0GpwrYY3DSKigEdh0AEvcAO/YDOTfuvaZQAM0bar4Xr30rILIZUMvbaqUSERFVGwxOFbDn4FRSYZHmuk879gF7jwC/XSrbp24wENkUaBMOtGgMNKkHeLo/8FKJiIisisGpAg9LcCot/S9g3xFNiDp6Gjj3h+F+tQM0AapZfaBJfaB+CFCvjuYTfERERPaIwakCD2twKi0nD/j1tCZEJZ/WrJP6M6P8/n4+QGhtTYgKq6251Q4Agv0Bv5qaC3YSERHZIganCjA4lS/nBnD2oiZEae8vXQEyr1e8nbOT5tN8tf2BIP979/61NKHK10cTvFxdHsz7ICIiqozKZAOnB1QT2QAvD83C8fat9Ntz84A/rgAXr2iC1KXLmvur14C0TM16qstpmltF3N3uhqia+vc+npqbd4mbjydQw1WzqJ2IiKi6YHCi+/J0B1o20dxKK1IDGX8BV9I1QerPDM3tagZwLevu7TqQXwDk3dLcLl0x7rjOToC3B+ClDVYemlDlUUMTwtxrAJ4lHnvUuPeaRw1N8OJ1rIiIyJwYnKhKnBw165yC/cvvIwLcuKmZ8ruWpbnPuHufeR24ngvk5Grus28A2TlAfqFmJOvadc3NVO5u94JVDVfNdKHb3ZurC+DmWuKxtt3VcB/tcxeVJtRxNIyI6OHD4EQWp1BoRq083TWf0jPG7TuaEHU95+793XCVfQPIuwncuHX3vtTjvLuP8ws1+9GOciHT/O9JpdSEKJXy7mNlicfGtJfq4+ysCWRKZ82t5HPdvTOgdLr3mrMTF+YTET1IDE5ULbneHd0JMvHLi3VTgzeB3Luh6vYd4NZt4NYdze323ftbt+891t7fvA3cNtC3uFizfxHgTr7mZm3aAFU6VBkMWs6AyhlwctKMFjo5lnhc+r7E646Omu0dHQHn+2yj19fAtobanRw1AdDRQXPP0Twiqq4YnMguaUdyzHl1dBHNSFZ+wb3bnXwjHlfUv/De48K705OFhUBBkeZxQUGptkJAXaxfV+HdvrhjvvdqbY4OmkClDVPa50a3OQAOjvrPS7aVDmqGwlvpNu32pdsUDoCDQtOmuy/9+O5zbd+SAVH7mqOj/vPS+1Hc7VPyua5ehX4dZeoqr8YK6iIiwxiciIykUGim1FyU1q1DrQYK1YZDVWGR5qt3CkqEML22Es+1+1GrNW1qNVBUVLat9H2R9lZU6t5Au/YYZfoWlQ2Aeu+xuOLXyfLKC2UKxb2bg+JegFNA00+vXQGgxOvlbq9tK9VPt70Rx1GU09fBoVS7olQN96vN4d75MHV7QzUrFJp2lHpe+jEMvVbqOSz8ujZIl369ZMiu7Laln5d7Hkq97utt/e9dZXAisjHaURZrB7iqEtEPUepioFh977Fard9WpNZMlZbXpr77uDJt6mL9fZZuU6tL1GGgTUTTXiyatpLPi0Wzz+KSbcX3Hkvxvf1qnxfL3f2W6F/6OKX3qdvP3Ta5W0vJfWr3UxnaWomqk4nDgBH9rVsDgxMRWYVCcW99Fj0YekGvVEArGeoqCnoid/cj9x5r9yu4G7ZKvF5c4nXgXrgrvb0xfYvl7r5LHMvQ9hX11bWX3sZAP0PvrzJ9DdWnbdP+eehupZ5X+Do076G8bbXPUep5yfdQ+rX7bWvS6wZe0/58lLddRduKVI//MPKfLCKih4R2nZQjAGdrF0Nko/hBZiIiIiIjMTgRERERGYnBiYiIiMhIDE5ERERERmJwIiIiIjJStQhOy5YtQ1hYGFxcXBAdHY2DBw9W2P/rr79G06ZN4eLigpYtW2Lr1q0PqFIiIiJ6mFk9OK1btw4JCQmYPn06jhw5goiICMTFxSEjI8Ng/3379qF///4YPHgwjh49it69e6N37944ceLEA66ciIiIHjYKEe2ltqwjOjoa7dq1w9KlSwEAxcXFCAkJwSuvvIKJEyeW6d+vXz/cvHkTmzdv1rU98sgjiIyMxIoVK+57vNzcXHh5eSEnJweenp7meyNERERkkyqTDaw64lRQUIDDhw8jNjZW1+bg4IDY2Fjs37/f4Db79+/X6w8AcXFx5fYnIiIiMherXjk8MzMTarUaAQEBeu0BAQE4c+aMwW3S0tIM9k9LSzPYPz8/H/n5+brnubm5VayaiIiIHlZWX+NkaXPnzoWXl5fuFhISYu2SiIiIyEZZdcTJ19cXjo6OSE9P12tPT09HYGCgwW0CAwMr1X/SpElISEjQPc/JyUHdunU58kREREQA7s1GGbPs26rBSalUIioqCklJSejduzcAzeLwpKQkjB492uA2MTExSEpKwtixY3Vt27dvR0xMjMH+KpUKKpVK91x7cjjyRERERCXduHEDXl5eFfaxanACgISEBAwaNAht27ZF+/btsXjxYty8eRPx8fEAgIEDB6J27dqYO3cuAGDMmDHo3LkzFixYgO7du2Pt2rU4dOgQVq5cadTxgoODkZqaCg8PDygUCrO/n9zcXISEhCA1NZWf2nuAeN6th+feOnjerYPn3Xosee5FBDdu3EBwcPB9+1o9OPXr1w/Xrl3DtGnTkJaWhsjISCQmJuoWgKekpMDB4d5SrA4dOuCLL77Am2++icmTJ6NRo0bYtGkTWrRoYdTxHBwcUKdOHYu8l5I8PT35l8oKeN6th+feOnjerYPn3Xosde7vN9KkZfXrONkbXifKOnjerYfn3jp43q2D5916qsu5t/tP1RERERGZC4OTmalUKkyfPl1vQTpZHs+79fDcWwfPu3XwvFtPdTn3nKojIiIiMhJHnIiIiIiMxOBEREREZCQGJyIiIiIjMTiZ0bJlyxAWFgYXFxdER0fj4MGD1i7JpuzevRs9evRAcHAwFAoFNm3apPe6iGDatGkICgqCq6srYmNj8fvvv+v1ycrKwoABA+Dp6Qlvb28MHjwYeXl5en2OHTuGTp06wcXFBSEhIXj77bct/daqtblz56Jdu3bw8PCAv78/evfujbNnz+r1uXPnDkaNGoVatWrB3d0dzz33XJmvPkpJSUH37t3h5uYGf39/jB8/HkVFRXp9du3ahTZt2kClUqFhw4b45JNPLP32qq33338frVq10l2TJiYmBt99953udZ7zB2PevHlQKBR630bBc28ZM2bMgEKh0Ls1bdpU97rNnHchs1i7dq0olUr5+OOP5eTJkzJ06FDx9vaW9PR0a5dmM7Zu3SpTpkyRDRs2CADZuHGj3uvz5s0TLy8v2bRpk/z666/Ss2dPqVevnty+fVvX56mnnpKIiAj5+eefZc+ePdKwYUPp37+/7vWcnBwJCAiQAQMGyIkTJ+TLL78UV1dX+eCDDx7U26x24uLiZPXq1XLixAlJTk6Wp59+WurWrSt5eXm6PsOHD5eQkBBJSkqSQ4cOySOPPCIdOnTQvV5UVCQtWrSQ2NhYOXr0qGzdulV8fX1l0qRJuj4XLlwQNzc3SUhIkFOnTsl7770njo6OkpiY+EDfb3Xxv//9T7Zs2SK//fabnD17ViZPnizOzs5y4sQJEeE5fxAOHjwoYWFh0qpVKxkzZoyunefeMqZPny7NmzeXq1ev6m7Xrl3TvW4r553ByUzat28vo0aN0j1Xq9USHBwsc+fOtWJVtqt0cCouLpbAwEB55513dG3Z2dmiUqnkyy+/FBGRU6dOCQD55ZdfdH2+++47USgUcuXKFRERWb58ufj4+Eh+fr6uz4QJE6RJkyYWfke2IyMjQwDIjz/+KCKa8+zs7Cxff/21rs/p06cFgOzfv19ENKHXwcFB0tLSdH3ef/998fT01J3rN954Q5o3b653rH79+klcXJyl35LN8PHxkVWrVvGcPwA3btyQRo0ayfbt26Vz58664MRzbznTp0+XiIgIg6/Z0nnnVJ0ZFBQU4PDhw4iNjdW1OTg4IDY2Fvv377diZfbj4sWLSEtL0zvHXl5eiI6O1p3j/fv3w9vbG23bttX1iY2NhYODAw4cOKDr89hjj0GpVOr6xMXF4ezZs7h+/foDejfVW05ODgCgZs2aAIDDhw+jsLBQ79w3bdoUdevW1Tv3LVu21H1VEqA5r7m5uTh58qSuT8l9aPvw7wigVquxdu1a3Lx5EzExMTznD8CoUaPQvXv3MueH596yfv/9dwQHB6N+/foYMGAAUlJSANjWeWdwMoPMzEyo1Wq9P0wACAgIQFpampWqsi/a81jROU5LS4O/v7/e605OTqhZs6ZeH0P7KHmMh1lxcTHGjh2Ljh076r7/MS0tDUqlEt7e3np9S5/7+53X8vrk5ubi9u3blng71d7x48fh7u4OlUqF4cOHY+PGjQgPD+c5t7C1a9fiyJEjui+PL4nn3nKio6PxySefIDExEe+//z4uXryITp064caNGzZ13q3+Jb9EVH2MGjUKJ06cwE8//WTtUh4KTZo0QXJyMnJycrB+/XoMGjQIP/74o7XLsmupqakYM2YMtm/fDhcXF2uX81Dp1q2b7nGrVq0QHR2N0NBQfPXVV3B1dbViZZXDEScz8PX1haOjY5nV/+np6QgMDLRSVfZFex4rOseBgYHIyMjQe72oqAhZWVl6fQzto+QxHlajR4/G5s2bsXPnTtSpU0fXHhgYiIKCAmRnZ+v1L33u73dey+vj6elpU/9ompNSqUTDhg0RFRWFuXPnIiIiAkuWLOE5t6DDhw8jIyMDbdq0gZOTE5ycnPDjjz/iP//5D5ycnBAQEMBz/4B4e3ujcePGOHfunE39zDM4mYFSqURUVBSSkpJ0bcXFxUhKSkJMTIwVK7Mf9erVQ2BgoN45zs3NxYEDB3TnOCYmBtnZ2Th8+LCuzw8//IDi4mJER0fr+uzevRuFhYW6Ptu3b0eTJk3g4+PzgN5N9SIiGD16NDZu3IgffvgB9erV03s9KioKzs7Oeuf+7NmzSElJ0Tv3x48f1wuu27dvh6enJ8LDw3V9Su5D24d/R+4pLi5Gfn4+z7kFdenSBcePH0dycrLu1rZtWwwYMED3mOf+wcjLy8P58+cRFBRkWz/zZltm/pBbu3atqFQq+eSTT+TUqVMybNgw8fb21lv9TxW7ceOGHD16VI4ePSoAZOHChXL06FH5448/RERzOQJvb2/59ttv5dixY9KrVy+DlyNo3bq1HDhwQH766Sdp1KiR3uUIsrOzJSAgQF588UU5ceKErF27Vtzc3B7qyxGMGDFCvLy8ZNeuXXofE75165auz/Dhw6Vu3bryww8/yKFDhyQmJkZiYmJ0r2s/Jty1a1dJTk6WxMRE8fPzM/gx4fHjx8vp06dl2bJlD/XHsydOnCg//vijXLx4UY4dOyYTJ04UhUIh27ZtExGe8wep5KfqRHjuLeW1116TXbt2ycWLF2Xv3r0SGxsrvr6+kpGRISK2c94ZnMzovffek7p164pSqZT27dvLzz//bO2SbMrOnTsFQJnboEGDRERzSYKpU6dKQECAqFQq6dKli5w9e1ZvH3/99Zf0799f3N3dxdPTU+Lj4+XGjRt6fX799Vd59NFHRaVSSe3atWXevHkP6i1WS4bOOQBZvXq1rs/t27dl5MiR4uPjI25ubvLMM8/I1atX9fZz6dIl6datm7i6uoqvr6+89tprUlhYqNdn586dEhkZKUqlUurXr693jIfNyy+/LKGhoaJUKsXPz0+6dOmiC00iPOcPUungxHNvGf369ZOgoCBRKpVSu3Zt6devn5w7d073uq2cd4WIiPnGr4iIiIjsF9c4ERERERmJwYmIiIjISAxOREREREZicCIiIiIyEoMTERERkZEYnIiIiIiMxOBEREREZCQGJyIiIiIjMTgRERERGYnBiYjs2ksvvYTevXtbuwwishMMTkRERERGYnAiIruwfv16tGzZEq6urqhVqxZiY2Mxfvx4rFmzBt9++y0UCgUUCgV27doFAEhNTUXfvn3h7e2NmjVrolevXrh06ZJuf9qRqpkzZ8LPzw+enp4YPnw4CgoKrPMGiahacLJ2AUREVXX16lX0798fb7/9Np555hncuHEDe/bswcCBA5GSkoLc3FysXr0aAFCzZk0UFhYiLi4OMTEx2LNnD5ycnPDvf/8bTz31FI4dOwalUgkASEpKgouLC3bt2oVLly4hPj4etWrVwpw5c6z5donIihiciMjmXb16FUVFRXj22WcRGhoKAGjZsiUAwNXVFfn5+QgMDNT1/+yzz1BcXIxVq1ZBoVAAAFavXg1vb2/s2rULXbt2BQAolUp8/PHHcHNzQ/PmzTFr1iyMHz8es2fPhoMDB+yJHkb8m09ENi8iIgJdunRBy5Yt8Y9//AMffvghrl+/Xm7/X3/9FefOnYOHhwfc3d3h7u6OmjVr4s6dOzh//rzeft3c3HTPY2JikJeXh9TUVIu+HyKqvjjiREQ2z9HREdu3b8e+ffuwbds2vPfee5gyZQoOHDhgsH9eXh6ioqLw+eefl3nNz8/P0uUSkQ1jcCIiu6BQKNCxY0d07NgR06ZNQ2hoKDZu3AilUgm1Wq3Xt02bNli3bh38/f3h6elZ7j5//fVX3L59G66urgCAn3/+Ge7u7ggJCbHoeyGi6otTdURk8w4cOIC33noLhw4dQkpKCjZs2IBr166hWbNmCAsLw7Fjx3D27FlkZmaisLAQAwYMgK+vL3r16oU9e/bg4sWL2LVrF1599VVcvnxZt9+CggIMHjwYp06dwtatWzF9+nSMHj2a65uIHmIccSIim+fp6Yndu3dj8eLFyM3NRWhoKBYsWIBu3bqhbdu22LVrF9q2bYu8vDzs3LkTf/vb37B7925MmDABzz77LG7cuIHatWujS5cueiNQXbp0QaNGjfDYY48hPz8f/fv3x4wZM6z3RonI6hQiItYugoiounnppZeQnZ2NTZs2WbsUIqpGON5MREREZCQGJyIiIiIjcaqOiIiIyEgccSIiIiIyEoMTERERkZEYnIiIiIiMxOBEREREZCQGJyIiIiIjMTgRERERGYnBiYiIiMhIDE5ERERERmJwIiIiIjLS/wM1mRLJkSEeigAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# viz: the cross-entropy loss curve for softmax regression on iris\n",
    "plt.figure(figsize=(6, 4))\n",
    "plt.plot(losses_iris, color=\"#1E40FF\")\n",
    "plt.xlabel(\"step\"); plt.ylabel(\"cross-entropy\")\n",
    "plt.title(f\"softmax regression on iris ({len(losses_iris)} steps)\")\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5c2356e",
   "metadata": {},
   "source": [
    "> **Interpretation.** The cross-entropy falls smoothly and the train accuracy reaches about 0.98, matching scikit-learn's softmax fit on more than 99% of examples. The same gradient-descent machinery that fit a line now fits a three-class classifier; only the loss changed.\n",
    "\n",
    "> **Key takeaways.** Logistic and softmax regression are linear regression with a different loss and a squashing function. A stable softmax subtracts the per-row max before exponentiating. The cross-entropy gradient is the same `Xᵀ(P − Y)/m` residual form as MSE, which is why one optimizer fits all of them.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f93c6aa1",
   "metadata": {},
   "source": [
    "### Experiment log\n",
    "\n",
    "Every method here recovers the same problem's answer or matches a reference. The table records the expected numbers for both the full run and the `FAST` smoke run, so you know what \"working\" looks like.\n",
    "\n",
    "| Method | What it fits | Expected (full) | Expected (FAST) |\n",
    "|---|---|---|---|\n",
    "| Normal equation (4 routes) | y = 4 + 3x | intercept ≈ 3.99, slope ≈ 2.93 | identical (closed form) |\n",
    "| Batch GD | y = 4 + 3x | → normal eq within 5e-2 | → within 5e-2 (fewer steps) |\n",
    "| SGD (schedule) | y = 4 + 3x | (intercept, slope) within 0.15 | within 0.15 |\n",
    "| Momentum vs GD | κ=50 quadratic | momentum reaches ‖w‖<0.05 sooner | same (step-count check) |\n",
    "| Our Adam vs torch.optim | κ=50 quadratic | match to ~1e-6 | match to ~1e-6 |\n",
    "| Softmax regression | iris (3-class) | acc ≈ 0.98, ≥95% agree w/ sklearn | ≥95% agree w/ sklearn |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6cc964c5",
   "metadata": {},
   "source": [
    "## Safety lens\n",
    "\n",
    "A linear model's decision surface is fully readable from its weight vector: every coefficient says exactly how much each feature contributes. Use that readability as a cheap safety check. Print and sort the weights after training; if a feature that should not matter carries a large weight, the model is shortcut-learning regardless of how good the loss looks. The cell below does this for the softmax fit on iris.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "9a093740",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:11.842264Z",
     "iopub.status.busy": "2026-06-10T18:46:11.841640Z",
     "iopub.status.idle": "2026-06-10T18:46:11.846268Z",
     "shell.execute_reply": "2026-06-10T18:46:11.845886Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "class 0 (setosa): strongest features -> petal length (cm) (-5.11), petal width (cm) (-4.81)\n",
      "class 1 (versicolor): strongest features -> petal length (cm) (-2.32), sepal length (cm) (+2.07)\n",
      "class 2 (virginica): strongest features -> petal length (cm) (+7.42), petal width (cm) (+6.65)\n"
     ]
    }
   ],
   "source": [
    "# inspect what the model learned: per-class weight magnitudes (bias row dropped)\n",
    "feat_names = iris.feature_names\n",
    "W = Theta_iris[1:]                                    # (4 features, 3 classes), skip bias\n",
    "for k in range(K):\n",
    "    order = np.argsort(-np.abs(W[:, k]))\n",
    "    top = \", \".join(f\"{feat_names[j]} ({W[j, k]:+.2f})\" for j in order[:2])\n",
    "    print(f\"class {k} ({iris.target_names[k]}): strongest features -> {top}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59bb9452",
   "metadata": {},
   "source": [
    "> **Caveat:** the probabilities a logistic or softmax model outputs are calibrated on the training distribution and nowhere else. Any decision that acts on $\\hat{p}$ crossing a threshold must check calibration on held-out data (a reliability diagram, or `sklearn.calibration.calibration_curve`). Two more habits worth keeping: always hold out a sealed test set you touch exactly once, because reusing a validation set for model selection quietly fits the model to it; and report which features drive the model, not just the headline metric.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "778005e5",
   "metadata": {},
   "source": [
    "## Test yourself\n",
    "\n",
    "Three parts: concept self-checks, two auto-checked problems, and a capstone. Every answer is in this notebook; if unsure, re-run that section. Solutions are folded, so try before you peek.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ba218bf",
   "metadata": {},
   "source": [
    "### Part A — Concepts\n",
    "\n",
    "1. The normal equation needs $(\\mathbf{X}^T\\mathbf{X})^{-1}$. For a model with a million parameters, why is that a non-starter? <details><summary>Answer</summary>Inverting a $(d+1)\\times(d+1)$ matrix costs about $O(d^3)$, so $d \\approx 10^6$ means roughly $10^{18}$ operations per fit. Gradient descent is $O(md)$ per step and converges in hundreds of steps, so it wins by many orders of magnitude at scale.</details>\n",
    "\n",
    "2. In the deliberate-failure cell, the loss went to `inf` within a few dozen steps at $\\eta = 1.5$. In one sentence, what fixes it and why? <details><summary>Answer</summary>Cut the learning rate (we used $\\eta = 0.1$): the step was larger than the $2/L$ convergence threshold, so each step overshot the minimum onto a steeper point. A smaller step stays inside the basin and descends.</details>\n",
    "\n",
    "3. Look at the SGD-vs-batch loss plot in Part 4. Why is the orange (SGD) curve jagged while the blue (batch) curve is smooth? <details><summary>Answer</summary>Batch GD uses the exact full-data gradient every step, so the loss falls monotonically. SGD uses a one-example gradient, a noisy estimate, so the loss wobbles step to step even while trending down. The decaying schedule shrinks the wobble over time.</details>\n",
    "\n",
    "4. Why subtract the per-row max inside `softmax` if it does not change the result? <details><summary>Answer</summary>Mathematically softmax is shift-invariant, so subtracting any constant (we use the max) leaves the output identical. Numerically it stops `np.exp` from overflowing on large logits: without it, a logit of 1000 gives `exp(1000) = inf` and the row becomes `NaN`.</details>\n",
    "\n",
    "5. What is the one-line difference between Adam and AdamW, and why does it matter? <details><summary>Answer</summary>AdamW applies weight decay directly to the parameters (`theta -= lr * wd * theta`) instead of folding it into the gradient. In plain Adam the decay passes through the adaptive `1/sqrt(v)` rescaling, which weakens it inconsistently across parameters; decoupling keeps the decay uniform.</details>\n",
    "\n",
    "6. Do-it-now (one line): given `logits = np.array([[2.0, 1.0, 0.0]])`, write the expression that returns the predicted class index. <details><summary>Answer</summary>`int(np.argmax(logits))` (here it is 0). The softmax is monotonic, so you can take the argmax of the logits directly; you do not need to normalize to predict.</details>\n",
    "\n",
    "7. Momentum helped a lot on the $\\kappa = 50$ quadratic. When would it help only a little? <details><summary>Answer</summary>On a well-conditioned problem (all Hessian eigenvalues similar, $\\kappa$ near 1). Momentum's advantage grows with the condition number, because its job is to dampen the oscillation that ill-conditioning causes. On a round bowl there is little oscillation to dampen.</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a89eed2c",
   "metadata": {},
   "source": [
    "### Part B1 — The MSE gradient from scratch\n",
    "`Difficulty 2/5 · ~8 min`\n",
    "\n",
    "Implement `mse_gradient(X_b, y, theta)` returning $\\frac{2}{m}\\mathbf{X}^T(\\mathbf{X}\\boldsymbol\\theta - \\mathbf{y})$. The check compares it against a finite-difference estimate of the gradient, so the reference is independent of your formula.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "a873de7f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:11.847476Z",
     "iopub.status.busy": "2026-06-10T18:46:11.847374Z",
     "iopub.status.idle": "2026-06-10T18:46:11.852671Z",
     "shell.execute_reply": "2026-06-10T18:46:11.852328Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] B1 mse gradient: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def mse_gradient(X_b, y, theta):\n",
    "    # TODO: return (2/m) * X_b^T @ (X_b @ theta - y), where m = X_b.shape[0]\n",
    "    result = None\n",
    "    attempted(result)\n",
    "    return result\n",
    "\n",
    "def _mse_grad_vs_finite_diff():\n",
    "    th = rng.standard_normal((2, 1))\n",
    "    analytic = mse_gradient(X_b, y, th)\n",
    "    # independent reference: central finite differences on the MSE\n",
    "    def mse(t): return float(np.mean((X_b @ t - y) ** 2))\n",
    "    eps = 1e-6\n",
    "    numeric = np.zeros_like(th)\n",
    "    for i in range(th.size):\n",
    "        d = np.zeros_like(th); d[i] = eps\n",
    "        numeric[i] = (mse(th + d) - mse(th - d)) / (2 * eps)\n",
    "    check_close(analytic, numeric, atol=1e-4,\n",
    "                msg=\"analytic MSE gradient must match the finite-difference estimate\")\n",
    "\n",
    "check(\"B1 mse gradient\", _mse_grad_vs_finite_diff)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "087fe268",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1</summary>This is the gradient you used inside `batch_gd`. `m = X_b.shape[0]`; the residual is `X_b @ theta - y`.</details>\n",
    "<details><summary>Solution</summary>\n",
    "\n",
    "```python\n",
    "def mse_gradient(X_b, y, theta):\n",
    "    m = X_b.shape[0]\n",
    "    return 2 / m * X_b.T @ (X_b @ theta - y)\n",
    "```\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "4f827d09",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:11.853773Z",
     "iopub.status.busy": "2026-06-10T18:46:11.853609Z",
     "iopub.status.idle": "2026-06-10T18:46:11.857239Z",
     "shell.execute_reply": "2026-06-10T18:46:11.856893Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] B1 mse gradient\n",
      "analytic gradient at theta=0: [-14.42122117 -17.97086736]\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines mse_gradient; the check below re-verifies the reference.\n",
    "def mse_gradient(X_b, y, theta):\n",
    "    m = X_b.shape[0]\n",
    "    return 2 / m * X_b.T @ (X_b @ theta - y)\n",
    "\n",
    "check(\"B1 mse gradient\", _mse_grad_vs_finite_diff, required=True)\n",
    "print(\"analytic gradient at theta=0:\", mse_gradient(X_b, y, np.zeros((2, 1))).ravel())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5921c007",
   "metadata": {},
   "source": [
    "### Part B2 — Ridge regression's closed form\n",
    "`Difficulty 3/5 · ~12 min`\n",
    "\n",
    "Ridge adds an L2 penalty $\\alpha\\sum_{i\\ge1}\\theta_i^2$ (not on the bias) to the MSE, giving the closed form $\\hat{\\boldsymbol\\theta} = (\\mathbf{X}^T\\mathbf{X} + \\alpha\\mathbf{A})^{-1}\\mathbf{X}^T\\mathbf{y}$ where $\\mathbf{A}$ is the identity with its top-left entry zeroed (so the bias is unpenalized). Implement `ridge(X_b, y, alpha)`. The check compares your fit against scikit-learn's `Ridge` on a small problem.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "f19fb809",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:11.858381Z",
     "iopub.status.busy": "2026-06-10T18:46:11.858210Z",
     "iopub.status.idle": "2026-06-10T18:46:11.868313Z",
     "shell.execute_reply": "2026-06-10T18:46:11.867822Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] B2 ridge == sklearn: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def ridge(X_b, y, alpha=1.0):\n",
    "    \"\"\"Closed-form ridge. Do not penalize the bias (column 0). Returns (d+1, 1).\"\"\"\n",
    "    d_plus_1 = X_b.shape[1]\n",
    "    A = np.eye(d_plus_1)\n",
    "    # TODO 1: zero out A[0, 0] so the bias term is not regularized\n",
    "    A[0, 0] = 0.0\n",
    "    # TODO 2: return inv(X_b^T @ X_b + alpha * A) @ X_b^T @ y\n",
    "    result = None\n",
    "    attempted(result)\n",
    "    return result\n",
    "\n",
    "def _ridge_vs_sklearn():\n",
    "    from sklearn.linear_model import Ridge\n",
    "    r = np.random.default_rng(1)\n",
    "    Xr = r.standard_normal((40, 3))\n",
    "    yr = (Xr @ np.array([[1.5], [-2.0], [0.5]]) + 0.1 * r.standard_normal((40, 1)))\n",
    "    Xrb = np.c_[np.ones((40, 1)), Xr]\n",
    "    ours = ridge(Xrb, yr, alpha=1.0).ravel()\n",
    "    sk = Ridge(alpha=1.0).fit(Xr, yr)\n",
    "    ref = np.r_[float(sk.intercept_), sk.coef_.ravel()]\n",
    "    check_close(ours, ref, atol=1e-6,\n",
    "                msg=\"closed-form ridge must match sklearn's Ridge (same penalty, bias unpenalized)\")\n",
    "\n",
    "check(\"B2 ridge == sklearn\", _ridge_vs_sklearn)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c19cd23d",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>It is the normal equation with `alpha * A` added inside the inverse, where `A` is the identity with `A[0,0] = 0`. That added term also makes the matrix invertible even when `X_b.T @ X_b` is singular.</details>\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "return np.linalg.inv(X_b.T @ X_b + alpha * A) @ (X_b.T @ y)\n",
    "```\n",
    "</details>\n",
    "<details><summary>Help — matches sklearn on the weights but not the intercept</summary>You probably penalized the bias too. Make sure `A[0, 0] = 0` so the intercept column is excluded from the L2 penalty; sklearn does the same by centering.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "dd4935c3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:46:11.869582Z",
     "iopub.status.busy": "2026-06-10T18:46:11.869387Z",
     "iopub.status.idle": "2026-06-10T18:46:11.880350Z",
     "shell.execute_reply": "2026-06-10T18:46:11.876862Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] B2 ridge == sklearn\n",
      "ridge theta on the y=4+3x problem (alpha=1): [4.0777924  2.85689377]\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines ridge; the check below re-verifies the reference.\n",
    "def ridge(X_b, y, alpha=1.0):\n",
    "    A = np.eye(X_b.shape[1])\n",
    "    A[0, 0] = 0.0\n",
    "    return np.linalg.inv(X_b.T @ X_b + alpha * A) @ (X_b.T @ y)\n",
    "\n",
    "check(\"B2 ridge == sklearn\", _ridge_vs_sklearn, required=True)\n",
    "print(\"ridge theta on the y=4+3x problem (alpha=1):\", ridge(X_b, y, alpha=1.0).ravel())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "719f317d",
   "metadata": {},
   "source": [
    "> **Note:** lasso (L1) has no closed form because $|\\theta|$ is not differentiable at zero; it needs coordinate descent or a proximal step. That non-differentiable corner is exactly what drives some weights to *exactly* zero, giving lasso its feature-selection property that ridge lacks.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8daab40",
   "metadata": {},
   "source": [
    "### Part C — Capstone: early stopping on a problem that actually overfits\n",
    "\n",
    "Iris is too separable to overfit, so it cannot demonstrate early stopping; you would just train to convergence. Build instead a small, noisy, high-dimensional binary problem (80 examples, 40 features, label noise) where a softmax-regression model will start memorizing the training noise, and watch the validation loss turn back up. Then add **early stopping**: track validation cross-entropy, keep the best-so-far parameters, and stop when it has not improved for `patience` evaluations, restoring the best checkpoint at the end.\n",
    "\n",
    "Deliverables:\n",
    "1. A noisy synthetic binary problem with more features than is healthy, split train/validation (stratified, seeded).\n",
    "2. A training loop that records validation cross-entropy each step and restores the best parameters at the end.\n",
    "3. The stopping step, the best step, and the validation accuracy of the restored model versus the last-step model.\n",
    "\n",
    "Self-assessment (pass / partial / fail): (a) the loop restores the parameters from the best validation step, not the last; (b) validation loss is computed on the whole validation set, not a batch; (c) the early-stop triggers well before `max_steps` because the problem overfits; (d) the restored model's validation loss is below the last-step validation loss (early stopping actually helped); (e) the notebook still runs top-to-bottom.\n",
    "\n",
    "> **Common confusion:** early stopping restores the *best* checkpoint, not wherever you happened to stop. Track `best_Theta` separately from the live `Theta`. Also evaluate validation loss on the full validation set; per-batch validation noise causes spurious early stops.\n",
    "\n",
    "<details><summary>My solution (reference, runs in seconds on CPU)</summary>\n",
    "\n",
    "```python\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# a problem that overfits: 80 examples, 40 features, ~15% label noise\n",
    "gen = np.random.default_rng(SEED)\n",
    "n_cap, d_cap = 80, 40\n",
    "X_raw = gen.standard_normal((n_cap, d_cap))\n",
    "w_true = gen.standard_normal(d_cap)\n",
    "y_cap = (X_raw @ w_true + 1.5 * gen.standard_normal(n_cap) > 0).astype(int)  # noisy labels\n",
    "X_cap = StandardScaler().fit_transform(X_raw)\n",
    "\n",
    "Xtr, Xval, ytr, yval = train_test_split(\n",
    "    X_cap, y_cap, test_size=0.35, stratify=y_cap, random_state=SEED)\n",
    "Xtrb  = np.c_[np.ones((len(Xtr), 1)), Xtr]\n",
    "Xvalb = np.c_[np.ones((len(Xval), 1)), Xval]\n",
    "Ytr   = np.eye(2)[ytr]\n",
    "Yval  = np.eye(2)[yval]\n",
    "\n",
    "Theta = np.zeros((Xtrb.shape[1], 2))\n",
    "best_val, best_Theta, best_step, since = float(\"inf\"), Theta.copy(), 0, 0\n",
    "max_steps, patience, eta = 3000, 30, 0.3\n",
    "for step in range(max_steps):\n",
    "    P = softmax(Xtrb @ Theta)\n",
    "    Theta = Theta - eta * (Xtrb.T @ (P - Ytr) / len(Xtrb))\n",
    "    val_loss = cross_entropy(softmax(Xvalb @ Theta), Yval)   # full val set\n",
    "    if val_loss < best_val - 1e-5:\n",
    "        best_val, best_Theta, best_step, since = val_loss, Theta.copy(), step, 0\n",
    "    else:\n",
    "        since += 1\n",
    "        if since >= patience:\n",
    "            break\n",
    "last_loss = cross_entropy(softmax(Xvalb @ Theta), Yval)\n",
    "val_pred = np.argmax(Xvalb @ best_Theta, axis=1)             # restore best\n",
    "print(f\"stopped at step {step}, best val at step {best_step}\")\n",
    "print(f\"best val loss {best_val:.3f} vs last-step val loss {last_loss:.3f} \"\n",
    "      f\"(early stopping kept the lower one)\")\n",
    "print(f\"restored-model val accuracy {(val_pred == yval).mean():.3f}\")\n",
    "```\n",
    "\n",
    "The lesson: training loss keeps falling the whole time, but validation loss bottoms out early (around step 15) and then climbs as the model memorizes noise. Early stopping ships the parameters from that validation minimum. The restored model's validation loss is below the last-step model's, which is the whole point; the validation accuracy lands around 0.79 because ~15% of the labels are noise the model should not fit.</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1709458",
   "metadata": {},
   "source": [
    "## Reflection\n",
    "\n",
    "Write ~150 words on the dumbest bug you hit in this notebook and how you found it. The classic one here is a flipped gradient sign that makes the loss climb instead of fall, or forgetting to keep `xi` two-dimensional in the SGD loop so the matmul silently broadcasts wrong. Nobody grades this; writing it is the point. Naming the symptom (loss went up, shape error, predictions all one class), the cause, and the one print statement that confirmed the fix is how you build the debugging reflexes that the rest of the book leans on.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a63a79d",
   "metadata": {},
   "source": [
    "## Going further\n",
    "- Géron, *Hands-On Machine Learning* 3e, Ch 4 — the source for the four-route normal equation, the learning-rate figures, and the regularization geometry compressed here.\n",
    "- Goh, *Why Momentum Really Works* (Distill, 2017) — the best single visual on why momentum accelerates ill-conditioned problems.\n",
    "- Kingma & Ba, *Adam* (2014) and Loshchilov & Hutter, *Decoupled Weight Decay Regularization* (AdamW, 2019) — the two papers behind the optimizer toolkit's last two classes.\n",
    "- scikit-learn user guide, *Linear Models* — `Ridge`, `Lasso`, `ElasticNet`, and `SGDRegressor` with the same math, production-hardened.\n",
    "- Boyd & Vandenberghe, *Convex Optimization*, ch. 9 — why gradient descent converges, the $2/L$ step bound, and when to use Newton's method instead.\n",
    "\n",
    "## What this enables\n",
    "- **Ch 05 — SVMs, trees, kernels**: the L2 penalty geometry here is exactly the max-margin geometry there, and SVMs train with the same gradient-plus-regularization toolkit.\n",
    "- **Ch 06 — Ensembles**: gradient boosting is gradient descent in function space; this chapter's parameter-space GD is the prerequisite.\n",
    "- **Ch 09-11 — Neural networks**: every network trains by mini-batch SGD or AdamW with weight decay. This is the gentle case where you can verify the optimizer against a closed form before there is no closed form left to check against.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af33dc86",
   "metadata": {},
   "source": [
    "---\n",
    "*Built top-to-bottom. If every check above printed `[ ok ]`, you've reproduced the chapter. Runtime stamp written by CI.*\n",
    "\n",
    "Total running time: written by CI · Verified on: numpy 2.x, torch 2.x, Python 3.12\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "obvix-nb",
   "language": "python",
   "name": "obvix-nb"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "obvix": {
   "title": "Ch 04 — Training Models"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}