{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d224194e",
   "metadata": {},
   "source": [
    "# Ch 06 — Ensemble Methods (notebook)\n",
    "\n",
    "`[← 05 decision-trees-svms-and-kernels]` · **this notebook** · `[07 dimensionality-reduction →]`\n",
    "\n",
    "Runs top-to-bottom in ~3 min on free Colab CPU. Last verified 2026-06-11.\n",
    "\n",
    "**What you'll build**\n",
    "- A majority-vote simulator that turns 51%-accurate coin-flip classifiers into a 98%-accurate ensemble, and the assert that proves error shrinks as `1/sqrt(N)`.\n",
    "- Bagging from scratch (bootstrap resample, fit, majority vote) in ~15 lines, plus the proof that each bootstrap sample covers ~63% of the data.\n",
    "- The demonstration that a `RandomForestClassifier` *is* bagging of trees plus feature subsampling, checked by reconstructing its prediction from `estimators_` exactly.\n",
    "- AdaBoost from scratch in ~15 lines, agreeing with sklearn on the same stumps; gradient boosting from scratch as \"fit a tree to the residuals\", agreeing with `GradientBoostingRegressor`.\n",
    "- A stacking pipeline that you first break with target leakage (train accuracy 1.00, learns nothing), then fix with out-of-fold predictions.\n",
    "\n",
    "**How this notebook works.** Code cells with a `# TODO` are yours to fill in. Run the cell to grade yourself: `[ ok ]` passed, `[FAIL]` shows what went wrong, `[ -- ]` means not attempted yet. Every exercise has a hint ladder (open only as many as you need) and a folded solution below it. The notebook runs top-to-bottom even if you fill in nothing: the solution cells redefine the functions so later cells work. See Ch 00 for the full protocol.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ee6b482",
   "metadata": {},
   "source": [
    "## Before you start\n",
    "\n",
    "Three questions to set intention. Each is answerable from Ch 04-05; the answer is folded.\n",
    "\n",
    "1. A single decision tree grown to full depth has low bias and high variance. What does \"high variance\" mean for a learning algorithm? <details><summary>Answer</summary>Refit the same model on a slightly different sample of the same distribution and the learned function changes a lot. The predictions wobble from dataset to dataset. Ensembles attack exactly this: averaging many high-variance trees cancels the wobble.</details>\n",
    "2. You average `N` independent noisy estimates of the same quantity, each with standard deviation `sigma`. What is the standard deviation of the average? <details><summary>Answer</summary>`sigma / sqrt(N)`. The variance of a mean of `N` i.i.d. terms is `sigma^2 / N`. This single fact is why bagging works, and the word \"independent\" is why it sometimes does not.</details>\n",
    "3. Predict before you run: you draw a bootstrap sample (sample `m` rows with replacement from `m` rows). Roughly what fraction of the original rows appear at least once? <details><summary>Answer</summary>About 63%. The probability a given row is never picked in `m` draws is `(1 - 1/m)^m -> 1/e ≈ 0.368`, so ~63.2% are picked at least once. The missing ~37% become the out-of-bag set, a free validation fold per tree.</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8e195e6",
   "metadata": {},
   "source": [
    "## Setup\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0204e96f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:46.894592Z",
     "iopub.status.busy": "2026-06-10T18:50:46.894494Z",
     "iopub.status.idle": "2026-06-10T18:50:47.493110Z",
     "shell.execute_reply": "2026-06-10T18:50:47.492752Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "numpy 2.2.6 · pandas 2.3.3 · sklearn 1.7.2\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import sklearn\n",
    "print(f\"numpy {np.__version__} · pandas {pd.__version__} · sklearn {sklearn.__version__}\")\n",
    "if np.__version__ < \"2.0\":\n",
    "    print(\"WARN: written for NumPy 2.x; older versions may shift the last digit or two\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7b122a95",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:47.494383Z",
     "iopub.status.busy": "2026-06-10T18:50:47.494226Z",
     "iopub.status.idle": "2026-06-10T18:50:47.498753Z",
     "shell.execute_reply": "2026-06-10T18:50:47.498366Z"
    }
   },
   "outputs": [],
   "source": [
    "import os, random\n",
    "SEED = 0\n",
    "FAST = bool(os.environ.get('NB_FAST'))  # CI smoke mode: ~10x fewer estimators, same code paths\n",
    "# Estimator counts scale with FAST so the full run stays well under budget.\n",
    "N_TREES   = 50  if FAST else 300   # forest / bagging size\n",
    "N_ROUNDS  = 30  if FAST else 200   # boosting rounds\n",
    "N_TRIALS  = 300 if FAST else 2000  # Monte-Carlo repeats for the variance demo\n",
    "rng = np.random.default_rng(SEED)\n",
    "random.seed(SEED)\n",
    "\n",
    "# ── house self-check harness (identical across all chapter notebooks) ──\n",
    "import numpy as _np\n",
    "\n",
    "def check(label, test_fn, required=False):\n",
    "    \"\"\"Run one self-check. test_fn raises AssertionError (with a teaching\n",
    "    message) on failure, NotImplementedError if the stub is unfilled.\n",
    "    required=True is used only in solution cells; it is what CI grades.\"\"\"\n",
    "    try:\n",
    "        test_fn()\n",
    "    except NotImplementedError:\n",
    "        if required:\n",
    "            raise AssertionError(f\"{label}: reference solution incomplete\")\n",
    "        print(f\"[ -- ] {label}: not attempted yet — fill in the TODO above, then re-run.\")\n",
    "        return False\n",
    "    except AssertionError as e:\n",
    "        if required:\n",
    "            raise\n",
    "        print(f\"[FAIL] {label}: {e}\")\n",
    "        return False\n",
    "    print(f\"[ ok ] {label}\")\n",
    "    return True\n",
    "\n",
    "def attempted(*vals):\n",
    "    \"\"\"Treat None placeholders as 'not attempted'.\"\"\"\n",
    "    if any(v is None for v in vals):\n",
    "        raise NotImplementedError\n",
    "\n",
    "def check_shape(x, want):\n",
    "    assert tuple(x.shape) == tuple(want), \\\n",
    "        f\"shape {tuple(x.shape)}, expected {tuple(want)} — check your reshape/transpose order\"\n",
    "\n",
    "def check_close(got, want, atol=1e-5, rtol=1e-4, msg=\"\"):\n",
    "    g, w = _np.asarray(got, dtype=float), _np.asarray(want, dtype=float)\n",
    "    assert g.shape == w.shape, f\"shape {g.shape} vs expected {w.shape}. {msg}\"\n",
    "    bad = ~_np.isclose(g, w, atol=atol, rtol=rtol)\n",
    "    assert not bad.any(), \\\n",
    "        f\"{bad.mean():.2%} of values wrong (max diff {abs(g - w).max():.3g}). {msg}\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e592780",
   "metadata": {},
   "source": [
    "> **Note:** seeds make this notebook's printed numbers reproduce on CPU. Library versions and BLAS threading can shift the last digit or two; quoted numbers hold for the pinned environment. If the page says OOB 0.877 and you see 0.879, you did nothing wrong.\n",
    "\n",
    "> **Caveat:** the production gradient boosters (XGBoost, LightGBM, CatBoost) are not imported here. They are not in Colab's default image and installing them can trigger a runtime restart, which breaks run-all. Everything in this notebook runs on `sklearn` alone, and the from-scratch gradient booster in Part 6 is the same algorithm those libraries optimize. The \"Going further\" section points to them.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1ca02bf",
   "metadata": {},
   "source": [
    "## The map\n",
    "\n",
    "> **Part 1 — Why ensembles work.** Simulate majority voting over weak classifiers; watch accuracy climb with `N`; assert the `1/sqrt(N)` error shrink and see what correlation does to it.\n",
    "> **Part 2 — Voting classifiers.** Combine a logistic regression, an SVM, and a forest on moons; hard vs soft voting; when diversity beats individual quality.\n",
    "> **Part 3 — Bagging from scratch.** Bootstrap-resample, fit, majority-vote; prove the 63% coverage; compare to `BaggingClassifier`.\n",
    "> **Part 4 — Random forests are bagged trees.** Reconstruct an RF prediction from its `estimators_` exactly; show feature subsampling decorrelates the trees; read `feature_importances_` and its bias.\n",
    "> **Part 5 — Out-of-bag evaluation.** Build the free validation set the bootstrap leaves behind; check it tracks the held-out test score.\n",
    "> **Part 6 — Boosting.** AdaBoost from scratch agreeing with sklearn; gradient boosting as \"fit a tree to the residuals\"; the bias-vs-variance contrast with bagging.\n",
    "> **Part 7 — Stacking, broken then fixed.** Train a meta-learner on leaked in-sample predictions (train accuracy 1.00), then fix it with out-of-fold predictions.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e299dffb",
   "metadata": {},
   "source": [
    "## Part 1 — Why ensembles work: the law of large numbers applied to classifiers\n",
    "\n",
    "> **Objectives.** Simulate a majority vote over `N` weak classifiers and watch accuracy climb. Assert that the standard error of an averaged estimate falls as `1/sqrt(N)`. See that the whole argument rests on the word *independent*.\n",
    "\n",
    "Take a biased coin that lands heads 51% of the time. Flip it ten times and you might see eight heads. Flip it ten thousand times and the heads ratio is 51% give or take a fraction of a percent. Now read \"coin\" as \"classifier\" and \"heads\" as \"correct\". A majority vote over many independent classifiers, each right with probability `p > 0.5`, is wrong only when more than half of them are wrong, and that gets exponentially unlikely as `N` grows.\n",
    "\n",
    "The mechanism is the variance of an average. If each classifier's correctness is an independent Bernoulli draw with mean `p`, the fraction-correct over `N` of them has standard deviation `sqrt(p(1-p)/N)`, which shrinks as `1/sqrt(N)`. The bias (the gap from `p` to the truth) does not shrink. Averaging kills variance, not bias.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "757145ce",
   "metadata": {},
   "source": [
    "> **Predict:** with `p = 0.51` and a 1000-classifier majority vote over *independent* classifiers, will accuracy be closer to 0.51, 0.75, or 0.98? <details><summary>Answer</summary>Over 0.75, and with 10000 classifiers over 0.98. The simulation below shows it. The catch is that real classifiers trained on the same data are nowhere near independent.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0e6d28c0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:47.499665Z",
     "iopub.status.busy": "2026-06-10T18:50:47.499588Z",
     "iopub.status.idle": "2026-06-10T18:50:47.519276Z",
     "shell.execute_reply": "2026-06-10T18:50:47.518794Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "N=    1  majority-vote accuracy ≈ 0.509\n",
      "N=   11  majority-vote accuracy ≈ 0.525\n",
      "N=  101  majority-vote accuracy ≈ 0.588\n",
      "N= 1001  majority-vote accuracy ≈ 0.721\n"
     ]
    }
   ],
   "source": [
    "# Monte-Carlo majority vote over N independent weak classifiers, each correct w.p. p.\n",
    "def majority_vote_accuracy(p, N, trials, rng):\n",
    "    # correct[t, i] = 1 if classifier i is right on trial t\n",
    "    correct = (rng.random((trials, N)) < p).astype(int)   # (trials, N), independent\n",
    "    n_correct = correct.sum(axis=1)                        # (trials,)\n",
    "    majority_right = n_correct > (N / 2)                   # vote correct iff >half right\n",
    "    return majority_right.mean()\n",
    "\n",
    "p = 0.51\n",
    "for N in [1, 11, 101, 1001]:\n",
    "    acc = majority_vote_accuracy(p, N, N_TRIALS, rng)\n",
    "    print(f\"N={N:5d}  majority-vote accuracy ≈ {acc:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a438059",
   "metadata": {},
   "source": [
    "> **Interpretation.** Each classifier alone is barely better than a coin flip (0.51). The vote of a thousand of them lands well above 0.7, and the climb is monotone in `N`. This is not a metaphor: it is the literal recipe behind bagging and random forests. Note the numbers carry Monte-Carlo noise, so they wobble run to run; the trend does not.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76963aa8",
   "metadata": {},
   "source": [
    "### The variance of an average shrinks as 1/sqrt(N)\n",
    "\n",
    "The claim under the simulation is exact and worth asserting directly. Draw `N` i.i.d. samples with standard deviation `sigma`, take their mean, repeat many times, and measure the standard deviation of those means. Theory says it equals `sigma / sqrt(N)`. Quadruple `N` and the spread halves.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "81565aae",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:47.520203Z",
     "iopub.status.busy": "2026-06-10T18:50:47.520108Z",
     "iopub.status.idle": "2026-06-10T18:50:47.529516Z",
     "shell.execute_reply": "2026-06-10T18:50:47.529163Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "N=   1  empirical std 0.2964   theory sigma/sqrt(N) 0.3000\n",
      "N=   4  empirical std 0.1517   theory sigma/sqrt(N) 0.1500\n",
      "N=  16  empirical std 0.0747   theory sigma/sqrt(N) 0.0750\n",
      "N=  64  empirical std 0.0373   theory sigma/sqrt(N) 0.0375\n",
      "N= 256  empirical std 0.0186   theory sigma/sqrt(N) 0.0187\n",
      "[ ok ] 1.1 std of mean ≈ sigma/sqrt(N)\n",
      "[ ok ] 1.1 quadrupling N halves the spread\n"
     ]
    }
   ],
   "source": [
    "# Empirical std of the sample mean vs the theoretical sigma/sqrt(N).\n",
    "sigma = 0.3\n",
    "Ns = np.array([1, 4, 16, 64, 256])\n",
    "emp = []\n",
    "for N in Ns:\n",
    "    means = rng.normal(0.7, sigma, size=(N_TRIALS, N)).mean(axis=1)  # (N_TRIALS,) means of N draws\n",
    "    emp.append(means.std())\n",
    "emp = np.array(emp)\n",
    "theory = sigma / np.sqrt(Ns)\n",
    "for N, e, t in zip(Ns, emp, theory):\n",
    "    print(f\"N={N:4d}  empirical std {e:.4f}   theory sigma/sqrt(N) {t:.4f}\")\n",
    "\n",
    "# Spanish-inquisition rule: the markdown claim becomes an assert.\n",
    "_ = check(\"1.1 std of mean ≈ sigma/sqrt(N)\",\n",
    "      lambda: check_close(emp, theory, atol=0.02, rtol=0.0,\n",
    "          msg=\"std of the mean should match sigma/sqrt(N) within Monte-Carlo error\"))\n",
    "shrink = emp[0] / emp[-1]\n",
    "_ = check(\"1.1 quadrupling N halves the spread\",\n",
    "      lambda: check_close(shrink, np.sqrt(256), atol=0.0, rtol=0.10,\n",
    "          msg=f\"std should shrink by sqrt(256)=16x from N=1 to N=256; got {shrink:.2f}x\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd3cf816",
   "metadata": {},
   "source": [
    "> **Intuition:** bagging is this picture with \"noisy estimate of 0.7\" replaced by \"one tree's prediction\". Average many trees and the part of their error that is independent across trees cancels at rate `1/sqrt(N)`. The part that is shared (correlated) does not cancel, which is the whole subject of Part 4.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8573f2d",
   "metadata": {},
   "source": [
    "### Exercise 6.1 — Correlation is what ruins it\n",
    "`Difficulty 3/5 · ~12 min`\n",
    "\n",
    "Real classifiers trained on the same data make *correlated* errors, so they are not the independent voters Part 1 assumed. Model that. Each of `N` classifiers is right with probability `p`, but their correctness is correlated: with probability `rho` they all copy a single shared draw, and with probability `1 - rho` each draws independently. Implement `correlated_vote_accuracy(p, N, rho, trials, rng)` and confirm that at `rho = 1` the ensemble is no better than one classifier, while at `rho = 0` it recovers Part 1's independent case.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "606a8b36",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:47.530392Z",
     "iopub.status.busy": "2026-06-10T18:50:47.530319Z",
     "iopub.status.idle": "2026-06-10T18:50:47.533645Z",
     "shell.execute_reply": "2026-06-10T18:50:47.533100Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] 6.1 correlation extremes: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    }
   ],
   "source": [
    "def correlated_vote_accuracy(p, N, rho, trials, rng):\n",
    "    \"\"\"Majority-vote accuracy of N classifiers, each correct w.p. p, with a\n",
    "    'shared-draw' correlation rho in [0, 1].\n",
    "\n",
    "    For each trial: with prob rho all N classifiers share ONE Bernoulli(p) draw\n",
    "    (perfectly correlated); with prob 1 - rho each draws its own Bernoulli(p).\n",
    "    Return the fraction of trials whose MAJORITY vote is correct.\n",
    "    \"\"\"\n",
    "    # TODO 1: shared[t] = one Bernoulli(p) per trial, shape (trials, 1)\n",
    "    shared = None\n",
    "    # TODO 2: indep[t, i] = Bernoulli(p) per classifier, shape (trials, N)\n",
    "    indep = None\n",
    "    # TODO 3: use_shared[t] = Bernoulli(rho) per trial, shape (trials, 1)\n",
    "    use_shared = None\n",
    "    attempted(shared, indep, use_shared)\n",
    "    # TODO 4: correct = where(use_shared, broadcast(shared), indep)  -> (trials, N)\n",
    "    correct = None\n",
    "    attempted(correct)\n",
    "    # TODO 5: vote correct iff more than half the row is correct\n",
    "    majority_right = None\n",
    "    attempted(majority_right)\n",
    "    return float(majority_right.mean())\n",
    "\n",
    "def _corr_extremes():\n",
    "    # rho=0 recovers the independent ensemble (should be well above p for large N)\n",
    "    a0 = correlated_vote_accuracy(0.51, 201, 0.0, N_TRIALS, np.random.default_rng(SEED))\n",
    "    assert a0 > 0.60, f\"rho=0 should behave like independent voting (>0.60), got {a0:.3f}\"\n",
    "    # rho=1 collapses to a single classifier (≈ p, far below the independent ensemble)\n",
    "    a1 = correlated_vote_accuracy(0.51, 201, 1.0, N_TRIALS, np.random.default_rng(SEED))\n",
    "    assert abs(a1 - 0.51) < 0.06, f\"rho=1 should collapse to ≈p=0.51, got {a1:.3f}\"\n",
    "    assert a0 > a1 + 0.05, \"fully-correlated voters must lose to independent ones\"\n",
    "\n",
    "_ = check(\"6.1 correlation extremes\", _corr_extremes)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "853cc7c8",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>Build two arrays of correctness, `shared` (one column, broadcast across all `N`) and `indep` (full `(trials, N)`), then pick between them per trial with `np.where` driven by a Bernoulli(`rho`) mask. The majority test is the same `> N/2` from the cell above.</details>\n",
    "\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "shared     = (rng.random((trials, 1)) < p).astype(int)   # one draw per trial\n",
    "indep      = (rng.random((trials, N)) < p).astype(int)   # one draw per classifier\n",
    "use_shared = (rng.random((trials, 1)) < rho)             # bool, per trial\n",
    "correct    = np.where(use_shared, shared, indep)         # broadcasting fills N columns\n",
    "majority_right = correct.sum(axis=1) > (N / 2)\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — my rho=1 accuracy is ~0.5 even though p=0.51</summary>At `rho=1` every classifier copies the same draw, so the vote equals one Bernoulli(0.51) draw; over finite trials that estimates 0.51 with Monte-Carlo noise. That is the point: correlated voters give you one classifier, not `N`. If it is far from 0.51, check that `shared` is broadcast (shape `(trials, 1)`) and not redrawn per column.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4262c5e0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:47.534415Z",
     "iopub.status.busy": "2026-06-10T18:50:47.534347Z",
     "iopub.status.idle": "2026-06-10T18:50:47.545793Z",
     "shell.execute_reply": "2026-06-10T18:50:47.545450Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 6.1 correlation extremes\n",
      "rho=0.0  201-voter accuracy ≈ 0.622\n",
      "rho=0.3  201-voter accuracy ≈ 0.595\n",
      "rho=0.7  201-voter accuracy ≈ 0.556\n",
      "rho=1.0  201-voter accuracy ≈ 0.509\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines correlated_vote_accuracy; the check below re-verifies it.\n",
    "def correlated_vote_accuracy(p, N, rho, trials, rng):\n",
    "    shared     = (rng.random((trials, 1)) < p).astype(int)\n",
    "    indep      = (rng.random((trials, N)) < p).astype(int)\n",
    "    use_shared = (rng.random((trials, 1)) < rho)\n",
    "    correct    = np.where(use_shared, shared, indep)\n",
    "    majority_right = correct.sum(axis=1) > (N / 2)\n",
    "    return float(majority_right.mean())\n",
    "\n",
    "_ = check(\"6.1 correlation extremes\", _corr_extremes, required=True)\n",
    "for rho in [0.0, 0.3, 0.7, 1.0]:\n",
    "    a = correlated_vote_accuracy(0.51, 201, rho, N_TRIALS, np.random.default_rng(SEED))\n",
    "    print(f\"rho={rho:.1f}  201-voter accuracy ≈ {a:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8aaed2a8",
   "metadata": {},
   "source": [
    "> **Key takeaways.**\n",
    "> - Majority voting over weak-but-better-than-chance classifiers climbs toward 1.0 as `N` grows, because the variance of an average shrinks as `1/sqrt(N)`.\n",
    "> - The shrink only happens for the *independent* part of the error. Correlation between voters is dead weight; at full correlation `N` voters equal one.\n",
    "> - Every ensemble method in this notebook is, at heart, a different trick for making the base learners disagree.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ccb412f",
   "metadata": {},
   "source": [
    "## Part 2 — Voting classifiers: the cheapest ensemble\n",
    "\n",
    "> **Objectives.** Train three different model families on the moons dataset and combine them with `VotingClassifier`. Contrast hard voting (vote on labels) with soft voting (average probabilities). See why diversity matters more than individual quality.\n",
    "\n",
    "The moons dataset is the anchor for the rest of this notebook: two interleaving half-moons, 30% label noise, a boundary no single linear model can draw. We build it once and reuse the same split everywhere.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "203c1538",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:47.546764Z",
     "iopub.status.busy": "2026-06-10T18:50:47.546689Z",
     "iopub.status.idle": "2026-06-10T18:50:47.645704Z",
     "shell.execute_reply": "2026-06-10T18:50:47.645281Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train (375, 2), test (125, 2), positive rate 0.48\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAekAAAGGCAYAAABfbHkYAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAA1WdJREFUeJzsnXV8VeUbwL/nxroZ24gxRo1RozsVCenGoKQVERBRxEJRREL4EdKpSEkJ0imdo5sNBqxY592N9/fHhTsu21iwAs/389lHz3veeM69l/O88YQkhBDIyMjIyMjIFDoUBS2AjIyMjIyMTPrISlpGRkZGRqaQIitpGRkZGRmZQoqspGVkZGRkZAopspKWkZGRkZEppMhKWkZGRkZGppAiK2kZGRkZGZlCiqykZWRkZGRkCimykpaRkZGRkSmkyEpa5j9LYGAgkiQxbdq0ghbltWfdunW4uLgQHx+fZ2McPHgQSZI4ePBgno3xOqDVavH09GTevHkFLYpMFpCVtIyMTJ6i1+v59ttv+fjjj7GzsytocQqEa9eu0aZNG+zs7HBxcaFPnz6Eh4dnqe3o0aOpWbMmLi4u2NjY4Ovry3fffZfuhEej0fD5559TvHhxrK2tqVevHnv27DGro1arGTNmDD/++CPJycm58nwyeYckx+6W+a8SGBiIt7c3U6dOZezYsQUtzmvL5s2b6dq1K0FBQZQoUSLPxjEYDKSkpGBhYYFCUXjWHw8ePKBGjRo4OjoycuRI4uPjmTZtGqVKleLUqVNYWFi8sH3jxo2pVasW5cqVw8rKivPnz7N06VJq167N4cOHzZ71nXfeYcOGDYwaNYry5cuzfPlyTp8+zYEDB2jcuLGpXnR0NO7u7vz222988MEHefbsMrmAkJH5jxIQECAAMXXq1IIW5bWmY8eOonHjxgUtRoExfPhwYW1tLe7du2cq27NnjwDEggULctTntGnTBCCOHz9uKjt58mSa33NSUpIoW7asaNCgQZo+2rdvL5o0aZKj8WXyj8Iz3ZSRecJ3332HJEncvHmT999/H0dHR4oWLcrXX3+NEIKgoCA6deqEg4MDHh4eTJ8+3ax9SkoK33zzDbVq1cLR0RFbW1uaNGnCgQMHMh1bCMGQIUOwsLBg48aNpvLff/+dWrVqYW1tjYuLC7179yYoKChbz7V8+XIkSeLIkSOMHDmSokWL4uTkxNChQ0lJSSE6Opq+ffvi7OyMs7Mz48aNQzy30ZWQkMCnn36Kp6cnlpaW+Pj4MG3atDT1dDodP/zwA2XLlsXS0pLSpUvz5ZdfotFozOqVLl2a9u3bc+TIEerWrYuVlRVlypRh5cqVZvW0Wi0TJ06kfPnyWFlZUaRIERo3bpxmK/V5kpOT2blzJy1btkxzT5IkRowYwebNm6lSpQqWlpZUrlyZnTt3pql7/vx52rZti4ODA3Z2drz55pucOHHCrE56Z9K3bt2iW7dueHh4YGVlRcmSJenduzcxMTFmbXPj+82Iv/76i/bt21OqVClTWcuWLalQoQLr1q3LUZ+lS5cGjCvip2zYsAGlUsmQIUNMZVZWVgwcOJDjx4+neZ633nqLI0eOEBkZmSMZZPKJgp0jyMik5dtvvxWAqF69unjnnXfEvHnzRLt27QQgZsyYIXx8fMTw4cPFvHnzRKNGjQQgDh06ZGofHh4uihUrJsaMGSN+++038csvvwgfHx+hVqvF+fPnTfWeX0nrdDrRt29fYWlpKbZt22aqN2nSJCFJkujVq5eYN2+emDhxonB1dRWlS5cWUVFRWX6uZcuWmZ6rTZs2Yu7cuaJPnz4CEOPGjRONGzcW7777rpg3b55o3769AMSKFStM7Q0Gg3jjjTeEJEli0KBBYs6cOaJDhw4CEKNGjTIbq1+/fgIQ3bt3F3PnzhV9+/YVgOjcubNZPS8vL+Hj4yPc3d3Fl19+KebMmSNq1qwpJEkSly9fNtX78ssvhSRJYvDgwWLRokVi+vTp4p133hE///zzC5/5yJEjAhBbt25Ncw8Qfn5+olixYuKHH34QM2fOFGXKlBE2Njbi8ePHpnqXL18Wtra2pno///yz8Pb2FpaWluLEiROmegcOHBCAOHDggBBCCI1GI7y9vUXx4sXFpEmTxOLFi8XEiRNFnTp1RGBgoKldVr/fhIQEER4enulfZGSkqc2DBw8EIKZMmZLm+d9//33h4uLyws/vKVqtVoSHh4uHDx+KXbt2iYoVKwp7e3sRERFhqtOyZUvh6+ubpu3evXvT/Q6efjd///13lmSQKRhkJS1T6HiqpIcMGWIq0+l0omTJkkKSJDPFEBUVJaytrUW/fv3M6mo0GrM+o6KihLu7u/jggw9MZc8qaa1WK3r16iWsra3Frl27THUCAwOFUqkUP/74o1l/ly5dEiqVKk35i3iqpFu3bi0MBoOpvEGDBkKSJDFs2LA0z9usWTNT2ebNmwUgJk2aZNZv9+7dhSRJ4vbt20IIIfz9/QUgBg0aZFZv7NixAhD79+83lXl5eQlAHD582FQWFhYmLC0txaeffmoq8/PzE+3atcvysz5l8eLFAhCXLl1Kcw8QFhYWJrmFEOLChQsCELNnzzaVde7cWVhYWIg7d+6Yyh49eiTs7e1F06ZNTWXPK+nz588LQKxfvz5D+bLz/T79XWb25+XlZWpz+vRpAYiVK1emGfuzzz4TgEhOTs5QvqccP37cbAwfHx/Tcz6lcuXK4o033kjT9sqVKwIQ8+fPNyt/9OhRhhMImcKDvN0tU2gZNGiQ6f+VSiW1a9dGCMHAgQNN5U5OTvj4+HD37l2zuk+NcQwGA5GRkeh0OmrXrs25c+fSjJOSkkKPHj3Ytm0b//zzD61atTLd27hxIwaDgZ49e/L48WPTn4eHB+XLl8/SFvrzDBw4EEmSTNf16tVL81xPn/fZ5/rnn39QKpWMHDnSrL9PP/0UIQQ7duww1QMYM2ZMmnoA27dvNyuvVKkSTZo0MV0XLVo0zWfq5OTElStXuHXrVraeNSIiAgBnZ+d077ds2ZKyZcuarqtVq4aDg4NpbL1ez+7du+ncuTNlypQx1StWrBjvvvsuR44cITY2Nt2+HR0dAdi1axeJiYnp1snO99u3b1/27NmT6d8ff/xhapOUlASApaVlmrGtrKzM6ryISpUqsWfPHjZv3sy4ceOwtbVNY92dlJSUrXGefiePHz/OdHyZgkNV0ALIyGTEs2d4YHzpWllZ4erqmqb8qTJ4yooVK5g+fTrXr19Hq9Wayr29vdOMM3nyZOLj49mxYwfNmzc3u3fr1i2EEJQvXz5dGdVqdXYeCUj/uQA8PT3TlEdFRZmu7927R/HixbG3tzer5+vra7r/9L8KhYJy5cqZ1fPw8MDJyclULyN5wPgCf3bs77//nk6dOlGhQgWqVKlCmzZt6NOnD9WqVcvSM4sMnEgyGzs8PJzExER8fHzS1PP19cVgMBAUFETlypXT3Pf29mbMmDHMmDGDP/74gyZNmtCxY0eTnQNk7/stU6aM2UQhK1hbWwOksQUATO5PT+u8CAcHB9O5fqdOnVi9ejWdOnXi3Llz+Pn5mfrJzjhPv5NnJ4wyhQ9ZScsUWpRKZZbKwFwJ/P777/Tv35/OnTvz2Wef4ebmhlKpZPLkydy5cydN29atW7Nz505++eUXmjdvblp5gHElLkkSO3bsSHfsnPj9ZvQM6ZVnpNyyQlZfvln5TJs2bcqdO3fYsmULu3fvZvHixfz666/Mnz/fbMfjeYoUKQJAVFQUJUuWzNHYL8P06dPp37+/Se6RI0cyefJkTpw4QcmSJbP1/cbHx2cpGItSqaRo0aKAccUPEBwcnKZecHAwLi4u6a5+M6Nr16706dOHNWvWmJR0sWLFePjwYbrjABQvXtys/OlE6PlJr0zhQlbSMq8dGzZsoEyZMmzcuNFMUX377bfp1q9fvz7Dhg2jffv29OjRg02bNqFSGf9plC1bFiEE3t7eVKhQIV/kzwgvLy/27t1LXFyc2Wr6+vXrpvtP/2swGLh165ZplQ0QGhpKdHS0qV52cXFxYcCAAQwYMID4+HiaNm3Kd99990IlXbFiRQACAgKoWrVqtscsWrQoNjY23LhxI82969evo1Ao0uxAPE/VqlWpWrUqX331FceOHaNRo0bMnz+fSZMmZev7nTZtGhMnTsxUZi8vLwIDAwEoUaIERYsW5cyZM2nqnTp1iurVq2faX3poNBoMBoOZlXr16tU5cOAAsbGxODg4mMpPnjxpuv8sAQEBAGa/EZnCh3wmLfPa8XRF9Oxq7OTJkxw/fjzDNi1btmTNmjXs3LmTPn36YDAYAOOKRalUMnHixDSrOyFEmm32vOTtt99Gr9czZ84cs/Jff/0VSZJo27atqR7AzJkzzerNmDEDgHbt2mV77Oef087OjnLlyqW7vfostWrVwsLCIl0llRWUSiWtWrViy5YtJsUHxgnH6tWrady4sZlCepbY2Fh0Op1ZWdWqVVEoFCa5s/P95uRMGqBbt25s27bNzAVq37593Lx5kx49epjKtFot169fN1t1R0dHmx3XPGXx4sUA1K5d21TWvXt39Ho9CxcuNJVpNBqWLVtGvXr10kxmzp49iyRJNGjQIN3PT6ZwIK+kZV472rdvz8aNG+nSpQvt2rUjICCA+fPnU6lSpRduV3bu3Jlly5bRt29fHBwcWLBgAWXLlmXSpEmMHz+ewMBAOnfujL29PQEBAWzatIkhQ4bkW7SyDh060KJFCyZMmEBgYCB+fn7s3r2bLVu2MGrUKJMBlp+fH/369WPhwoVER0fTrFkzTp06xYoVK+jcuTMtWrTI9tiVKlWiefPm1KpVCxcXF86cOcOGDRsYMWLEC9tZWVnRqlUr9u7dy/fff5+j5540aRJ79uyhcePGfPjhh6hUKhYsWIBGo+GXX37JsN3+/fsZMWIEPXr0oEKFCuh0OlatWoVSqaRbt24A2fp+c3ImDfDll1+yfv16WrRowSeffEJ8fDxTp06latWqDBgwwFTv4cOH+Pr60q9fP5YvXw4Yfb9HjhxJ9+7dKV++PCkpKfz7779s3LiR2rVr8/7775va16tXjx49ejB+/HjCwsIoV64cK1asIDAwkCVLlqSRa8+ePTRq1Mh0JCFTSMlvc3IZmcx46uoSHh5uVt6vXz9ha2ubpn6zZs1E5cqVTdcGg0H89NNPwsvLS1haWooaNWqIbdu2iX79+pm5x2QUcWzevHkCEGPHjjWV/fXXX6Jx48bC1tZW2NraiooVK4qPPvpI3LhxI8vP9dQF6/Tp0zl+3ri4ODF69GhRvHhxoVarRfny5cXUqVPNXLqEMPrVTpw4UXh7ewu1Wi08PT3F+PHj07j7eHl5peta1axZMzP3r0mTJom6desKJycnYW1tLSpWrCh+/PFHkZKSkulzb9y4UUiSJO7fv29WDoiPPvooTX0vLy8zlzohhDh37pxo3bq1sLOzEzY2NqJFixbi2LFjZnWed8G6e/eu+OCDD0TZsmWFlZWVcHFxES1atBB79+5NM2ZufL8v4vLly6JVq1bCxsZGODk5iffee0+EhISY1Xn6e3z22W/fvi369u0rypQpI6ytrYWVlZWoXLmy+Pbbb0V8fHyacZKSksTYsWOFh4eHsLS0FHXq1BE7d+5MUy86OlpYWFiIxYsX58rzyeQdcuxuGRmZPEWv11OpUiV69uzJDz/8UNDiyGA8Cvnll1+4c+dOlqzLZQoO+UxaRkYmT1EqlXz//ffMnTs3T1NVymQNrVbLjBkz+Oqrr2QF/Qogr6RlZF6SpKSkNLGgn8fFxSXTbEcyMjIyzyMbjsnIvCRr1641MwBKjwMHDqQJlCIjIyOTGfJKWkbmJQkODubKlSsvrFOrVq0MQ2PKyMjIZISspGVkZGRkZAopsuGYjIyMjIxMIUU+k84Eg8HAo0ePsLe3lwPRy8jIyMjkGCEEcXFxFC9eHIUia2tkWUlnwqNHjzKNDSwjIyMjI5NVgoKC0k04kx6yks6Ep4kMgoKCMowRLCMjIyMjkxmxsbF4enqmSTf7ImQlnQlPt7gdHBxkJS0jIyMj89Jk5+hUNhyTkZGRkZEppMhKWkZGRkZGppAiK2kZGRkZGZlCiqykZWRkZGRkCimykpaRkZGRkSmkyEpaRkZGRkamkCIraRkZGRkZmUKKrKRlZGRkZGQKKXIwExmZAuL4iXNcvhJI1are1K9bo6DFkZGRKYTISlpGpgBYtHQrK/9KRCiKIW28Tr/uQQwa0LGgxZKRkSlkyNvdMjL5jBCCdZvvIhTFjNeK4sZrObW7jIzMc8hKWkYmn9HpdGj1SrOyFJ0Sg8FQQBLJyMgUVmQlLSOTz6jVanzLCYRBB4DBoKNyeQmlUplJSxkZmf8a8pm0jEwBMPOXoUyZsZbQcB0eRVWMGzO4oEWSkZEphEhCPgh7IbGxsTg6OhITEyOnqpSRkZGRyTE50SfydreMjIyMjEwhRVbSMjIyMjIyhRRZScvIyMjIyBRSZCUtIyMjIyNTSJGVtIyMjIyMTCFFVtIyMjIyMjKFFFlJy8jIyMjIFFJkJS0jIyMjI1NIeaWU9OHDh+nQoQPFixdHkiQ2b978wvoHDx5EkqQ0fyEhIfkjsIyMjIyMzEvwSinphIQE/Pz8mDt3brba3bhxg+DgYNOfm5tbHkkoIyMjIyOTe7xSsbvbtm1L27Zts93Ozc0NJyen3BdIRkZGRkYmD3mlVtI5pXr16hQrVoy33nqLo0ePvrCuRqMhNjbW7E9G5nXDYDBw48Yt7t4NkPNYy8gUYl5rJV2sWDHmz5/PX3/9xV9//YWnpyfNmzfn3LlzGbaZPHkyjo6Opj9PT898lFhGJu/RaDQMGj6T9z86xTtD/mXU2LlyLmsZmULKK5sFS5IkNm3aROfOnbPVrlmzZpQqVYpVq1ale1+j0aDRaEzXsbGxeHp6ylmwZF4bps38k4173FAojKddBn0Sg3ulMKBfxwKWTEbm9SYnWbBeqTPp3KBu3bocOXIkw/uWlpZYWlrmo0QyMvlLTKzepKABFEprwiNiClAiGRmZjHitt7vTw9/fn2LFihW0GDIyBUbVyh4IXYTpWjIEU7umdwFKJCMjkxGv1Eo6Pj6e27dvm64DAgLw9/fHxcWFUqVKMX78eB4+fMjKlSsBmDlzJt7e3lSuXJnk5GQWL17M/v372b17d0E9goxMgdOzW0vCwjZw+MQ1JATtW5XmjeZ1C1osGRmZdHillPSZM2do0aKF6XrMmDEA9OvXj+XLlxMcHMz9+/dN91NSUvj00095+PAhNjY2VKtWjb1795r1ISPzX2TE8O6MGF7QUsjIyGTGK2s4ll/k5KBfRkZGRkbmeXKiT/5zZ9IyMjIyMjKvCrKSlpGRkZGRKaTISlpGRkZGRqaQ8koZjsn8d4iNjWXF7zvR6aFTu/qUKVOqoEWSSYedu45y7kIQLs4WDP6gE0qlsqBFkpF5rZCVtEy+EngviCtXblGrZmU8PNzTrRMXF0f/YQsJja2NJEns2r+NXye3xrdi2XyWVuZFLF+5jcVrEkFZDINew4XLc5k382MkSSpo0WRkXhtkJS2Tb6xavZOFqx6gowTW6s0M61uKU2cfEB4BbkXhmy96Y29vz6rVuwiNrWV62cfrKvP7mqP8+J2spAsTO/c9AKUvAAqlJWevuHDr1h0qVChXwJLJyLw+yGfSMvmCVqtl+Z83EKryKFU2pIiK/PDLbo5fKs+d4Aocu1COT8cvB0CvNyBJ5j9Ng0FenRU20vpuKjDIHp0yMrmKrKRl8oW4uDiSNNama4NeA8oSptWyJCm4FSgQQtClYxMcLC6Z6qq5y9utK+e7zDJGtFot02et4fOvV/Hbwr9MGbPebFoM9OEAGAxa/CqEUaG8vNshI5ObyNvdMvmCs7MzpUskcv/xkwJJiVKRYFbHxkogSRIlSxZj9i8dWLn6MHoh0fbN6jRqVCP/hZYBYPS4BZy74YNCacHhs4k8fLSMSd8NZMjATri7HcT/4kOcHFV8OPQjFAp53i8jk5vIEccyQY44lnsEB4cyefoWIqOhuLsC3wruLFsbho5SWEj3GDm4LF07yyFbCxPx8fE0fXsRNo41TWW2qovs2vRxro4jhGD9X3sIvB9FxfLudOzQPFf7l5EpDMipKmUKNcWKufO/aUPMypo2DsT/4g1q12yNl5dnAUmWilarJSoqCldXV3lVCKzfuB9Nih6bZ8ri4+NzfZwfJq9g578uKFQebNkbye27axnzSa9cH0dG5lVDVtIyBUrZsqUpW7Z0QYsBwOa/DzF3sT/xSXYUd41j0jcd/vNuXw8exmPQp5CiicTC0oXkxGBKuSbm6hharZZ9R2JRqIzpMiWlC7sOXGD0SCG7c8n855GXCjIyGA3bZi+8RJKhOkrLcoTG1eCXmTsLWqwCp5SnA06u1UhOeEhU2ElSkkLo0rFBjvsLCAjk0qWr6HQ6U5nBYECI56z55UM4GRlAXknL/EfZs+8Yd+4GU8OvLPXqVufRo2BiE52xfGZfNzJaXsX1fa8dd+4u4/BJA2BF03p2vP9u22z3I4Tgy28Ws/+4GiFZUa7kLn6b+QGOjo5YWlpSx0/JsQtJKFXWGPTx1K9tK6+iZWSQDccyRTYce/2YNnMNm3apkFRFURoeMPi9InTr0oxu7y8iNsUPMCqVamWv8tusD7PVd1JSEt/9+AcPgvU4O0p8NqodXqVK5MVj5CtPV74qVc7m9Zs272XqgiQUKuO/ISEELevfY+JX/QDjavq3hRsJDkvGq6QdgwZ0kpW0zGuHbDgmI5MJSUlJbN0VjqSqBoBeUZJ1Wy7y/rtt+PqzZsz67V+iY8GzuMTEr3pnu/8vvlnOmWs+SJKSe2Hw2YT1/Ln841c+pnVOlfNTgkOjUaiKmq4lSSLuGfszhULBR8O6v9QYMjKvI7KSlvlPkZycjE5vieKZX36K1rhia1Dfjwb1/V6q/9uBeiQpVSHfC3YiJCSEEiVe/dX0y1C/ji/rt51FixcABl0cvhWcClYoGZlXANlwTOY/hbOzM75lExFCD4Ber6FGZctc69/O2vzayiIeJyenXOv/VaVmzcqMGuKFl9s1Srhco2urWAYN6FjQYsnIFHrkM+lMkM+kXz+SkpKY+ut6HkcZ8CppxagRPXJtO3r/gVP8NPMs8Rov1FII/XsVZWD/DrnSt4yMzKtNTvSJrKQzQVbS/w2EECxe9jfnLjzG0kIwZEBTKlUqn6O+wsPDOXPmMj4Vy1DG2+uFde/df8j6jUeQJMF7vd/Ew73oC+vLyMi8ushKOg+QlfR/g0VLtrBsg0ChcgbAwfISy+f1xs3NNc/GvHPnHiPG/U1cSlWEEDhb+7No9rsU83DLszFlZGQKjpzoE/lMWibX0Ov13L59h9DQ0IIWJducuRBhUtAAMcmV2LP3eJ6OuXrdEeJSqgJGa+eopOr88ef+PB3zWYJDwpg9bx2z560nMjIq38aVkZHJOrJ1t0yuEBMTw/BRS7h13w2VIok2zdV8O6F/QYuVZSwtzDeUDPp4XJzzdufk+S0sSZIQ5I9vcFDQI4aNXk+MpjpCCPYeWsaSuf1xdXXJl/FlZGSyhrySlskVps3axL3wmljaeKK0qsDuI078s/NQQYuVZQb3b4y9xSWEMKDXxVK38n3atGmWp2N2bl8ba8V107Wt6jJdO9bP0zGfsnL1AWI01QHj5OBxQk2W/747X8aWkZHJOvJKWiZXiIk1T4YgqZy5Z0oenTsIIfhjzQ4ePoqjkq8HHd7OPSVatYoPy+a5sGfvcYq4ONC2zUd5HvGqWlUfpv0g2PT3WRQS9OrWlrJlX2xollvoDebXkiRhMMgRvmRkChuykpbJFcqXseX01USUSmPwazX3aFCvdq6OMeG7JRw8XQyF0oO/94VzP2gjHw3tmmv9e7gXpc97+eu761etIn7VKubrmADt21Tn0InjaAxGC3Zr5TU6vP1WvsshIyPzYmQlLZMr9Hm3FVevL+JhqBXWVgre6VaZ6n6Vcq3/2NhYDp1IQWFhZyxQFmXHvot8NDTXhvhPUbNGZSZ/pWfL9gtIEvTs2vw/n5ZTRqYwIitpmZfm0L9n+XH6CaITymKtDuO9HuXo3LF5ro5hMBhAmP9chXj9tmcNBgNRUVE4OzujUOTMZEQIQUhICAAeHh4ZbtvXrVONunWq5VhWGRmZvEdW0jIvzZyFx0jUV8PCCvQUYdFKf7p0bIqFhUWujeHk5ETtqlpOXdOgVFpi0MXQpK5TrvVfGDh6zJ/JMw4QEWuHq2M8X376RrZjiev1ekaOnceZSzYgSdSrlsivvwx/5RN8yMj8V5Gtu2VemrgE85VaQpIdMTExuT7OjF+G8V6HWFrUecDw91V8Pva9XB+jIJn6v4NEa2qitKxAVHJNpv7vYLb7WLhkC+du+KC2Lofaqiynr5VnyfKtuS+sjIxMviCvpGVeGu+SEhfvplp3l3SPxdU19yN1KZVKRgx/PdMZpqSkEBGtBHVqWUSUhF6vz3QVfO/+Q/7adARJIREaFodSWcR0T6m0Iiw8Mq/ElpGRyWNkJf0fxmAwMH/RJgKDEnBzVfPJR91Rq9WZN3yOyd+/z3c/ruFhqMDZUfDF6G557r6UEXcDgvhr81Ek4N1ezSle3KNA5MguFhYWFHMz8OiZwF/F3ESmCvpuQBAfjd1MXEo1hBAYEk6hsHJCUhknSUIXTrUqxfNSdBkZmTxEjt2dCa9z7O6vJy5l78liKJU2GPQp1PS5wZxfRxS0WDnm1q1APh63jXidMdSmk9V5Fs7q/coo6mvXbjNp6j+ERYB7Efjq8/ZU9CnzwjaTfv6DnUdLmq6FEJRw3IdWFEMCWjZz58Oh3fJY8pdDCEFkZCQODg45miTKZJ/NWw+xcs0lkjRQuYKayd9/IH/2+cBrH7v78OHDdOjQgeLFiyNJEps3b860zcGDB6lZsyaWlpaUK1eO5cuX57mcrwqn/BNMfs0KpQX+V5XEx8eb7iclJbFsxWaWLN9EbGxsQYmZZf7ccNSkoAGik2uweu3BghMom/j6lmPRnEE0rO2Io5Mdm7YcQ6vVvrCN4bkptiRJ1KldhU2rP2Tj6g8LvYK+e/c+PfvOpP07G2nbbR4bNuZf7PL/Ktev32bWogDC4qoSl1KVYxfL8vO0PwtaLJkMeKWUdEJCAn5+fsydOzdL9QMCAmjXrh0tWrTA39+fUaNGMWjQIHbt2pXHkr4aPL+TqpB0qFTGE5C4uDj6Dp7H4vUOLN3gTN8hi3n8uJCfbaazJ/SqbRON/HQRe096cuFWWf4+5MHnXy15Yf1O7WphZRZa9Aqd29fLazFzjZ9nbCc42mgslyyqM2fJdaKi5GQfecnxk5dIEaVN1wqFmqBHKQUnkMwLeaWUdNu2bZk0aRJdunTJUv358+fj7e3N9OnT8fX1ZcSIEXTv3p1ff/01jyV9NejYqiQYjBmrhC6Ktm84Y2VlBcCipdt5FF0TSVIiSRLh8TVZuHRHQYqbKV071cFamaqw7NWX6Na5UQFKlD3i4+O5HmCBJBlnTwqFmkvXX7yS9qtWkakTm/JmvQe0rB/ErJ9bU7586TyR7+bNO2z7Zz/h4bkX7jUy2vw6QVOUuwH3c61/mbRU9vVGMgSbroUQODu+fjEHXhdea8Ox48eP07JlS7Oy1q1bM2rUqIIRqJAxbEgXypU9zqWrDyhT2pVOHTqb7ul0EpKUOoeTJImkZH0BSJl1qlSuwIxJgk1bzyBJ8E6P9pTx9ixosbKMpaUlaqWWZ9Xy89m50qNGdV9qVPdNU24wGFi9ZgdhjxOoV7s8jRrWyLFs8xZs5PeNsQiFO/aW6/j6s/o0aVQzx/09pZi7RHB0qmeAi30oFcq3eel+ZTKmbt3q9O5wh3VbL6LV21DWM5YvPh1Q0GLJZMBrraRDQkJwd3c3K3N3dyc2NpakpCSsra3TtNFoNGg0GtP1q3AW+zK0fLMBLd9MW/7Wm1X565/9qG2MZ7xx0Tc5d/4OBoMhx5Gwcps7dwO5HxRMnVpVsbMzhgutWsWHqlV8stReCMH0mX9y7HQMCqWgUxtv+rzXNi9FfiFqtZruHUry59ZQULqjFA94p2vWnuV5hBB8+vl8Tl7xRqF0Z8uuq3w0IJye3Vtlu6+IiAhWbwpDYWGcCCTqKzN/6YlcUdLffdmL8d+tJjDIgIMdfDy0Ifb29i/dr8yLGTG8Gx/0SyQhIQFXV9cC88aQyZzXWknnhMmTJzNx4sSCFqPAqVG9EvZWawkOOwmA2tKF8Phq3L59lwoVyuXZuFeu3OCc/3Vq1fClUqUKGdabMWst67cnYZBccLVfyuRvW1OtavYU2srf/2HjbjsUqhIALFr9CC+vszRtXOulnuFl+GhYV6pVOc21G/epWb0atWtVfWF9g8HAz9NWc/laAlZWMPD9ujRqVIPbt+9y9LwdFtZGw0C9oiQ/TN2DnZ0tb7fJ3hFAeHgEGq09ls8Y/yYm5c5LvUgRZxbO/ihX+pLJHjY2NtjY2BS0GDKZUDiWRHmEh4cHoaGhZmWhoaE4ODiku4oGGD9+PDExMaa/oKCg/BC1UGI8n5awtHbDzrE8kkjByir3Qn0+z6rVOxk+7iTz/3Rk+Ljj/PFn+gZ+t27dYf32JJSW3qgtHInR+DFnwYFsj3ftZiQKlaPp2qAoztlzd3Isf27RpHEdhgzslqmCBpjxv7VsO1iE+48rcvNBRb6fdoqHD4PRaDQIzF1qDDgwY945Mwv+rFC2rDee7uGma2HQUa60HGZURiY/eK2VdIMGDdi3b59Z2Z49e2jQoEGGbSwtLXFwcDD7+y+yc9dRImLtcSpaG0lSExHyL01qJ1GqVKk8GU8Iwe/rrmNQeCFJEnqpNKvWXSM9N/77QSEIyTyiWUIOVnbOjkqEQWe6NujjKOZR+LZa79y9z5Llm9i563CaezduJ6JQpq6G4rXlOXj4NL6+PviWfoQQRjuClOQIAKITXAgICCQ5OTnL46vVaqZN6kr1cjfwdr/OG3XvMenbfmZ17gc9ov/QObTs+D969pnNydMXc/KoMjIyz/FKKen4+Hj8/f3x9/cHjC5W/v7+3L9vtAYdP348ffv2NdUfNmwYd+/eZdy4cVy/fp158+axbt06Ro8eXRDiv1IsW+2P0qoqkqTA2q4kdnZ2TPi8d56Np9PpSEoxP31JTlEYs189R53aVXGxDzBdG/QplCud/RX+JyO6UqHkBbTJ99Al36ZJjUf06lG4jJYOHT7DsDG7WLrBme9nRTHh28Vm9+1szesLXQSeJdz5+vtlBIdBcMDvRIWdJCnhAU5F65ISd5IR4/bTqstixo6fn+7nmx5eXiWoXcOD6lWdebtNLSwtLc3uT/xpI7cfVSVZ+PEouho//LIfnU6XQW8yMjJZ5ZVS0mfOnKFGjRrUqGG0Uh0zZgw1atTgm2++ASA4ONiksAG8vb3Zvn07e/bswc/Pj+nTp7N48WJat25dIPK/SiQ9t9BSKJ2Ji4vLs/HUajU+3gaEMCoNIQz4lFGkGxbTwcGBn75+C99S1ynlep23Gjzkqy/ez/aYVlZWLPltFOuXNGfzqnZM+XHISxvQ3Ll7n3kLNrDqj79zRUktX32WJH0FJElCoXRi7zFLrl69Ybr/8bBWuNr6o9VEo02+R5umiZy7EMiB054kidq4luiIXheFva0GErdh6/ImepUfBlVVjvqXZsHiTZnKIIRg5KdzWbDGkk173fli0iU2bz1oVudRmHmb8CgnwsKeK5SRkck2r5ThWPPmzdPd/nxKetHEmjdvzvnz5/NQqteTShUsOXxOi0KhRgiBd/EISpQokadj/jrlA36auo6wxwbciyqYMO6DDOtW96vIorkVX3pMhUKRa1v4585fYfz3R0jQVcSg13DwyGwWzhlpNtHQaDRotVqTNXpmaJ6LMSEkeyKjok3XZbw9+XPZEM6eu0jRopXxqVCO0V+sQKk0+rtbWDrjWrwNvd8OIzGpFFsPOJnaKlXWPAxJ9XkWQhAXF4e9vb3ZZMX/wmVOX3HD4okPvR5PNv59zSxnuKszxD1j/uFkF5MnSVZkZP5rvFJKWib/+OGbfkyZvobAIA2ODvDZqD55npPYzs6OnyZmrJgLAiEE301azvGz8SgUgrffLMbIj3qkW3f5HydI0BknDgqlJVcDfdm8ZS/duhp3bqZMW832fREYhJqqPinMmjos05zbHq7J3H0Uh0ptPCsvUSSA2rXMt+Stra1p3Cg1ylgRZwXCoENSGP95C30E5cp6kKLRs2VvJJLSBTCewZf2NPZ77PgFfp65n8hoa1ydk/h6XCtq1awMgEaTApjLqTeY7zh8MaY130zaTkiEDfY2CYz5sH6O84nfvXuf//22m/hEqFjOljGf9Co0bn8yMvmNrKRl0kWtVvPVF30KWowM+WvTfk6ceYCF2sDwQa0pWbLYC+unpKTw3aRV3L2vw94WPh7WPEsuW4uXbWXPcTcUyjKgh7Xbwilb5jDt2jZNU1enN1dcCqUl8YlGP/vt/xxi635rJFU1AC7c1vHr7A18/um76Y4bGvaYn6f9yd5Dj9CkhKG2cEQyhPPdz51MUeEyYuwnPbh/fyGXb1uiklJo86Yj+w895vptLfqEO6gsPbC2tqZhXTsG9u+PEIIpMw8QmVgTLOBxAkz+dS8bVhmVdO1afpT3PERgmDOSpEDoI2jawDz+QOVK5dnwxyemxAE5VaqJiYmM/nITEQnVAbhyN4mUlD/48vPC+1uUkclLZCUtk+/s2XeCoyfuYm0JHw7tkO3gFavX7mTeihhQegFw6eqfrFo0EEdHxwzbfPvDSg6d80ahUEEEfPn9LtYuL4mtrW2GbQAC7sWiUKZuh0uqoly7Hky7dGKeNKlfgos3QkBpzLrlYnOJ9m2NyuVOYBiS0s1UV6FQER6RfsjPyMgoBo9YSURiTRzdqhEXdQWlyhYb+3qcPvcg3eAzz2JlZcXCuSOJjo7G0tKSqTPXc8S/DAqFGpVDFSylmyya1YbSpY3R2JKSkoiMsTDLZf04UkIIYyQwlUrFwtlDmTVnE/FJgupVi9GzW8s040qShJOT04uFy4QzZy8SElka9RO7NKXKmss3El+qTxmZVxlZScvkKxs3H+DXRSEIRQmEMHDuwgJWLv44jbXwi/j32CNQljVdRyT4sX3Hv7zbu32Gbe7c1xoV9BPCokty4eI1Gjao/cKxPNws0es1KJVG+fS6GEqXckm37ju9WqNS7eX46QAsLWDwgC4UKeIMQLUqpVm//S5CYVTgBn0SpT3TDySxdsM+IhJrms6F7Z0rExV2Ems7LxSKrKcMeaowA4NSUChSNXCyoSzHjvublLS1tTVuRVIIfSa4nkdRzM6lbW1t+fLz7BvnPc+jRyHs3X8SdzdnWr3VJI2hnoe7K4ibgJOpzOoFP43o6GjOnrtC+XJelCpVMuOKMjKvKLKSlslXdu67g1CUB0CSFNyPqMrOXYfp1PGtLPehVJorKr0hBVubFyt5exsJnknipVJE4lmybqZjfTSsGwGBCzh7GUBPqyYOdO/WMcP6Pbq1pEc62SGbN61DvzvBbPrnInqdRK2aVnw4NOvn70IIXKzP0+/97CtKp+c2KiR9ML6+1c3Kvvm8LZOn7yb0sYFibhLffN4u2+NkxrnzV/jyh8PEpVREpw3n5xmf4lmqIk728Okn7fAqVYIKFcrRrsVRdhwKQaEuip36FoP6pp/V6/CRs0yadoLYJC9U0jUGvuvBgL4ZT9TyguTkZCZPXUtYhB53VyVfjO2V6XGETOFAq9WSmJj4wh24woAkXmQuLZOjJN0yGTN05BKuBKSGFdXpEpkwwoIO7TLZw32GYyf8+fbn0yTpK6DXJWGh20Glyn7Y2wrGjOyIh3vRNG3O+1/jq0l7CY/1RCVF8G5nJ0YMz3qu5aSkJBQKRbZW/DklKiqa/sOW8DihFpIkoU28QJO6esaN7oObW/YspiMjo5g5ewO79l0nMcUNRyd7enZ0Y+SH3dOt/3SLOy8YPmoRl+4YQ70+fnSQIh6NTcZtHg7nWLPiY1Oq1KPHznD/fgjNmtWheDH3dPt7Z8Bcgh5XMV0r9ZfZtrZvvsb+Hj5yNhfvVkaSjIFxqpW7xm+zRuTb+DI5Y/P/5nNzxmIs45LQ1KnEkD8X4uTsnOfj5kSfyCtpmXyl09sVuDE3EB2ljOEli12lTauR2eqjYf3qzP3Fkd37znL0+CUCH7fh0h0bhBDc/XQVq5eOSGNZXKO6L2uXe+J/4QqlPGtRqlT2smNlFEY2L3B2dmLJ3AEsXbkTvV5B21atqO6XfXezuLg4Bn20lNC4WqgdK+Csv83YD73p2P6NDNtkpKCFEOzafZiIyFjatGps2sbPDhpNat9KlbVJQQPcC3Hl/v37lClTBoBGDWvTqOGL+4tPML9O0tgRFRWVb0paq9Vy9TZIT7weJIWKU+cTSEhIyNTWQabguHbhImET51I51ujfKPZeYO24iQxdNLNgBcsAWUnL5Ctvt2mCs5MDh49ex9paYujAEajV6swbPkeFCt5UqODNvyceo3wSFlOSJO6HenL5ynVq1qiWpo2dnZ2Zq1JhxtXVhXFj0rf8ziobNu4jNC71bNugLMf+wwF0zMaO8MOHIUycvIlzF8OIS9Dj4FSFPzf+zvRJ7fDxKZMteWpUdeR6YCwKlQMGg3mgFyt1HC4u6Z/1Z0QZLyVnrhlMKVVLeURRvHjxbPXxMqhUKizUepKeCdqWmGTgswnLmDdTXk0XVm6du0CxGA08+XchSRKG56PxFCJk50OZNBgMhhcGjXlZGtT34/NPezHyw54vvX1soTaXU0E8zk7ysQQ8XRU//z1m73sd/916rt6rhJVjc4oWf5O46GtEJ1djwdKD2ZZnxPDuvN9ZS2XvO1T10aEyXEKv1yC0gbzbtVi2LcMnT+xLI7/blHK9TtUy1/nlh66m7fL8QJIkencpS2LcXQAS4++hUllz+abR5U+mcFK1UX3uu6UGE9ILgbpc3uQkyA3klbSMCZ1Ox+cTFnP+ihal0kDntiX5aFjWz20Lgj69qzNl9lWS9WUR+ijaNpfw9i5d0GJlGSEE+/Yf43FENK1aNsTFJffOxXp2f4ttu34jOMZ4tq3mLl06ZJ5Z6ykGg4H7j4TZVF5l4YhBn5ImElpWkCSJD4d2NV2HhoZx4uQFfHwaUdEn++lPbWxs+OXHwdkXJB3+PXqOFavPoEmBGlUcGD2yV5bO5j/o154lKycQFRaOpbU7jq41UOOfr5MFmezhXaE8vjMmcG7qfIhLwLJeVQZP+a6gxcoQ2XAsE/5LhmPTZ63hr10uKJ64G2EIY+JYT95sUb9gBcuEW7cCOHzUH88SRWn1VuOCFidLPD3jnf3bZkKiq2FpWwoXm6s52kZ+EbGxsSxcup2UFHjrjSrUqZ11JQ3QqfcsU2ARgIiQIzgV8WXwO2o+6Nch1+QsSO4GBDF09E6S9EajNr0ujj6dU8wmFC9i0dKtrPwrAaEoDvpQ3u2s5sMhXfJSZJlXFNlwTOaleBSsSVXQAAo3rl0PKvRKunx5b8qX986VvjZu2c/N2+F4lXSkd8/WeWLpLITg868WcviMO0p1JzQafwQPiVZUY/7SA/w6JfeUtIODA2NHvZPj9iOHNmDKrJNExbmi0wRSvpSOju1Vr42CBth/8DSJuvJPjyhRquy5fD3recUHf9ARX5+zXL4aSKWK3jRtUiePJJX5LyIraRkTxdwtMFxMQaF8YhmtD6di+bxNqlGYmDbzTzbttkJSemDQx3Lz9gq+ndA/3bo6nY7pM9dy/5EGF0eJL8b2zLJF74kTZzl02gW1pXFr28m1BpGhx7Cx9yJZkzfuTznlzRZ1qV+3MvfuBVGqVIcsJwZ5lShezAWhj0VSpfrL2mbTmL9xo1o0blQrlyWTkZENx2SeYdTHPahT6RZK/UUs8afn24KWLTPxg3lNEEKw+2CYKfmEQunA/iOxGRoAffntErbsL8qFW2XZf7o0H41emGVju7DwKBRKczchSVKi18VQo2qRl3uQPMDW1pZKlSq+lgoaoG3rZjSrHYpW8whdShyuducZObxw5RWX+e8ir6RlTKhUKmZN+xCtVotSqSx0mYd0Oh3nz1/CysqSKlV8c30rWgjz5xUoMlS8l65rTUcDkqTgRqAdkZGRFCmSuZJt0bweS/5YRmRidQC0KTEUdXxMry46Bn8gn2XmN5IkMfmHwfhfuEzE42gaNBiCjU36IVtlZPIbWUnLpCEnfst5TVxcHMNGLuRmkBcSKdTz28+sqR9maSIhhGDhki1cuR6FjbVg1IgOpqhkGo0GCwsLJEmiUR17dh+LQ6G0x6BPpEENFSH377P50+8wBIWgLONJz1mT8ChZ0uj69cwiW63UZPnF7uDgwK8/dWbuwn0kJUPVSk4MHzIlzyJ9FQTh4REsX7UbvUGiQ9uaVK5cIVvtb94MYNfeM1hbqej7frscp73MDtX9qmReSUYmn5GtuzPhv2TdXZiZ9PMqdhwpaQpcodfGUcf3AnVqV6VDu2Y4OWUcf/fX/61l/Q4bFCoHhBB4OJ7j18k9+eqHDQQEqbC11jJ0QA06d2jG8pXbCLgfS3EPG4YM7MT0Fp2pdOyGqa+brWowevsa/ly3m/krQ9FLpUAfRrc2gtEje+b55/AqEBkZxYDhy3icYAykYq28zi/fNaVGdd8stT977jJffH+UJH1FDAYdZdz9WbZgZKGcPMrIZAfZulvmtSU+AZOCNhh0RIQe57SiGWdvqdiwdSnzZ75LsQxiPJ+5EI1CZcw+JUkSD8LL8vlXK7kf2RBJLRGvg/8tuE7Der4MeMZqOT4+HvW1QLO+tNfuIoTgnZ6t8Cl/mbPnb1K+bCmaN3s1IpnlB6vX7jUpaIAkfUXWbzqdZSX9+5rTJOmNYVAVChV3Q6qx7q+dvNf79bEol5HJKrKSlil0HDx0mt/XnUerlahXy4UPh3ajWpWi/Hs2EknpQszjc7gWb2FKvxiRWJPfFu3k+2/6pdufxXMLMCESSNBYm20vJ+qKc+XKbdzdUxW9ra0tGjcnkiOD+atIaaLtS6K1SqJPSBjFirlTs0YVatbI+hapEILo6GgcHR0L3Xl/rpLO3pwg61v5Or35taRQo9HEZ0uEFb9v58z5UCwtBcMHvUXZMgUfUSo4JIzde49TtIgjbds0e62ON2TyDllJy+QbR46e449159DqJJrU96Bfn7TpEK/fuMOkGRdJNvgAcHNzFJcuT8bT0xu/soHEJDpDYigozDWvRpvxC69Prxr8NOsSyfpyGHSxtKgbT5LGhojrqRmfbNQPqVrV3B9ckiQa/fg5341dRJLPSCTJmEhhzPjVrF42KssvWSEE585d4qcZewkOt8XBNpFPhtWlbetGWWr/lNjYWPyPHqdEWW/KVsjeGW9+0qN7C3Ye+J3o5BoAWEq36Noh6zsNbzbzxv/aI4TSGIfbxcafTu3Tn4Clx9IVf7N4jQ6Fyphz/MYXm1m5oC/Ozk5Zf4hc5sLF63zx3X5iU3wx6GPZe3A+038eJitqmUyRlbRMvnD9xh2+++UcyQZjLulrd8OxstpNrx6tzOodPOxPsiE1RGRU2AXOi1pcCrRH6C3p2lrDV58NZ+QXB0jSG/tC/4gmDTIOZvJGi7qUKF6EQ/9ewN3dgY7tBxMREcVnX/1OQBDY2RgY2r8WRYumTXHZuFM7FEvvIBmUprLAR/ZERETg6pp52sijW//hyIQppNx5hLVNMUSVIcRbVuTXeWdp0axWlnMPXzl9lh39R+N54xFXbNQ4fTaAd74el6W2+cHseeuNyU4UEh3berNwVm+W/b4Pg0GiXesG1KxROct9de7YHAuLIxw6GoCFWjBkQM80WbcMBgPHj59Bp9fTsEFts/PqE2dCUajKm64jEiqza89RevfM/RzZWWXR8iPEaSsjSaBUOXL8gmDvvqO81fLViJAnU3DISlomX9h34LxJQQNIqqKcOhdIrx7m9YoUscOgT0ChtEWv16BSO6BSG32KJaULew9d4NNPyvL9+Hj+XH8GvUGiReNStH+7yQvH9/Epi49PWdO1q6sLy+aPRK/Xo1QqX9AS7GwhMS712kqdmKV0iElJSRwbO4kqARGAgpoxoay6/jsP6ownKt6J0NBQvLy8Mu0H4MAPM/C9GQqSErskA4HTlnHv/R54eb98pDUhBHPn/8W5i1FYWsKgvg2oVTPr2/ir1+7kz79VKFSVAPht1UNKlHjIhHHv5Vimt9s05u026SswrVbL8JFzuHTHC1BStvj/WDxvuMm6/nn7MmFIwMkxY8PC/CDluZ0ehdKOx4+jC0YYmVeK1/hgTKYw4ehghV6fbFZmZZn28LJ7l1bUqxyINjkUnTYOhcL8gFII48uuQT0//jdtIHNnfED3ri1zLFdmChpgcL9aWEo3MOg1SLq79O1VNkvZu0JCQnC4Z54CzzU5EgA35yiKFSuWYVutVmt2LaLjzK6dElIIuf8gUxmywoLFm1m91YKbDypw6U4Fvpz0L48ehZjVuRsQxOKlG9m8dW8a3/HL18JRqFLTTBqkEpw+eztXZEuPJcu3cvVeFdSWRVBbOnHvcQ1m/7YZvV5PbGwsH7xfH1vVNYQQ6PWJ1Kx4j9atmuaZPFmhZrUi6HUxpmsnm2u89R8JFCTzcsgraZl84Z1ebfn3+Bwu3CqDQmGJm8MVRgxNmy9ZkiRm/DKck6fOExMTz849IZy8koRCaY1BF0PTxk7p9n/12k0CAx/SoH51nJ1zL5MUQNvWjfCrWpZTpy9SpXJzypXL2uq1ePHixJYrDjdDTWXh1mqKOV1g9IfN0/X9vXnhEltHjIc7DzCUdKf1jO/wa9wAq5qV0B67jvrJGWZwlVJ0rl0zV57vwuUo0/ktQHxKRfbsO0m/Pp0AOH7iAt/+fJIEXQWEIZ5DR35jxpThpvNURzslQuhNZ/Z6fSJFi2QtRGpOiI3VpoauxRit7fyFe7TpMpsEjTVexRL5/otmXLoSiJOjLV27fFTgZ79DB3VCodjK+Yt3sLQUDBvYFlfX7OXPlvlvIvtJZ4LsJ517GAwG9uw7QnxcEq1bNcpSmEmDwcCipVsIDk2inLcz773TJs0Ld9rMNfy1Q4tQuOJsc5dJE5pla7v2KZGRUcycs5X4RKji68KAvu3TjJWcnIxer89ynO6z+w9yYMIUDKERqCp603v+NIqX8syw/vTmnah49Lrp+oqfJ5+d3oNOp2Pl59+S5H8dycmelt+OxccvexmtMmLM50s5dTVVSRu00Xw9yom2bZoBMPTjhVwJNBryCaHn8cM9eJUqQhEXG/r19qN5s1oM/Xge1wI8kNDSsGYi038emmcW7EePnufLyVfQS8bPUehCSIy9jK1L6o5K9fLXmTNjaJ6MLyOTU3KiT2QlnQmyki7cBATc492h+1FapmaO8vG8xpJ5w7LVT0pKCu8Pms3DSGPuZb0uht7tkvlkRGqAkh8mr2TP4RgMQkWNygZ+nTI0y3mDhRBZWs39UqYulYNSt0VvOar54O6xPP3t3bwZwJgJW4lMqIRBH0vjGmFMnZxqeTxg+GJuPTDaE0SGHsPRtSZKpdHgzVJxm/nT36SMtyc3bt7G0kJNmTLeeb5y3bT1AFu230IAxYsmceB0WVQWqXYCnq7X+XOZrKRlChdyMBOZ/xwPH4Wgx5lnT5YTErPfz9lzlwgM9sbCyqhclCpHTp4LNt3/a/NedvzriEJlNPQ6e03DnN/+YtTHvbLUf1aVlrJ0CXhGSWtKe2TJSO1lqFDBmz8W92fv/hMUdS1Jk8ZdzeSt7efM9cAYlCpHJElpUtAAyfqyHDnqT/lypank65Oncj5Ll44tqFKpNGfPXqF0aQ9OXjxNijAargkhKOYmuzbJvB7ISlrmlaZG9Sq4O50kKsl4Dm3Qp+BTNmtuTc/iYG8LJJmVPbtIDrwXiUKZGuhEobQkNDz9DFkvQ4MJo1jy3hhcYxIJtbLFuVbDfDlPdXR0pFuX1une+3BoVywt/8b/8l0upUSge3ZXQP+YMt75Hyjkz3W7mb/iATq8UEmXaFzTwNVbF4hLgLKlFXwzvk++yyQjkxfISlrmlcbW1pZfvm/H/+bvIz4Byntb8uW497PdT6VKPjStdYBjF40JNiykQN7pmurbW9m3OJt2BYPSDQCDPp7SpXI/dePGvbe5UWcG1w06JIUKcf8xBw6eoEXz+vy5dhfnL4ZiYw2ffNQx34JzSJLEoAEdAQgNe8zHn64k8JE7SmUinVrZ0qxp3XyR4ylCCFasuYpeUQMJ0FOGs5fOs2PjJwVuICYjk9vIZ9KZ8F84kw4Ne8zylbvRGxS83bo61f0qFrRIBYIQgr827SEsPJbGDatQrar55zB3wUa273mIXieoW92G778dmOtKYdgnS7l8N9WISwjB4J4xaLWwfIMeSVUEIQQlnM6yasmILLmC5TY6nY47dwJwcnIwC6Oan+O/2XEeeqWfqUypv8jeLcNemITj3v2HzJy7k7h4qFDWmk8/6ZUlF7z85OCh02zadhlJkujQxpc335Bjwr9OyGfSMtkmMjKKwSNWEpFoTIiw/+gRJn+tpU7t3LEczis2z15A8M5DYKmmzscfULtFs5fuU5IkundtleH94YM7k3RiBI92/4ulvwW/3DuDQ7wGkrUUa9WEzh+/vKFShTK2XLyViEJpDMxho7pN0yat+G7yTiSVj0nOe+EVOHb8LC2a552vrRCCvRs2EX3/Ab5NG1GlTi3AmHfcx6d8Jq3zDpVKRQVvuHrPgCQpEMJA+dLihQo6OTmZUZ+vJzzeGKr0yt0kNJo/+Hp83/wSO1NOnb7ExGmX0WI0gvS/eg0bGysa1PfLpKXM64yspP/DpKSksGb9PpOCBkg2lGfjVv88U9JBD4L5dtJfPAqDIk4wbtRb+FXLnsHRP0tWEff5r5TSGjeBTp67jvOOFZT1zdsdgM8GjefIBQ8Mvl+jDj5GjT+38bbeuBKL3nea7ZYWtBsy4KXGGPVxDxKTfufStUSsLKH/u7Uo410K5fPeTCIZG5vsn71nh8WffIHdws046OFokWWEzfqKN3p1y9Mxs8qMnwcwaco6wiIERV0EX33e/4X1L1y8ysPHpVBZ6FAoVChV1ly+kfTCNvnN3gOpChpAJ3mz7+BVWUn/x5GV9H+Qq9du893kbQSHKUmOv4WFU9d8G/vrHzZw+1E1AOLD4duf/mHj6vLZ8qkNPnwcD23qKU3ph9Gc3b7bpKRDQkI5f+Ea1apUoESJ4rkid2JiIv/esMWixJMVe5ku3E6OhPsnAXDSQvChE/CMkhZC8NvCTVy4Eo2NteDDQW9SvnzpF46jUCj46ou0q7teXSrzy9zbpIjSGPRJ1K4cQt063XPl2dLjwYMHaJZtpviTgG8lIxK5OG9FoVHS9vb2TJk0EIBHj0JYtnInQsC7vd7AzS1tTPWAgGBiI8+jsnBGp43H3sk33Yh3BYmFhbmrnhACC3XhklEm/5GV9H+QSb/8w6OoGqAGpZ03MWG7cHRrjSRJWClu0bVj3p2DPRdtktAIeyIjI7OUrOIpwtbG7DoJgU1RY/Smbf/8y/TfrpKk9cRSuZ2PBnjTs3vOw4Y+JTIyEoNFCbOyRMvnIkY9J9fs3zaw5m8rFKqyCCG4OX4jq5cMxDEHcaTbtG6Eu7sL/x67jLOTNe/1ztsoWonx8Vil6OEZ5zZJkxqq1GAw8H7/r7l+OwFHpyL06FSZD4fm32TvKfeDHjF89HpiNNURQrDv399ZNPs9PNxTk6VotVpWrL2Fa/HU30FUyD8M6ps197n8YsgH7ThxehHBMcYteXeH8wz+YGABSyVT0Lxysbvnzp1L6dKlsbKyol69epw6dSrDusuXL0eSJLO/rGYdep0Ji0j9f5XaHmsbV95qEEDbJo+YOrFRnp5HuziZXzvaxWcYxlMIweSpv9Pt/d/o1X8eGzfvB6D9hNFcruGFRhiIkQwE9WzK232MIUbnLzuLloqo1LboFRVYvOpymljTOcHDwwNPj1gz2aKTQ4lGj1YYuOrnSZsvR5m1OX8pBoXKgbioq0SGHiPwgaB33+mEh0eQE2pU92Xkhz3o8277PM9HXaZcObZXrsxcr4YsKFGD82olTm82AIzP3r7rl9wJbYhNkS5ExDix+Pd77Nx9NE9lSo/f/zxIjKY6YDyrj0yszqrVe83qhIWFERlrPjEqVsyDJo1yJ6xqbuHg4MDKRcMZ0iuOwT3jWLVoWIGm15QpHLxSK+m1a9cyZswY5s+fT7169Zg5cyatW7fmxo0buLm5pdvGwcGBGzdumK5lFw1wd5W4F556XcxNwbdf9s+Xz+azT95g4pTdhEY64mQXy+jhdTO0sJ2/aDN/H3ACqRhRYce5cu0I6zae5dsvuzLiwCaO/rMLN0cH3nnrTSRJQghBQpICnnmMJI0FycnJWFtbv5TcKpWKSV+3p0ffRUhKV/T6ZBzLv88c1yJMGNmIoR3bpbHWtFQbSNFEIYSOIh7G3NGxesEPUzbwv2mFOxrW/MVbCHUfjKKEMZDKjoTT/D7YuPI8ePgEjxPrYmFpdEFzcKlGZOgx1i7aQPz1i3QY+kG+WZ3rzfOvIEkSBoP5BMbNzY0ijjHEPuPWXtIjYyOzgsTGxsYUM11GBl6xlfSMGTMYPHgwAwYMoFKlSsyfPx8bGxuWLl2aYRtJkvDw8DD9FYTLSGHj63FvU9LFH4XuAm72/kwY+2auKGitVsvKP/5m4eKNabIoPaVWzcpsWj2SLaveZtu6j2jdKmPr5Jt3Y1Eo7YkKPY6zW32c3JrxIKou477ZjiRJtOrRlcatWppklySJsqUwrZyFEJQuqX1pBf2USr7lKFG8BM5u9XAt1gy1hR12HuVp917vdN0pBvdviKQ5iY1dakIOSZLI4UI6X7lyPQaF6plIZza1OXjoLADR0XGoVOY+4sJgoNiO/ViM+ZVfO7xLSkruB3pJj07ta2GpSI11bqO8Rqf2dczqqNVqxo9uiqutP0rdBUq5XuCrcR3yRb7skJKSwqIlG/nf3PXcuHG3oMWRKSS8Mko6JSWFs2fP0rJl6rmSQqGgZcuWHD9+PMN28fHxeHl54enpSadOnbhy5Up+iFuoqVixLGtWfMKBbR+xcfUn1Ktb7aX71Gq1DBz+P+avtmblliIMGrmB6zfupFtXqVTi4eHxQpcZAAdbEMKAQmmBQpFaNzSqJBcvXUu3zbSf+lGv8k1KFb1Obd+bTPsp5zmNn0eSJLp18EIyGCcgkiGUru1LZTjBqVWzCotmv4fQpaZtFEKQjl1TocPG2vyIwKCPxcPdeCzR8o0GuNqn/jvSJD3G7cF+GiYnoZQkfPdf4p8Vf+SZbAaDgVV/bOPX/60jPiGJqd81oUWdIFrUuc/0SS2o6FMmTZsmjWuyec0n7Pv7Q8Z+3ILjJ/25c/dejmUQQuTKMcpTjP9+ZrNsoyPrdrox4vO9nD5zKdf6l3l1eWW2ux8/foxer0+zEnZ3d+f69evptvHx8WHp0qVUq1aNmJgYpk2bRsOGDbly5QolS5ZMt41Go0Gj0ZiuY2Nj0633OpCbgRxWr9nB7WA/kzKN1VRl2ap/mTKpbCYtM2bsqK7c/mQRZ4LjzcqVUhTFi9VJt42TkyMzpgzJ8ZiZMWxwZypVPM2Va/ep7OtN0ybpy/GUatWq8OWoSBauOEdCohJvT8HXn2c/Ilp+8/Gw1twet4HQmPIY9LFYc4Z1Wypy5vw9vhj7DrOn9mTsyJmknL2LMuoGAzVaeDJZUQIaTfZX0lFR0Uyasp6wCIGbi8RXX/RIcyYrhGD0Z/M4fa0sCqU7m3dfZuTAYvzwTdY+09GfzeLkRVfU1qVYtmYvY4ZXoP3bTdKtm5KSwvqNu0jR6OnSqQVOTo4IIfj+pxX8e9L4XnijsUuOItw9z7q/dnInuBoKpfGVnKT3YfW6M4U+XkFh5NS+AwScPEeRcqVp2bNweCO8DK+Mks4JDRo0oEGDBqbrhg0b4uvry4IFC/jhhx/SbTN58mQmTpyYXyK+NiQl68xWuwA63cttodvb27Ny0Sds2bqb35adJy6lPAoRTs8ODpR6QbrHvKZpkzqZKudn6dShKZ06NEWv12d5YmQwGPLcOOxFlPYqyeqlgzn872l+nXOYRKkjtx7AjftaEhNXMPmHQXSrXRT77Vu5IZIIQoGnZDyHvlalBB/0yr6l99gvV3I9qBqSJHHl9gOOdvmJhvUr0fqNCqZjkevXb3L8ggsW1kZLer3kyabt1+jW5c1M+5/+6yoOHAenoqUBSKEcy1dfSFdJazQaPhg2h4AwPyRJyeZ/ljJ/5rvs3neaXUdcUaiMRxjbDsZQpvQuevdMP+55VknR6JGe+/ejNxj//RgMBpYs30pwaCLly7jwTq82LzVWfhJ4+zbXzpynSv06eJYunefj/T1vMY8nzMQtXku4SmLJqfMMnDYpz8fNS16Z7W5XV1eUSiWhoaFm5aGhoXh4eGSpD7VaTY0aNbh9+3aGdcaPH09MTIzpLygo6KXkLgycO3+F/81Zz6rV2zAYDLne/6atB1m36TJRYUdSC/UPadLQmDEqIPABv85ez6w564iMjMpW30qlkq5d2rJh1QC+G2PHijlNGP1xj9wUP9/IioI+tGEzv/i1YHrpOszo+B5REQV3gG1ra0v5cp5EJacehygUaq7dMa6Sq7dvzb1iDvhI1mgQ/KtM5HwrP97ZvJwiRYtm1G266HQ6bt8zemBoksJISQ5DZd+OU1e8+WnWXXbvNR5pJWtSQHpOmemzNhncvuceKgtzK+/EpPS3rJet3EZgWHUUCjWSpCAisSYLlu4m4J75Wb1C6cidgJf/jjp1aEYRmwuma0n/iDealgbg868WsWyDJbuPlWD2Cg2/zFj90uNlh4SEBHQ6Xbbb7Vyyiu2NuqF5/0u2NOrK/jUb8kA6c24tWYNbvNFV0EEniFm55ZXfDX1llLSFhQW1atVi3759pjKDwcC+ffvMVssvQq/Xc+nSJYoVK5ZhHUtLSxwcHMz+XmX+2fkvn35zmnW73Ji3SsXHn87N1bO0hw+D+d/Cm6Qom2FtV5aIkGMoNHsYO6w4nTs259atQD4cs5m/druxbqcbA4YtzbaiBmOWpjatmlG+fM63z/ODA4dO02/IfHr3X8CPU1Zla1IUFhbG+TGTqHz1EZWC46nwz2n+HPNVHkqbOQ4O9giduRJ6GgSkYnU/6q6YRlDXJlh1eoOWS2fw1fZ1lCztle1xlEolNlZGU+3EuACcXFPdo/QKT/YfNk6sq1bxpYJnEEIYP1eDLpJGdYtkbQy1HUnx902/f4NBh0+Z9F+BSckGJIX5RqNWC54l7DDoU49fDLpYvDydsvaQL8DFxZl5M3ryRp17NK5+n/EjPencsTkJCQkcP6c3hYlVKJ04cDR/Jm5JSUlM79aXBWUaMKNcfdb/MjPLbQ0GAxemzKN0ZDJqSYF3WAJnJ8/NdRnj4+PZtHApmxctIykpCZFiPplQpejzzYgxr3iltrvHjBlDv379qF27NnXr1mXmzJkkJCQwYIAxylPfvn0pUaIEkydPBuD777+nfv36lCtXjujoaKZOncq9e/cYNGhQQT5GvrJ+8zW0GMNuKlXWnLniwekz/tStUyNX+ve/cI0kvScqFVjZFMPKphiebtfp3LE5AH+sO0Kc1niuJkkSjxNrsfKPPYz6uGeujF+YCAgM4sdf/UnW+wJw/1AiNjYbGJ3FZ73pfxGPR9EgGVfckiShD3yUbt2/f1tM8P5jYGvNm59/nK2QqEIIYmNjsbW1RaV68SvgxynrSUiIxyDdx9qmONrESwwcVd90v3aLZrkSNz0lJYV3upZl6Z+3MBi0iCdZwJ6iVBiVskqlYsHsocycs4m4BEG1ym682ytrW80NajkQnVCMyNAjSJIKN6dQpvw4Jd26bVrWYPu+AyTrKwCgMDygaaNytGpZn7sBSzh6OgkkQbNG9rz3Tv+Xe/gnlCxZjO+/SSeW+HNz6vxyIv1zwiTKbzmBUpIADeE/LuJ0nRrUycL3rdFoUMU+l9g9Nt4sotrLEhUZyfx271Ll9F0EMGvlXzi2qEfijU3YCAmdEKS0rEORIlmbxBVWXikl3atXL8LDw/nmm28ICQmhevXq7Ny502RMdv/+fbNzvKioKAYPHkxISAjOzs7UqlWLY8eOUalSpYJ6hHxH95wfKZIFiYnJuda/X7WKWCn+QYcx4YLRejn1O0jPj/V1zbt28NBZknTln9pPoVDacP1W+ko2Pcr7VeWsuwMJYXHcVqvx1qag8Ex7lLN13iJiP51GiSeLhk1nLzNg/4YsbTE/fBjC2K/WcO+RNbaWyQzsU4nePd7i+I7d3N5/BJWTA13HfoylpSVxcXGcvaKkiEcjEuPuER1xHjcXDW+9WT/TcbLDjFlr2bIzFK3BEu8S0Qx+rxLrt5wgPL4+SEqcrC4ysF8XU31bW1smZMH4btbcdRw/FYlSBd06+DDx6wGUXLqFwPteuLtZMmLYZxkeQfj6lmPS+AT+/OssBr1EqxZlaP2Wccfux4mD0Gq1SJKU6STnZbG1taVxHTWHziagVNli0EXyxpv54x6gfRDyREEbKZqoI/DilSwpaWtra0QtX8Suc0bfdSFQ16qUq7EYts+YR9XTd019Vj5+k9i3mqCY8gmhV26gLu7OyK8+e+VjY7xSShpgxIgRjBgxIt17Bw8eNLv+9ddf+fXXX/NBqsJL0/ru3N3wGEnlihACL7fbNGzQNtf6L1myOCMGlmHxqvMkJivx9jQw4bM+pvsd3/bj6OmTpDxR4nbqy3Tq0DHXxtfpdBw4eAK9Qc+bLRpm6taVl5Qq5Qb6EFClztztbbPe3t3dnah3uvP7UQvULjXYG3uFgU3SHs0E7z1KyWd29SpcC+bI1n/oNLBfpmNMnr6FoMc1UFhAkoDflt9AETKfuG/n4hGXgl4IZh45xad/r0apVCKhRwA29l7GP8uLWX+gLLB3/3E27lKCyg8FEBim5869B6xZ8Skz/7eKazcf0qxxFbxLp++NkREr//iHtdstUSiNuxr/WxJEyRKXGDKwc5b7qF/Pj/r10k9ukZ+/s8k/DGL5qm08Cn6ET/midO/aJfNGuYBVOS+04l/UT5RcsKMlderVznL7gavmsWbsN4gHYShKF2fg9PSNdXOKSNaYKWCFJEGyhm7ffP5S/RoMBrYtWUlCcCjlGtejzhvNX1LSl+OVU9Iy2WPwwE7Y2e3C//J9bKwEn3w0CAsLi1wdo3vXN+jWpQVarTZN33VqV+XnbwTbdl5Ckgy826MtZbxzxzJbo9Ew6MM53HrgAxKsXP0/lvz2Ya4FL8kub7ZoyJFjy9lxMBY99pQqGsToEVmLDx0VEcHWKf/jyuaTWDq/AQoVlk5+/PXPeYYMNrf0Ftbm0bySJEGRIumHVn2eiOfMAZJ1xbjx+xxqxhnP7ZSShOeec5w+9C8N3mxBi4bW7D0Ri0LpAPpw3m6ZOwlLnnL7ziNQpu4ASJKSx5EGTp+9yr5jEkn6FtxaF8+lK78x45fhZskntvy9j4jIOFq9WQ9PT3O5Ll8NR6EsbbrW4cnxk9epW+flYgJotVoCA+9RpIgLLi4umTfIBSRJYkDf/A++8u7E8SwIDiVm7zE0CMoNfRe/+nWz3N7ZxYXhS+fkmXxVOrTm5O9/4xlhzGZ2z82OJh1fzvJdCMG8QSNxX7UbByQuOvxBxLRxtBlQcG6TspL+D/BOr9a8k8e5BCRJylD5161T7aVfjs9mlLK2Enw46A127jnN3ZDqqCyMP+N7j2swd8Fmxo5656XGesqdq9fY8slXGAIfofQqRoeZP1CuSuUXtvl2Qn/e7x3A44goqvu1z1J4zMTERBZ0eJ8qp+7QX5K4HbSM9fpkKNGMZI2alJQUs5jzzcYOZ+e5K5S/FUaSJHjUqwU9O2XtJe5RFO6Hp54L2lk+wN7e7rlagnmL/qF05cpM/HoAlTbsIuhBCJUrFufttun7FOeU6tXKsHrLNQySMXmJXq+hVAlLVv55hiS90ZZCobTj+IWinDhxlgYNaiOE4JOxczlzzRtJ4cz6rVv56esm1KyR+t3Y2UkIYcw3DWDQJ+Dm+vxzZp2Tpy4ye/5uLl+PQGVVCSuLeN7p7M5Hw/I/qUh+oVar8ahUAYuN+/FI0HN/wTr+rVKJJp3bF7RoANRo1pikpVO48sdGkCRq9+9F1WxMItLjXmAgyjW7sXly8u8Rq+HGsnWykpaRyYxnM0oBjJnwNzWr2iEpUoPbSJIyQ5eanLDxwy+odPRJoJzASDZ9+AWfHf4703Zly3pTtqx3pvWecuCvLfieumNSnOUMBrxCThNYvCnlvfRpksL41qiO8961HNu6A0dXF7p375Llc7dvv+zN51//zp17OuxtJYYNqI0U6kzQuZum7e71RbwJi2vDqM//YPXSkfTukXd+ufXr1WDQOyGs33KRFC34VbNg9MeDGPjRMvOKSjuiouMA2LPvCKeveKJUG5VuvNaXJauOmynp0SO6cOPmIm4GlURCQ+Na8fTuNSxHMgbee8A3k49z/4GGIh6pn8XqzYG82eI2FX3K5ajfwk58fDz3fl1GxUQDSBLeIXGc/uW3QqOkARq+3ZqGb7+cj/qzaFNSUOkNmGV/y4H7WW4iK2mZPCE5OTlXM475X4pBoUpNohKZUBEL9R2UhiD0CuP2uWR4RIO6aUNC5gS9Xo+4fd+sTLodREpKSq4fFyjVKpIRqJ6x27W2SaBh1Vt8OS4da1/Ao0QJug7PvpeCk5MjC2Z/9FxpQ04ULcIvo2cTSSlSvDuhUKgIfOjCw4cPKVWqVLbHyQ5932tL3+eit9ap7syNe9EoVU4AFHO6QfNmxkhyUVFxSErzVbFWaz5Jsbe3Z8WikVy+fA0bG2vKly+bYwOifftPE6/1QaGIMSs3KDy4cSMgT5T04Y1bub1jP5KVBa0/G0HxPP4O0iMuLg7r2KTnChPyXY78pGz58mxqVQf3HWdRShIxagm3DpkHyslLZCUtk2UuXb7JtP/tJSpGUMJD4oeve+Pqan4ud/zEBabMPEBEtAp3Vx3ffvE2VatUyNY4ycnJ/LZwM/EJBurV8aZVywZYWqaNJd2iWW2qVE7g7503EQLatCydZcvjlJQUrl27iaurMyVKlEhzX6lUQgk3CA00lelLFM11BQ3wZrfOTF++Ft+9F1FJEvfc7Bjy21fUfeuNXB8rI+q3eQvVnzfRRVYxBU+wUCXkKPc1pCY5kSSJ9Rv3ceL0AywtBB8Obk3JkhnHKXjKR8O6YWO9jfOXA7CxEnw8rBc2NkZf4VYtG7Jq3Uqik41GXQZdFHWqp7V4VqlUVK+ecVjNLX8fIPDeY6pV9aJFs4y3SV2L2GPQx2MwpGAw6FA8cQ2zswigQf3cDzu578/1BH44EY8nQTl+P3KGIfv/wimDlK55hbu7O3F1fRGHryJJRpcmy7qvVphSIQQpKSlZzsqmUCgYuX4Z63+ajj4sErc6frw9MP2Jcn4hidyMbPEaEhsbi6OjIzExMa98YJOXQafT0f292TxONAaZEEJQtcwV5v8vdVUmhKBjr1+JSqplKvMscoE/l4/M8jh6vZ4BQ2dyJ7g6kkKFZHjER/1cKF+uBBMm/UtcSkUM+lia1gzl50mDc7Q6uh/0iNGfr+N+WHFUiji6tLZm3Kfvpql34d9j7Pzka6wDgknycqfVrO+p0Sx3z2SfotFo2DJnIfr4BPzataJS7bS5jjUazQvP/l+WXbuPMXXuVZL0ZUD/iO5tVRRLDkYkJlO9U1t8a1TPtA8hBBN/XMGRU8YoT27OkdwLq4SkMh5LuNicY9XCD3Ks/J8yf+F6fl9/CYNB0LyROz99n77HR0ZMmrKKHQftkVTOSIYQ+vewZmD/9L0OhBCM+fw3jp4vQkzEFSwtlZQrbc8nw5vSrGnWrZ2zyqLegyn+12HTtUEIdPO+oMuQD3J9rMwIDw3lr/GTEKERWFb05v3J3+bZ7y+32fv7Gs7+OAdVTDyiug8frJyLi2vBZrfJiT6RV9IyL0QIQWhoKA8ePODhYycsjYsZJEni/kPz+V1cXByRMTZIz/wbDn1MtgIYHD12muv3y2JhafxpCkVxdu6/Se+erVk2tyh795/E3c2Ft1p2zPH25YzZ/xAaVwNLawB3tuwJoVnj89SrZx7gxa9JQ6qe2UNUVBTOzs55Fks7Li6On6evJzLaFs9iTnT3M1+tCCFY8PE44jftAcChW2sGz/o51/0/W7dqSEWfUhw74U9Z78rsGz8Rm0NXUUoS+5duJHHldGo1b/rCPv439092HSmCUm08k/e/dgSXZ5LiPI6vxj87jvBO73Y5lnPnrqP8vlkLVm+iAE743+bipRtUq+qTpfYJCQnsOhCLpDZuIQuFB5v/ucjA/unXlySJGVOGc+jfk0RGOtHyjQZ5OmEXanPfbR0CdS4eHWWHou7uDFs6O8v1Y2Njue5/kVIVymU5XHNe8OjBA659NoUqj40BVcTu86z99BuGr5hXYDLlFFlJy2SIwWBg7PgFHDtngUBJfNQ51FalTNt9gfce0G/I//h5Yi+KFXPH3t4eV6dEIp4JNOThSraUyYvqFivmTp/3cu5jrdfruX37LiEhz/khqYpy+25QGiUNxu2vvIxYJITg408Xc+tRdSRJwYVbWiKjlzJt8lBTnS3zFlNk4RY8hfGziZ+/ib+rVKTjkAG5Lo+XV0m8vEqydelKKj1R0AClg2M5t+iPFyrpe/cfsnz1OWyLPGtpbj6xMeg12NpmbesxI/49HgDKVDe+FFGOPfvOc/T4FSKjNdSoVpK322S846HVajEY1GaRu9IE/XkOSZJo3jR3g7hkRP2PBnDomD9l7kehFYKbrWow5p3CH6/+3KF/OTDkCzzuhHCiiC1lvx9Fu6H5v/oHuH7OH4/weHhi3S9JEvqg9HPcF3ZemdjdMvnPilXbOHGpNCqrcqitvHH26Ehc+DYS4+8RGXoMK/vK3An2Y+LkjYDxH8JXn7XCw8Eflf4CnkUu8PXnb2drzIYNalOp9F0MBqNFpWR4xNstS7/0s8TGxtJ30Cz6fHiKW/cdCX+433Ruaqu6TbMmqduW5w8fZW7H95nzZjd+//7nXI11/jzR0dHcDLQ1uQopFGou39Ca1Ym7ew8bkapS7IREzO3APJMJQK/TpX05ZPI5rPjjAFpDMXTaVOMilYU9hmRj4gi9XkPVsjdo3+7lztotLcxzORsMOv7ZfYJVm+3Y8W9xfpodzNIVGVvhOzk5UcUn2fQb0+sTqeOXtagz6zbs4f1B83n3gwUsWrrlpZ4jI6rVr0e77SuInzQMw8yxjN78+0sFT9Hr9az842/mzl/PjRt3c1FScw59O42Kdx/jJKkoG6nh2o9ziY+PT1MvPj6evZu3cubosTz7t1WpTi2CPVIToQghUHplbgtRGJFX0jIZEhKeiEKZ+sNWKC1oWMeTY+ejcHKtjUJp3NcOe5zapk7tKmz4o0q20jIC/HvkLOf871KyhDPz//chCxZvJT5BT4O6ZXmjxcv5PgLM+N9mAsNrYGFjVD1qK3cUyf9QurQ3A/vWoWRJYzCMh/fu8e8HYyl7LxKAhH8vsd7Kkp7jRr+0DOlhbW2NWpXMsws5S7V5Ug6XShWIVYLDk0oxKomilbO2tZtTWr7Tg3lL1lD5bCCSJPHQ0RLfni/exTAYFDi61iIi+BAqtR0CQZkSEUyfPJi9B85hZ2fJe71HvvSxwdBBbTl/cSVhcX4IocfD8ST3Q6ti9XSFrnBnx94rfPCCAGyzpw/j19kbeBypw7uULcOzsCtx6PAZ5iwPxyAZo5gtXx+Oa5GDdOnU/KWeJz3KVPShTMWX/44NBgPDR87mUkAllEoHNu/cx8TPY2nYoPrLC/k8kebZpuzCY4iMjMTOLtUSPyggkDXdB1Hmwj0eqiRO9mnLhwtn5vrRjUexYlSdPoFTP81Gio5DVaMiH8x4NVNWykpaJkNqVPNk+4GH8MQX2aCLpGH98twJuk+ySD14drRP5v79+xQvXtwUyzg7Cnrl7/+w6M8YhKI4Bl0MFy6t5vtvcnebLDZemFarACq1E7071+WjYd3N6p3cuZcygRE8DcBtKyQenPDPVVmexcrKim7ti7NuWwgoPVBxn/d6+JrVefuDPqy4HcDVv3YB4NajDW36pTV0y03s7e0Z+PfvbJ82B5JTqNauJfVbt3xhm7ZvVeHwidO4Fm+OwaDDRnWDaT91pUKFMlSo8GLXuIDAIA4eOounZ1FavtHohXXd3VxZuWgwW7cdxMJCTWXfHvQfedqsztXrYXz86Vx+nTI03fjaFhYWfJ6OseCLOHn6tinoCoCkKsq5Cw/o0ilb3eQ6iYmJCCGwtU27G7Bj1yEu3imHSm2cwCTpK/D72jN5oqRV1SpguPrQGJ4TiK1RnuLFzSPB7Zg0g0oXg0BSYKUHq5U72f/237zZJfdCBT+lRa9utOjVrcBzs78sspKWyZA2rRrx4OFWduy9gkFAswZF6NenC0WLHmXOolPEJVgi6e9yK9CT7gMO4OkexS8/dKFs2eylKty0/S5CYTSWUqgc2XskmFGRkbkadrFCWQeOX4hD+SQXsJog6tVJG5fZ1bMkQWoFTrpntuEccx6pKiuM/LA7taqf4frNIGrXqIWfn3kCGEmS6D/5W8RP35iuc4JWq0WlUmXYXghBcHAwarWaokWL4urmRr9fvs9y//XqVmPSBAN//3MJhULQu/ub+Phk7rd+6N8zTJp+nkRdeYQ+jH+PLmPi1y9e2drb2/PeOx1Mctfw3c2lOx4olBZokkKRlPacvVaBWXM38OknvbP8DC/CxdkSvV6DUmn5ZFw9DnYF9/IXQrBgxGck/LUHCYFVpzcY9tsMM4UUH5+EQmkeJldnyJuEEwPmT2eV5Xi0NwKR3FzoMXlC2glSrHH72yAEZ0UCkgE085fSqG2rXI2r8CyvsoIG2QUrU2QXrPQRQnD3bgD9P9qLUKduy1Xxvsb8/2UvslPHXnOITEy1aNYm32HLqrdz1TpUCMEv01dzxj8WlRp6dvbNcJtywYjPUC7ZjL1WcK+6N++sX4Snd+lckyW/iY6KYkm/jxBnr6J3tKfm+A95q4956FSNRsPsdwdjs+skOpUCi/feZuicafmSQWjAsAXcepiaalObfI9V8xpRIRu5w3U6He8PmMilm/ao1PbYOxsnOg397vHLpPT9XJ/15c4Ker2eEWPmcvZKUZBUVPS6z8I5H2VLufw1cx7Bf+8FpYpKg3rzRs+chxXdOn8JySOnYPvEXiEJgWL6GLqOTP33FxsbS5/BS4lINBpFCl04Q9+3oe972bMVyS3+mjkX7eezuKCLp7Zki5WkQC8EtzrV59O/VhaITPmJ7IIlk29IksT9B6GkCA+eNWmJicv+S71uDXv+ORyHQmmPMOioUi7GlH70KStWbeeUfxjWFoKhH7xB+fKlM+zPYDCw/ItviTt0GmFtSY2PP6BZt058Pva9DNs8y9A5U7k86H2iQkNp37hRutuIrxJ/jvkan3/OGJVRWBLXx/1CxaYN8fRK3fFY99N0ym85gUqSQGMgfvE2dtStydv9svaZvQzJGvNrIdnzODwyW0papVIxsF9bJs54AMonxzP6REqVsElTVwjBtz8s4/jZeBQStHnTI0s5v5VKJfNmfszFS1fR63T4+XXN1rHO7pWrSZwwG+8Uo83BnQs3KVLCA79GDbPcx7PEBgTh/IxBoTUSkYFBZnUcHByYN6M38xbtJFkjUa92CXp2e/GxRV7S9ZMPWatJQTVpDlbJxhWuUpJQHTxLdHQ0Tk5OBSZbYUVW0q8Rd+/eZ9vOk6iUEv37vG2K0PSUhw+D2b33JO5uTrRt0+ylV0k1/HxxsjlHgs4YmEIIAyU9st/nhM/74l50K7fu3sPFWcXID4eaybZk+VaWrNWjUBm3Tq9/uZWVC/rg4pJ+BKZ1U37F/te1uD9xsrl263s8ypfBp1rWoyVVqZ5+isJXEUNQiNnnWSI8nuunz5kpaV3wY6OCfoKdAWLuPcwX+fwq2RF4IBGl0vh7LVEkkJo106ZTjY6OJioqCk9Pz3TPmd9q2ZBbdzaydZc/er1E7erWfDh0YJp6i5ZuYe8JdxRK4yRgw44IypY5RMd2medJliQJv2ovTrKSEQ+PnsUtJdUosHhkEtf2H8mxknavVokwNTg9cQaIVUm4Vq2Ypl6JEh78+F3/HI2R20iSRO/PRzPtz7/hSurvS2upznJUsP8aspJ+Tbhy5SZjvt5HgrYSQug5eOQ3VixMTdt45uwlvvrxKHEpFRG6GA4cXsAvPw19KUXt5OTEhE/rM2/xcRISJcp4Kfl2QvazxUiSxOCBnbh1K5BvftpK226LcXESfDayOQ0b+HHq3GNTYg2AqMTK7Nx9lHd7px/oP87/GsWf8YL1DIvn0sEj2VLSrxOKUh4IcdH0XT90s6NzHfOIZi7VfYlTbsf+iQV5mJ0a3/q1nu8qTxj36TtYW2/g2q1g7GwEn3zYK80W8rwFG/lzUwjJOjtKF4tgxk+90qSnBPhwaFc+HPriADr3guJRPONnLSmLcONmCOQ8vkqWUDjbYxDCZFiVLAxYu+U8AtZb7/Xkjzt3ubbmH0Dg1rUVbQf0ybRdYaDi0PcImTATj7gUIi0UuA/qUWApZgs7spJ+TVi19jgJWuM5nCQpeRhVg9//3MHgD4xnXguWHSVe64skgaR24vDZFI4dP0OjhnVeatymjWvRtHHuvMy/n7KNoMd+oILwePhpxn62rq2KWmXukmQwJOLsZJ9BLyC5Opu9pKPVULJM9ozZXifemf4DSyOj0J+9hsHRjlrjR5itogE6Dh/E7yGh3Nx2EKFWUWFAD+q3yp/EAgqFgk9GZLzdfPXqDf7YlICkroylGoKjvZg6axv/mzYkwzYvDIrjboVen4xSaZwIGPQxeJfK+9zQXSd8ytwzFyh58CIpCojp2YJPBvd/qT7f++YL+OaL3BEwlzl78DA3Dx/HxqMoHQb1NzPgaj98IGd8K3D3zHlK+ZSjWYeCOSN/FZCV9GuCQW/+UpIkJTptqk3glWvBKGxSXXsUKkcePAxFq9Wy/q8dpGi0dOvaCnv7jJVfXhP62NyGMSLagYiICD7o05Avfzj8JG53IrUqBtKmdcaxmrv9MJ6F12/hcuQyydZq7AZ3p1n7/+5LwMnZmTGb/0Cn06FUKtNVYJIk0ef7r+D7rwpAwhdz9VoAQuFhFiEsJjbnO0DDh3QhIHAhpy8akCQ9rZs40r1b7rsAPY+9vT2f7ljHhdNnUFtaUqW63ytveZwRe1b9SeDoyRSP0ZCEYM6x03y8bJ7Zb6928ybUbp43sfBfJ2Ql/ZrQ6s0KnLxwA71UGgAHy4t06pAaSlCvS0CfEoPawnh+HBV6gqJFmtKw+SgS9caz3lnzx7Nj87d4uBfNd/kB3IpAYFjqtYtjLC4uLhQtWpRFs5zYve8kzk52dOk04oUrJWcXFz7dtYGAO3ewd3Qs0BjChYn0znFfBRo1rMGClZtJMhjPWw0GHaVK5vxZlEol06cMJzk5GaVS+VLRvLKLWq3m4dmL3Ji3kl1JGqya1mHY4lmvTNKKrHJ96TrKxBgtAq2RsFqzl9vjb1LeJ2+D8LyOyC5YmfAquWDtP3CKvQdvoFQK+r/XzMxfud+QuZy5mATCgMGgw85GolypOALCG6NSG62XtSmxeLmeYt0fPxSI/Nev3+Hbn7bxKEyBi6Oezz5pRuOGaeNpy/z32H/gFAtXnCIxWcKnjJofv+v/Siq2s4eP4N95OB5xKQBohSD6s/foP/nbfJdl24KlBG3Zi1ArqTGsDw3atsq1vv/XpCNlT9wwXYdKOuqd3khlv1fPIHP/uo3c+XsPWKppPmoo5avkzHAQZBes/zxvtKibYQjNUR+24KtJewiLLo6VKoIP3i3J5r/PmBQ0gNrCgdg445xNq9WiUCiy5WLyslSsWJa1Kz8hJSXllXwBy+QdL/ptv0rcOXvBpKAB1JJESkDQC1rkDXv/WEfEp1MppTHae1w4ew2nv92ylI40KxTr+CbRZ2/gpAW9EIS3rEmFSpVY+8uvJN6+j03ZUnQfOzJf3y854eCGTQQM/YbiT3J7/33kHL12raaEV6l8kyHXlPS1a9do164dd+/mXQB3meyh1+sJDg7G2dmZGtUrsX5laW7cuE3JksUoWrQooaHhbNqbugWekhxBrQbufPrFfM5c1KJQCNq0cM12+MSXRVbQ6RMUEMiJLf9g7eRA2z7vFPoXXEakpKRw8NBJrKzUNG5U97U9l00P7+pVuGyrxi3B+NLXCYGFV1or9bzm/sFjuGtSDTI9Q+O4uHNfrinpHp99ws6iRQg5eR6FixMjvxrLohGf4b5kO26SgmRhYP7tQD5aNCtXxssr7vy9F4/41IQ3PnfCObbxb3qM/ijfZMg1JZ2SksK9e/dyqzuZl+Te/YeMnbCOe8HOWFnE06+nFwP7d6BGjWqmOp+P7U/Avamc8tcCghqVFLi7l+PP7U4olZboga37IqlY/gCdOrYosGeRgSunz7LnvY8pHxBBsjDw67Y9jFm3LFMFp9FouH75Cs6uRSjlVfAW7gF37tDv/emkOL6NwZCCX7lZ/Pa/j1/Z8/LsUqdFM+5/PZQr835HmZyCoklNPvx+Qv4LYmdj5gGhEQYsXZxydYg2/d+H/kaXTL1eT8KuI1g9iZ9vJSlI2n0UnU5XqL97YWlur6AVApVt/rqKZfnTGTNmzAvvh4eHv7QwMrnH1JnbCY6uiYU1GIBla2/Totk9yninvqglSWLBnHFm4RG/+HqlKTYxgKR04XZAaH6L/5/jQVAQQbfvUKV2rXQt7I/MXEj5gAjA+ILz2nyMPRs20bpntwz7DA4KYmWvobifuk68tRr7Ee/SLxfOPo/t2UvI3XvUfusNSpXxznK7lJQURnb9FL3nKFPglCuBdqxYtY2BAzq/tFyvCt0+/RjD6I/Q6XQFtmvUecIYFp72p9yJW6RIguAujRk1qH+ejSdJEuK5nR+DUmmaZJ7asx//pX+CEPj27kyTzunHQMhvWowexuaj56h4MxStENxoVZ0x+eyLnmUlPWvWLKpXr57hYXd6eUNlCo6oWHN7QJ0oxo0bAWZK+inPWkp7lrDBcDYRxZPoT0L3mIoVXi/r6EVLtnDy3GMs1IL+79Wlbp1qmTfKQzZOn82jH+fjEpvMsQolaLV0GlXrP3f+qkkxu7RCIik2ju0LlnF9/ipITsGmeT2GzPnFtA2+5dspVDl9ByQ1RZMhfNZqTrVuQd3mTXMs69KxX6Geux5nrWCL50LqL/qZOi2ztsty6dw5LMNSkEql/t4USkuiY6NzLM+rikKh4G7AA36a+g9hkeDhCl9/3pGyZfPnrNPF1ZVP9vzFsd17sbKxofebLfL02EGhUFCyX1cipyzHJcVApIWCkv27olAouHTyNGf6f4ZXmFGHXNt3GktbG+q+9XJ5x3ODsr4VeW/3n/y7YQtqWxvG9HsvX70BIBtKuly5cowePZr3308/opS/vz+1auVPhCKZzCnpoSQwRI8kGV/Ydpb3qFO7RyatYPiQrjx4uIRT/kkoFIJ27YrTrm3OX+qFjWUrt7FsgwGFqhwAX/10nIUzXSjtVbJA5AkNDSXo54WUi9OBpMLxVigHJk6n6o61ZvVKtmnB450ncH1yjnijcglqe7hxu98XVIw1urpobm1lbXF33v16nLFRhDG/b7IwcEokYKORuNN3JLE/T6Dlu5n/Fp7nzq1baH5bj7sOkCTKPIjmxPQFWVbSDs7OuCc+5EHsPVQOxsmiNuYSTRtlHo7zdeSbH//mUVR1AG4Hw7c/bub3pSPzbXxra2ve7NQh38Z795vPOVS1Eo8uXcOzakWad+nIueMn2D51DjXDUhd5JaKTubZ9b6FQ0gAeJUrQ45MPC2z8LCvp2rVrc/bs2QyVtCRJyN5chYfvJrzHF9+s4E6gDltrGPZBPVxdMw9BqFAomPzDYAwGA5IkvdAfOSoqmmkzNxIdL1HOy4ZPRvQo9EZAZy+Em2KAAyRofdi77xSDPigYJR3y8BGOUQkgpc7ORVRsmnrtBvVlp0rBg71HEFZW9Ph8BKe27qBYbGp2CktJQeT1O6Zruxq+JG8/zjmRQGPJzhiOMjiJ62MnU6F+LUqVyTyN5LNEPX6MncY4mTCRkJTl9uV9fKj6/ltIv8/iXpEqJFlIdH2/JnVq//fCtSYlJfEoTMWz2WmCQnjlcx9nRrMuHaCLcWKwePR4VAs2kqKJQy/ZoXzyrhFCgLVsPPqULCvp6dOno9FoMrzv5+eHwWDI8L5M/mJtbc2sqdlLGfksmb0o9Ho9H41Zyr3wmkiSxLmrSUTHrMg0D3BBY21lPpEUhkRcXLIWZU0Iwe/fTSbin0OgVlJh4Du8PfDlzqcq+FZkdzVvil56ABjdVSz90iZJAHNDHID7vuUJtFTg8mR1rRcCVfHU7GHvfvM5yxMS0S1Zg+KZ7GQlw+K5ePxUtpV0lRo12F+3Ai6njR4ciQgcshkSduicqex/cytRDx7h92YzyleulHmj1xArKyucHVKIemaOU8RJvNYK+ln8T51CsWAjRbVgL9nyr4ijEfYogKt1yzL0s/zbUSjsZPkX4eHhgZeXFwcOHMiwzoIFC3JFKJnCz/3797kd5GpaaStV1ly4lvVVVX4RHBzM0T37iIyMBGD4oLdwsfHHoE9Bp42mtu9dunTKWuq+v39bjPrn5ZT3D6T86TsEj5vK2YOHX0o+a2trOq+YRUD7etxp4EPIkI4M+N/PWWrb5O02KD7rx1U3a247qrnZoS69f/jSdF+hUPDB1B/w7tLWbJcr2MUa39o10+vyhVhZWdFn7UIe9G3Fgw710X03mD6TshdGVJIk3uzaie4jh+eagl75xw5Gjl3OuAnLuFsAPsc5QZIkxnzUBHv1eXTJd3C0PMenI/472/6h9x7g9CQjmJWkoJFkx0FfV8SCLxmxez3OLnkfS/1VIdsRxywtLRk5ciQ//fST6QD98ePHDBgwgCNHjhAVFZUnghYUr1LEsfxACMHefUcJCHzAoj8eYWmXupIq7nyZdSvzz38wM3YuXcXNL6dT9HE8oaWKUHf29zRs15qYmBh27z2Os5M9b77RMMuZwJYOHoX78h1mZXHfDaL3hM/yQvwsk5ycjEajwdHRMd370VFRLOkzHM5cRedgS9XPhvL2SyZ2KCwsWb6VpWt1SKoiADhZnWfVwv44OztluY/Y2FiWrtiJTido16YWvr7lclXGlJQUvvlhJbcDtNjawvCBDalf1xh5S6fTER4eTtGiRXPkiiSEYMfKP4h7EEKFpg2p0SRnaS/zi6CAAM4fOoqbdymOjviaitdDAOOuTMpXH/Det4UzWUhukS8Rxw4cOEDfvn3Zs2cPq1evJiAggIEDB+Lj44O/v392u5N5hRBCMObz3zhxsTgKZVEU3CYp9hrWDr6opUDe61GloEU0odVqufjjHCpHJIGkxCEomuM//ErDdq1xdHSkR7c22e5T5eaCXgjT2Vm8ApxKFcxZ9rNYWVmlSe34LE7Ozny6bQ1JSUlYWlrmeEs1Jjqajd9PxRAVg3v9mrQf+kFORc41TpwJQ1KVN11HJlZh156j9O6ZtbyT8fHxDBi2iJDYWkiSxK5D+5nyrYaaNXIe+vF5Jv64isPnvFEoVBAD3/18hDVLvXByckKlUlGsWLEc9z1vyCiKrtiBnZA45bCCxzMn8Faf3i8lb2JiIv9u34mljQ1N27bKtS34o1u2c+7jb/F6FMtFOzU2H3TkfqVwiI3HsUEN3vkqbya7Wq2W4OBg3NzcXvjvpLCSbSXdsGFD/P39GTZsGDVr1sRgMPDDDz8wbty4l8pNLFP42bXrMCcvlkCpdgJAbdcEL6djvNU8gnp16lKtmu+LO8hH4uLisIqMMysTUXEZ1M4aPb8ex/8uXcNh7xm0aiXq99vR8/2XeyHmJy+Tr1er1fJb135UOXwNSZKIWbOXNVHR9P7CGD/h9P6DnF/0BxgEFXq0p3n3zrkk9YuxUKe1MXDKYEchPVav2U1IbE3TuytJ78Oav87kqpIOCNIZFfQTohO9OHX6Eq3eakJKSgpqtTpH787AgACkP/7BVhiVaMnYFA5/O+2FSlqv17No6WYio7TUrO5Jm1aNzO5HhIezqFNfKpy6TRSCGZ0bMnrtslyJbnfyl3n4BMeBJFEiQcf1NTsYcuPfPM28d/H4CXYNH4/DjSDiShal3pTxNOma9xnPcpMcTZFu3rzJmTNnKFmyJCqVihs3bpCYmJjbsskUMsIjYpBU5ls0zs7uDB7YtVApaABnZ2eS/VJXWEII1H4VXqpPKysrxm7+g7bX9tLz5iGGzZ3+n5mYXjp7juKHL5ue11EneLzTeB5/7dx5TvUfS4kNhymx8V9uDf2GHX+u4+HDh3nu8dH/vfrYqq4DYNAnUrPiPVq3yrrLoN4ggOe+Q5G736mDnfm1QjzGWqFnaovO/K90XX6p1ZKTu/aa1XkcFsb21Wu5cPp0hv1qkpNRaXVmZfoHoTwISv9cXgjBiDFzWbHJnn/+LcakmQ9ZsWq7WZ0tP86gyqk7WEoK7CUlZTcf5+/Fy7P+sC8iLsHs0jomkdjYtJ4Mucne8ZOpdOURJXVKfAMjOfblFHQ6XeYNCxHZVtI///wzDRo04K233uLy5cucOnWK8+fPU61aNY4fP54XMsoUEt5q2QAHy6uma70uhprVCqeBhyRJvL9yNoHdGnOngQ9BfVsxYNHMl+5XoVDg5eWFu7t75pVfI2zsbNGozF8X4olNyvmtOykdnLpLUTxWw94Bn7LV5w2md+9HSop5IJbcpE7tqsz/tR39OkcweqDE7OkfZWvi1KNrC1xs/E3Xau7Srk3uWpyP/ugtitj6o0kKRaTc5N3O9pydsYBKR67hG55ElUsPODx6osl75sKx4/zepCvavl9zpkUfVk1IPytduQoV8HdQoH8yEYoROnR6HcH301fSt2/f5ewVl9SIgkp3/tl737xSQpLZ52chKdBExrzkJ2DEqm41dE9kFUIQX7fSS231ZwUR/Njs2jYk0mRE+qqQbSU9a9YsNm/ezOzZs7GysqJKlSqcOnWKrl270rx58zwQ0Zy5c+dSunRprKysqFevHqdOnXph/fXr11OxYkWsrKyoWrUq//zzT57L+Lri4V6UXya2pEb5W/h63aZvFx1DBnbKUV//7DzEgsUbuXjxWi5LmUrxUqX4aM0SRh7eyvClc3DIxjaojDk+lSuT/G5rEiXjSzbI1YZqw43uZ0pbG5OiAOML2E5roJQGym85wdqfpuepbN6lSzJ4YFd6dGuT7fNTV1cXFszsTeuGQbSs/5Afx9egWZPauSqfj08Z1i4fyqJpvmxc2ZERw7thuB9sVschIJjQUGP43X8nz6H83XDUkoSHRhD/vz+4c+tWmn6VSiW+3dpzSsRz1pDAPZGCW5WKVKqefjpIvcHA86/85/c5Sr3RiMdWqVvb94o5UP3tt7L1vI/Dwpjz3hD+17gDc98ZTOijRwAMmjeNiJE9uP9WLR70aUX/P+fnucuZoqJ52NpEn1JZihdRmMi2dffjx48zfMhDhw7RrFneuRGsXbuWvn37Mn/+fOrVq8fMmTNZv349N27cwM3NLU39Y8eO0bRpUyZPnkz79u1ZvXo1U6ZM4dy5c1SpkjUjJ9m6++UQQhAREYGlpaXp7Om7ScvZfdQJhcoZC+keo4Z40f7tJhz69yQaTQpvNG8gZ8LKA0IePmTjF99jePQYdflS9JnxIzY2NunWjYuLY903PyHCIrGr6kPPcaOQJIkdq/4kLiSUKm80o/ITN67k5GRmdniXigcuowCOi3j8JBvsn0S7Cx/Ynv7z81ZRv2rM7PAO5XeeM11fqeDOqPN7sbCwYHazTpQ5dt10L0boKLtvGfWapd3GT0lJYcWYCSRfvIHk7EDLiZ9RMQMlbTAYGPLRLK7er4pCoQZ9OH27qdNMtLctXEbQ3/tAraTmsL7Ua/Vmtp5tRod3qLDjrGlFfu2NqozdtSFbfeQWoY8esWbEF+hv3UcqXpS3f/kaH7+CCwOcE32SbSVdkNSrV486deowZ84cwPij8/T05OOPP+aLL9Ka7vfq1YuEhAS2bdtmKqtfvz7Vq1dn/vz5WRpTVtI5JykpiY9GzefaXXuUSg3t33Si3/tv0KXvdlRWqefFxZ0v4mCr49KdMkgKFV5u11k0Z7D8eeciQgimvtGFykeMOxcGIQjs3YKPf08b28BgMDC9fW8q7vZHIUkkIYgZ2YOB03/MsP+UlBR2/7meWxcuYrVkK6USjT6wsUqwm/U5HQqBJXhh4v7tO6wb9hmGG4HgXoQ3fp5A7ZbNAVg27mscZ6zB8knGqKvVPPno360ZTqiyg1arZd6CTUTH6KherTidOjR/6T6fxWAwML10HSoFp4b5vOZqxceBp7C0tHxBy7T8+eNUgtcbXR5L9GpPr/EZJ3mKjIzkwrETlCpflrI+PjkTPh/IFxesgiIlJYWzZ88yfvx4U5lCoaBly5YZnoUfP348Tfau1q1bs3nz5gzH0Wg0ZpHV8tqw4VVDp9Mxf+EmIqJTqFGtJB3bZ7xzMvXX9dx46IfyyfbZ3/sjcC1yAIPB3Mr4/r1AFLZtUFsaV88PImsyZ/7ffDnuvUzlEUKwbupMHu85BpZq6nwyqNDE/C1MPH78GJvzN0zXCkki5eyVNPV2r/yTs7/8RsL1uxwVBhpgh7UkEXTgpKlOYmIiKpXKbLfDwsKC9v3eA95jW4UK3Fy2Hkmnp0j7FrQfUrij0BUERYp50GziWNw9S1KqlHlSjb6Tv+MPtQWh564gnO3p/u1nuaKgAdRqNZ+M6JkrfaWHQqFAFHGEZ5S03sUh2ztje35fi+HHxfg8SeUc9cNC9pb2pOU7aWPO+x85xv6B4/C8E8J1OwuKfDGYnl+MfqnnKEy8Mkr68ePH6PX6NAY77u7uXL9+Pd02ISEh6dYPCQnJcJzJkyczceLElxf4NUQIwUej53LpTiUUSgv2HAknOGQzQwd1Trd+ZLTBlOADQFIVIVkTSlnPIO4/LoEkSej1ibgXtSJCk/qPWJIUJCZlbYNny9xFGL6ZT2m98fr05dsU2bGSsr7ph9b8r+Lg4ECiky0kPOOG5mhudnztvD+Bn02hWmQSSLZoEZwVCdST7MBSbVyF9f8Q/b6T6C3UFPugO+9/N57naT9sIAwbmNeP9Mpy7dx5NvQajndAOEdsVHh+/SHdP/vEdF+pVNL3x68LUMKXo8E3ozkz+nvcHkQSXsyJet+OzrYXRMj5S7hoU6+dtYLg85cgHSV9+IeZ+NwNB0lJqQQ9d6YuJnTAu6+Nced/I1BsNhg/fjwxMTGmv6AM3BkKCwkJCVy+co24uJfzAc4Kd+8GcP6aCwrlE4WqLMrew8EZ1vcuZYNelxoqVGl4QO2aFZj360Ba1A6gpk8A77aPZ/y4nqi4l9pQH0Kdmp5Zkin82Fkc9KnXpR/GcHbnvmw913+B/7d35+ExXf8Dx993JntIIrITS4LYonZiV1pBbVVrKYqq2imlRbX9+inaUqVU1Vp7La2dhtiFIvYtttgSJLLvM+f3R5gYEZKYZCac1/Pkedw799z7ycTM5y7nfI6lpSUVxnzKDQcLkoSWkGJ21P1Kvz7y+UNHKRaZ8fcyVxTUKNx1tMZ3UC9WfTeNkqv3UuhhLIl3wrn2/Tz2/p3+KGnWnDV07TOPHn3nsuGfrEsHS7D0k7FUv/GIIooZ5RLhxjdzuHPnTr4cOyoqilu3buXpPAsNO7Sh14kdlAlYTM/gnTTp/H6mbTbNXcDsdzox278zAasyP6+28fQgSWTEmIigkGex5x8wRn+a5MLRiYTfzfp7qaApMFfSTk5OqNVqXQ/IJ8LDw3Fze/58x25ubjnaHtK/zHL67CQ77t69y9274VSqVP6Viko8bc/e//h+5mEiYpxxsA1gxMCqtG7ZwCD7fp70s2H9K9wXnR8P+ewD7j9cSPDZJMzNBB+0LUvdOlUB+N8k/VugowfFsP6fiwih0LSBB+3aZK8DorCz1VuOUwQO7vqdCM+fDObC4WN4VatCNb862drv66jNZ/240aIpIafP0sCvDq7PfA5KVanIGRsznBPSx5FqhSCxXmXq/PAdVerU4o8+g7lGEuYo1FDZEpGWxtYZc7kZp2LVZgtUZulj5Wf9fgvPYmeoWePNm90qO2JCI/SW3ZI0hJw7T7FiWSQhA1k9+QfuzFqKdWwiMXUr0W/17xR1ds6TYzk6OlKnccPnvrZ7zXoiv5hB6cT0/2fXjl+iiJsr1ZtkbN9h6KfMPnUO8XcgAoFZh2Z8Nqg/CQkJaDQavQIoFtUqkHYsBLPHV+sPq5Wh3Gt0J63AJGkLCwtq1KhBQEAA7du3B9I7KQQEBDB48ODntvHz8yMgIIDhw4fr1u3atQs/P798iDjDrNl/sWpTFKkaBzyK7mfat+/h45OzGYieu9+5B4lPq4qVDSQJd2b/foxW/vXzrMBG6dKlqFFpE6euuKFSW4L2Pu82zfqLRaVSMXlSP1JTUxnz1QIWr7rMinWX6Ny+DL176pdtbO3fkNb+z/9Qv0i7iZ+z+NR5Sh0LIVGtEPthCzp3+UD3+vZFf3Lzi+kUf5TE8UIW3Pj6UzoMN5364vmtlLc3pby9n/tajYYNuPZVf87NXoZlfBLJflX4auV83ReibTkvbog06qrSl4sqZrieDOHof9dRmWWMlkjFk0NHLhotSV+/cZsfZm7hUSyU9DBj4pfdDXZibAh37IqiiYzRlZc9ZFOYz8s8/29iKCcPBxE/dSE+iRpAhdh3nr/GfceABbPy9LjPc2vfEZwSMwqKeEQlcXHPAb0krVKpGLpoDpGRkSiKgoODA4vGTOTRn/+gTtNCCz8GL/4VMzMzPp45haUWFiSduYRStAgffPdFgSz/mZUCk6QBRo4cSa9evahZsya1a9dm5syZxMfH06dP+lXZRx99RLFixZgyZQoAw4YNo3Hjxvz444+0bt2aVatW8d9//zF//vx8i/ny5aus2hSHyqIslkBEgjMzf93F3J8HvNJ+hRBExyl6l7KxCZYkJCRga2ubdcNXoCgKs38axO8L/yYi8iFV3/LMVmL98ee1BJ0tg0ptQVIK/LHqLuXLnaRunWqvHJOrhwdDd2/gv737KFSkCFVr19I7STkz8w8qPkoCwD0uhXM/LyZt8IBcTWaQG6cOHubQjPmQnIJr8wa8P2xgvhw3tzqNGU7ysIEkJibi4OCg91rnL4YzYeYCiMx4vmCbosFCnYwQWpTHvZG1mnicnZ4ps5VP0tLSGD3+L8Ki0/9v3QzX8OXXS5kx7dU+b4ZUodv7zFkaQMm4MGLMbdD6VaZ06dIvb/gKbp47j0tCGjz+bCiKgva+cSZDUjnYoRUifX5zIEVosSzq8NxtHR/PhrVjxWqsf1mD6+P/eqmr9rC2wgy6fTUaCwsL+s2ckh+hG0WBStJdunThwYMHTJw4kbCwMKpWrcr27dt1HQRCQ0P1BsfXq1ePFStWMH78eL788kvKli3Lxo0bsz1G2hCuXruFVnHRe/gfG//qV7qKolCimELI3Yx1xV1T8ixBP6FWq/m0f+ZnTC9yNyw54zk2IFQenDpz3SBJGtJrUjf0b/H8F+P1p880T0ifMSo/kvTNkKvs/Wgk3qHpFY6iAv5jk6UFbZ7pVKXRaFj65bfEBwUjCtnSaNwQ3qqfv3d7nvb0Ix+NRsPGf/4lPj6J91o1onSPDiT9vAYrRYUQgqiGb/H1hL4MHLaAkDslQCRTv3oMXTsb52Tkzp07hIY5YvH4wllR1ITc1C8DKYRg3U+/EHEkGMWuEO99/Tkez/SwzkuTv+nPLBcHrt9KxMNOYfSID/K8vGzVxg3Z4j6bUmHpz29ThRabSrm7ej+4ZTunfl8OGi2l3/enZZ+czane8cuRzDp6Es/dwaQp8KB9A4Z/1v+FbSJDbuj1PTFXVDwKzboD8OukQCVpgMGDB2d5ezswMDDTuk6dOtGpU+Yegfmlbp23cLBZQVxq+omBVpuKVwnDFOqYMqkz30xZR9hDcC6iMG5UzpJnfnEqqkZo01CeTDKgvU/5ciXz5diW9aqSdmM3ZoqCVgi0fm/l+YnME8e27tQlaACHVLi3LyhTz+cVk6Zg9+MKXB5fie65/Dke+9fjbOTeqRqNhgGDZ3HuRnlU6sKs/WcpP08dwFHHIkSevoji4sin332Jvb09SxcM4/SZ81hbW1Hep6zRapoXKVIEK4tYnu4WVeiZO92rJk9H9e0Cij+u0f3nmYsMDtxosGFOL6NSqRg+pEu+HOuJUmXL4DtrEsd/mg+xCVj5vUW/b7967rZCCFJSUp7bN+dM0DHO9PuSkg/T63CH7Qtmr50djTtmv/KgjY0No7as4sShI5hZmlOtdu2XVh7zrP4WV63NdLfJ4xWBQ6WyL2zzuihwSbqgKVKkCN+Oa8TcPw4Qn6hQtpQFE8bl7MzziX8272Ph8mASEsHH24zpkz9m3qzPDByx4Y0Z0Znbd37jQog5ZupU3n+vGI0b1c6XYw/4/WdWuP6PlKuhqDxcGPD91/lyXABbJ0fiENg89UxC2GZOBHH/ncVRyfiSKnXtAUcDAmndPX+/yJ+1cvU2zt+shNos/cv6UWJVfl+8hynffJ5pWzMzM6pXS6/ktGXBEkIWrwWtFrf279BlTP6NWbWzs6PnB54sWRNCGm7YWd3Cv1kx+n42l+hYQQkPNaWOHsL7qUk0Sp64xvH9B2jY4t18i9MQIiMjMTMze25RjOTkZDb/vhhtcgr1O7XDo0QJGnVoQ6MObV64z20LlnB6+m+YxSai1K5Ev6W/6u3/7I7deD7MmCjDJSGN6wEHXpqkzwQd4/iytaAo1Ovfg3JVfLPsWPY8Dd7zJ3xSCBcX/wWpGhxaNeLjIabzCCMvySSdD2rX8qV2rVfrRHP9eigzfrtMKukl/05c0vC/qSv539emXyjCysqK+bOHkZCQgIWFRb49D4b0W7d9pj9/goK85t+tMz9v243L6t1YayHkrZJ0++o5CauI/pdspJWK6t55+4wyO2LjklGp9acRTEnW30ar1epdBQXtDCB8zA+UiU2fVCMq+BrbXVzw7/3ywjTP7vevH2YRd+UG1iU96DR2ZLb/3/T/uC3Nmt7k0uXrVK3Sjk9HrCEivioA9x5pSb61jKdv9MaZqXBwdiYtLY379+/j5ORk0mVpU1NTmd1rIOrth9GYqSjUrRX9Zn6vu3uRnJzMT627UinwHGpFYeWC1bRfv+CltQOunL/AzS9/ovLjPhzazUdZNWYSn8z7SbeNhYMdqUJg/vhYQggonHHiGRsby78r1qCoVLTo0RVra2vOHz9JYOdBeN1Nn6hj67Z9qDctynEtg44jB8PI599FfZ3JcdIFRNDR06RoM764FUXNnXsFa8o1GxubfE3QxqYoCsOWzqPE5l+xWjaZAXvWU7xU5tv8/l9/zvkqnkSLNEItQT2kG1Xr5M+dhhd5t1lNbM0zqpSJtHDq1i4OwK5lK5lauRE/lK7FTx/00o3Tv3rwKK6xGbNeOaQKwo+dyvGxfx8yBvVXv+K6eBtWk35nbt8hOWrvVbokLVs0ITo6lrsPMuYaUBQV10rW5ppnEYQQxKgh5eM2KKlp/FS7BX+Xa8rPvk048PeWF+zduP6aNpPSa/biHZtGuUcp2M3byPZlK3Wvb16wRJegAcqHPCBwZubyr8+6cPQ4Hk+Nk1cpCppnnvu2HdiPy61rESc0JAktZ/3K0X7scAAeRUYy591OmA2aimrgFGa26ERsbCzHV6zXJWiAsjcjCVqx7lXegjfKm/ONWcBV8S2Loj0CqlJA+hmso4NRQ5KyQVEUGrz74lmESvuUY9D+fzh99BhOHu6UKfdq817nVFJSEmeOn6CIs5Pesb29S/J/4+uxZEUQaRqFBnU96PR+M65fCeHy6KlUjkj/QtduPMwKp4kMmDcDW3cXEhFYP77FnyYEauciOYpHCEHM9v0Uf7wPC0VFyr9HSE5OznENAzc3F6wto9FSXLfO2asM7eYP5r+dAbgV86Brm1bMeLsDlc7cTt/gWgQHv/g/6rZuYVInlXvX/8OdU+e4ejCI6k89HimkhahrGcWAtEnJugT9hJKq4WUq+dVip5MNJZ78XYXArJSH3jbm5uaMWr+U/dt3kZacxPBW/rrhTpt/nEPlY9d0V/SVD11m86x5CDO13j6EEJnWSVkznf+B0gtVrOhDny6X+XPtKZJTrfAqHs+4z3sZOyzJQGxsbKjbJO9mkMvKvVu3WNq5P8WOXSHOQk3ggI70m5ExnKVG9crUqK4/GuLckaN4PkzQDedRKQqaG+nDDNp80odfjvyHxZoAzLSCRy1qM+wFEyNkRZjrfzVpzFSo1Tn/YndwcKBP15IsXHGBpDQnnB3uMfTTppTw9qLEwIxaBeLuA712tvfS5x12dnbm4cOHWFtbU6iQcYaVAaya/AOp//cHjilaUkggFisKP74R+tBaTZla1XXb1u/UjlV/rMHnyn0A7tpZUqHdy5+3e/v44P39GIJ/mIcSl4B5LV/6T89cIlmtVtOktX/mHSSl6HUYVCkKIiGJtwf2YdXWQCpcvIcQgvNvedL3M1k2NrsK1CxYxmBqs2ClpKQQFxdHkSJFjNaDVnp9zO87lGJLd+iWo8yhxLpfaNjyXaKiohBCUKSI/pXw1UuX2NmwMyUeP7sUQnDn41YMmD9Tt3zzxg3SUlPxKlMmV3MGr5k2g6TvfqdokoYocwXt5z3pmUVv5Ox48OABN2/epkKFcs/t3T+jbXfKbTuuWz5XrSQDA9Yxt9snWB8IJsXKAtcBXejxzZe5jiE3oqOjsbS0ZHa15lQIST+REEKwxSaFMiVLgZmK0r060npgP727DCHnLxD48++oUtMo1+5dGrR7L0fH1Wg0OT4pOrn3AEGdB+P5+Jb5TZdCNFw/D986tQi7c4fApatAUXinb888q3Rm6l77qSqNwdBJWqvV8uDBA4oWLWpSt9Ik0xV25w4H1m/CqnAhWvbslqsryqz89n5vim/KmEVOCEHyL6O5d+IMaesDUAClbRMG//GLXrLd9sdSTk2bh3lsItSsQL9lc7G3tzdYXAD7N2/jzqmzuJYvS9OO7Q2672eF3b7NqkHp8w6rPJxpOX0CB5esxmX2Ol3RjUhLFaX/+pkG/nnfCzwxMZE53ftjtj+YVCtzbiTF0So64/2/6GTFoOtB3A65yvzuA0i7chMzCwtcO7Zg5B9zjHYCf2jrDs4tXw8qFVV7d6ZWs6ZGicNUySSdBwyZpE+dvsTX/7eNsIjC2BeKZfintWjZor6BIpVeR1fOnGVzl4GUvRxOCoJrbeow4q8lBkvUq6b8iHriPGwe3zq9Vtweq17tcJi8RDd0LEloEdOH88EI/XKqLxpP+zr4vdsnePy1V29dwvRhdBqe98MeF30+gaIzV+ueLYertdxPS8ZXsSZJaLn7UQuGLJzN1/X9sQk6R2UlvYd1rNCgmjKUzqOHvmj3r+xu6C3+Gv4Vmht3UZd05/2fvqN46VJ5eszXQW7yiezdnY+mzdzJw/hqmFmVIT6tGjPnHiMlJeXlDaU3VuBP8yh35T6KomCpqPDaFMS25asNtv/OX4wgaWwvrtUrz7Xm1aj7+1SsElL0xnZbKSoSb2eeVUhRlNc2QQPYVPAi+amZmO4WscbHr1a+HDvtbrhe5y9XjYr4Ng140L8tmv8bxGe//wzA/SvXqURGtZbCipqw/UGZ9mdoK/uPwHtTEOXO3MJ781FW9895vwMpe+T91nwUEaV/Cyo63p6IiAjc3d2NFJFk8pL1T+IsUEiOMdy0pCqVip7fjddbF/0wgjCLNRRJSU9QUebgUrWSwY5ZUHT9ajQL7kcQH3gUrCyoNLg3VerkT5K2LleaFLEHi8e9uO/ZW9Jx9BCq1tMvF2vp4UJM5D3sybizYulUNE9j02g0aM5f0195/jopKSkmPb68oJJJOh95uMLl2xnLLo7RuLi4ZN1AeuOV9G/C3c0HcE5MH0JzuZwbH77/4qpRr6p51w9Ydf0mF1dtQUHg2tGfFj275ekxX+bKufOcPRyET41qVKxWNV+OqVarGTB7er4c61ndJoxhfth9EgKPIqwsqfBZz0wJGmDIyt/54Z2OVA9LwA41F8u60n9S5opwuZWWlsbq//uR5NA72JYrTafPh6FWq1FcHCEso/KYxsUBc3Nzgx1XyiCfSb+EIZ9J37kTxsT//cXtMEFRBxg15G1qVH/zrlCknNm1bCU3twcirCxoOnIgZSpVNHZIryQhIYHbt25RrHjxbNVR37lkBde+mEbxhwncdbDEbdIQ2g7+JB8iLTiCjx0j5mEkNRs3NGgN8p8/GkjJFf9irqhIElruf9KWgb/+yJFtuzgwfBIO1+4RXcoVvxkTqfdeS4Md93UlO47lAVMbgvUi166FsnXHUaws1fT8sNVr/bxQKpj++zeQwCHjsQ+5S0wJF+r8NIEG7TLmFj99+Aj/jpuCCHuIunxpus+dzhL/7lQ8nzHd29kSDnx++ZBBe7nnByEEG2fP58GBY1DYlpZfjcDThDtbJSUlMcerLuUfZFQhu1CyCKOuHEZRFOLj47l96xbFPT3zbdKagi43+UTe7n5NnD5zkdET9xCfVhGhTWP3vtks+m2wTNSSSdnz1RQqhjwAzPEIfcTBr6ZSv20rFEUhJSWF7Z+Oo9KThHz1ISsHj0XExuvtwywukeTk5HybtcpQ1v00m5TxcyiWln5dtOrkOT4N3EDhwoVf0tI41Go1GvNnqoWZm+mGd9na2uJTPmf1t1/VhbNneXDrDtUb1jdqcZn8JHt3vyaWrjxMfFr6bVBFZcaNB1VZuXq7kaOSpAxCCMT9SL11Fg8ekZCQAMDdu3exu3Rb73XNlVAs61ZB8/iGn1YIqFO5wCVogAcBh7BPy7hx6X3qJkd2/GvEiF7M3Nwc917v88g8PSk/tDaj1MevNu3vicD9zOv0Mb+178WWBUty1HbR6Ansr/M+99oMYm7Dtly9cPGVYiko5JX0a0KrfaZWr6ImJeXl9Xol0xf410ZuHwvG1tOddp/1z1UFL1OgKArmFb3h9kndurQKpXW3Sl1cXIgtXhRuPtK9rvJwZsDC2Sx3+YaUq7dQe7rSPx+nGzUkYaXf8zlBDcWcHI0UTbq0tDT+/OpbEoIvQVE7Wn/zBSXLltG93vPbrwis5kv4+ct4V61Mg+eVA82mkPMXONT7c0rfiQLgYcAx/rW2pvmHnV/a9vihIzBnDcVSFVDUVDx7h53f/cTAFfNzHU9BIZO0kQghDFoVqFkTb46duYlQeQLgYH2Gtm26Gmz/knH89eMsEr6eS9FkLUkIfj1xhsELZxs7rFzr9tsPrBk+Hs2126g93ej8wyTdazY2NtT+fixBX03DJuwRCeVL0G7qBKysrOg7c0rWOy0g6o0cwL7Tl/G6EUG8CiK7v0vtxo2MGtOiUV/hPGc9zoqKRKHl+4NBNB7+CS379MDewQGAJh3aQodXP9bxzTt0CRrAKUnDzYD9kI0kHXb9Bo4pQlcvHoCouFcPqgCQSTqfnTx1gf/7YScPIxXcnGHCF62oWKHMyxu+RGv/hqgVFXsOXMNMLejbqz1urm9mfdzXya2VmymXnD5e2QoF7foALo+9TNmyZQtk7Xa34sUZ+tfiLF9v/EF76rVrrZvcIqu7Bif3HyJ4+TpQqfD7pCflq1bJo4gNp2qDejjuXMGxrTsp4upM5w86GP1vmHjsDOaKigSh4T+RQKs7CsroWcxd+Q/9tyw3aI1tSwc7koUWy6dm8MLGOusGT6nzztss855NuWvp9csThZZCtSq/pNXrQfbufglD9+7u0H0GD2IzZqzxLHqKlYvztoSfVHDNqPEO5U6H6pYvkURCISssalSi95+/4uLmZsTojOPUocMc6DyYUuHpV1JXSjrSdvNivMr7GDmygmeWf2e8A05xTBtHTcVWd9IghODRFz3oOXmiwY6l0Wj4qXMfSv59CAtUXK5eio82LMKtWLFstT8TdIy9U2ZBTDx29arR49uvCtyjH9m728QlJCTwIMISnno0FfbA8Le+pdeH+wf+nDv7C0lpaQgEcUJD41hLCDzLX6Mn8dmyecYOMd+dWrNJl6AByt6MJGj1Bry+HmvEqAqmhuOGsvfqGLTXYvW+gxRFgaRUgx5LrVYzau1i9vyzhaTYOAa2fy9HPdt969TCd+Myg8ZUEMgknY+sra0p6pBMRELGOmdHZIKWsuRUwpNIK2sqJWhJ0Go4SvpwJEVR0D4zB/KbQpjpXz0JIeAVZ5QTQrBt8Z9EXQvFs+ZbNMzh1I75RaPRsGXxcpKjoqje6l28K7zaEKhqjRtQ/MAG/l64lMuzllPu4eNpJp1tqdfB8MVJVCoVzdrrV8zTarVotVo5K2AW5LuSjxRFYdzIt/l+xh4ePrLAxTGFcSPzfto7qeC68MdKvBLSn0nbqNSUEVbcIxU3YY6Zt6eRozOOZoP7s2rHfipcDEMIwZkapRj4Wd9X2uf8oWNw+O1v7IXCNWszwr++zgejhhgoYsPQarXM7NqXUhsOYqOo2Dr7TxosmUG1Rq82k56zqyv9xo3mZIP6HF+4AgTU7NGRqg3qGSjyrC3/5nvuLFmPKjUNq3frM3D+zAJXpCavyWfSL5EXFceEECQmJhbIsZ5vun3r/ubYtF8hJh6L2r70/20GVlZWeXa8Xxq1w+twxnjQcJHKhWKFKV6vJn1///mNKejwrLA7d9i7fA2KSkWLfr10PZFzIyYmhoXe9SkblTGZycVKxRgVvNsAkRrOnk1buP/+cAo9Vd7iVocGfLrmDyNGlXt7N2witMcYHB9P5JIstKR89yldx40ycmR5R05VWUAoiiITdAF06/p1Tg//lvInblA+5AEllwew/ItJeXpMl/eaEPV43gKNENxvVp3vr51g2Mo/3tgEDeBWrBhF3Ny4vXYbCxt1YP6wL9BqtS9v+BwajQZFo9/22WVTkBgTi6V45tFYsmGfG+ene+cu6BI0gKWiIuHaLSNGZJpkkpakbAref4gS9zKmiVQrCimXb+TpMbuMGUHRORO436c1j0Z/yNANSwvU7cDU1FRu3bpl8HnTzxz9jxufT8XnxHXKXbhL0TnrWTPlx1ztq0iRIohW9Uh5PHd0rBqKtn3bkOEaRMO2rblUraRu+aGliuKtmhoxolfj4VuRCMuMFJQktNh4lzBiRKZJPpOWpGwqV+0tAu0tKR6TnnCEECjueT8W3b9PD+jTI8+PY2jHdwcSMHQiha6FEVfajSYzJ1H7HcMkv4sHgyj2KGPiBy2CC/sOEz04Gnt7+xzvb/DiufxVaRYxt8NwrFKeLp/0yVa78yeDuRD0Hz61qlG5Ro0st0tNTeXy+fMUsrenZKlSOY4PoHDhwvTasIgtU2ZCfCKeb9fn3Y+652pfpqBRu/e4O+4i5xb/hZKqwbZFfT4dM9zYYZkc+Uz6JQrSLFi5JYRgxartXLn2CA83a/r1aVfgxh/ml40/z+XyTwuwik4kqVZ5+q74zaAFH14n02u3oOLJG7rls77F+eJEgEH2vX/rdm52GkGRFC0hIolYocFTseBhCWfqzJioN7PW07YsWMr11ZsAKNujAy165T7Jbf19Mbe/nIHHo0Tu2Vvh+u0Q2g7qn2m7R5GRzO/YB+cDZ0myNMO8b3v6/zw118d93Qgh0Gq1BeoOUW7JqSrzwJuQpL+f/if/7LZDZWaPVpNAvSo3+OH7gcYOy2QlJSURFxdH0aJF5fC5LGg0Gn4sVYtKYRkzWF10tmbw9SCDzcy2bPz/eDh/NWGPImlExvP5sxXdGRO8J9Pf5sDfm7nSaywu8enPccMKW1Bp5Qzqtmie42MLIZhWqRGVr9zXrTtXypHRlw9lOu6CwaNxm/e3bn2MmYLrimm83aFtjo8rFWyy45iUK3uPPEJlln6LUKW24UiwID4+/iWt3lxWVlY4OTnJBP0CarUalU8pvXXaciVznaCFEGz+Ywkrv53Ksd2BAPT833g+uhBIEScnvW1VD6NIS0vTW6fRaNi2YCl3Y6NJevzs2S02hZA9B3Icy7oZc/i1ZVfCb4ZyTSTr1qvjk0hNzdyRSxsRrfd/xS5NEHHzdqbtnrXrz9XMatqBWQ3bsmbazJduf+HESZZ/+S1/TprCo8jIl24vFQzymbSEwrPJRlZAk9Ldu3WLHTPmoSSnUqG9f46eKXeeN531w8ejuXEXsxLufDDj21zFIIRgTr+huC7dgR0Kp+2WE/HDGPz79KBo0aKYv1UOAk7rtlcql8Hc3Fy3HBcXx5zu/am67STmijVBIo4q2GCNCnNHhxzFsvm3RSR9NYtSqVAKKy6IBMJIxQUzlLq+WFhYZGrjUL0S8esCsX3cM/uWSyGaNHnx2OYTew9wc8RkvKPSTwKijl9iu7NTev+E5zh16DD7ug3F624MQgjmbdvDZzvX5ur5vGRaZJKWaN7ImXU7IlHUjmg1cTSuZSGHiElEPnzIsra9qHT2DgAn1+4gddFU6mdzusISZbwZvnnlK8cRevMmqlU7sHl8MukWk8ylRWt0CevDBTP4a9RENDfuoirhTo8Z3wGQmJjIvJ4DCdm1j3fi1Zg/ntihPoXYK2Jxfu9tRowYlKNYwg4cxf2pi+UKKhv+9SlK5bcb8mkWU2h2+nwoy6JjCNtzBGFpQbUhH1O+6lsvPM7lfYdwj8q4SndIhXtHTmbZgfC/RavwuhsDpA/x9D1+g52LltNp+Gc5+v0k0yOTtMTIYV0oWSKAS1fuUcy9EB/16GfskAo0IQT7t+8kOvwB9Vq3KLAdy3avWEvFM7d10wMmRUaxaehXnF+5kap9ulCrWd4N/0lISOCf2fMhOQWX6pUxT9MCGR2LlNSM29luxYszePVCAG5cucKB9Zso7OLE7cPH8d54iFiRgrnKNqOtouDcuimfb1iW4w6Sip2tXq39BATNvxhC657dsm6jKHz0vwk5Oo6NmwuJCKwfn5hohEBTOOuiOcpzehbJ7kavB5mkjeBk8HkOHTmPq4sdHTu8YxK3ljt2aGbsEF4LQghm9x2C0/Kd2GpgSfnfabNyDmV9DT+tXtSjR+xevQ4zCwv8e3R97q3WV6EyN+dJarwpklEBDUPjIHQPx/Ycw2r9PHzr1Mr1/mNjY9FoNDg8Uy0sMTGRn1t2pdLBi6gVhbPFHbhTqwyuR66hVhSizBSc38t82/3k3gMc6PM5XrceEaMILnnY4qYoFMOcSyIRHyV9WsT7NmbU6tEpVyMY2kz4nEUnzuJ1LIQEFTzq2pxhPQw/b/t7fT/ilwNBWK7aiblGEKyNx2P9Tg42qk/9tq0ybV+1ZycO7zhAqbA4hBCcrVqCAb2yPnGQCg7Zu/slDN27e8u2/fw49xopohSatFgaVLvN9P/71ACRSqYgcPNW7nUcjp0248TrdpcmDPjzN4Me596tWyxr34eKp0JJAy687cvITSsM1nMa0nuxz2jRmUoHLxIsEqjx1NUoQPSYHnSfnLMrREg/kfl9+FjiVm5FlaYB/3oMXjpPN8HC+rkLMBsyDfVTJ6+3uzfDpoQHmgePcKlVlVZ9e2ba768delFy8xHdcqCIoSGFUSsKN0QyN800OPtVx7dXZ/x7f5jjuJ9ITEzkSMAebO3tqdWgXp6dZAshGFu5HqUv3cMDi/QTlix6rgOcOXKU4HWbwcIM/6EDcHZ1JerRI1aOHI/m+m0o5kqn6ZM4+W8gkZeu4lzZh3e6dc6T2KXne62nqoyMjGTIkCFs2rQJlUpFx44d+fnnF9cubtKkCXv37tVbN2DAAObNM970fqvWnyNFVABAbVaYvUdtuXQ5BJ9yZYwWk2Q40eEPsNWAXl+8+CSDH2f7j79S6fQtUBTMgUq7z/DP3D8M+gzSysqKYdtWsfnX30ncvBNx4IouOWiFID4tlfVzF2DtYEeLLh9k+8p0+7KV2M/bSLHHFSFT1+xlbaUZdPtqdPq+U1MzDTsxUxR6fjf+hftVkvWrmlUXNhyq54XjrYcIGxcaD/uY1gOyV6TkRaytrWn6XuarWUPTarUUjU3BU8k48TJ7EEVKSspzT8Z869bGt25t3fKjiAjGv9Oe4qeuU1mxQaVcZNLBljQKS8JeA/fNYdnFEHp+82We/y5S7hWYIVgffvgh586dY9euXWzevJl9+/bxySefvLRd//79uXfvnu5n2rRp+RBt1tLS9M+ABVbEx8nhTq+Leu/5c9XHTbccowanRrVf0CJ3lCT9hKRWFLQJiVlsnXs2NjZ0/nwYY9Yv42wtL7RCoBGC/6oW58E/AVgMmUZ8z6+Y0eXjbNfOfnTtJoWe2tRcUZF8O0y3/Hb3Tpz3La5bvlXEigqdXj6m2KV5faLMMj5ft6p7MWH7Oj6/dpQx5/YZJEHnJ7Vajbqit946UdErW3dLbly5wh/NPqDF6XDKKFbsFbGkaLW43InATpO+jUMq3FuxST67NnEF4kr6woULbN++nWPHjlGzZk0AfvnlF1q1asUPP/yAh4dHlm1tbGxwc3PL8vX8VruaAze3RaMys0cIgbdHKL6+7Y0dlmQgzq6utFw+m8DpsyEuCecmdXg/D3rYlm3djAtrduIem56sr3o60Kqd4ef/faKIoyODdq7l3xVrQFHhfCCI0iv3gKJgjUKpDQfZvnINrT588fPZfev/IWTtFlKJR6vVUkcpRKJaoUjljHmRHZ2c6PnPEnbO/A2SU6jeLntDvzqOHMw/VpbcPXAMCtnQY/xIbG1tX9rOFB3YuJkr2/Zg6eHCyeZVKHQ/CnVJd7r+mL1hbLumzqHCubugKNigxo9CnNYmolJU8FROVnI5KYmUfwpEkj58+DAODg66BA3QvHlzVCoVQUFBdOjQIcu2y5cv588//8TNzY02bdowYcKEFw4vSk5OJjk5Y+hDTEyMYX6Jx4YP6Yyd3SbOX7yBrY1gxJC+emM6pYKvfNUqlF8+P0+PUb9NK1LnJxOyfhvCTE3jgb0oU6linh6zcOHCdBiQPm/z73sO671miUJi1Is/K1cvXuLs0G+pER4L2JKiaNnpCNX7dKXzZ/ojCtw9Pen14/9eGlNycjIBf20AoNkHHWj7WX/4LHNpzoJkz9oNXP1kAu5x6WO9zlXxpPe/a3EoUiTb+1AS9e+qWCkqIgqZU7hWZRL3XcIahThF4ND27Vd+pp6UlMSRgEBs7Arn6TP6N1WBSNJhYWG4uLjorTMzM8PR0ZGwsLAsWkH37t0pWbIkHh4enD59mi+++IJLly6xfv36LNtMmTKFb775xmCxP0tRFPr2luUApVfX5IMONPkg6xPUvFT8nYbc37Qfp+T0K7HLZV3p3v759bKfOLVnPyXCYnRDuiwUFb716/Lx1Nx93uLi4vjlve5UOHABgJ8WrGDo5pUF9ur5iSvrtlIsLmMwdsVToexZs153gpQdxZo14MHGvRRNSb9sPuNqQ6+/llKtdi02zJrHw2uhOPh4Zzo5yqnIhw+Z3/4jvI5cJkkFBzu/zbCl82TtfwMyapIeO3YsU6e+uND8hQsXcr3/p59Z+/r64u7uTrNmzbh69Sre3t7PbTNu3DhGjhypW46JicHT0zPXMUjS66jlxz3ZqhXc3rUPrCxoPXIgbsWKZdru4JbtXP5nF1iZ41qvJg+szXBKSn8oKoRAcXHM0XHXTv+Z8C27EWZmRBaxofqBi+m3cIHK+y+y8cfZfDjxi1f/BY3JXH+iiTQEZpY5G17X8uOebNFouL19L1ia02bEACrVSp+lq6MBH79s/N+PVD4SgqKosRVguzqQLW8vp83HmXvfS7lj1CQ9atQoevfu/cJtvLy8cHNz4/79+3rr09LSiIyMzNHz5jp16gAQEhKSZZK2tLQ06DAWSXpdter3EfT7KMvXD2zczPl+4ygWnf7c/OzeIBwHduTKoo1YxyUS7VeZfpO/yvbxti5YSurXcymd+vihqqXgDil4kv55VSkK2riE3P9CJqLOZ73Zd+AkXrejSBOCS82rMvIlz/qfp3X/3tC/d65iCA0JYf3Qr0i7fgd1CXfazfwOrwrlM28YHad3e9sKhbiHD3N1TOn5jJqknZ2dcc5GNSY/Pz+ioqI4fvw4NR7P2bp79+70jiePE292BAcHA+Du7p6reCVJyr6L67boEjRAxbN3SO5XnI6X9xETE0Px4sVzND1h2JHjuKZm9HrySlbYbQ2ej0e43XG0pkarZqybOYdHx06Dgx3vf/MFjs9MwGHq3vKri+3mRexZ+RdWdoUZMXxQvvdbWfvpGMrvPZe+EPKA9Z+O5vO9mzJt59GoDg9W78Lx8d8l1LUwTd+VhZEMqUA8k65QoQL+/v7079+fefPmkZqayuDBg+natauuZ/edO3do1qwZS5cupXbt2ly9epUVK1bQqlUrihYtyunTpxkxYgSNGjWiSpUqRv6NJOkNYK7/9aIB1BYWFClShCI56AT1hMrRHq0QqJ4qyVmsd0duhz0CoGKP97m47zBmkxfirk2/nT7/9AVGBqw3eDW2rNy+fZvE+Hi8ypTJ9fzIycnJ/D3xe8wD/iPWQs3yh5H0mfZdtttfOXee4J17sPNw5d3OHXPckUur1aK5fFNvnXIplOTk5Ex3Gf379GBddAw3t+8FczOqDvyI8lXl96shFZiKY5GRkQwePFivmMmsWbN0xUxu3LhB6dKl2bNnD02aNOHWrVv06NGDs2fPEh8fj6enJx06dGD8+PE5qhz2JswnLUl54dyx4/zbdRBlQh+hEYLzjSsxYuuqHD9O0mg0qNVqYmNjmdPhI9wDT5GiVpHY7R2GLPpVLwn90rgdXocu6pYjRRqnKrjhgTmWNSvRb95PefI460kVtZSFG7FM0RDRqAqfrVucq++MpV99i8PUP3UV16LNFFyX/h/NOr3/0rZHdvzLif7jKHkvhjhF8KCXP4Pm/5zjRD29TgsqnrihWz7nW5zRx/+VPbdfUW7ySYFJ0sYik7Qk5d61i5c4tmEzipUlbQf2w8oqY5KI+Ph4bt28SfESJZ5bOfBBeDjLPh6G5vRlhKM9fpNG4temJWdPBmNlY41PxYqZksYv73bCa0/GtJWhIgUzwEOxQCME9we9T7+fvzf47/nvXxt40GOsrlCIVgjuD+5I35lTcryvhR8PxXXZDt1+VIpC3LcD6DJu5EtawuxWXSm966RuOUylpfaRtfhWq5qjGE7uPcDOoeMpdO0ecSVdqf71CG4dPApx8Xg2qc87PbrkaH9Sute6LKgkSQWPV3kfvMb5ZFoftONf9g+dSJFrYUSVcKHuTxOo305/CNfqEV9RbseJ9EQcFs+xEd9SuUkDqtaqmWl/T1Qb0ocz5ydSIjyWWEVwQ5tEI1X6l6FaUUi5dN2wv+Bjj27e0iVoSO/EJiKicrUvu0pluSQ2Eik0WKAQrRY0K5l1wSY9z5RGtdYI4h49ynEM1Ro3wPf4vzx48IBChQoxx78zlYOuoigK91b/y6a4ONp8mv0hYVLuycFskiTlu33jp1HhWgRumFM+9BEHxk/LVJ5Se/Oe3pWyy+1Irpw998L9NmjTimbbFpMwbQjJUwZR2k5/iJfKrWi2Y7xzM5SZbbrzY4WGzHjnAy6fPpvlthUa1ed20YwiSXGKoEi1Stk+1tPeG9SfiCK2+KkKUUNly9vaQlxevTlbbZ2aNyBGlf4+CiG4Vacs1er55SoOMzMz3N3dCdoVQNmgEN3fomiShtubA3K1T2MQQrB92UpWTJjM7rVZ18gwVfJKWpKkfKXVahH3I/TWmT2IIikpCWtra906VQl3OBqiW77vUQTNnv2cXrACMzcnukwap7f9E+V8K1Pu8dSgG9RmXJy+AOvoBOJqlafXlOzP2rV64Gh8dgWnL4TcZ+OA0Yw+tPW5z2Ur16rB/Z/Hc/rXJZCUikNzP3qNGJTtYz0tLCwMz5hUnp4/W1y/k622XcaO4O9CNtw7chLsCvHxpDF6jxhyw6qQLfcVwdN7EWYFJ3Us+mIi1rPWYK+Bu+YKKy5coXsBGktfcN5pSZJeCyqVCrOK3nD3lG6dqOiVKeF2+vFb/nw0DHH6Ctqi9iR6ueP53UJsUKERgl8uXGH03yte2Jmpw/BBxPfvTXR0NG5ubjmqhKW5dENv2fJSKNHR0Znmv37i7S4debtLx2zvPytubm7ElHKBaxknMkqJ7NWDUBSF9kM+hSGvHIZOvebNCGpfH6sNh7BUVNx0K0z1gVmPjzclqampRKzaSsXHjyKKpArOL/8bMWFMgekEJ5O0JL1Gdi5dQdiJs1gVc6XjyMG5HgaU17r//hNrho9He/0OqhJudHnOxBGuHh6M2r6WlJQUzM3NmVmtOTaPn9CpFQXbPScIDw9/aUEjW1vbXJUKVTycITRKt5zkUTRfOo9aWVnRcMbX7Bs3BbPwR2jLl6TzjOxNrJEXVCoVw1ct5J8Fi4mNjKapfzMqVHvLaPHkhFarRaXRn0RE0RSsvtIySUvSa2Ll5B/QTv6doqmQIrTMCj7D8D9/N8krBrfixRn61+JsbftkjLOw0P+6SrU0z9PqgO9OHc+2QV9S+Mod4jydaPj9uHyrSe3XqgV1W75LfHw8tra2Rv8bqtXqHNUONxWWlpZYtWpE0qKtWCkq4hWBfZsmRn8/c0ImaUl6TYSt2065x/MyWCgqzDcd4P79+7i6uho3MAMpP6AHd7+Yjnt0MtFmCnZ9OuSqKEp2ValXl4rHdvLgwQOcnJzytepXcnIy8z4eTOr+E2itLCk3qCfthw3Mt+PnByEEYWFhmJmZZavyZG4N/G0G68uX4eG1W9iV96bf4E9e3siEyCQtSa8Llf7VgValvFazEbXq25OT5by4fPgort6ladaxfZ4f80kP5/y2cuJkSq/e+7igSRzhk+ZwvJovNRo1yPdY8kJqaiq/dO+P5fbDaNRqzLu24NO5P+XJFa5KpeKDUQZ8SJ/PXp9PsCS94Up0b0eEVfoz6EQEqo7N8/QKxRiqNaxPlzEj8iVBG1Pqjbu6imMArnGpXA8+Y8SIDGvNlJ/w2nCIUkngHa+h6MItbFm41NhhmSR5JS1Jr4mOIwezr3RJQo+epLCnO5+9JsUmTgTu5/CM+SiJyRR924/OXwwvUM8UnwgJucHKtQcRQIc2NfGtnLnIyxPmpYqRJgRmj3/P8MIWVK3++tTETrn3AIen/oa2QiEqNHvDzN40MklL0mukUYc20KGNscMwmNCr1zjYdzReoelVs2L3BrPe0oKOuRyDbCzXb9xmyBebiU3xBWB/UCA/fCt4q8pzpn8Eun37JfNu3yV1/0k0Vub4DO5F9Qb18zPkPOVcvTIx6k26Km0PbM0pVzfrSnJvMpmkJUkyusiHD1n1+US0oWGoS3rQ7afvcChShGPb/6X0zUh4fNVVWKtw5/AJGGHkgHNo3cYDugQNkKgpz4ZNx7NM0paWlgxb/jsajQaVSlUg7xy8SOt+vVh+5x6XN+0GtRqvXh2p1/JdY4dlkmSSliQpz105d54z+w5RopIPNRs1zPT6ot6D8dmeXqdb7DvLwshHjPx7OQ4ebjxUKxR6eqhr4ZyPeTa25+XY7PTpM9Vx7q9KURR6TBoHk8YZOxSTJzuOSZKUp/asXkfAOz2wGDKNs+8NYNXk6Xqvp6SkwIlLuqtFRVHQnLyIRqPh7fZteNjLnwdqLQlCw7mqJWg1YZTBYwy7e5dNS5dz4nCQwfcN0L1zUxysMmanKmR+lq4d6+XJsaTXi5yq8iXkVJWS9Gp+qtsSn+PXdMsXi1rxyZUDFC5cGEgfLzutUiMqX7mv2+ZcBXfGnA7UvR589ChRDyKo3bRxrqqHvciJwP3s6zcGrxsRPLRSYTbsQz6anP0a39l17144y1ftQaDwftu6eHuXNPgxJNMmp6qUJCnHhBD8+fX/EbF1L5ibUa5vF1r162W4A8Qn6i1axScRHx+vS9KKolB93Gdc+GIaHvfjuOtamBrjMjqGKYpCtTp1DBfPMw5MnU3Zx8+9XZMFN35Zzs3+PSlZqpRBj+Pu7srnI7oadJ/S608maUl6QwkhWPzlN5xato7aYYmUVdKff9679APHvEpS6+0mBjmOTYMapF74B3NFhRCCOD/fTFXQ3unZjYpNGnDheDD1a1XHvVgxgxw7W2IT9BYLJ6QSERae6ySdnJzMP78uQJOQQI22LSn7eEYuScoNmaQl6Q218ZffsPlxBY6aBOxVGbeQ3WJTuHr4mMGSdL9fprLC0Z7EC9dQuRZlwP+Nf25v5WKenhTz9DTIMXOikF9Vko9cxFJJ76ITVr0MnavmbgKJ1NRUZrT9kIoBp1ErCjt+X0vc0hlUa5T94VNarZbU1NQ8rUtuig5v3cmhb35CGxGFmW9Zei+YSZGi2Z//+3Ulk7QkvaEenb6Aq1BIQ6ARQlfhKk4FDp6Gu5I1MzPjo8kTDbY/Q+v1/SSWW1oQfvICONrR9RXmYN6xci3lA06hfpzwS9+J5thvS7OdpDfOmsflWYtRJySjqvcWny6b+9w5s183UVFRHB4ykfKPx8OLG0EsH/4Vg5fNM3JkxieTtCS9oczcnNAIQTXFhn0iltJYkmpphlXvdnTu2S1Pj/3w/n2CdgbgUrIEtRoat0iHWq3mo/8ZpqNYanIyZujfJVDSNNlqG3w4iIdfz6FSXAoAmo2HWPHlt/SdMcUgsZmyK+fO43LzASjpKUlRFDRXbxk5KtMgh2BJ0huqy4QxXG5bhzvWKorZFyGuVyu6XTvIp3N+zNPiGef/O8HSRu+T1vtrzr/bhwUjXp+xss26dORCtVI8GTRzr7AF3h1bZavttZOncX+coCF9zuy00LA8idPUeJX34aGbvd46dXEXI0VjWuSVtCS9oSwtLRm1bin379/HwsIiT6d9fFrglJ/xufoAFAWnNAj/bR2nP/yAKjVr5Mvx85KdnR29Ny1l67RfIDGZCi2bUr9t62y1LVOrGkH2lnhEJwOgEQLz0oZ77BAXF8cf/YeRdvIiFLHDb/ww6rX2N9j+X0XRokWp/P0XHP92JpaRsaT6lqH79G+MHZZJkOOkX0KOk5Ykw5rzTidKBZ7WLScJLfZ//Ujz9m2NGFXeuHb5CuePHce3bi1Kenu/dPstvy3k3KzFKAmJWNarxoCFvxisA9ncfkMpvng7qsd3Sa4Ut+fDo1tMaqY0jUZDXFwc9vb2L9+4AMpNPpG3uyVJyld29auTQMa1wfWKxanVtLERI3q56OhoLl24QHJycrbbbF+4jG0NPyCt10Q2N/yAf/9c/dI2rQd8zOizexl1NYihy+cbtIe35uotXYIGKHYrknNHjxts/4agVqtf2wSdWzJJS5KUrz6c+AWpE/sR+k4NQj9oRPuVc0z6i3nbgiX8Uakp+6u0Zlbdllw4fvKlbYQQnJo6l1KRSZgrCqUeJHBy2txsHU9RFFTZKeydQ4q7k95yeFFbyvhWNPhxJMOSz6QlScpXKpWK7hPGGDuMbImOjubCpJ/xCY8HzHE/e4ed46dQYduaF7ZLTk7GLCZeb52IiUOr1eZJAs6O96dOZNndwVgHXybRsTCVxg6keIkSRolFyj6ZpCVJkrIQdvcuduFRgLlunbj/6KXtrKysoFYlxNb/UBQFrRBY1KxktAQN4O7pyeg9G4mIiMDOzg4LCwujxSJln7zdLUmSlIVSXl48qlxKtyyEwLzSyzuAAfRdModbvf258fZb3O3bmr6LZudRlNmnKApOTk56CVr2HTZtsnf3S8je3ZL0fJEPH7J7xVpU5ua06tMj11W6TN2F4yfZOX4K4v4jzCt589Gc6brJQQqyjb/M48rc5ZCUgnXT2nz62wzMzOTN1byUm3wik/RLyCQtSZndDQ3lz3a9qXjmNlrgXMMKDN+6yiRKWJ47dpy9U2cj4hIo0qAG3b4anafFWQqiY3v2cfb9z3CJSwUgRWiJG/8xPSe9PoVlTJEcgiVJUr7YMWMelc7eQVEU1IpCpf0X+GfOfGOHxf2wMHb2HEbJvw9RKiAYs28XsOb7n5677fXLl1k/bwFHAnbnc5TGd+NEsC5BA1goKpJDQo0YkZQVmaQlScoxJTlVb1mtKIjklCy2zj9BOwLwDrmvW7YVCpGHMg+ZOrh5G9uaf4jF4GlcaTuIJV9+m59hGl2xyhV5aJ1xaztNCMw83YwYkZQVmaQlScqx8u1acMch4xn01RJFqNk+e+Uv85KzZzGizZ+5tW1nm2m7/36cT8l7MSiKgmOKlphfV3Lv3r18itL46rVojtW4jzlX3I6LTlZc79SQ7t/IW92mSPYSkCQpx+q2aI5m4RQurN2EUJvR9NOPKFPJ+IUx6r7dhFN926L542/sUgXXKhWnw/gRmTdMSNRbtElIIfrRI9zd3fMpUuPrMm4UmjHDSU1NfW07/b0OCsyV9OTJk6lXrx42NjY4ODhkq40QgokTJ+Lu7o61tTXNmzfnypUreRuoJL0h6rdpRb+lc+m/6Bd869Qydjg6A2ZPp8rupRRaOZV+ezfgXaF8pm0KN6xJktAC6d8TD+tWxLts2fwO1ejUarVM0CauwFxJp6Sk0KlTJ/z8/Pjjjz+y1WbatGnMmjWLJUuWULp0aSZMmECLFi04f/68/I8pSa+x6n51X/h672nfssquMA+Dz6MULcLH/xuHubn5C9tIkjEUuCFYixcvZvjw4URFRb1wOyEEHh4ejBo1is8//xxIL/Hn6urK4sWL6dq1a7aOJ4dgSZIkSYYgh2A95fr164SFhdG8eXPdOnt7e+rUqcPhw4ezbJecnExMTIzejyRJkiQZw2ubpMPCwgBwdXXVW+/q6qp77XmmTJmCvb297sfT0zNP45QkSZKkrBg1SY8dOxZFUV74c/HixXyNady4cURHR+t+bt26la/HlyRJkqQnjNpxbNSoUfTu3fuF23h5eeVq325u6QPzw8PD9YZVhIeHU7Vq1SzbWVpaGnSidUmSJEnKLaMmaWdnZ5ydnfNk36VLl8bNzY2AgABdUo6JiSEoKIiBAwfmyTElSZIkyZAKzDPp0NBQgoODCQ0NRaPREBwcTHBwMHFxcbptypcvz4YNG4D0KdmGDx/O//73P/755x/OnDnDRx99hIeHB+3btzfSbyFJUkFXwAbESAVcgRknPXHiRJYsWaJbrlatGgB79uyhSZMmAFy6dIno6GjdNmPGjCE+Pp5PPvmEqKgoGjRowPbt2+UYaUmScuxRRASLPh5K2qlLKI721J0wnIYd2hg7LOk1V+DGSec3OU5akiSAXz/6lBIrduumvbziYUfPE9spWrSokSOTCgo5TlqSJCmPaG7c1ZuX2vXOIy4HnzZiRNKbQCZpSZKkbFA9M5XjfVc7vH0rGSka6U0hk7QkSVI2fDB9Eueb+nKxqCVnyzhTefo4XFxcjB2W9JorMB3HJEmSjMnVw4PRO/8iISEBa2trvVvfkpRXZJKWJEnKARsbG2OHIL1B5O1uSZIkSTJRMklLkiRJkomSSVqSJEmSTJRM0pIkSZJkomSSliRJkiQTJZO0JEmSJJkomaQlSZIkyUTJJC1JkiRJJkomaUmSJEkyUTJJS5IkSZKJkklakiRJkkyUTNKSJEmSZKJkkpYkSZIkEyWTtCRJkiSZKJmkJUmSJMlEySQtSZIkSSZKJmlJkiRJMlEySUuSJEmSiZJJWpIkSZJMlEzSkiRJkmSiZJKWJEmSJBMlk7QkSZIkmSgzYwcgSZJkDFfOnmPf7D9Q0jSU79CKeq1bGDskScpEJmlJkt44t2/cZPMHAyh39QEAZ/8ORFmi4NfqXSNHJkn65O1uSZLeOIfWbtQlaIBiUUlc2LjNiBFJ0vPJJC1J0htHZWGBRgj9lWZq4wQjSS8gk7QkSW+c1p/05lzDCrpEfbGsK02HfWLkqCQpM/lMWpKkN461tTXDt65iy++LEampdO/cAXdPT2OHJUmZFJgr6cmTJ1OvXj1sbGxwcHDIVpvevXujKIrej7+/f94GKklSgWBtbc0HQwfSadRQmaAlk1VgrqRTUlLo1KkTfn5+/PHHH9lu5+/vz6JFi3TLlpaWeRGeJEmSJBlcgUnS33zzDQCLFy/OUTtLS0vc3NzyICJJkiRJylsF5nZ3bgUGBuLi4oKPjw8DBw4kIiLC2CFJkiRJUrYUmCvp3PD39+f999+ndOnSXL16lS+//JKWLVty+PBh1OrnD7dITk4mOTlZtxwTE5Nf4UqSJEmSHqNeSY8dOzZTx65nfy5evJjr/Xft2pW2bdvi6+tL+/bt2bx5M8eOHSMwMDDLNlOmTMHe3l734yk7lEiSJElGYtQr6VGjRtG7d+8XbuPl5WWw43l5eeHk5ERISAjNmjV77jbjxo1j5MiRuuWYmBiZqCVJkiSjMGqSdnZ2xtnZOd+Od/v2bSIiInB3d89yG0tLS9kDXJIkSTIJBabjWGhoKMHBwYSGhqLRaAgODiY4OJi4uDjdNuXLl2fDhg0AxMXFMXr0aI4cOcKNGzcICAigXbt2lClThhYt5Gw3kiRJkukrMB3HJk6cyJIlS3TL1apVA2DPnj00adIEgEuXLhEdHQ2AWq3m9OnTLFmyhKioKDw8PHj33Xf57rvv5JWyJEmSVCAoQjxbZV56WnR0NA4ODty6dQs7OztjhyNJkiQVUE/6OEVFRWFvb5+tNgXmStpYYmNjAWTnMUmSJMkgYmNjs52k5ZX0S2i1Wu7evUvhwoVRFOWF2z45SyqoV90yfuOS8RtfQf8dZPzG9bL4hRDExsbi4eGBSpW9LmHySvolVCoVxYsXz1EbOzu7Avkf7AkZv3HJ+I2voP8OMn7jelH82b2CfqLA9O6WJEmSpDeNTNKSJEmSZKJkkjYgS0tLvv766wI7xEvGb1wyfuMr6L+DjN+48iJ+2XFMkiRJkkyUvJKWJEmSJBMlk7QkSZIkmSiZpCVJkiTJRMkk/YomT55MvXr1sLGxwcHBIVttevfunWnebH9//7wNNAu5iV8IwcSJE3F3d8fa2prmzZtz5cqVvA00C5GRkXz44YfY2dnh4OBA37599SZdeZ4mTZpkev8//fTTfIl3zpw5lCpVCisrK+rUqcPRo0dfuP3atWspX748VlZW+Pr6snXr1nyJMys5iX/x4sWZ3mcrK6t8jFbfvn37aNOmDR4eHiiKwsaNG1/aJjAwkOrVq2NpaUmZMmVYvHhxnseZlZzGHxgYmOn9VxSFsLCw/An4GVOmTKFWrVoULlwYFxcX2rdvz6VLl17azlQ+A7mJ3xCfAZmkX1FKSgqdOnVi4MCBOWrn7+/PvXv3dD8rV67MowhfLDfxT5s2jVmzZjFv3jyCgoKwtbWlRYsWJCUl5WGkz/fhhx9y7tw5du3axebNm9m3bx+ffPLJS9v1799f7/2fNm1anse6evVqRo4cyddff82JEyd46623aNGiBffv33/u9ocOHaJbt2707duXkydP0r59e9q3b8/Zs2fzPNbnyWn8kF7U4en3+ebNm/kYsb74+Hjeeust5syZk63tr1+/TuvWrWnatCnBwcEMHz6cfv36sWPHjjyO9PlyGv8Tly5d0vsbuLi45FGEL7Z3714GDRrEkSNH2LVrF6mpqbz77rvEx8dn2caUPgO5iR8M8BkQkkEsWrRI2NvbZ2vbXr16iXbt2uVpPDmV3fi1Wq1wc3MT06dP162LiooSlpaWYuXKlXkYYWbnz58XgDh27Jhu3bZt24SiKOLOnTtZtmvcuLEYNmxYPkSor3bt2mLQoEG6ZY1GIzw8PMSUKVOeu33nzp1F69at9dbVqVNHDBgwIE/jzEpO48/JZyK/AWLDhg0v3GbMmDGiUqVKeuu6dOkiWrRokYeRZU924t+zZ48AxKNHj/Ilppy6f/++AMTevXuz3MbUPgNPy078hvgMyCtpIwkMDMTFxQUfHx8GDhxIRESEsUPKluvXrxMWFkbz5s116+zt7alTpw6HDx/O11gOHz6Mg4MDNWvW1K1r3rw5KpWKoKCgF7Zdvnw5Tk5OVK5cmXHjxpGQkJCnsaakpHD8+HG9902lUtG8efMs37fDhw/rbQ/QokWLfH+fIXfxQ/q87iVLlsTT05N27dpx7ty5/AjXIEzp/X8VVatWxd3dnXfeeYeDBw8aOxydJ9MKOzo6ZrmNKf8NshM/vPpnQCZpI/D392fp0qUEBAQwdepU9u7dS8uWLdFoNMYO7aWePM9ydXXVW+/q6prvz7rCwsIy3bozMzPD0dHxhbF0796dP//8kz179jBu3DiWLVtGjx498jTWhw8fotFocvS+hYWFmcT7DLmL38fHh4ULF/L333/z559/otVqqVevHrdv386PkF9ZVu9/TEwMiYmJRooq+9zd3Zk3bx7r1q1j3bp1eHp60qRJE06cOGHs0NBqtQwfPpz69etTuXLlLLczpc/A07IbvyE+A3KCjecYO3YsU6dOfeE2Fy5coHz58rnaf9euXXX/9vX1pUqVKnh7exMYGEizZs1ytc+n5XX8eS278efW08+sfX19cXd3p1mzZly9ehVvb+9c71fS5+fnh5+fn265Xr16VKhQgd9++43vvvvOiJG9GXx8fPDx8dEt16tXj6tXrzJjxgyWLVtmxMhg0KBBnD17lgMHDhg1jtzKbvyG+AzIJP0co0aNonfv3i/cxsvLy2DH8/LywsnJiZCQEIMk6byM383NDYDw8HDc3d1168PDw6latWqu9vms7Mbv5uaWqdNSWloakZGRujizo06dOgCEhITkWZJ2cnJCrVYTHh6utz48PDzLWN3c3HK0fV7KTfzPMjc3p1q1aoSEhORFiAaX1ftvZ2eHtbW1kaJ6NbVr1zZ6Yhw8eLCuk+fLZhg0pc/AEzmJ/1m5+QzIJP0czs7OODs759vxbt++TUREhF7SexV5GX/p0qVxc3MjICBAl5RjYmIICgrKcQ/3rGQ3fj8/P6Kiojh+/Dg1atQAYPfu3Wi1Wl3izY7g4GAAg73/z2NhYUGNGjUICAigffv2QPots4CAAAYPHvzcNn5+fgQEBDB8+HDdul27dumdmeeX3MT/LI1Gw5kzZ2jVqlUeRmo4fn5+mYb7GOv9N5Tg4OA8/X/+IkIIhgwZwoYNGwgMDKR06dIvbWNKn4HcxP+sXH0GXqnbmSRu3rwpTp48Kb755htRqFAhcfLkSXHy5EkRGxur28bHx0esX79eCCFEbGys+Pzzz8Xhw4fF9evXxb///iuqV68uypYtK5KSkkw+fiGE+P7774WDg4P4+++/xenTp0W7du1E6dKlRWJiYr7H7+/vL6pVqyaCgoLEgQMHRNmyZUW3bt10r9++fVv4+PiIoKAgIYQQISEh4ttvvxX//fefuH79uvj777+Fl5eXaNSoUZ7HumrVKmFpaSkWL14szp8/Lz755BPh4OAgwsLChBBC9OzZU4wdO1a3/cGDB4WZmZn44YcfxIULF8TXX38tzM3NxZkzZ/I8VkPE/80334gdO3aIq1eviuPHj4uuXbsKKysrce7cOaPEHxsbq/v/DYiffvpJnDx5Uty8eVMIIcTYsWNFz549ddtfu3ZN2NjYiNGjR4sLFy6IOXPmCLVaLbZv314g4p8xY4bYuHGjuHLlijhz5owYNmyYUKlU4t9//zVK/AMHDhT29vYiMDBQ3Lt3T/eTkJCg28aUPwO5id8QnwGZpF9Rr169BJDpZ8+ePbptALFo0SIhhBAJCQni3XffFc7OzsLc3FyULFlS9O/fX/dFZ+rxC5E+DGvChAnC1dVVWFpaimbNmolLly7lf/BCiIiICNGtWzdRqFAhYWdnJ/r06aN3gnH9+nW93yc0NFQ0atRIODo6CktLS1GmTBkxevRoER0dnS/x/vLLL6JEiRLCwsJC1K5dWxw5ckT3WuPGjUWvXr30tl+zZo0oV66csLCwEJUqVRJbtmzJlzizkpP4hw8frtvW1dVVtGrVSpw4ccIIUad7MiTp2Z8nMffq1Us0btw4U5uqVasKCwsL4eXlpfc5yG85jX/q1KnC29tbWFlZCUdHR9GkSROxe/du4wQvxHNjf/a7xZQ/A7mJ3xCfATkLliRJkiSZKDkES5IkSZJMlEzSkiRJkmSiZJKWJEmSJBMlk7QkSZIkmSiZpCVJkiTJRMkkLUmSJEkmSiZpSZIkSTJRMklLkiRJkomSSVqSJEmSTJRM0pIkGUxgYCDVq1fH0tKSMmXKsHjxYmOHJEkFmkzSkiQZxPXr12ndujVNmzYlODiY4cOH069fP3bs2GHs0CSpwJK1uyVJypYHDx7g6+vL0KFD+fLLLwE4dOgQTZo0Ydu2bezcuZMtW7Zw9uxZXZuuXbsSFRXF9u3bjRW2JBVo8kpakqRscXZ2ZuHChUyaNIn//vuP2NhYevbsyeDBg2nWrBmHDx+mefPmem1atGjB4cOHjRSxJBV8ZsYOQJKkgqNVq1b079+fDz/8kJo1a2Jra8uUKVMACAsLw9XVVW97V1dXYmJiSExMxNra2hghS1KBJq+kJUnKkR9++IG0tDTWrl3L8uXLsbS0NHZIkvTakklakqQcuXr1Knfv3kWr1XLjxg3dejc3N8LDw/W2DQ8Px87OTl5FS1IuydvdkiRlW0pKCj169KBLly74+PjQr18/zpw5g4uLC35+fmzdulVv+127duHn52ekaCWp4JO9uyVJyrbRo0fz119/cerUKQoVKkTjxo2xt7dn8+bNXL9+ncqVKzNo0CA+/vhjdu/ezdChQ9myZQstWrQwduiSVCDJJC1JUrYEBgbyzjvvsGfPHho0aADAjRs3eOutt/j+++8ZOHAggYGBjBgxgvPnz1O8eHEmTJhA7969jRu4JBVgMklLkiRJkomSHcckSZIkyUTJJC1JkiRJJkomaUmSJEkyUTJJS5IkSZKJkklakiRJkkyUTNKSJEmSZKJkkpYkSZIkEyWTtCRJkiSZKJmkJUmSJMlEySQtSZIkSSZKJmlJkiRJMlEySUuSJEmSifp/MjTpNBhW7d0AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 500x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.datasets import make_moons\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X, y = make_moons(n_samples=500, noise=0.30, random_state=SEED)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=SEED)\n",
    "print(f\"train {X_train.shape}, test {X_test.shape}, positive rate {y_train.mean():.2f}\")\n",
    "\n",
    "# viz: the two moons, colored by label\n",
    "fig, ax = plt.subplots(figsize=(5, 4))\n",
    "ax.scatter(X_train[:, 0], X_train[:, 1], c=y_train, cmap=\"coolwarm\", s=12, edgecolor=\"k\", linewidth=0.2)\n",
    "ax.set_title(\"make_moons (noise=0.30)\"); ax.set_xlabel(\"x0\"); ax.set_ylabel(\"x1\")\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "180a59a3",
   "metadata": {},
   "source": [
    "> **Interpretation.** The classes interleave and the 30% noise smears the boundary, so a straight line cannot separate them and a single deep tree will overfit the wiggles. This is the regime where ensembles earn their keep.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1ae22e6",
   "metadata": {},
   "source": [
    "A voting classifier aggregates several fitted models. **Hard voting** takes the majority predicted label. **Soft voting** averages the predicted class probabilities and takes the argmax, so a confident-and-right model can outweigh a hesitant-and-wrong one. We use three deliberately different inductive biases: a linear model, a kernel SVM, and a forest.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "15bc5d50",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:47.646770Z",
     "iopub.status.busy": "2026-06-10T18:50:47.646696Z",
     "iopub.status.idle": "2026-06-10T18:50:47.853847Z",
     "shell.execute_reply": "2026-06-10T18:50:47.853429Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "lr   test accuracy 0.816\n",
      "svc  test accuracy 0.880\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "rf   test accuracy 0.880\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.ensemble import RandomForestClassifier, VotingClassifier\n",
    "\n",
    "def base_estimators():\n",
    "    # three DIFFERENT model families -> uncorrelated errors -> useful ensemble\n",
    "    return [\n",
    "        (\"lr\",  LogisticRegression(random_state=SEED)),\n",
    "        (\"svc\", SVC(probability=True, random_state=SEED)),  # probability=True enables soft voting\n",
    "        (\"rf\",  RandomForestClassifier(n_estimators=N_TREES, random_state=SEED)),\n",
    "    ]\n",
    "\n",
    "# individual scores first, so the ensemble has something to beat\n",
    "for name, est in base_estimators():\n",
    "    est.fit(X_train, y_train)\n",
    "    print(f\"{name:4s} test accuracy {est.score(X_test, y_test):.3f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "c3ab4a2e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:47.855076Z",
     "iopub.status.busy": "2026-06-10T18:50:47.854966Z",
     "iopub.status.idle": "2026-06-10T18:50:48.213602Z",
     "shell.execute_reply": "2026-06-10T18:50:48.213219Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hard voting test accuracy 0.880\n",
      "soft voting test accuracy 0.872\n"
     ]
    }
   ],
   "source": [
    "# hard vs soft voting over the same three families\n",
    "hard = VotingClassifier(base_estimators(), voting=\"hard\").fit(X_train, y_train)\n",
    "soft = VotingClassifier(base_estimators(), voting=\"soft\").fit(X_train, y_train)\n",
    "print(f\"hard voting test accuracy {hard.score(X_test, y_test):.3f}\")\n",
    "print(f\"soft voting test accuracy {soft.score(X_test, y_test):.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a09a330",
   "metadata": {},
   "source": [
    "> **Common confusion:** \"soft voting always beats hard voting.\" It usually does when the base models are well calibrated, but not always, and on this particular split the two are within a point of each other. The honest claim is weaker and is what we assert next: the *ensemble* should be no worse than its *worst* base model, because a vote cannot be dragged below the floor by one outlier the way a single model can.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "92cf260a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:48.214851Z",
     "iopub.status.busy": "2026-06-10T18:50:48.214742Z",
     "iopub.status.idle": "2026-06-10T18:50:48.416311Z",
     "shell.execute_reply": "2026-06-10T18:50:48.415839Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "worst base model 0.816 · soft ensemble 0.872\n",
      "[ ok ] 2.1 ensemble ≥ worst base model\n"
     ]
    }
   ],
   "source": [
    "# Assertable claim: the soft-vote ensemble is at least as good as the weakest base model.\n",
    "base_scores = []\n",
    "for name, est in base_estimators():\n",
    "    est.fit(X_train, y_train)\n",
    "    base_scores.append(est.score(X_test, y_test))\n",
    "worst = min(base_scores)\n",
    "ens = soft.score(X_test, y_test)\n",
    "print(f\"worst base model {worst:.3f} · soft ensemble {ens:.3f}\")\n",
    "_ = check(\"2.1 ensemble ≥ worst base model\",\n",
    "      lambda: (_ for _ in ()).throw(AssertionError(\n",
    "          f\"ensemble {ens:.3f} dropped below the worst base model {worst:.3f}\"))\n",
    "          if ens < worst - 1e-9 else None)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b7d9aba",
   "metadata": {},
   "source": [
    "> **Note:** three random forests with different seeds would barely improve on one, because their errors are correlated. A logistic regression plus an SVM plus a forest help because their *biases differ*: the linear model gets the global trend, the SVM curves around the margin, the forest carves boxes. Diversity, not raw quality, is the currency.\n",
    "\n",
    "> **Key takeaways.**\n",
    "> - `VotingClassifier` aggregates fitted models; soft voting averages probabilities, hard voting averages labels.\n",
    "> - Soft voting usually wins with calibrated models but is not guaranteed to; assert the safe claim, not the folklore.\n",
    "> - The base models must be *diverse* for the vote to help.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fb8e947",
   "metadata": {},
   "source": [
    "## Part 3 — Bagging and the 63% bootstrap\n",
    "\n",
    "> **Objectives.** Implement bagging in ~15 lines: bootstrap-resample the rows, fit a tree on each sample, majority-vote. Prove that a bootstrap sample of size `m` covers about 63% of the rows. Compare your bagger to sklearn's.\n",
    "\n",
    "**Bagging** (bootstrap aggregating) makes the base learners diverse by feeding each one a different resample of the data. Draw `m` rows *with replacement* from the `m` training rows, fit a tree, repeat `N` times, and vote. The resampling is the only source of diversity, and it is enough to convert a high-variance tree into a low-variance ensemble.\n",
    "\n",
    "First the arithmetic that the whole method leans on.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6d5cc5d",
   "metadata": {},
   "source": [
    "### Exercise 6.2 — The 63% bootstrap coverage\n",
    "`Difficulty 2/5 · ~10 min`\n",
    "\n",
    "Draw a bootstrap sample (`m` integer indices in `[0, m)`, with replacement) and compute the fraction of the `m` original rows that appear at least once. Implement `bootstrap_coverage(m, rng)`. The probability a specific row is *never* chosen is `(1 - 1/m)^m`, which converges to `1/e`, so the covered fraction converges to `1 - 1/e ≈ 0.632`. Your empirical number should land there.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8fc935c0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:48.417534Z",
     "iopub.status.busy": "2026-06-10T18:50:48.417419Z",
     "iopub.status.idle": "2026-06-10T18:50:48.420362Z",
     "shell.execute_reply": "2026-06-10T18:50:48.420115Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] 6.2 bootstrap coverage ≈ 63%: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    }
   ],
   "source": [
    "def bootstrap_coverage(m, rng):\n",
    "    \"\"\"Fraction of the m row-indices [0, m) that appear at least once in a\n",
    "    size-m draw WITH replacement.\"\"\"\n",
    "    # TODO 1: draw m indices in [0, m) with replacement -> use rng.integers\n",
    "    idx = None\n",
    "    attempted(idx)\n",
    "    # TODO 2: count how many distinct indices appeared, divide by m\n",
    "    frac = None\n",
    "    attempted(frac)\n",
    "    return float(frac)\n",
    "\n",
    "def _coverage():\n",
    "    g = np.random.default_rng(SEED)\n",
    "    # average coverage over several large samples should sit at 1 - 1/e\n",
    "    cov = np.mean([bootstrap_coverage(2000, g) for _ in range(30)])\n",
    "    target = 1 - 1 / np.e\n",
    "    assert abs(cov - target) < 0.01, \\\n",
    "        f\"mean coverage {cov:.3f} should be ≈ 1 - 1/e = {target:.3f} (the 63% rule)\"\n",
    "\n",
    "_ = check(\"6.2 bootstrap coverage ≈ 63%\", _coverage)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9493482e",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>`rng.integers(0, m, size=m)` gives the bootstrap indices (with replacement, since it can repeat). `np.unique` counts the distinct ones.</details>\n",
    "\n",
    "<details><summary>Hint 2 (the lines)</summary>\n",
    "\n",
    "```python\n",
    "idx  = rng.integers(0, m, size=m)\n",
    "frac = len(np.unique(idx)) / m\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — my coverage is ~1.0, not ~0.63</summary>You probably sampled without replacement (e.g. `rng.permutation(m)` or `rng.choice(m, size=m, replace=False)`), which covers everything by construction. The defining feature of a bootstrap is replacement: `rng.integers(0, m, size=m)` lets indices repeat, leaving ~37% never drawn.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "9d2de0e8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:48.421368Z",
     "iopub.status.busy": "2026-06-10T18:50:48.421284Z",
     "iopub.status.idle": "2026-06-10T18:50:48.426985Z",
     "shell.execute_reply": "2026-06-10T18:50:48.426591Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 6.2 bootstrap coverage ≈ 63%\n",
      "empirical coverage (m=2000) ≈ 0.631\n",
      "theory 1 - 1/e            = 0.632\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines bootstrap_coverage; the check below re-verifies it.\n",
    "def bootstrap_coverage(m, rng):\n",
    "    idx = rng.integers(0, m, size=m)            # with replacement: indices may repeat\n",
    "    return float(len(np.unique(idx)) / m)\n",
    "\n",
    "_ = check(\"6.2 bootstrap coverage ≈ 63%\", _coverage, required=True)\n",
    "g = np.random.default_rng(SEED)\n",
    "print(f\"empirical coverage (m=2000) ≈ {np.mean([bootstrap_coverage(2000, g) for _ in range(30)]):.3f}\")\n",
    "print(f\"theory 1 - 1/e            = {1 - 1/np.e:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2a0c766",
   "metadata": {},
   "source": [
    "### Exercise 6.3 — Bagging from scratch\n",
    "`Difficulty 3/5 · ~18 min`\n",
    "\n",
    "Now the ensemble. Implement `fit_bag` (train `n_estimators` trees, each on its own bootstrap sample) and `predict_bag` (majority vote across the trees). Use `DecisionTreeClassifier` as the base learner. Each tree gets a distinct `random_state` so the trees themselves vary, but the dominant source of diversity is the resampling.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "2d77e2f4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:48.428110Z",
     "iopub.status.busy": "2026-06-10T18:50:48.428023Z",
     "iopub.status.idle": "2026-06-10T18:50:48.431447Z",
     "shell.execute_reply": "2026-06-10T18:50:48.431187Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] 6.3 bagging from scratch: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "def fit_bag(X, y, n_estimators, rng):\n",
    "    \"\"\"Train n_estimators decision trees, each on a bootstrap resample of (X, y).\n",
    "    Returns a list of fitted trees.\"\"\"\n",
    "    m = len(y)\n",
    "    trees = []\n",
    "    for t in range(n_estimators):\n",
    "        # TODO 1: bootstrap row indices (size m, with replacement)\n",
    "        idx = None\n",
    "        attempted(idx)\n",
    "        # TODO 2: fit a DecisionTreeClassifier on X[idx], y[idx];\n",
    "        #         give it a distinct random_state so the tree itself varies\n",
    "        tree = None\n",
    "        attempted(tree)\n",
    "        trees.append(tree)\n",
    "    return trees\n",
    "\n",
    "def predict_bag(trees, X):\n",
    "    \"\"\"Majority vote across the trees. Returns an int label per row of X.\"\"\"\n",
    "    # TODO 3: stack each tree's prediction -> shape (n_trees, n_samples)\n",
    "    votes = None\n",
    "    attempted(votes)\n",
    "    # TODO 4: per column, take the most frequent label (axis=0).\n",
    "    #         Hint: scipy.stats.mode, or np.apply_along_axis with np.bincount.\n",
    "    out = None\n",
    "    attempted(out)\n",
    "    return np.asarray(out).astype(int)\n",
    "\n",
    "def _bag_works():\n",
    "    g = np.random.default_rng(SEED)\n",
    "    trees = fit_bag(X_train, y_train, 25, g)\n",
    "    assert len(trees) == 25, \"should return one fitted tree per estimator\"\n",
    "    pred = predict_bag(trees, X_test)\n",
    "    assert pred.shape == y_test.shape, f\"prediction shape {pred.shape} != {y_test.shape}\"\n",
    "    acc = (pred == y_test).mean()\n",
    "    # a single deep tree on noisy moons is ~0.80; the bag should clear that\n",
    "    assert acc > 0.82, f\"bagged accuracy {acc:.3f} should beat a single tree (~0.80)\"\n",
    "\n",
    "_ = check(\"6.3 bagging from scratch\", _bag_works)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a4e26d8",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>`fit_bag` is the loop you already have the pieces for: bootstrap indices from Exercise 6.2, then `DecisionTreeClassifier(random_state=...).fit(X[idx], y[idx])`. `predict_bag` collects all the trees' label predictions and returns the per-row majority.</details>\n",
    "\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "# fit_bag\n",
    "idx  = rng.integers(0, m, size=m)\n",
    "tree = DecisionTreeClassifier(random_state=int(rng.integers(0, 2**31 - 1))).fit(X[idx], y[idx])\n",
    "\n",
    "# predict_bag\n",
    "votes = np.array([t.predict(X) for t in trees])      # (n_trees, n_samples)\n",
    "out   = np.apply_along_axis(lambda col: np.bincount(col).argmax(), axis=0, arr=votes.astype(int))\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — \"object of type ... has no len\" or a shape error in predict_bag</summary>Stack the per-tree predictions into a 2-D array first: `np.array([t.predict(X) for t in trees])` has shape `(n_trees, n_samples)`. Vote *down the columns* (`axis=0`), one majority per sample. `np.bincount` needs non-negative ints, so cast labels with `.astype(int)`.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "a5ff9540",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:48.432402Z",
     "iopub.status.busy": "2026-06-10T18:50:48.432319Z",
     "iopub.status.idle": "2026-06-10T18:50:48.618367Z",
     "shell.execute_reply": "2026-06-10T18:50:48.618039Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 6.3 bagging from scratch\n",
      "single tree   test accuracy 0.856\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bag of 300   test accuracy 0.848\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines fit_bag / predict_bag; the check below re-verifies them.\n",
    "def fit_bag(X, y, n_estimators, rng):\n",
    "    m = len(y)\n",
    "    trees = []\n",
    "    for t in range(n_estimators):\n",
    "        idx = rng.integers(0, m, size=m)\n",
    "        tree = DecisionTreeClassifier(random_state=int(rng.integers(0, 2**31 - 1)))\n",
    "        tree.fit(X[idx], y[idx])\n",
    "        trees.append(tree)\n",
    "    return trees\n",
    "\n",
    "def predict_bag(trees, X):\n",
    "    votes = np.array([t.predict(X) for t in trees]).astype(int)   # (n_trees, n_samples)\n",
    "    out = np.apply_along_axis(lambda col: np.bincount(col).argmax(), axis=0, arr=votes)\n",
    "    return out.astype(int)\n",
    "\n",
    "_ = check(\"6.3 bagging from scratch\", _bag_works, required=True)\n",
    "g = np.random.default_rng(SEED)\n",
    "single = DecisionTreeClassifier(random_state=SEED).fit(X_train, y_train)\n",
    "trees = fit_bag(X_train, y_train, N_TREES, g)\n",
    "print(f\"single tree   test accuracy {single.score(X_test, y_test):.3f}\")\n",
    "print(f\"bag of {N_TREES:3d}   test accuracy {(predict_bag(trees, X_test) == y_test).mean():.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2813edc",
   "metadata": {},
   "source": [
    "Now line your bagger up against sklearn's `BaggingClassifier`. They are not bit-for-bit identical (different RNG plumbing), but they are the same algorithm, so their accuracies should sit within a couple of points.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "d194eb59",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:48.619484Z",
     "iopub.status.busy": "2026-06-10T18:50:48.619360Z",
     "iopub.status.idle": "2026-06-10T18:50:49.053855Z",
     "shell.execute_reply": "2026-06-10T18:50:49.053365Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "my bagging      0.848\n",
      "sklearn bagging 0.848\n",
      "[ ok ] 3.x scratch bagging tracks sklearn\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import BaggingClassifier\n",
    "\n",
    "sk_bag = BaggingClassifier(\n",
    "    DecisionTreeClassifier(random_state=SEED),\n",
    "    n_estimators=N_TREES, bootstrap=True, random_state=SEED,\n",
    ").fit(X_train, y_train)\n",
    "g = np.random.default_rng(SEED)\n",
    "mine = (predict_bag(fit_bag(X_train, y_train, N_TREES, g), X_test) == y_test).mean()\n",
    "print(f\"my bagging      {mine:.3f}\")\n",
    "print(f\"sklearn bagging {sk_bag.score(X_test, y_test):.3f}\")\n",
    "_ = check(\"3.x scratch bagging tracks sklearn\",\n",
    "      lambda: check_close(mine, sk_bag.score(X_test, y_test), atol=0.04, rtol=0.0,\n",
    "          msg=\"same algorithm, different RNG: accuracies should agree within ~4 points\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e53654dc",
   "metadata": {},
   "source": [
    "> **Key takeaways.**\n",
    "> - Bagging trains each base learner on a size-`m` bootstrap resample; resampling alone supplies the diversity.\n",
    "> - A bootstrap sample covers ~63% of the rows (`1 - 1/e`); the missing ~37% return in Part 5 as the out-of-bag set.\n",
    "> - The full method is ~15 lines; the library adds parallelism, OOB tracking, and probability aggregation, not new ideas.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68d9d155",
   "metadata": {},
   "source": [
    "## Part 4 — A random forest is bagged trees plus feature subsampling\n",
    "\n",
    "> **Objectives.** Show, with an exact assert, that `RandomForestClassifier` predicts by averaging its member trees. Add the one thing that separates a forest from plain bagging: at each split, consider only a random subset of features. Measure that this decorrelates the trees. Then read `feature_importances_` and its cardinality bias.\n",
    "\n",
    "A random forest is bagging of decision trees with one extra randomization: at every split each tree may only look at a random subset of the features (`sqrt(d)` of them by default for classification). The bagging part is identical to Part 3. To make that concrete, reconstruct the forest's prediction from its member trees and assert it matches exactly.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "ab83b98a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:49.054782Z",
     "iopub.status.busy": "2026-06-10T18:50:49.054645Z",
     "iopub.status.idle": "2026-06-10T18:50:49.241623Z",
     "shell.execute_reply": "2026-06-10T18:50:49.241256Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "forest agrees with hand-averaged member trees on 100.0% of rows\n",
      "[ ok ] 4.1 RF predict == mean of member-tree probabilities\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "rf = RandomForestClassifier(n_estimators=N_TREES, random_state=SEED).fit(X_train, y_train)\n",
    "\n",
    "# A forest predicts by averaging the per-tree predict_proba, then argmax.\n",
    "# Reconstruct that by hand from rf.estimators_ and assert EXACT agreement.\n",
    "member_proba = np.mean([est.predict_proba(X_test) for est in rf.estimators_], axis=0)\n",
    "reconstructed = rf.classes_[member_proba.argmax(axis=1)]\n",
    "print(f\"forest agrees with hand-averaged member trees on \"\n",
    "      f\"{(reconstructed == rf.predict(X_test)).mean():.1%} of rows\")\n",
    "_ = check(\"4.1 RF predict == mean of member-tree probabilities\",\n",
    "      lambda: None if (reconstructed == rf.predict(X_test)).all()\n",
    "              else (_ for _ in ()).throw(AssertionError(\n",
    "                  \"a forest's prediction is exactly the argmax of the average member-tree \"\n",
    "                  \"probability; the reconstruction should match every row\")))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44c53e8d",
   "metadata": {},
   "source": [
    "> **Interpretation.** The forest is not a black box on top of the trees. It *is* the trees, averaged. Bagging supplies the resampling; the forest adds feature subsampling per split. Strip the feature subsampling (`max_features=None`) and a forest reduces to exactly the bagging of Part 3.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00e36989",
   "metadata": {},
   "source": [
    "### Exercise 6.4 — Feature subsampling decorrelates the trees\n",
    "`Difficulty 3/5 · ~15 min`\n",
    "\n",
    "This is *why* forests beat plain bagging. If two trees always split on the same dominant feature first, they make correlated errors and the averaging in Part 1 buys little. Forcing each split to consider only a random feature subset makes the trees disagree more, which is exactly the independence the `1/sqrt(N)` argument wants. Moons has only two features, so switch to a 20-feature problem where subsampling has room to act, and measure the *mean pairwise agreement* of the member trees under `max_features=None` (full bagging) versus `max_features=\"sqrt\"` (a real forest). Implement `mean_pairwise_agreement` and confirm the forest's trees agree *less*.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "ac614ddd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:49.242722Z",
     "iopub.status.busy": "2026-06-10T18:50:49.242630Z",
     "iopub.status.idle": "2026-06-10T18:50:50.240258Z",
     "shell.execute_reply": "2026-06-10T18:50:50.239839Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] 6.4 sqrt subsampling decorrelates trees: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import make_classification\n",
    "\n",
    "def mean_pairwise_agreement(estimators, X):\n",
    "    \"\"\"Average over all distinct tree-pairs of the fraction of rows where the\n",
    "    two trees predict the same label. Higher = more correlated = less useful.\"\"\"\n",
    "    preds = np.array([e.predict(X) for e in estimators])   # (n_trees, n_samples)\n",
    "    n = len(estimators)\n",
    "    # TODO 1: collect (preds[i] == preds[j]).mean() for every pair i < j\n",
    "    agreements = None\n",
    "    attempted(agreements)\n",
    "    # TODO 2: return the mean over all pairs\n",
    "    out = None\n",
    "    attempted(out)\n",
    "    return float(out)\n",
    "\n",
    "# a higher-dimensional dataset so sqrt(d) subsampling actually removes features\n",
    "Xc, yc = make_classification(n_samples=600, n_features=20, n_informative=8,\n",
    "                             random_state=SEED)\n",
    "Xc_tr, Xc_te, yc_tr, yc_te = train_test_split(Xc, yc, test_size=0.25, random_state=SEED)\n",
    "rf_full = RandomForestClassifier(n_estimators=N_TREES, max_features=None,   # = bagging of trees\n",
    "                                 random_state=SEED).fit(Xc_tr, yc_tr)\n",
    "rf_sqrt = RandomForestClassifier(n_estimators=N_TREES, max_features=\"sqrt\", # = real forest\n",
    "                                 random_state=SEED).fit(Xc_tr, yc_tr)\n",
    "\n",
    "def _decorrelation():\n",
    "    a_full = mean_pairwise_agreement(rf_full.estimators_, Xc_te)\n",
    "    a_sqrt = mean_pairwise_agreement(rf_sqrt.estimators_, Xc_te)\n",
    "    assert 0.0 <= a_sqrt <= 1.0 and 0.0 <= a_full <= 1.0, \"agreement is a fraction in [0,1]\"\n",
    "    assert a_sqrt < a_full, \\\n",
    "        f\"feature subsampling should DEcorrelate: sqrt agreement {a_sqrt:.3f} \" \\\n",
    "        f\"must be below full-feature agreement {a_full:.3f}\"\n",
    "\n",
    "_ = check(\"6.4 sqrt subsampling decorrelates trees\", _decorrelation)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53724385",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>You already have `preds` of shape `(n_trees, n_samples)`. For each unordered pair of trees `(i, j)`, `(preds[i] == preds[j]).mean()` is their agreement. Average those over all pairs. A double loop over `i` and `j > i` is fine.</details>\n",
    "\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "agreements = [(preds[i] == preds[j]).mean()\n",
    "              for i in range(n) for j in range(i + 1, n)]\n",
    "out = np.mean(agreements)\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — agreement is 1.0 for both settings</summary>You may be comparing each tree to itself (`j` starting at `i`, not `i + 1`), which always agrees. Restrict to `j > i`. Also confirm you passed the *test* features `Xc_te`, not the training features, where overfit trees can all be near-perfect and trivially agree.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "d80bb83f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:50.241436Z",
     "iopub.status.busy": "2026-06-10T18:50:50.241345Z",
     "iopub.status.idle": "2026-06-10T18:50:50.771407Z",
     "shell.execute_reply": "2026-06-10T18:50:50.770852Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 6.4 sqrt subsampling decorrelates trees\n",
      "max_features=None (bagging) pairwise agreement 0.847\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "max_features=\"sqrt\" (forest) pairwise agreement 0.766\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines mean_pairwise_agreement; the check below re-verifies it.\n",
    "def mean_pairwise_agreement(estimators, X):\n",
    "    preds = np.array([e.predict(X) for e in estimators])\n",
    "    n = len(estimators)\n",
    "    agreements = [(preds[i] == preds[j]).mean()\n",
    "                  for i in range(n) for j in range(i + 1, n)]\n",
    "    return float(np.mean(agreements))\n",
    "\n",
    "_ = check(\"6.4 sqrt subsampling decorrelates trees\", _decorrelation, required=True)\n",
    "print(f\"max_features=None (bagging) pairwise agreement {mean_pairwise_agreement(rf_full.estimators_, Xc_te):.3f}\")\n",
    "print(f'max_features=\"sqrt\" (forest) pairwise agreement {mean_pairwise_agreement(rf_sqrt.estimators_, Xc_te):.3f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4001ab7",
   "metadata": {},
   "source": [
    "### Feature importances and their bias\n",
    "\n",
    "A forest reports `feature_importances_`: for each feature, the average impurity decrease at splits that use it, weighted by how many samples pass through, averaged over all trees, normalized to sum to 1. On Iris this recovers a real fact about the data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "16f41ab9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:50.772515Z",
     "iopub.status.busy": "2026-06-10T18:50:50.772435Z",
     "iopub.status.idle": "2026-06-10T18:50:50.927618Z",
     "shell.execute_reply": "2026-06-10T18:50:50.927137Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "petal length (cm)   0.438\n",
      "petal width (cm)    0.434\n",
      "sepal length (cm)   0.100\n",
      "sepal width (cm)    0.028\n",
      "[ ok ] 4.x importances sum to 1\n",
      "\n",
      "petal features carry 87% of the importance\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "\n",
    "iris = load_iris(as_frame=True)\n",
    "rf_iris = RandomForestClassifier(n_estimators=N_TREES, random_state=SEED).fit(iris.data, iris.target)\n",
    "imp = pd.Series(rf_iris.feature_importances_, index=iris.feature_names).sort_values(ascending=False)\n",
    "print(imp.to_string(float_format=lambda v: f\"{v:.3f}\"))\n",
    "_ = check(\"4.x importances sum to 1\",\n",
    "      lambda: check_close(rf_iris.feature_importances_.sum(), 1.0, atol=1e-6,\n",
    "          msg=\"Gini importances are normalized to sum to 1\"))\n",
    "print(f\"\\npetal features carry {imp.iloc[:2].sum():.0%} of the importance\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee4003f7",
   "metadata": {},
   "source": [
    "> **Interpretation.** Petal length and width dominate, matching the textbook structure of Iris: the petals separate the three species, the sepals barely help. The number is a genuine readout of how the model splits.\n",
    "\n",
    "> **Caveat:** impurity-based importances are biased toward high-cardinality features (continuous, or categoricals with many levels) because such features offer the tree more candidate split points. For any deployment decision, prefer **permutation importance** (shuffle one column, measure the accuracy drop), which does not have this bias. We use it in the Safety lens.\n",
    "\n",
    "> **Key takeaways.**\n",
    "> - A forest's prediction is exactly the average of its member trees; it is bagging plus per-split feature subsampling.\n",
    "> - Feature subsampling decorrelates the trees, which is what makes a forest beat plain bagging.\n",
    "> - `feature_importances_` is a real signal but cardinality-biased; permutation importance is the deployment-grade alternative.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6db21c00",
   "metadata": {},
   "source": [
    "## Part 5 — Out-of-bag evaluation: the free validation set\n",
    "\n",
    "> **Objectives.** Use the ~37% of rows each tree never saw to score the ensemble without a held-out split. Build OOB voting from scratch, then check it against `RandomForestClassifier(oob_score=True)` and the held-out test score.\n",
    "\n",
    "Each tree is trained on a bootstrap sample that misses ~37% of the rows (Part 3). For a given row, the trees that did *not* see it form an unbiased committee: ask only those trees, and you get an out-of-sample prediction for that row, for free, with no separate validation split. Average that over all rows and you have the **out-of-bag (OOB)** score.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96674761",
   "metadata": {},
   "source": [
    "### Exercise 6.5 — Out-of-bag score from scratch\n",
    "`Difficulty 4/5 · ~20 min`\n",
    "\n",
    "Track which rows each tree did *not* see, predict each row using only its out-of-bag trees, and report the accuracy. Implement `fit_bag_with_oob` (returns trees and, per tree, the set of row indices it was trained on) and `oob_score` (for each row, majority-vote the trees that excluded it). Rows that happened to be in every bootstrap sample are skipped (rare for reasonable `N`).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "b500596d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:50.928564Z",
     "iopub.status.busy": "2026-06-10T18:50:50.928469Z",
     "iopub.status.idle": "2026-06-10T18:50:51.103003Z",
     "shell.execute_reply": "2026-06-10T18:50:51.102463Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] 6.5 OOB tracks the test score: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    }
   ],
   "source": [
    "def fit_bag_with_oob(X, y, n_estimators, rng):\n",
    "    \"\"\"Like fit_bag, but also record, per tree, the boolean mask of which rows\n",
    "    were IN its bootstrap sample. Returns (trees, in_bag) where\n",
    "    in_bag[t] is a length-m boolean array.\"\"\"\n",
    "    m = len(y)\n",
    "    trees, in_bag = [], []\n",
    "    for t in range(n_estimators):\n",
    "        idx = rng.integers(0, m, size=m)\n",
    "        tree = DecisionTreeClassifier(random_state=int(rng.integers(0, 2**31 - 1)))\n",
    "        tree.fit(X[idx], y[idx])\n",
    "        mask = np.zeros(m, dtype=bool)\n",
    "        mask[idx] = True                 # rows that appeared at least once\n",
    "        trees.append(tree); in_bag.append(mask)\n",
    "    return trees, in_bag\n",
    "\n",
    "def oob_score(trees, in_bag, X, y):\n",
    "    \"\"\"For each row, majority-vote ONLY the trees whose in_bag mask is False for\n",
    "    that row (the trees that never trained on it). Return accuracy over the rows\n",
    "    that had at least one such tree.\"\"\"\n",
    "    m = len(y)\n",
    "    in_bag = np.array(in_bag)            # (n_trees, m)\n",
    "    all_preds = np.array([t.predict(X) for t in trees]).astype(int)  # (n_trees, m)\n",
    "    correct, total = 0, 0\n",
    "    for i in range(m):\n",
    "        # TODO 1: which trees did NOT see row i?  (a boolean over trees)\n",
    "        oob_trees = None\n",
    "        attempted(oob_trees)\n",
    "        if oob_trees.sum() == 0:\n",
    "            continue                     # row was in every bag; skip it\n",
    "        # TODO 2: majority label among all_preds[oob_trees, i]\n",
    "        vote = None\n",
    "        attempted(vote)\n",
    "        # TODO 3: tally correct/total\n",
    "        total += 1\n",
    "        if vote == int(y[i]):\n",
    "            correct += 1\n",
    "    return correct / total if total else 0.0\n",
    "\n",
    "def _oob_tracks_test():\n",
    "    g = np.random.default_rng(SEED)\n",
    "    trees, in_bag = fit_bag_with_oob(X_train, y_train, N_TREES, g)\n",
    "    oob = oob_score(trees, in_bag, X_train, y_train)\n",
    "    test = (predict_bag(trees, X_test) == y_test).mean()\n",
    "    assert 0.6 < oob <= 1.0, f\"OOB accuracy {oob:.3f} is implausible\"\n",
    "    assert abs(oob - test) < 0.08, \\\n",
    "        f\"OOB {oob:.3f} should track held-out test {test:.3f} within ~8 points\"\n",
    "\n",
    "_ = check(\"6.5 OOB tracks the test score\", _oob_tracks_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e23309f4",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>`in_bag[:, i]` is the boolean \"tree saw row i\". Its logical NOT picks the out-of-bag trees for row `i`. Index `all_preds[oob_trees, i]` to get just those trees' predictions, then take the majority with `np.bincount(...).argmax()`.</details>\n",
    "\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "oob_trees = ~in_bag[:, i]                       # bool over trees, True = didn't see row i\n",
    "votes     = all_preds[oob_trees, i]             # only the OOB trees' labels for row i\n",
    "vote      = np.bincount(votes).argmax()\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — my OOB is way below the test score</summary>Two usual causes. (1) You voted with *all* trees, not only the OOB ones, which leaks training labels and inflates nothing here but breaks the estimate. (2) You inverted the mask: `in_bag` is True for rows the tree *did* see, so the OOB trees are `~in_bag[:, i]`. Confirm `oob_trees` selects roughly 37% of the trees per row.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "07069400",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:51.104030Z",
     "iopub.status.busy": "2026-06-10T18:50:51.103934Z",
     "iopub.status.idle": "2026-06-10T18:50:51.476908Z",
     "shell.execute_reply": "2026-06-10T18:50:51.476392Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 6.5 OOB tracks the test score\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "my OOB score      0.877\n",
      "my held-out test  0.848\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines oob_score; the check below re-verifies it.\n",
    "def oob_score(trees, in_bag, X, y):\n",
    "    m = len(y)\n",
    "    in_bag = np.array(in_bag)\n",
    "    all_preds = np.array([t.predict(X) for t in trees]).astype(int)\n",
    "    correct, total = 0, 0\n",
    "    for i in range(m):\n",
    "        oob_trees = ~in_bag[:, i]\n",
    "        if oob_trees.sum() == 0:\n",
    "            continue\n",
    "        vote = np.bincount(all_preds[oob_trees, i]).argmax()\n",
    "        total += 1\n",
    "        correct += int(vote == int(y[i]))\n",
    "    return correct / total if total else 0.0\n",
    "\n",
    "_ = check(\"6.5 OOB tracks the test score\", _oob_tracks_test, required=True)\n",
    "g = np.random.default_rng(SEED)\n",
    "trees, in_bag = fit_bag_with_oob(X_train, y_train, N_TREES, g)\n",
    "print(f\"my OOB score      {oob_score(trees, in_bag, X_train, y_train):.3f}\")\n",
    "print(f\"my held-out test  {(predict_bag(trees, X_test) == y_test).mean():.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9089fe0f",
   "metadata": {},
   "source": [
    "sklearn computes the same quantity when you ask for it. The two should agree within a couple of points (different RNG, same estimator).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "1992f46c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:51.478187Z",
     "iopub.status.busy": "2026-06-10T18:50:51.478100Z",
     "iopub.status.idle": "2026-06-10T18:50:51.693491Z",
     "shell.execute_reply": "2026-06-10T18:50:51.693008Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sklearn OOB  0.885\n",
      "sklearn test 0.880\n",
      "[ ok ] 5.x sklearn OOB tracks its own test score\n"
     ]
    }
   ],
   "source": [
    "rf_oob = RandomForestClassifier(n_estimators=N_TREES, oob_score=True, random_state=SEED)\n",
    "rf_oob.fit(X_train, y_train)\n",
    "print(f\"sklearn OOB  {rf_oob.oob_score_:.3f}\")\n",
    "print(f\"sklearn test {rf_oob.score(X_test, y_test):.3f}\")\n",
    "_ = check(\"5.x sklearn OOB tracks its own test score\",\n",
    "      lambda: check_close(rf_oob.oob_score_, rf_oob.score(X_test, y_test), atol=0.06, rtol=0.0,\n",
    "          msg=\"OOB is an unbiased generalization estimate; it should sit near the test score\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9320c057",
   "metadata": {},
   "source": [
    "> **Caveat:** OOB is not magically better than a validation set. Each tree sees ~37% of the data as OOB, so for small forests the estimate is noisy and slightly pessimistic (each row is judged by fewer trees than the full ensemble). And if you *tune* hyperparameters against OOB, it stops being held-out, exactly like reusing a validation set. It is a convenience, not a free lunch.\n",
    "\n",
    "> **Key takeaways.**\n",
    "> - The ~37% of rows each tree misses form an out-of-sample committee for that row; averaging gives the OOB score.\n",
    "> - OOB tracks the held-out test score and costs no extra data split.\n",
    "> - It is noisier for small forests and is no longer held-out once you tune against it.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26400868",
   "metadata": {},
   "source": [
    "## Part 6 — Boosting: train each learner to fix the last one\n",
    "\n",
    "> **Objectives.** Implement AdaBoost from scratch and confirm it matches sklearn on the same stumps. Implement gradient boosting as \"fit a tree to the residuals\" and confirm it matches `GradientBoostingRegressor`. Contrast the bias-vs-variance role of boosting against bagging.\n",
    "\n",
    "Bagging makes trees diverse by perturbing the *data*. Boosting makes them diverse by perturbing the *loss*: each new learner concentrates on the examples the previous ones got wrong. Where bagging averages independent low-bias learners to cut variance, boosting stacks high-bias learners additively to cut bias.\n",
    "\n",
    "**AdaBoost** (Freund and Schapire, 1996), for labels in `{-1, +1}`:\n",
    "1. Start with uniform example weights `w_i = 1/m`.\n",
    "2. For `t = 1..T`: fit a weak learner on the weighted data; compute its weighted error `eps_t`; give it voice `alpha_t = 0.5 * log((1 - eps_t) / eps_t)`; multiply the weight of each example by `exp(-alpha_t * y_i * h_t(x_i))` and renormalize.\n",
    "3. Predict with `sign(sum_t alpha_t * h_t(x))`.\n",
    "\n",
    "The `0.5 * log((1 - eps) / eps)` is not arbitrary: it is the step size that minimizes the exponential loss for that round, larger for more accurate stumps and zero at `eps = 0.5` (a coin-flip learner gets no voice).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1b014b9",
   "metadata": {},
   "source": [
    "### Exercise 6.6 — AdaBoost from scratch\n",
    "`Difficulty 4/5 · ~25 min`\n",
    "\n",
    "Implement `fit_adaboost` (returns the stumps and their `alpha` weights) and `predict_adaboost`. The base learner is a depth-1 tree (a \"stump\"), fit with `sample_weight=w`. Work in `{-1, +1}` label space. Clip the error away from 0 and 1 so the `log` never blows up.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "9e5d2e34",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:51.694420Z",
     "iopub.status.busy": "2026-06-10T18:50:51.694346Z",
     "iopub.status.idle": "2026-06-10T18:50:51.697989Z",
     "shell.execute_reply": "2026-06-10T18:50:51.697757Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] 6.6 AdaBoost matches sklearn: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    }
   ],
   "source": [
    "def fit_adaboost(X, y_pm, T):\n",
    "    \"\"\"AdaBoost over depth-1 stumps. y_pm in {-1, +1}. Returns (stumps, alphas).\"\"\"\n",
    "    m = len(y_pm)\n",
    "    w = np.ones(m) / m\n",
    "    stumps, alphas = [], []\n",
    "    for t in range(T):\n",
    "        stump = DecisionTreeClassifier(max_depth=1, random_state=SEED)\n",
    "        # TODO 1: fit the stump on (X, y_pm) WEIGHTED by w (sample_weight=...)\n",
    "        # TODO 2: pred = stump.predict(X)\n",
    "        # TODO 3: weighted error eps = sum(w * (pred != y_pm)) / sum(w); clip to [1e-10, 1-1e-10]\n",
    "        # TODO 4: alpha = 0.5 * log((1 - eps) / eps)\n",
    "        # TODO 5: w *= exp(-alpha * y_pm * pred); renormalize so w sums to 1\n",
    "        # TODO 6: stumps.append(stump); alphas.append(alpha)\n",
    "        raise NotImplementedError  # replace this whole loop body per TODOs 1-6\n",
    "    return stumps, np.array(alphas)\n",
    "\n",
    "def predict_adaboost(stumps, alphas, X):\n",
    "    \"\"\"Weighted vote, then sign. Returns labels in {-1, +1}.\"\"\"\n",
    "    # TODO 7: score = sum_t alpha_t * stump_t.predict(X)\n",
    "    score = None\n",
    "    attempted(score)\n",
    "    return np.where(np.sign(score) >= 0, 1, -1)\n",
    "\n",
    "def _ada_matches_sklearn():\n",
    "    y_pm = np.where(y_train == 0, -1, 1)\n",
    "    stumps, alphas = fit_adaboost(X_train, y_pm, N_ROUNDS)\n",
    "    assert len(stumps) == N_ROUNDS and alphas.shape == (N_ROUNDS,), \"one stump+alpha per round\"\n",
    "    pred_pm = predict_adaboost(stumps, alphas, X_test)\n",
    "    acc = (np.where(pred_pm == -1, 0, 1) == y_test).mean()\n",
    "    from sklearn.ensemble import AdaBoostClassifier\n",
    "    sk = AdaBoostClassifier(estimator=DecisionTreeClassifier(max_depth=1, random_state=SEED),\n",
    "                            n_estimators=N_ROUNDS, random_state=SEED).fit(X_train, y_train)\n",
    "    assert abs(acc - sk.score(X_test, y_test)) < 0.05, \\\n",
    "        f\"scratch AdaBoost {acc:.3f} should match sklearn {sk.score(X_test, y_test):.3f} within 5 points\"\n",
    "\n",
    "_ = check(\"6.6 AdaBoost matches sklearn\", _ada_matches_sklearn)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3612af48",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>The loop body is a direct transcription of the four update lines. Fit the stump with `stump.fit(X, y_pm, sample_weight=w)`. The reweighting `w *= np.exp(-alpha * y_pm * pred)` increases weight on misclassified examples (where `y_pm * pred = -1`) and decreases it on correct ones.</details>\n",
    "\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "stump.fit(X, y_pm, sample_weight=w)\n",
    "pred  = stump.predict(X)\n",
    "eps   = np.clip((w * (pred != y_pm)).sum() / w.sum(), 1e-10, 1 - 1e-10)\n",
    "alpha = 0.5 * np.log((1 - eps) / eps)\n",
    "w     = w * np.exp(-alpha * y_pm * pred)\n",
    "w    /= w.sum()\n",
    "stumps.append(stump); alphas.append(alpha)\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — \"sample_weight\" error, or accuracy stuck near 0.5</summary>If the fit errors, you passed `sample_weight` positionally; use the keyword `stump.fit(X, y_pm, sample_weight=w)`. If accuracy is stuck near chance, you likely forgot to renormalize `w` (it must keep summing to 1) or you reused labels in `{0,1}` instead of `{-1,+1}`, which breaks the `y * pred` sign trick.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "ecc4e72c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:51.698734Z",
     "iopub.status.busy": "2026-06-10T18:50:51.698668Z",
     "iopub.status.idle": "2026-06-10T18:50:52.007351Z",
     "shell.execute_reply": "2026-06-10T18:50:52.006801Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 6.6 AdaBoost matches sklearn\n",
      "scratch AdaBoost (200 stumps) test accuracy 0.864\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines fit_adaboost / predict_adaboost; the check re-verifies them.\n",
    "def fit_adaboost(X, y_pm, T):\n",
    "    m = len(y_pm)\n",
    "    w = np.ones(m) / m\n",
    "    stumps, alphas = [], []\n",
    "    for t in range(T):\n",
    "        stump = DecisionTreeClassifier(max_depth=1, random_state=SEED)\n",
    "        stump.fit(X, y_pm, sample_weight=w)\n",
    "        pred = stump.predict(X)\n",
    "        eps = np.clip((w * (pred != y_pm)).sum() / w.sum(), 1e-10, 1 - 1e-10)\n",
    "        alpha = 0.5 * np.log((1 - eps) / eps)\n",
    "        w = w * np.exp(-alpha * y_pm * pred)\n",
    "        w = w / w.sum()\n",
    "        stumps.append(stump); alphas.append(alpha)\n",
    "    return stumps, np.array(alphas)\n",
    "\n",
    "def predict_adaboost(stumps, alphas, X):\n",
    "    score = sum(a * s.predict(X) for a, s in zip(alphas, stumps))\n",
    "    return np.where(np.sign(score) >= 0, 1, -1)\n",
    "\n",
    "_ = check(\"6.6 AdaBoost matches sklearn\", _ada_matches_sklearn, required=True)\n",
    "y_pm = np.where(y_train == 0, -1, 1)\n",
    "stumps, alphas = fit_adaboost(X_train, y_pm, N_ROUNDS)\n",
    "acc = (np.where(predict_adaboost(stumps, alphas, X_test) == -1, 0, 1) == y_test).mean()\n",
    "print(f\"scratch AdaBoost ({N_ROUNDS} stumps) test accuracy {acc:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa0677e2",
   "metadata": {},
   "source": [
    "> **Caveat:** AdaBoost is famously sensitive to label noise. A mislabeled example is wrong for every stump, so its weight grows round after round and the ensemble bends to fit it. Gradient boosting, next, is more robust because its leaf values are regularized and its loss is not exponential.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "353bd882",
   "metadata": {},
   "source": [
    "### Gradient boosting: fit a tree to the residuals\n",
    "\n",
    "Gradient boosting generalizes AdaBoost. Treat the ensemble output `F(x)` as a function and take a gradient-descent step in *function space*: at each round fit a weak learner to the negative gradient of the loss with respect to `F`, then add it scaled by a learning rate. For squared-error regression the negative gradient at example `i` is just the residual `y_i - F(x_i)`. So the algorithm is: predict the mean, fit a tree to the residuals, add a shrunken slice of it, recompute residuals, repeat.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9771b0ae",
   "metadata": {},
   "source": [
    "### Exercise 6.7 — Gradient boosting from scratch\n",
    "`Difficulty 3/5 · ~18 min`\n",
    "\n",
    "Implement `fit_gbm` (squared-loss regression: start at the mean, fit trees to residuals) and `predict_gbm`. Then check it against `GradientBoostingRegressor` on a synthetic regression. The two use slightly different split criteria, so compare test MSE within a tolerance, not exactly.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "00911711",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.008503Z",
     "iopub.status.busy": "2026-06-10T18:50:52.008401Z",
     "iopub.status.idle": "2026-06-10T18:50:52.013374Z",
     "shell.execute_reply": "2026-06-10T18:50:52.013029Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] 6.7 gradient boosting matches sklearn: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeRegressor\n",
    "\n",
    "def fit_gbm(X, y, n_estimators, learning_rate, max_depth):\n",
    "    \"\"\"Squared-loss gradient boosting. Returns (trees, initial_prediction).\"\"\"\n",
    "    y = y.astype(float)\n",
    "    initial = float(np.mean(y))\n",
    "    F = np.full(len(y), initial, dtype=float)   # start at the mean (the const that min. MSE)\n",
    "    trees = []\n",
    "    for _ in range(n_estimators):\n",
    "        # TODO 1: residuals = negative gradient of 0.5*(y-F)^2 w.r.t. F  ==  y - F\n",
    "        # TODO 2: fit a DecisionTreeRegressor(max_depth=max_depth) to (X, residuals)\n",
    "        # TODO 3: F += learning_rate * tree.predict(X);  trees.append(tree)\n",
    "        raise NotImplementedError  # replace this loop body per TODOs 1-3\n",
    "    return trees, initial\n",
    "\n",
    "def predict_gbm(trees, initial, learning_rate, X):\n",
    "    F = np.full(X.shape[0], initial, dtype=float)\n",
    "    # TODO 4: add learning_rate * tree.predict(X) for every tree\n",
    "    for tree in trees:\n",
    "        pass\n",
    "    return F\n",
    "\n",
    "def _gbm_matches_sklearn():\n",
    "    from sklearn.datasets import make_regression\n",
    "    from sklearn.ensemble import GradientBoostingRegressor\n",
    "    Xr, yr = make_regression(n_samples=500, n_features=5, noise=10, random_state=SEED)\n",
    "    Xr_tr, Xr_te, yr_tr, yr_te = train_test_split(Xr, yr, test_size=0.25, random_state=SEED)\n",
    "    trees, init = fit_gbm(Xr_tr, yr_tr, N_ROUNDS, 0.1, 3)\n",
    "    mse_mine = np.mean((predict_gbm(trees, init, 0.1, Xr_te) - yr_te) ** 2)\n",
    "    base_mse = np.mean((yr_te - yr_tr.mean()) ** 2)\n",
    "    assert mse_mine < 0.5 * base_mse, \\\n",
    "        f\"boosting should cut the mean-only MSE by >half; {mse_mine:.0f} vs baseline {base_mse:.0f}\"\n",
    "    sk = GradientBoostingRegressor(n_estimators=N_ROUNDS, learning_rate=0.1, max_depth=3,\n",
    "                                   random_state=SEED).fit(Xr_tr, yr_tr)\n",
    "    mse_sk = np.mean((sk.predict(Xr_te) - yr_te) ** 2)\n",
    "    assert abs(mse_mine - mse_sk) / mse_sk < 0.20, \\\n",
    "        f\"scratch GBM MSE {mse_mine:.0f} should be within 20% of sklearn {mse_sk:.0f}\"\n",
    "\n",
    "_ = check(\"6.7 gradient boosting matches sklearn\", _gbm_matches_sklearn)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "161851a5",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>The residual `y - F` *is* the negative gradient of squared loss, so each tree literally learns \"where am I wrong, and by how much\". `predict_gbm` replays the same additive sum: start at `initial`, add `learning_rate * tree.predict(X)` for each tree.</details>\n",
    "\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "# fit_gbm body\n",
    "residuals = y - F\n",
    "tree = DecisionTreeRegressor(max_depth=max_depth, random_state=SEED).fit(X, residuals)\n",
    "F = F + learning_rate * tree.predict(X)\n",
    "trees.append(tree)\n",
    "\n",
    "# predict_gbm body\n",
    "for tree in trees:\n",
    "    F = F + learning_rate * tree.predict(X)\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — my predictions are NaN or way off</summary>If you see NaNs, you let `F` inherit an int dtype; force float with `np.full(..., dtype=float)` and `y.astype(float)`. If MSE barely drops, confirm you add `learning_rate * tree.predict(X)` (not the raw residuals) and that you recompute `residuals = y - F` *every* round, not once.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "60c975be",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.014377Z",
     "iopub.status.busy": "2026-06-10T18:50:52.014286Z",
     "iopub.status.idle": "2026-06-10T18:50:52.224039Z",
     "shell.execute_reply": "2026-06-10T18:50:52.223695Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 6.7 gradient boosting matches sklearn\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines fit_gbm / predict_gbm; the check re-verifies them.\n",
    "def fit_gbm(X, y, n_estimators, learning_rate, max_depth):\n",
    "    y = y.astype(float)\n",
    "    initial = float(np.mean(y))\n",
    "    F = np.full(len(y), initial, dtype=float)\n",
    "    trees = []\n",
    "    for _ in range(n_estimators):\n",
    "        residuals = y - F\n",
    "        tree = DecisionTreeRegressor(max_depth=max_depth, random_state=SEED)\n",
    "        tree.fit(X, residuals)\n",
    "        F = F + learning_rate * tree.predict(X)\n",
    "        trees.append(tree)\n",
    "    return trees, initial\n",
    "\n",
    "def predict_gbm(trees, initial, learning_rate, X):\n",
    "    F = np.full(X.shape[0], initial, dtype=float)\n",
    "    for tree in trees:\n",
    "        F = F + learning_rate * tree.predict(X)\n",
    "    return F\n",
    "\n",
    "_ = check(\"6.7 gradient boosting matches sklearn\", _gbm_matches_sklearn, required=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "051c3509",
   "metadata": {},
   "source": [
    "Boosting's signature is the diminishing-returns curve: the first few trees do most of the work, and each later tree nudges the residual a little less. Plot the test error as trees accumulate.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "df72ec83",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.225089Z",
     "iopub.status.busy": "2026-06-10T18:50:52.224994Z",
     "iopub.status.idle": "2026-06-10T18:50:52.402296Z",
     "shell.execute_reply": "2026-06-10T18:50:52.401887Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeoAAAGGCAYAAAC0W8IbAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAXL5JREFUeJzt3XlcVOX+B/DPsA3rDDsDCogr4pYrkbtyRS+VpuWS5a5pWG6pl9vV1G5Z2nI1FStL7JdpektLcwkX8KaEipKmRi4opgyUyowKsj6/P8Y5MoIyo8AZ4fN+vc5rZp7nmXO+5wzMd55znnOOQgghQERERFbJRu4AiIiI6N6YqImIiKwYEzUREZEVY6ImIiKyYkzUREREVoyJmoiIyIoxURMREVkxJmoiIiIrxkRNRERkxZioqdopFArMmzdPeh0fHw+FQoHz58/LFtPdjDEdPnxY7lAskpiYCIVCgcTERLlDqVSPHj3Qo0cP6fX58+ehUCgQHx9fZct4mO3xoPE0aNAATz75ZLXGRnUbEzXVSnl5eZg3b16t+VJcsWJFlSY0Ikt99dVX+M9//iN3GHWSndwBUN3z4osvYujQoVAqldW2jLy8PMyfPx8ATHpxj6oVK1bA29sbo0aNMinv1q0b8vPz4eDgIE9gDyE4OBj5+fmwt7evsnk+zPaojnjKepQ/K8CQqH/99VdMnTpV7lDqHPaoqUKlpaW4detWtczb1tYWjo6OUCgU1TL/usTGxgaOjo6wsXn0/pUVCgUcHR1ha2tbZfN8mO1RHfGUZW2f1c2bN+UOAYDhRzXdn3X8xVC1SUxMRIcOHeDo6IhGjRrh448/xrx588olSYVCgcmTJ2Pt2rVo0aIFlEolduzYAQB477338MQTT8DLywtOTk5o3749/vvf/5ZbVkFBAaZNmwYfHx+4ubnh6aefxh9//FGu3b2OUW/fvh1du3aFi4sL3NzcEB0djRMnTpi0GTVqFFxdXXHp0iUMGDAArq6u8PHxwWuvvYaSkhIAhmONPj4+AID58+dDoVCUO05+L3l5eXjppZfg5eUFlUqFESNG4Nq1a+XarVixQtpOAQEBiImJQW5ubrl2GzduRPv27eHk5ARvb2+88MILuHTpkkkbrVaL0aNHo379+lAqlfD390f//v2l7dOgQQOcOHECSUlJ0roY9xJUdNyzR48eaNmyJU6ePImePXvC2dkZ9erVw6JFi8rFd+HCBTz99NNwcXGBr68vpk2bhp07d5abZ15eHn777Tf89ddflW5DAPjkk0/QqFEjODk5oVOnTvjf//5Xrk1Fx4SNn29mZiaefPJJuLq6ol69eli+fDkA4Pjx4+jVqxdcXFwQHByMr776ymSeD7M9Koqnss+mrJ9++gmdOnWCo6MjGjZsiC+++KLKYgPM/6wqYvyfP3nyJJ5//nl4eHigS5cuUv2XX34p/Z16enpi6NChuHjxokmcP/zwAy5cuCD9DTZo0ADAvf+f77e+qamp6NatG5ydnfHPf/5T2vbvvfee9LejVCrRsWNHHDp0yGS+lnwmtQV3fddiR48eRd++feHv74/58+ejpKQECxYskJLY3fbs2YMNGzZg8uTJ8Pb2lv4RlyxZgqeffhrDhw9HYWEh1q9fj+eeew5bt25FdHS09P5x48bhyy+/xPPPP48nnngCe/bsMam/n//7v//DyJEjERUVhXfffRd5eXmIi4tDly5dcPToUSkWACgpKUFUVBTCw8Px3nvvYdeuXXj//ffRqFEjTJo0CT4+PoiLi8OkSZPwzDPPYODAgQCA1q1bVxrH5MmT4e7ujnnz5iE9PR1xcXG4cOGC9KUDGL705s+fj8jISEyaNElqd+jQIezfv1/adRofH4/Ro0ejY8eOWLhwIbKzs7FkyRLs378fR48ehbu7OwBg0KBBOHHiBF555RU0aNAAOTk5SEhIQGZmJho0aID//Oc/eOWVV+Dq6orXX38dAODn53ff9bh27Rr69u2LgQMHYvDgwfjvf/+L2bNno1WrVujXrx8AQ4+qV69eyMrKwpQpU6DRaPDVV19h79695eZ38OBB9OzZE2+88UalP3g+++wzvPTSS3jiiScwdepUnDt3Dk8//TQ8PT0RGBhY6WdQUlKCfv36oVu3bli0aBHWrl2LyZMnw8XFBa+//jqGDx+OgQMHYuXKlRgxYgQiIiIQEhLy0NujIpV9NkZnzpzBs88+i7Fjx2LkyJH4/PPPMWrUKLRv3x4tWrR46Ngs+azu57nnnkOTJk3w9ttvw3iH47feegtz5szB4MGDMW7cOPz555/46KOP0K1bN+nv9PXXX4dOp8Mff/yBDz/8EADg6upq0bKNrly5gn79+mHo0KF44YUXTP6Wv/rqK1y/fh0vvfQSFAoFFi1ahIEDB+LcuXPS/5W5n0mtIqjWeuqpp4Szs7O4dOmSVHb69GlhZ2cn7v7oAQgbGxtx4sSJcvPJy8szeV1YWChatmwpevXqJZWlpaUJAOLll182afv8888LAOKNN96QylavXi0AiIyMDCGEENevXxfu7u5i/PjxJu/VarVCrVablI8cOVIAEAsWLDBp27ZtW9G+fXvp9Z9//lluufdjjKl9+/aisLBQKl+0aJEAIL777jshhBA5OTnCwcFB9OnTR5SUlEjtli1bJgCIzz//XNpGvr6+omXLliI/P19qt3XrVgFAzJ07VwghxLVr1wQAsXjx4vvG16JFC9G9e/dy5Xv37hUAxN69e6Wy7t27CwDiiy++kMoKCgqERqMRgwYNksref/99AUBs3rxZKsvPzxehoaHl5mlcTmXb07jejz32mCgoKJDKP/nkEwHAZB0yMjIEALF69WqpzPj5vv3221LZtWvXhJOTk1AoFGL9+vVS+W+//VYupofZHnfHY+5nExwcLACIffv2SWU5OTlCqVSKGTNmVElslnxWFXnjjTcEADFs2DCT8vPnzwtbW1vx1ltvmZQfP35c2NnZmZRHR0eL4ODgcvO++//ZnPVduXKlSVvjtvfy8hJXr16Vyr/77jsBQGzZskUIYf5nUttw13ctVVJSgl27dmHAgAEICAiQyhs3bnzPHkT37t0RFhZWrtzJyUl6fu3aNeh0OnTt2hVHjhyRyrdt2wYAePXVV03ea87Ak4SEBOTm5mLYsGH466+/pMnW1hbh4eEV9homTpxo8rpr1644d+5cpcuqzIQJE0wGE02aNAl2dnbS+u3atQuFhYWYOnWqybHG8ePHQ6VS4YcffgAAHD58GDk5OXj55Zfh6OgotYuOjkZoaKjUzsnJCQ4ODkhMTKxwF/uDcnV1xQsvvCC9dnBwQKdOnUy20Y4dO1CvXj08/fTTUpmjoyPGjx9fbn49evSAEKLS3rRxvSdOnGgyaGrUqFFQq9Vmxz9u3Djpubu7O5o1awYXFxcMHjxYKm/WrBnc3d3N+tzN2R53s+SzCQsLQ9euXaXXPj4+aNasWZXFZslndT93/998++23KC0txeDBg03+9zQaDZo0aWJxj90cSqUSo0ePrrBuyJAh8PDwkF4bt6lxW1TX/4u1Y6KupXJycpCfn4/GjRuXq6uoDMA9dx9u3boVjz/+OBwdHeHp6SntWtbpdFKbCxcuwMbGBo0aNTJ5b7NmzSqN9fTp0wCAXr16wcfHx2T68ccfkZOTY9Le0dGx3O57Dw+PKvnHbdKkiclrV1dX+Pv7S8e/Lly4AKD8ejk4OKBhw4ZS/b3aAUBoaKhUr1Qq8e6772L79u3w8/OTdvdqtdqHWo/69euXG4dw9za6cOECGjVqVK7dvf4+zGFcr7u3o729PRo2bGjWPCr6fNVqdYXrpFarzfrczdked7PkswkKCipXZu7fZE1+Vnf/j58+fRpCCDRp0qTc/96pU6fK/e9VhXr16t1z5Pvd29GYtI3borr+X6wdj1GTpGzP2eh///sfnn76aXTr1g0rVqyAv78/7O3tsXr16nIDeR5UaWkpAMNxao1GU67ezs70z7S6RuXKZerUqXjqqaewefNm7Ny5E3PmzMHChQuxZ88etG3b9oHmea9tJG4fl7Rm94r9YdbpQd9r7mcjR2wP4u7/8dLSUigUCmzfvr3COMw5Dn2vszeMgzsri6Esc7ZFdfy/WDv2qGspX19fODo64syZM+XqKiq7l2+++QaOjo7YuXMnxowZg379+iEyMrJcu+DgYJSWluLs2bMm5enp6ZUuw9gL9/X1RWRkZLnpQc6DftBTv4y9e6MbN24gKytLGqQSHBwMoPx6FRYWIiMjQ6q/VztjmbHeqFGjRpgxYwZ+/PFH/PrrrygsLMT777//0OtzP8HBwTh79my5hGDJ30dF8wTKb8eioiJkZGQ88HzlVNlnUxOq47MCDOsmhEBISEiF/3uPP/641PZef4PGXu/dZz0Y965UB2v4TGoSE3UtZWtri8jISGzevBmXL1+Wys+cOYPt27dbNB+FQmHy6/j8+fPYvHmzSTvjce+lS5ealJtzJaOoqCioVCq8/fbbKCoqKlf/559/mh2vkbOzM4DyXx6V+eSTT0xiiIuLQ3FxsbR+kZGRcHBwwNKlS02+ND/77DPodDpplHuHDh3g6+uLlStXoqCgQGq3fft2nDp1SmqXl5dX7nz1Ro0awc3NzeR9Li4uFq9LZaKionDp0iV8//33UtmtW7fw6aeflmtr7ulZHTp0gI+PD1auXInCwkKpPD4+vsrjr27mfjY1wZLPyhIDBw6Era0t5s+fX+5HgBACV65ckV67uLiYHO4yMv7Q3rdvn1RWUlKCTz755KFiq4g1fSY1ibu+a7F58+bhxx9/ROfOnTFp0iSUlJRg2bJlaNmyJdLS0syaR3R0ND744AP07dsXzz//PHJycrB8+XI0btwYx44dk9o99thjGDZsGFasWAGdTocnnngCu3fvNusXv0qlQlxcHF588UW0a9cOQ4cOhY+PDzIzM/HDDz+gc+fOWLZsmUXr7uTkhLCwMHz99ddo2rQpPD090bJlS7Rs2fK+7yssLETv3r0xePBgpKenY8WKFejSpYs0iMfHxwexsbGYP38++vbti6efflpq17FjR2lQkL29Pd59912MHj0a3bt3x7Bhw6TTsxo0aIBp06YBAH7//XdpeWFhYbCzs8OmTZuQnZ2NoUOHSnG1b98ecXFx+Pe//43GjRvD19cXvXr1smib3O2ll17CsmXLMGzYMEyZMgX+/v5Yu3atNPitbA/K3NOz7O3t8e9//xsvvfQSevXqhSFDhiAjIwOrV682+xi1tTD3s6kJlnxWlmjUqBH+/e9/IzY2FufPn8eAAQPg5uaGjIwMbNq0CRMmTMBrr70GwPA3+PXXX2P69Ono2LEjXF1d8dRTT6FFixZ4/PHHERsbi6tXr8LT0xPr169HcXFxla2/kTV9JjVKjqHmVHN2794t2rZtKxwcHESjRo3EqlWrxIwZM4Sjo6NJOwAiJiamwnl89tlnokmTJkKpVIrQ0FCxevVq6XSPsvLz88Wrr74qvLy8hIuLi3jqqafExYsXKz09y2jv3r0iKipKqNVq4ejoKBo1aiRGjRolDh8+LLUZOXKkcHFxKRdjRfEcOHBAtG/fXjg4OFR6apExpqSkJDFhwgTh4eEhXF1dxfDhw8WVK1fKtV+2bJkIDQ0V9vb2ws/PT0yaNElcu3atXLuvv/5atG3bViiVSuHp6SmGDx8u/vjjD6n+r7/+EjExMSI0NFS4uLgItVotwsPDxYYNG0zmo9VqRXR0tHBzczM5zelep8C0aNGiXCwjR44sd3rNuXPnRHR0tHBychI+Pj5ixowZ4ptvvhEAxM8//yy1M/f0LKMVK1aIkJAQoVQqRYcOHcS+fftE9+7dzTo9q6LP917rFBwcLKKjo8vF+SDb4+54zP1s7o6h7HLLrm9NfVYVMf5//PnnnxXWf/PNN6JLly7CxcVFuLi4iNDQUBETEyPS09OlNjdu3BDPP/+8cHd3FwBM4jt79qyIjIwUSqVS+Pn5iX/+858iISHB7PU1bvuKTrsq+3dn7mdS2yiEeARGl1CVGjBgAE6cOFHuOCIRYDhcMW3aNPzxxx+oV6+e3OHQffCzqhuYqGu5/Px8k1GWp0+fRosWLTBy5MiHPr5Fj767/z5u3bqFtm3boqSkBL///ruMkdHd+FnVXTxGXcs1bNgQo0aNks7xjYuLg4ODA2bNmiV3aGQFBg4ciKCgIDz22GPQ6XT48ssv8dtvv2Ht2rVyh0Z34WdVdzFR13J9+/bFunXroNVqoVQqERERgbfffrvcBSmoboqKisKqVauwdu1alJSUICwsDOvXr8eQIUPkDo3uws+q7uKubyIiIivG86iJiIisGBM1ERGRFeMxajOUlpbi8uXLcHNzq5ZLORIRUd0ihMD169cREBBgcie+ijBRm+Hy5ctm3fCeiIjIEhcvXkT9+vXv24aJ2gxubm4ADBtUpVLJHA0RET3q9Ho9AgMDpfxyP0zUZjDu7lapVEzURERUZcw5nCrrYLKSkhLMmTMHISEhcHJyQqNGjfDmm2+a3MVFCIG5c+fC398fTk5OiIyMLHfpy6tXr2L48OFQqVRwd3fH2LFjcePGDZM2x44dQ9euXeHo6IjAwEAsWrSoRtaRiIjoYciaqN99913ExcVh2bJlOHXqFN59910sWrQIH330kdRm0aJFWLp0KVauXImUlBS4uLggKirK5FZnw4cPx4kTJ5CQkICtW7di3759mDBhglSv1+vRp08fBAcHIzU1FYsXL8a8efOq5TZsREREVUq224EIIaKjo8WYMWNMygYOHCiGDx8uhBCitLRUaDQakzuq5ObmCqVSKdatWyeEEOLkyZMCgDh06JDUZvv27UKhUIhLly4JIQx38vHw8BAFBQVSm9mzZ4tmzZqZFadOpxMAhE6ne7AVJSIiKsOSvCJrj9p4z2LjBeV/+eUX/PTTT+jXrx8AICMjA1qtFpGRkdJ71Go1wsPDkZycDABITk6Gu7s7OnToILWJjIyEjY0NUlJSpDbdunWDg4OD1CYqKgrp6em4du1aubgKCgqg1+tNJiIiIjnIOpjsH//4B/R6PUJDQ2Fra4uSkhK89dZbGD58OABAq9UCAPz8/Eze5+fnJ9VptVr4+vqa1NvZ2cHT09OkTUhISLl5GOs8PDxM6hYuXIj58+dX0VoSERE9OFl71Bs2bMDatWvx1Vdf4ciRI1izZg3ee+89rFmzRs6wEBsbC51OJ00XL16UNR4iIqq7ZO1Rz5w5E//4xz8wdOhQAECrVq1w4cIFLFy4ECNHjoRGowEAZGdnw9/fX3pfdnY2HnvsMQCARqNBTk6OyXyLi4tx9epV6f0ajQbZ2dkmbYyvjW3KUiqVUCqVVbOSRERED0HWHnVeXl65S6fZ2tqitLQUABASEgKNRoPdu3dL9Xq9HikpKYiIiAAAREREIDc3F6mpqVKbPXv2oLS0FOHh4VKbffv2oaioSGqTkJCAZs2aldvtTUREZE1kTdRPPfUU3nrrLfzwww84f/48Nm3ahA8++ADPPPMMAMOJ4FOnTsW///1vfP/99zh+/DhGjBiBgIAADBgwAADQvHlz9O3bF+PHj8fBgwexf/9+TJ48GUOHDkVAQAAA4Pnnn4eDgwPGjh2LEydO4Ouvv8aSJUswffp0uVadiIjIPDUwCv2e9Hq9mDJliggKChKOjo6iYcOG4vXXXzc5jaq0tFTMmTNH+Pn5CaVSKXr37i3S09NN5nPlyhUxbNgw4erqKlQqlRg9erS4fv26SZtffvlFdOnSRSiVSlGvXj3xzjvvmB0nT88iIqKqZEleUQhR5jJgVCG9Xg+1Wg2dTvdQlxA9ng7ELAC8PYBvl1VhgERE9EixJK/wWt81yMYWuHAZyLtVeVsiIiJA5mPUdY2bi+FRf+P+7YiIiIyYqGuQ2tXwWFBomIiIiCrDRF2DXJ3vPL9+U744iIjo0cFEXYNsbe8kayZqIiIyBxN1DeNxaiIisgQTdQ1T3T5OzR41ERGZg4m6hhl71Dr2qImIyAxM1DVM6lEzURMRkRmYqGuYdIyau76JiMgMTNQ1jD1qIiKyBBN1DTMmao76JiIiczBR1zDV7V3fHPVNRETmYKKuYexRExGRJZioa5gbe9RERGQBJuoaxh41ERFZgom6hrkZEzV71EREZAYm6hrGHjUREVmCibqGGY9R38gDSkvljYWIiKwfE3UNU9/uUZeWAjfz5Y2FiIisHxN1DVM6APZ2hucc+U1ERJVhoq5hCgWPUxMRkfmYqGXAe1ITEZG5mKhlIN1Biz1qIiKqBBO1DLjrm4iIzMVELQP2qImIyFxM1DLgMWoiIjKXrIm6QYMGUCgU5aaYmBgAwK1btxATEwMvLy+4urpi0KBByM7ONplHZmYmoqOj4ezsDF9fX8ycORPFxcUmbRITE9GuXTsolUo0btwY8fHxNbWKFWKPmoiIzCVroj506BCysrKkKSEhAQDw3HPPAQCmTZuGLVu2YOPGjUhKSsLly5cxcOBA6f0lJSWIjo5GYWEhDhw4gDVr1iA+Ph5z586V2mRkZCA6Oho9e/ZEWloapk6dinHjxmHnzp01u7JlsEdNRERmE1ZkypQpolGjRqK0tFTk5uYKe3t7sXHjRqn+1KlTAoBITk4WQgixbds2YWNjI7RardQmLi5OqFQqUVBQIIQQYtasWaJFixYmyxkyZIiIiooyOy6dTicACJ1O9zCrJ/n8v0IE9RDi5XlVMjsiInrEWJJXrOYYdWFhIb788kuMGTMGCoUCqampKCoqQmRkpNQmNDQUQUFBSE5OBgAkJyejVatW8PPzk9pERUVBr9fjxIkTUpuy8zC2Mc6jIgUFBdDr9SZTVeKobyIiMpfVJOrNmzcjNzcXo0aNAgBotVo4ODjA3d3dpJ2fnx+0Wq3UpmySNtYb6+7XRq/XIz+/4ottL1y4EGq1WpoCAwMfdvVMuKsMj7nXq3S2RERUC1lNov7ss8/Qr18/BAQEyB0KYmNjodPppOnixYtVOn9jor5WtR11IiKqhezkDgAALly4gF27duHbb7+VyjQaDQoLC5Gbm2vSq87OzoZGo5HaHDx40GRexlHhZdvcPVI8OzsbKpUKTk5OFcajVCqhVCofer3uxcOYqHXVtggiIqolrKJHvXr1avj6+iI6Oloqa9++Pezt7bF7926pLD09HZmZmYiIiAAARERE4Pjx48jJyZHaJCQkQKVSISwsTGpTdh7GNsZ5yMGYqG/kAYVFsoVBRESPANkTdWlpKVavXo2RI0fCzu5OB1+tVmPs2LGYPn069u7di9TUVIwePRoRERF4/PHHAQB9+vRBWFgYXnzxRfzyyy/YuXMn/vWvfyEmJkbqEU+cOBHnzp3DrFmz8Ntvv2HFihXYsGEDpk2bJsv6AobBZDa3tzyPUxMR0f3Inqh37dqFzMxMjBkzplzdhx9+iCeffBKDBg1Ct27doNFoTHaP29raYuvWrbC1tUVERAReeOEFjBgxAgsWLJDahISE4IcffkBCQgLatGmD999/H6tWrUJUVFSNrF9FbG0B9e2R37nc/U1ERPehEEIIuYOwdnq9Hmq1GjqdDiqVqkrm2XMEcO4isOE/QHibKpklERE9IizJK7L3qOsqjvwmIiJzMFHLhCO/iYjIHEzUMvFgj5qIiMzARC0TD7XhkYmaiIjuh4laJtz1TURE5mCilokxUeeyR01ERPfBRC0TjvomIiJzMFHLRDpGzV3fRER0H0zUMvHkYDIiIjIDE7VMyt6TurRU3liIiMh6MVHLxDiYrLQUuH5T3liIiMh6MVHLxMEecLl9O2zu/iYionthopYRz6UmIqLKMFHLyHic+ioTNRER3QMTtYyMI7950RMiIroXJmoZ8aInRERUGSZqGfHGHEREVBkmahl58upkRERUCSZqGXlwMBkREVWCiVpGXu6Gx6u5ckZBRETWjIlaRsZj1FfYoyYiontgopaRsUfNY9RERHQvTNQy8nQ3PF7T88YcRERUMSZqGZW9MUfudXljISIi68RELSN7O0Dlanh+JVfWUIiIyEoxUcuMx6mJiOh+mKhlZrzoCXvURERUEdkT9aVLl/DCCy/Ay8sLTk5OaNWqFQ4fPizVCyEwd+5c+Pv7w8nJCZGRkTh9+rTJPK5evYrhw4dDpVLB3d0dY8eOxY0bN0zaHDt2DF27doWjoyMCAwOxaNGiGlm/yvBcaiIiuh9ZE/W1a9fQuXNn2NvbY/v27Th58iTef/99eHh4SG0WLVqEpUuXYuXKlUhJSYGLiwuioqJw69Ytqc3w4cNx4sQJJCQkYOvWrdi3bx8mTJgg1ev1evTp0wfBwcFITU3F4sWLMW/ePHzyySc1ur4V4bnURER0X0JGs2fPFl26dLlnfWlpqdBoNGLx4sVSWW5urlAqlWLdunVCCCFOnjwpAIhDhw5JbbZv3y4UCoW4dOmSEEKIFStWCA8PD1FQUGCy7GbNmpkVp06nEwCETqezaP3M8c4nQgT1EGLeR1U+ayIislKW5BVZe9Tff/89OnTogOeeew6+vr5o27YtPv30U6k+IyMDWq0WkZGRUplarUZ4eDiSk5MBAMnJyXB3d0eHDh2kNpGRkbCxsUFKSorUplu3bnBwcJDaREVFIT09HdeuXSsXV0FBAfR6vclUXXiMmoiI7kfWRH3u3DnExcWhSZMm2LlzJyZNmoRXX30Va9asAQBotVoAgJ+fn8n7/Pz8pDqtVgtfX1+Tejs7O3h6epq0qWgeZZdR1sKFC6FWq6UpMDCwCta2YtIxau76JiKiCsiaqEtLS9GuXTu8/fbbaNu2LSZMmIDx48dj5cqVcoaF2NhY6HQ6abp48WK1LcuDPWoiIroPWRO1v78/wsLCTMqaN2+OzMxMAIBGowEAZGdnm7TJzs6W6jQaDXJyckzqi4uLcfXqVZM2Fc2j7DLKUiqVUKlUJlN14XnURER0P7Im6s6dOyM9Pd2k7Pfff0dwcDAAICQkBBqNBrt375bq9Xo9UlJSEBERAQCIiIhAbm4uUlNTpTZ79uxBaWkpwsPDpTb79u1DUVGR1CYhIQHNmjUzGWEuB+Mx6qu5gBCyhkJERNaoBga33dPBgweFnZ2deOutt8Tp06fF2rVrhbOzs/jyyy+lNu+8845wd3cX3333nTh27Jjo37+/CAkJEfn5+VKbvn37irZt24qUlBTx008/iSZNmohhw4ZJ9bm5ucLPz0+8+OKL4tdffxXr168Xzs7O4uOPPzYrzuoc9Z2Xbxj1HdRDiOs3q3z2RERkhSzJK7ImaiGE2LJli2jZsqVQKpUiNDRUfPLJJyb1paWlYs6cOcLPz08olUrRu3dvkZ6ebtLmypUrYtiwYcLV1VWoVCoxevRocf36dZM2v/zyi+jSpYtQKpWiXr164p133jE7xupM1EII0TTKkKgvXKqW2RMRkZWxJK8ohOAO18ro9Xqo1WrodLpqOV79xFDgUjaweTnQNqzy9kRE9GizJK/IfglRKnOcmgPKiIjoLkzUVsDT3fDIRE1ERHdjorYCXsZzqctfJI2IiOo4Jmor4O1pePyTiZqIiO7CRG0FfIyJ+qq8cRARkfVhorYC3revufIXe9RERHQXJmor4MseNRER3QMTtRUw7vpmj5qIiO7GRG0FjLu+r+qAomJ5YyEiIuvCRG0FPFSA7e1P4mqurKEQEZGVYaK2AjY2gNftXnUOj1MTEVEZTNRWgqdoERFRRZiorQRP0SIiooowUVsJnqJFREQVYaK2EuxRExFRRZiorQSPURMRUUWYqK2EsUfNG3MQEVFZTNRWgj1qIiKqCBO1lWCiJiKiijBRWwmf27u+9TeAgkJ5YyEiIuvBRG0l1G6AvZ3hOUd+ExGRERO1lVAoeIoWERGVx0RtRYzHqXOuyBsHERFZDyZqK+LnZXhkoiYiIiMmaivieztRZzNRExHRbUzUVoQ9aiIiuhsTtRXx9TY8skdNRERGsibqefPmQaFQmEyhoaFS/a1btxATEwMvLy+4urpi0KBByM7ONplHZmYmoqOj4ezsDF9fX8ycORPFxcUmbRITE9GuXTsolUo0btwY8fHxNbF6FvPjrm8iIrqL7D3qFi1aICsrS5p++uknqW7atGnYsmULNm7ciKSkJFy+fBkDBw6U6ktKShAdHY3CwkIcOHAAa9asQXx8PObOnSu1ycjIQHR0NHr27Im0tDRMnToV48aNw86dO2t0Pc1hPEad85e8cRARkRURMnrjjTdEmzZtKqzLzc0V9vb2YuPGjVLZqVOnBACRnJwshBBi27ZtwsbGRmi1WqlNXFycUKlUoqCgQAghxKxZs0SLFi1M5j1kyBARFRVldpw6nU4AEDqdzuz3PIicK0IE9RAiuKcQRcXVuigiIpKRJXlF9h716dOnERAQgIYNG2L48OHIzMwEAKSmpqKoqAiRkZFS29DQUAQFBSE5ORkAkJycjFatWsHPz09qExUVBb1ejxMnTkhtys7D2MY4j4oUFBRAr9ebTDXByx2wtQGEAP7iNb+JiAgy7/oODw9HfHw8duzYgbi4OGRkZKBr1664fv06tFotHBwc4O7ubvIePz8/aLVaAIBWqzVJ0sZ6Y9392uj1euTn51cY18KFC6FWq6UpMDCwKla3UjY2PEWLiIhM2cm58H79+knPW7dujfDwcAQHB2PDhg1wcnKSLa7Y2FhMnz5deq3X62ssWft5AVl/8hQtIiIykH3Xd1nu7u5o2rQpzpw5A41Gg8LCQuTm5pq0yc7OhkajAQBoNJpyo8CNrytro1Kp7vljQKlUQqVSmUw1hT1qIiIqy+xEvWjRIpNdxfv370dBQYH0+vr163j55ZcfKpgbN27g7Nmz8Pf3R/v27WFvb4/du3dL9enp6cjMzERERAQAICIiAsePH0dOTo7UJiEhASqVCmFhYVKbsvMwtjHOw9pIiZojv4mICDB/1LeNjY3Izs6WXru5uYmzZ89Kr7VarbCxsbFo1NuMGTNEYmKiyMjIEPv37xeRkZHC29tb5OTkCCGEmDhxoggKChJ79uwRhw8fFhERESIiIkJ6f3FxsWjZsqXo06ePSEtLEzt27BA+Pj4iNjZWanPu3Dnh7OwsZs6cKU6dOiWWL18ubG1txY4dO8yOs6ZGfQshxJIvDCO/Zy6q9kUREZFMLMkrZh+jFkLc9/WD+OOPPzBs2DBcuXIFPj4+6NKlC37++Wf4+PgAAD788EPY2Nhg0KBBKCgoQFRUFFasWCG939bWFlu3bsWkSZMQEREBFxcXjBw5EgsWLJDahISE4IcffsC0adOwZMkS1K9fH6tWrUJUVNRDx18deBlRIiIqSyHMzLg2NjbQarXw9fUFALi5ueGXX35Bw4YNARiO+wYEBKCkpKT6opWJXq+HWq2GTqer9uPViQeBkbOBsMbA9k+rdVFERCQTS/KKVQ0mI8CX96QmIqIyLDo9a9WqVXB1dQUAFBcXIz4+Ht7ehjtJXL9+veqjq4P8bt+Y469rQFExYC/rCXRERCQ3s3d9N2jQAAqFotJ2GRkZDx2UtanJXd+lpUDTKEOSTv4aCPCt1sUREZEMLMkrZvfXzp8//7BxkRlsbACND3AxC7iUzURNRFTX8Ri1FQo0XKsFF7PkjYOIiORndqJOTk7G1q1bTcq++OILhISEwNfXFxMmTDC5AAo9uEB/wyMTNRERmZ2oFyxYIN2RCgCOHz+OsWPHIjIyEv/4xz+wZcsWLFy4sFqCrGukRK2VNw4iIpKf2Yk6LS0NvXv3ll6vX78e4eHh+PTTTzF9+nQsXboUGzZsqJYg6xru+iYiIiOzE/W1a9dMbheZlJRkcverjh074uLFi1UbXR3FHjURERmZnaj9/PykU68KCwtx5MgRPP7441L99evXYW9vX/UR1kHGRJ31p+E0LSIiqrvMTtR///vf8Y9//AP/+9//EBsbC2dnZ3Tt2lWqP3bsGBo1alQtQdY1Ph6Ao9JwTvXl7MrbExFR7WV2on7zzTdhZ2eH7t2749NPP8Wnn34KBwcHqf7zzz9Hnz59qiXIukahAOobj1Nz9zcRUZ1m9gVPvL29sW/fPuh0Ori6usLW1takfuPGjdLlRenhBWqAMxeAzMsA2ssdDRERycXiK0mr1eoKyz09PR86GLqDA8qIiAiwIFGPGTPGrHaff/75AwdDd/CiJ0REBFiQqOPj4xEcHIy2bdvCzPt40EPgudRERARYkKgnTZqEdevWISMjA6NHj8YLL7zA3d3ViLu+iYgIsGDU9/Lly5GVlYVZs2Zhy5YtCAwMxODBg7Fz5072sKtB0O1EfSUXuH5T1lCIiEhGFt09S6lUYtiwYUhISMDJkyfRokULvPzyy2jQoAFu3LhRXTHWSSpXwNvD8Pz8JXljISIi+TzwbS5tbGygUCgghEBJSUlVxkS3hdQ3PGb8IW8cREQkH4sSdUFBAdatW4e//e1vaNq0KY4fP45ly5YhMzOT51BXgwb1DI9M1EREdZfZg8lefvllrF+/HoGBgRgzZgzWrVsHb2/v6oytzmsYaHhkoiYiqrvMTtQrV65EUFAQGjZsiKSkJCQlJVXY7ttvv62y4Oo6Y4/6PBM1EVGdZXaiHjFiBBQKRXXGQncx9qjPXgSEMFwDnIiI6haLLnhCNSs4wPCovwFc0wOeFV+9lYiIarEHHvVN1c/JEQjwNTzncWoiorqJidrKSadoXZQ3DiIikofVJOp33nkHCoUCU6dOlcpu3bqFmJgYeHl5wdXVFYMGDUJ2drbJ+zIzMxEdHQ1nZ2f4+vpi5syZKC4uNmmTmJiIdu3aQalUonHjxo/UbnwpUfOiJ0REdZJVJOpDhw7h448/RuvWrU3Kp02bhi1btmDjxo1ISkrC5cuXMXDgQKm+pKQE0dHRKCwsxIEDB7BmzRrEx8dj7ty5UpuMjAxER0ejZ8+eSEtLw9SpUzFu3Djs3LmzxtbvYbBHTURUxwkLJSUliaKionLlRUVFIikpydLZievXr4smTZqIhIQE0b17dzFlyhQhhBC5ubnC3t5ebNy4UWp76tQpAUAkJycLIYTYtm2bsLGxEVqtVmoTFxcnVCqVKCgoEEIIMWvWLNGiRQuTZQ4ZMkRERUWZHaNOpxMAhE6ns3j9HtauA0IE9RCi79gaXzQREVUTS/KKxT3qnj174urVq+XKdTodevbsafEPhZiYGERHRyMyMtKkPDU1FUVFRSbloaGhCAoKQnJyMgAgOTkZrVq1gp+fn9QmKioKer0eJ06ckNrcPe+oqChpHtbOeIrWuT+A0lJ5YyEioppn9ulZRkKICs+nvnLlClxcXCya1/r163HkyBEcOnSoXJ1Wq4WDgwPc3d1Nyv38/KDVaqU2ZZO0sd5Yd782er0e+fn5cHJyKrfsgoICFBQUSK/1er1F61WVAv0BB3vgVgHwR/adu2oREVHdYHaiNh4bVigUGDVqFJRKpVRXUlKCY8eO4YknnjB7wRcvXsSUKVOQkJAAR0dHC0KufgsXLsT8+fPlDgMAYGdrOE6dngGcucBETURU15i961utVkOtVkMIATc3N+m1Wq2GRqPBhAkT8OWXX5q94NTUVOTk5KBdu3aws7ODnZ0dkpKSsHTpUtjZ2cHPzw+FhYXIzc01eV92djY0Gg0AQKPRlBsFbnxdWRuVSlVhbxoAYmNjodPppOniRXlHcjUOMjyeyZQ1DCIikoHZPerVq1cDABo0aIDXXnvN4t3cd+vduzeOHz9uUjZ69GiEhoZi9uzZCAwMhL29PXbv3o1BgwYBANLT05GZmYmIiAgAQEREBN566y3k5OTA19dwZZCEhASoVCqEhYVJbbZt22aynISEBGkeFVEqlSZ7DOTWONjweOaCvHEQEVHNs/gY9axZsyCEkF5fuHABmzZtQlhYGPr06WP2fNzc3NCyZUuTMhcXF3h5eUnlY8eOxfTp0+Hp6QmVSoVXXnkFERERePzxxwEAffr0QVhYGF588UUsWrQIWq0W//rXvxATEyMl2okTJ2LZsmWYNWsWxowZgz179mDDhg344YcfLF112TBRExHVXRaP+u7fvz+++OILAEBubi46deqE999/H/3790dcXFyVBvfhhx/iySefxKBBg9CtWzdoNBqTu3PZ2tpi69atsLW1RUREBF544QWMGDECCxYskNqEhITghx9+QEJCAtq0aYP3338fq1atQlRUVJXGWp3K7vou8xuJiIjqAIUQln31e3t7IykpCS1atMCqVavw0Ucf4ejRo/jmm28wd+5cnDp1qrpilY1er4darYZOp4NKparx5d8qAEL7GZL04W8AH88aD4GIiKqQJXnF4h51Xl4e3NzcAAA//vgjBg4cCBsbGzz++OO4cIH7ZquDo9JwmhbAAWVERHWNxYm6cePG2Lx5My5evIidO3dKx6VzcnJk6W3WFdLub/4WIiKqUyxO1HPnzsVrr72GBg0aoFOnTtLo6R9//BFt27at8gDJgAPKiIjqJotHfT/77LPo0qULsrKy0KZNG6m8d+/eeOaZZ6o0OLqjaQPD46+nZQ2DiIhq2APdPUuj0cDNzQ0JCQnIz88HAHTs2BGhoaFVGhzdEX77xmJpp4Cb+fLGQkRENcfiRH3lyhX07t0bTZs2xd///ndkZWUBMJzzPGPGjCoPkAyCAgwDyopLgJRf5I6GiIhqisWJetq0abC3t0dmZiacnZ2l8iFDhmDHjh1VGhyZ6tzO8Lj/iLxxEBFRzbE4Uf/444949913Ub9+fZPyJk2a8PSsamZM1AeYqImI6gyLE/XNmzdNetJGV69etarrY9dGT9weVH/yLHAlV9ZQiIiohlicqLt27SpdQhQw3PaytLQUixYtQs+ePas0ODLl7QGENjQ8Z6+aiKhusPj0rEWLFqF37944fPgwCgsLMWvWLJw4cQJXr17F/v37qyNGKqNzO+C3c0DKMeCpXnJHQ0RE1c3iHnXLli3x+++/o0uXLujfvz9u3ryJgQMH4ujRo2jUqFF1xEhlPNbc8Pjr7/LGQURENcPiHnVmZiYCAwPx+uuvV1gXFBRUJYFRxVo1NTyePGs4VcvOVt54iIioelncow4JCcGff/5ZrvzKlSsICQmpkqDo3oIDAFdnoKAQOH1e7miIiKi6WZyohRBQKBTlym/cuAFHR8cqCYruzcYGaNnE8Pw4d38TEdV6Zu/6nj59OgDDKO85c+aYnKJVUlKClJQUPPbYY1UeIJXXsinw8y+GRD24n9zREBFRdTI7UR89ehSAoUd9/PhxODg4SHUODg5o06YNXnvttaqPkMoxHqdmj5qIqPYzO1Hv3bsXADB69GgsWbKE956WkTFRn+KAMiKiWs/iY9SrV69mkpZZSH3DgLJbBbw/NRFRbfdAt7kkednYAC0aG55z9zcRUe3GRP2Ian371t9pp+SNg4iIqhcT9SOqbZjh8SgTNRFRrcZE/Yhqe/tSor+dBfJvyRsLERFVHybqR5S/D+DnDZSU8jg1EVFtxkT9iFIo7tyg48hJeWMhIqLqw0T9CGt3O1FzQBkRUe3FRP0IM/aoj7JHTURUa8maqOPi4tC6dWuoVCqoVCpERERg+/btUv2tW7cQExMDLy8vuLq6YtCgQcjOzjaZR2ZmJqKjo+Hs7AxfX1/MnDkTxcXFJm0SExPRrl07KJVKNG7cGPHx8TWxetWudTPDOdXav4Cs8jc0IyKiWkDWRF2/fn288847SE1NxeHDh9GrVy/0798fJ06cAABMmzYNW7ZswcaNG5GUlITLly9j4MCB0vtLSkoQHR2NwsJCHDhwAGvWrEF8fDzmzp0rtcnIyEB0dDR69uyJtLQ0TJ06FePGjcPOnTtrfH2rmrMT0LyR4fnBY/LGQkRE1URYGQ8PD7Fq1SqRm5sr7O3txcaNG6W6U6dOCQAiOTlZCCHEtm3bhI2NjdBqtVKbuLg4oVKpREFBgRBCiFmzZokWLVqYLGPIkCEiKirK7Jh0Op0AIHQ63cOsWrV4K06IoB5CTF8odyRERGQuS/KK1RyjLikpwfr163Hz5k1EREQgNTUVRUVFiIyMlNqEhoYiKCgIycnJAIDk5GS0atUKfn5+UpuoqCjo9XqpV56cnGwyD2Mb4zwedd06Gh6TDgFCyBsLERFVPbPvnlVdjh8/joiICNy6dQuurq7YtGkTwsLCkJaWBgcHB7i7u5u09/Pzg1arBQBotVqTJG2sN9bdr41er0d+fj6cnJzKxVRQUICCggLptV6vf+j1rC4dWwFOjsCfVw130wprLHdERERUlWTvUTdr1gxpaWlISUnBpEmTMHLkSJw8Ke8w5oULF0KtVktTYGCgrPHcj9IBeLyN4fm+Q/LGQkREVU/2RO3g4IDGjRujffv2WLhwIdq0aYMlS5ZAo9GgsLAQubm5Ju2zs7Oh0WgAABqNptwocOPrytqoVKoKe9MAEBsbC51OJ00XL16silWtNt3L7P4mIqLaRfZEfbfS0lIUFBSgffv2sLe3x+7du6W69PR0ZGZmIiIiAgAQERGB48ePIycnR2qTkJAAlUqFsLAwqU3ZeRjbGOdREaVSKZ0yZpysmfE49eFfges35Y2FiIiqlqyJOjY2Fvv27cP58+dx/PhxxMbGIjExEcOHD4darcbYsWMxffp07N27F6mpqRg9ejQiIiLw+OOPAwD69OmDsLAwvPjii/jll1+wc+dO/Otf/0JMTAyUSiUAYOLEiTh37hxmzZqF3377DStWrMCGDRswbdo0OVe9SjUMBBoFAYVFwH8f/bPOiIiorBoYhX5PY8aMEcHBwcLBwUH4+PiI3r17ix9//FGqz8/PFy+//LLw8PAQzs7O4plnnhFZWVkm8zh//rzo16+fcHJyEt7e3mLGjBmiqKjIpM3evXvFY489JhwcHETDhg3F6tWrLYrTmk/PMlqzyXCaVrfhQpSUyB0NERHdjyV5RSEET+qpjF6vh1qthk6ns9rd4DfzgccHA/obwOdvA73vvWefiIhkZklesbpj1PRgXJyAodGG55/9V95YiIio6jBR1yIjBxiu/b3/CJCeIXc0RERUFZioa5H6GiCqi+H56m/kjYWIiKoGE3UtM/ZZw+O3CcBVnbyxEBHRw2OirmU6tARaNgEKCoF1W+WOhoiIHhYTdS2jUABjBhmeb9h+/7ZERGT9mKhroT5dADtb4Pwl4MIluaMhIqKHwURdC7m5AO1bGJ7vOyxvLERE9HCYqGsp4/W/eUctIqJHGxN1LWVM1AeOAkXF8sZCREQPjom6lmrZBPBUAzfygKPy3t6biIgeAhN1LWVjA3Rpb3jO+1QTET26mKhrsZ7hhsetewHeeoWI6NHERF2LRXUFnB0Np2kd/lXuaIiI6EEwUddiLk5AdA/D8407ZA2FiIgeEBN1LfdcX8Pj1r1AXr68sRARkeWYqGu5Tq2B4ADgZj6wLUnuaIiIyFJM1LWcQgEM+bvh+acbOaiMiOhRw0RdB7zwtOF49W/ngD0/yx0NERFZgom6DlC7GZI1ACxfy141EdGjhIm6jhj7HKC0B1JPAAePyR0NERGZi4m6jvDzAp69PQJ8+VfyxkJEROZjoq5DXhpiuLRo0kHg19NyR0NEROZgoq5DgusBT/YwPI9jr5qI6JHARF3HvPy84fGHJOBspryxEBFR5Zio65jmjYDIJwwjv5d8IXc0RERUGSbqOmjaKMPj93uA9AxZQyEiokrImqgXLlyIjh07ws3NDb6+vhgwYADS09NN2ty6dQsxMTHw8vKCq6srBg0ahOzsbJM2mZmZiI6OhrOzM3x9fTFz5kwUFxebtElMTES7du2gVCrRuHFjxMfHV/fqWa2WTYC/dzf0qj9YLXc0RER0P7Im6qSkJMTExODnn39GQkICioqK0KdPH9y8eVNqM23aNGzZsgUbN25EUlISLl++jIEDB0r1JSUliI6ORmFhIQ4cOIA1a9YgPj4ec+fOldpkZGQgOjoaPXv2RFpaGqZOnYpx48Zh586dNbq+1mTaKMPlRXf8D/g5Te5oiIjonoQVycnJEQBEUlKSEEKI3NxcYW9vLzZu3Ci1OXXqlAAgkpOThRBCbNu2TdjY2AitViu1iYuLEyqVShQUFAghhJg1a5Zo0aKFybKGDBkioqKizIpLp9MJAEKn0z3U+lmb2PeFCOohRK+RQhQUyh0NEVHdYUlesapj1DqdDgDg6ekJAEhNTUVRUREiIyOlNqGhoQgKCkJycjIAIDk5Ga1atYKfn5/UJioqCnq9HidOnJDalJ2HsY1xHnXV7PGAtwdw5gLw8ddyR0NERBWxmkRdWlqKqVOnonPnzmjZsiUAQKvVwsHBAe7u7iZt/fz8oNVqpTZlk7Sx3lh3vzZ6vR75+eVv0lxQUAC9Xm8y1UZqN+BfkwzP/xMPHDgqazhERFQBq0nUMTEx+PXXX7F+/Xq5Q8HChQuhVqulKTAwUO6Qqs2ASODpXkBxCfDSXJ5bTURkbawiUU+ePBlbt27F3r17Ub9+falco9GgsLAQubm5Ju2zs7Oh0WikNnePAje+rqyNSqWCk5NTuXhiY2Oh0+mk6eLFiw+9jtZKoQAWzwbahgH6G8CEuUD+LbmjIiIiI1kTtRACkydPxqZNm7Bnzx6EhISY1Ldv3x729vbYvXu3VJaeno7MzExEREQAACIiInD8+HHk5ORIbRISEqBSqRAWFia1KTsPYxvjPO6mVCqhUqlMptrM0QFY9W/A18twvPrtj+WOiIiIJNU/tu3eJk2aJNRqtUhMTBRZWVnSlJeXJ7WZOHGiCAoKEnv27BGHDx8WERERIiIiQqovLi4WLVu2FH369BFpaWlix44dwsfHR8TGxkptzp07J5ydncXMmTPFqVOnxPLly4Wtra3YsWOHWXHW1lHfd0s6aBgFHtRDiF0H5I6GiKj2siSvyJqoAVQ4rV69WmqTn58vXn75ZeHh4SGcnZ3FM888I7Kyskzmc/78edGvXz/h5OQkvL29xYwZM0RRUZFJm71794rHHntMODg4iIYNG5osozJ1JVELIcT8ZYZE3elZIfQ35I6GiKh2siSvKIQQQq7e/KNCr9dDrVZDp9PV+t3gtwqAPmOAC5eBEQOAN6fIHRERUe1jSV6xisFkZD0clcDb0w3P/+87IPGgvPEQEdV1TNRUTpf2wOB+hmuBj/0n8NUWuSMiIqq7mKipQm9OBfr3NpxfHfsBsPZ7uSMiIqqbmKipQo4OwJLXgZjnDa/nLgVSfpE3JiKiuoiJmu5JoQBmjrtz5bKJb/DKZURENY2Jmu5LoQAWzQRaNQWu6oBhM4Dzl+SOioio7mCipko5OQJr3gWaNgCy/wIGTwWOpcsdFRFR3cBETWbxcge+ev9Osn72VWDDdsPIcCIiqj5M1GQ2H0/g22VA7wigoBCYuQgYHQto/5Q7MiKi2ouJmizi5mK4gcescYCDPbA3BXh+BnAjT+7IiIhqJyZqspiNDRAzHNj2KaDxBs5eBGYv5m5wIqLqwERND6xJMLD8DcDOFtiaCLy5wrBLnIiIqg4TNT2UDi2Bf00yPP/sv8DTk4BzF+WNiYioNmGipoc2ehDw8QLDyPDfzgEDJwNHT8odFRFR7cBETVWib1dg52dAm1Dgmh4YOh3YnSx3VEREjz4maqoyPp7Aug+A7p0M97Ue/y/g621yR0VE9GhjoqYq5eIEfPYW8GwUUFIKzFoMfPQlR4QTET0oJmqqcvZ2wHuz79x5673PDBdH4YVRiIgsx0RN1UKhAGaNB+a/ani+cQfQdTjwwWr2romILMFETdVq1DOGa4R3ag0UFgFLvgCWrZU7KiKiRwcTNVW7J9oCG5cAc2MMr9/7DFj9jbwxERE9KpioqcaMffbOcet5y4B//QcoKpY1JCIiq8dETTVq5jhg5ljD8//7DngmBkjPkDcmIiJrxkRNNUqhACa/AHz6JqByBY7/DkRPACYvAP53mAPNiIjuxkRNsujTBdgVD/ztCcPu7y17gRdmAv3GAZt3AcUlckdIRGQdFEKwD1MZvV4PtVoNnU4HlUoldzi1zvHfDVcw+2YnkHfLUFZfA0wYDAzuBzg5yhsfEVFVsySvMFGbgYm6ZuTqgS++M4wIv6ozlHm5Ay8NNZzmpXSQNTwioipjSV6Rddf3vn378NRTTyEgIAAKhQKbN282qRdCYO7cufD394eTkxMiIyNx+vRpkzZXr17F8OHDoVKp4O7ujrFjx+LGjRsmbY4dO4auXbvC0dERgYGBWLRoUXWvGj0AdxXw6ovAgfXAglcNveorucDbK4E+YwyDz7L/kjtKIqKaJWuivnnzJtq0aYPly5dXWL9o0SIsXboUK1euREpKClxcXBAVFYVbt25JbYYPH44TJ04gISEBW7duxb59+zBhwgSpXq/Xo0+fPggODkZqaioWL16MefPm4ZNPPqn29aMH4+QIjHwGSPoSWDzLcLOP85cMp3N1eg4YHQv8lMqBZ0RURwgrAUBs2rRJel1aWio0Go1YvHixVJabmyuUSqVYt26dEEKIkydPCgDi0KFDUpvt27cLhUIhLl26JIQQYsWKFcLDw0MUFBRIbWbPni2aNWtmdmw6nU4AEDqd7kFXjx7C9ZtCxK0Tov8kIYJ63Jn6jBFi3VYhbuTJHSERkWUsyStWO+o7IyMDWq0WkZGRUplarUZ4eDiSkw03Ok5OToa7uzs6dOggtYmMjISNjQ1SUlKkNt26dYODw50DnFFRUUhPT8e1a9dqaG3oYbg6AxOHAptXAHu/AEYMMPS6fzsHzH4P6DDQcJeu85fkjpSIqOpZbaLWarUAAD8/P5NyPz8/qU6r1cLX19ek3s7ODp6eniZtKppH2WXcraCgAHq93mQi69AwEHhzCpCyAfjnRMPrvFuGUeO9RgAT3zCMHr+SK3ekRERVw07uAKzRwoULMX/+fLnDoPtQuwEvDTGcwnX4V2D5WmBvCrB9n2FSKIDHQoH2LYEWTYDO7QA/L7mjJiKynNUmao1GAwDIzs6Gv7+/VJ6dnY3HHntMapOTk2PyvuLiYly9elV6v0ajQXZ2tkkb42tjm7vFxsZi+vTp0mu9Xo/AwMCHWyGqFgoF0LEVEP8O8OtpYMf/gD0/AydOA0dPGSaj1s2AAZFA/96At4d8MRMRWcJqd32HhIRAo9Fg9+7dUpler0dKSgoiIiIAABEREcjNzUVqaqrUZs+ePSgtLUV4eLjUZt++fSgqKpLaJCQkoFmzZvDwqPjbWqlUQqVSmUxk/Vo2AV4bA2z7xLBr/L3ZhvOvWzcz1B9LBxYsBzo+CwyIAZasAdJOAaWl8sZNRHQ/sl7w5MaNGzhz5gwAoG3btvjggw/Qs2dPeHp6IigoCO+++y7eeecdrFmzBiEhIZgzZw6OHTuGkydPwtHRcLmqfv36ITs7GytXrkRRURFGjx6NDh064KuvvgIA6HQ6NGvWDH369MHs2bPx66+/YsyYMfjwww9NTuO6H17w5NH351VgWxLwzY/AL7+Z1nmqgW4dgR6dDI9e7rKESER1yCNzZbLExET07NmzXPnIkSMRHx8PIQTeeOMNfPLJJ8jNzUWXLl2wYsUKNG3aVGp79epVTJ48GVu2bIGNjQ0GDRqEpUuXwtXVVWpz7NgxxMTE4NChQ/D29sYrr7yC2bNnmx0nE3XtkvUnkHgQSEwB9h8Brt80ra+vAZoEA22bA+1aAI81B9xc5ImViGqnRyZRPyqYqGuvomLgyIk7ifvk2fJtFAogNMQwMK1pA8BDDQRqgGYhgLNTjYdMRLUAE3UVY6KuO67qgNPngVPngKMngdQTwMWsitsqFEBwABDaEGjeyJC4Q+oDQf5M4ER0f0zUVYyJum7LvmLodR85CWReBq7pgHN/GI5734uPBxAUYEjkdz96exiSPBHVXUzUVYyJmiry1zXD1dGMU3oGcOEyoLt+//c5O95J2oH+QH0/oJ4fUE9jeK5yZSInqu2YqKsYEzVZQnfd0PO+cHvKzLr9eBm4nFP5zURcnQ3HwBvUBxrUM0waH8PNSXw8AS81YGtbM+tCRNWDibqKMVFTVSkoBC5l30ncmVmG18bJnEuf2tgYkrW3x53kbZzKlvl6Gq7gxt45kfWxJK9Y7ZXJiGojpYPh+uQN73Ghu/xbwKUc4MIlw5Rx+zHnimFX+1+5hgu0/HnNMJ06d//l2dvdSdq+Xrefe5V/7eYCOCnZUyeyRkzURFbEyRFoHGSYKlJcYhiZ/udV4K+rhsc/r91+vOu57rrh9LPLOYbJHEp7wMnJkLSdHQFHR8Ojk9KQzD3UgLsK8FABLk6GHx5KJeDoYHjuYA/Y2QH2toZHOzvDjwU7W8DeHnB3M7QjIvMxURM9Quxsb/eGPStvW1Bo6IXnXDEk7pyrhuc5VwwJ3fj8r2uGHwAAUFBkmHKrcR3UbobE7+BgSOJKe0MSd7g92dvfLrMrk/hvJ30He8N7nRxvT0rDqXBly5wd78zHwc7waG93+0cFfyTQI4iJmqiWUjrcHk3ud/92QhiSet4tIC8fyC8w7ILPv2UoMz7X3zScmnZND+TqDWUFhXdNRYZefPHtqajk9uPtqbTU0NOvbGR8dbG3A1ycAVcnw96Cson87sRuZ2s4FHD3o62NYZyA8bWNjaHM1gawsTXsTbAv80PD4a4fIcbndrfnaWd3Z/52xuXeXkbZ5dnaGH7c2Npw3EFdw0RNVMcpFICj0jB5qqtvOUIYEvyf1wxJvbDIMBUV3XleeDvRFxYChcWG12UTfVGR6Q+J/ALDj4lbxh8VBUB+vuG9RUV33mdUVGyIIfcRvsW8QmH4QWHcI2H8AWBbyS2WFArARgHg9qNCYfgBoACgsLlTX/bRkraK2/U2d9Ur7mpnczvOe7a7ez5l2pUts1EY1vnuQyzG58YfObY2hthsb7/H5nZ52R9Gxj03draGZaNsTGW2n3FydwP8vKvl460QEzUR1QiFwnCM26MafwxURAhDwr9VCNzMA27kGR6NPxaKiu88FhXdSfIlpUDJ7T0CxSW3n99+LCk1TKWld16X3i4rLr5rvrd/gBTc9aPk7nkal1N2GfdaH+MhCpLH0Gjg3ddqbnlM1ERUqykUd45Pq10rb29NjMnf+KPB+EOisPDOD4uC24n/fifaGuuEMMxT3J63EHemUnGnHmXqS8vUw8z3CwCigvcLcbscFc+nbBluv+de7Yw/jqRDLSV3nhsPuZTeblMqDNuv3Pa8670lJWXW6a7thjJlLjV8iWAmaiIiK2Vz+/i0vbGA15Cvkyo5qkFERERyYqImIiKyYkzUREREVoyJmoiIyIoxURMREVkxJmoiIiIrxkRNRERkxZioiYiIrBgTNRERkRVjoiYiIrJiTNRERERWjNf6NoO4fWV2vf4RvjceERFZDWM+Efe7m8ptTNRmuH7dcJf7wMBAmSMhIqLa5Pr161Cr73/vV4UwJ53XcaWlpbh8+TLc3NygUCgqf0MF9Ho9AgMDcfHiRahUqiqOsPo8qnEDj27sjLvmPaqxM+6aV1WxCyFw/fp1BAQEwMbm/keh2aM2g42NDerXr18l81KpVI/cHybw6MYNPLqxM+6a96jGzrhrXlXEXllP2oiDyYiIiKwYEzUREZEVY6KuIUqlEm+88QaUSqXcoVjkUY0beHRjZ9w171GNnXHXPDli52AyIiIiK8YeNRERkRVjoiYiIrJiTNRERERWjIm6BixfvhwNGjSAo6MjwsPDcfDgQblDKmfhwoXo2LEj3Nzc4OvriwEDBiA9Pd2kTY8ePaBQKEymiRMnyhSxwbx588rFFBoaKtXfunULMTEx8PLygqurKwYNGoTs7GwZIzZo0KBBubgVCgViYmIAWNe23rdvH5566ikEBARAoVBg8+bNJvVCCMydOxf+/v5wcnJCZGQkTp8+bdLm6tWrGD58OFQqFdzd3TF27FjcuHFDtriLioowe/ZstGrVCi4uLggICMCIESNw+fJlk3lU9Dm98847ssUNAKNGjSoXU9++fU3ayLG9zYm9or95hUKBxYsXS21qepub891nzvdIZmYmoqOj4ezsDF9fX8ycORPFxcVVEiMTdTX7+uuvMX36dLzxxhs4cuQI2rRpg6ioKOTk5MgdmomkpCTExMTg559/RkJCAoqKitCnTx/cvHnTpN348eORlZUlTYsWLZIp4jtatGhhEtNPP/0k1U2bNg1btmzBxo0bkZSUhMuXL2PgwIEyRmtw6NAhk5gTEhIAAM8995zUxlq29c2bN9GmTRssX768wvpFixZh6dKlWLlyJVJSUuDi4oKoqCjcunVLajN8+HCcOHECCQkJ2Lp1K/bt24cJEybIFndeXh6OHDmCOXPm4MiRI/j222+Rnp6Op59+ulzbBQsWmHwOr7zyimxxG/Xt29ckpnXr1pnUy7G9gcpjLxtzVlYWPv/8cygUCgwaNMikXU1uc3O++yr7HikpKUF0dDQKCwtx4MABrFmzBvHx8Zg7d27VBCmoWnXq1EnExMRIr0tKSkRAQIBYuHChjFFVLicnRwAQSUlJUln37t3FlClT5AuqAm+88YZo06ZNhXW5ubnC3t5ebNy4USo7deqUACCSk5NrKELzTJkyRTRq1EiUlpYKIaxzWwshBACxadMm6XVpaanQaDRi8eLFUllubq5QKpVi3bp1QgghTp48KQCIQ4cOSW22b98uFAqFuHTpkixxV+TgwYMCgLhw4YJUFhwcLD788MPqDe4+Kop75MiRon///vd8jzVsbyHM2+b9+/cXvXr1MimTe5vf/d1nzvfItm3bhI2NjdBqtVKbuLg4oVKpREFBwUPHxB51NSosLERqaioiIyOlMhsbG0RGRiI5OVnGyCqn0+kAAJ6enibla9euhbe3N1q2bInY2Fjk5eXJEZ6J06dPIyAgAA0bNsTw4cORmZkJAEhNTUVRUZHJ9g8NDUVQUJBVbf/CwkJ8+eWXGDNmjMm15K1xW98tIyMDWq3WZBur1WqEh4dL2zg5ORnu7u7o0KGD1CYyMhI2NjZISUmp8ZjvRafTQaFQwN3d3aT8nXfegZeXF9q2bYvFixdX2e7Mh5GYmAhfX180a9YMkyZNwpUrV6S6R2V7Z2dn44cffsDYsWPL1cm5ze/+7jPneyQ5ORmtWrWCn5+f1CYqKgp6vR4nTpx46Jh4re9q9Ndff6GkpMTkwwMAPz8//PbbbzJFVbnS0lJMnToVnTt3RsuWLaXy559/HsHBwQgICMCxY8cwe/ZspKen49tvv5Ut1vDwcMTHx6NZs2bIysrC/Pnz0bVrV/z666/QarVwcHAo98Xr5+cHrVYrT8AV2Lx5M3JzczFq1CipzBq3dUWM27Giv3FjnVarha+vr0m9nZ0dPD09reZzuHXrFmbPno1hw4aZXL/51VdfRbt27eDp6YkDBw4gNjYWWVlZ+OCDD2SLtW/fvhg4cCBCQkJw9uxZ/POf/0S/fv2QnJwMW1vbR2J7A8CaNWvg5uZW7lCUnNu8ou8+c75HtFpthf8DxrqHxURN5cTExODXX381OdYLwOQYV6tWreDv74/evXvj7NmzaNSoUU2HCQDo16+f9Lx169YIDw9HcHAwNmzYACcnJ1listRnn32Gfv36ISAgQCqzxm1dWxUVFWHw4MEQQiAuLs6kbvr06dLz1q1bw8HBAS+99BIWLlwo21W1hg4dKj1v1aoVWrdujUaNGiExMRG9e/eWJaYH8fnnn2P48OFwdHQ0KZdzm9/ru09u3PVdjby9vWFra1tudGB2djY0Go1MUd3f5MmTsXXrVuzdu7fSO4aFh4cDAM6cOVMToZnF3d0dTZs2xZkzZ6DRaFBYWIjc3FyTNta0/S9cuIBdu3Zh3Lhx921njdsagLQd7/c3rtFoyg2eLC4uxtWrV2X/HIxJ+sKFC0hISKj0bkjh4eEoLi7G+fPnayZAMzRs2BDe3t7S34Y1b2+j//3vf0hPT6/07x6ouW1+r+8+c75HNBpNhf8DxrqHxURdjRwcHNC+fXvs3r1bKistLcXu3bsREREhY2TlCSEwefJkbNq0CXv27EFISEil70lLSwMA+Pv7V3N05rtx4wbOnj0Lf39/tG/fHvb29ibbPz09HZmZmVaz/VevXg1fX19ER0fft501bmsACAkJgUajMdnGer0eKSkp0jaOiIhAbm4uUlNTpTZ79uxBaWmp9ANEDsYkffr0aezatQteXl6VvictLQ02Njbldi3L6Y8//sCVK1ekvw1r3d5lffbZZ2jfvj3atGlTadvq3uaVffeZ8z0SERGB48ePm/xAMv7wCwsLq5IgqRqtX79eKJVKER8fL06ePCkmTJgg3N3dTUYHWoNJkyYJtVotEhMTRVZWljTl5eUJIYQ4c+aMWLBggTh8+LDIyMgQ3333nWjYsKHo1q2brHHPmDFDJCYmioyMDLF//34RGRkpvL29RU5OjhBCiIkTJ4qgoCCxZ88ecfjwYRERESEiIiJkjdmopKREBAUFidmzZ5uUW9u2vn79ujh69Kg4evSoACA++OADcfToUWl09DvvvCPc3d3Fd999J44dOyb69+8vQkJCRH5+vjSPvn37irZt24qUlBTx008/iSZNmohhw4bJFndhYaF4+umnRf369UVaWprJ37xxlO6BAwfEhx9+KNLS0sTZs2fFl19+KXx8fMSIESNki/v69evitddeE8nJySIjI0Ps2rVLtGvXTjRp0kTcunVLmocc27uy2I10Op1wdnYWcXFx5d4vxzav7LtPiMq/R4qLi0XLli1Fnz59RFpamtixY4fw8fERsbGxVRIjE3UN+Oijj0RQUJBwcHAQnTp1Ej///LPcIZUDoMJp9erVQgghMjMzRbdu3YSnp6dQKpWicePGYubMmUKn08ka95AhQ4S/v79wcHAQ9erVE0OGDBFnzpyR6vPz88XLL78sPDw8hLOzs3jmmWdEVlaWjBHfsXPnTgFApKenm5Rb27beu3dvhX8bI0eOFEIYTtGaM2eO8PPzE0qlUvTu3bvcOl25ckUMGzZMuLq6CpVKJUaPHi2uX78uW9wZGRn3/Jvfu3evEEKI1NRUER4eLtRqtXB0dBTNmzcXb7/9tklCrOm48/LyRJ8+fYSPj4+wt7cXwcHBYvz48eV++MuxvSuL3ejjjz8WTk5OIjc3t9z75djmlX33CWHe98j58+dFv379hJOTk/D29hYzZswQRUVFVRIj755FRERkxXiMmoiIyIoxURMREVkxJmoiIiIrxkRNRERkxZioiYiIrBgTNRERkRVjoiYiIrJiTNRERERWjImaSAY9evTA1KlT5Q6jnMTERCgUinI3IKgLGjRogP/85z9yh0FUDhM1UR1V0Y+FJ554AllZWVCr1fIERUTlMFETkcTBwQEajQYKhaJGl1tSUoLS0tIaXSbRo4KJmkgmxcXFmDx5MtRqNby9vTFnzhyUvfT+tWvXMGLECHh4eMDZ2Rn9+vXD6dOnTebxzTffoEWLFlAqlWjQoAHef/99k/oVK1agSZMmcHR0hJ+fH5599lkAwKhRo5CUlIQlS5ZAoVBAoVDg/Pnz5XZ9x8fHw93dHTt37kTz5s3h6uqKvn37Iisry2Q9Xn31Vbi7u8PLywuzZ8/GyJEjMWDAgHuuu3G+33//PcLCwqBUKpGZmVnpOs+bNw+PPfaYybz+85//oEGDBtLrUaNGYcCAAXjvvffg7+8PLy8vxMTEoKioSGqTk5ODp556Ck5OTggJCcHatWtN5imEwLx58xAUFASlUomAgAC8+uqr91wfourERE0kkzVr1sDOzg4HDx7EkiVL8MEHH2DVqlVS/ahRo3D48GF8//33SE5OhhACf//736WEk5qaisGDB2Po0KE4fvw45s2bhzlz5iA+Ph4AcPjwYbz66qtYsGAB0tPTsWPHDnTr1g0AsGTJEkRERGD8+PHIyspCVlYWAgMDK4wzLy8P7733Hv7v//4P+/btQ2ZmJl577TWp/t1338XatWuxevVq7N+/H3q9Hps3b650/fPy8vDuu+9i1apVOHHiBHx9fStdZ3Pt3bsXZ8+exd69e7FmzRrEx8dL28W4bS9evIi9e/fiv//9L1asWGFyL+FvvvkGH374IT7++GOcPn0amzdvRqtWrSyKgajKVMk9uIjIIt27dxfNmzcXpaWlUtns2bNF8+bNhRBC/P777wKA2L9/v1T/119/CScnJ7FhwwYhhBDPP/+8+Nvf/mYy35kzZ4qwsDAhhBDffPONUKlUQq/X3zOGKVOmmJQZb1N47do1IYQQq1evFgBMbh26fPly4efnJ7328/MTixcvll4XFxeLoKAg0b9//3uuv3G+aWlpUpk56/zGG2+INm3amMzrww8/FMHBwdLrkSNHiuDgYFFcXCyVPffcc2LIkCFCCCHS09MFAHHw4EGp/tSpUwKA+PDDD4UQQrz//vuiadOmorCw8J7rQFRT2KMmksnjjz9uciw4IiICp0+fRklJCU6dOgU7OzuEh4dL9V5eXmjWrBlOnToFADh16hQ6d+5sMs/OnTtL8/jb3/6G4OBgNGzYEC+++CLWrl2LvLw8i+N0dnZGo0aNpNf+/v5S71On0yE7OxudOnWS6m1tbdG+fftK5+vg4IDWrVtLr81ZZ3O1aNECtra2FcZsXE7ZGENDQ+Hu7i69fu6555Cfn4+GDRti/Pjx2LRpE4qLiy2KgaiqMFET1VJubm44cuQI1q1bB39/f8ydOxdt2rSx+NQre3t7k9cKhcLkWPqDcnJysnjQmo2NTbllV7RbvKKYLRmsFhgYiPT0dKxYsQJOTk54+eWX0a1bN4t3wRNVBSZqIpmkpKSYvP7555/RpEkT2Nraonnz5iguLjZpc+XKFaSnpyMsLAwA0Lx5c+zfv99kHvv370fTpk2l3qSdnR0iIyOxaNEiHDt2DOfPn8eePXsAGHq0JSUlD7UOarUafn5+OHTokFRWUlKCI0eOWDwvc9bZx8cHWq3WJFmnpaVZtJzQ0FAUFxcjNTVVKktPTy/3A8bJyQlPPfUUli5disTERCQnJ+P48eMWrxfRw7KTOwCiuiozMxPTp0/HSy+9hCNHjuCjjz6SRm03adIE/fv3x/jx4/Hxxx/Dzc0N//jHP1CvXj30798fADBjxgx07NgRb775JoYMGYLk5GQsW7YMK1asAABs3boV586dQ7du3eDh4YFt27ahtLQUzZo1A2C4wEdKSgrOnz8PV1dXeHp6PtB6vPLKK1i4cCEaN26M0NBQfPTRR7h27ZrFvWVz1rlHjx74888/sWjRIjz77LPYsWMHtm/fDpVKZfZymjVrhr59++Kll15CXFwc7OzsMHXqVDg5OUlt4uPjUVJSgvDwcDg7O+PLL7+Ek5MTgoODLVonoqrAHjWRTEaMGIH8/Hx06tQJMTExmDJlCiZMmCDVr169Gu3bt8eTTz6JiIgICCGwbds2abduu3btsGHDBqxfvx4tW7bE3LlzsWDBAowaNQoA4O7ujm+//Ra9evVC8+bNsXLlSqxbtw4tWrQAALz22muwtbVFWFgYfHx8kJmZ+UDrMXv2bAwbNgwjRoxAREQEXF1dERUVBUdHR4vnVdk6N2/eHCtWrMDy5cvRpk0bHDx40GQEuiXLCQgIQPfu3TFw4EBMmDABvr6+Ur27uzs+/fRTdO7cGa1bt8auXbuwZcsWeHl5WbwsooelEFVxsImI6LbS0lI0b94cgwcPxptvvil3OESPPO76JqKHcuHCBfz444/o3r07CgoKsGzZMmRkZOD555+XOzSiWoG7vonoodjY2CA+Ph4dO3ZE586dcfz4cezatQvNmzeXOzSiWoG7vomIiKwYe9RERERWjImaiIjIijFRExERWTEmaiIiIivGRE1ERGTFmKiJiIisGBM1ERGRFWOiJiIismJM1ERERFbs/wGAmrRgDx2UlwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 500x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MSE after 1 tree 7819 -> after 200 trees 551\n"
     ]
    }
   ],
   "source": [
    "# viz: test MSE vs number of boosting rounds (the diminishing-returns curve)\n",
    "from sklearn.datasets import make_regression\n",
    "Xr, yr = make_regression(n_samples=500, n_features=5, noise=10, random_state=SEED)\n",
    "Xr_tr, Xr_te, yr_tr, yr_te = train_test_split(Xr, yr, test_size=0.25, random_state=SEED)\n",
    "trees, init = fit_gbm(Xr_tr, yr_tr, N_ROUNDS, 0.1, 3)\n",
    "\n",
    "F = np.full(Xr_te.shape[0], init, dtype=float)\n",
    "errs = []\n",
    "for tree in trees:\n",
    "    F = F + 0.1 * tree.predict(Xr_te)\n",
    "    errs.append(np.mean((F - yr_te) ** 2))\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(5, 4))\n",
    "ax.plot(range(1, len(errs) + 1), errs, color=\"#1E40FF\")\n",
    "ax.set_xlabel(\"boosting rounds\"); ax.set_ylabel(\"test MSE\"); ax.set_title(\"gradient boosting: diminishing returns\")\n",
    "plt.tight_layout(); plt.show()\n",
    "print(f\"MSE after 1 tree {errs[0]:.0f} -> after {len(errs)} trees {errs[-1]:.0f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e111e16",
   "metadata": {},
   "source": [
    "> **Interpretation.** The curve drops steeply then flattens. The first handful of trees capture the bulk of the signal; the rest fine-tune. This is also where overfitting can creep back (variance re-entering), which is why production boosters use early stopping on a validation fold.\n",
    "\n",
    "> **Intuition:** bagging and boosting reach the same destination from opposite directions. Bagging averages many *low-bias, high-variance* deep trees to kill variance. Boosting stacks many *high-bias, low-variance* shallow trees to kill bias. Both end low-bias and low-variance.\n",
    "\n",
    "> **Key takeaways.**\n",
    "> - AdaBoost reweights examples toward the current mistakes; the `alpha` weights are the loss-minimizing step sizes, not magic constants.\n",
    "> - Gradient boosting fits each tree to the residual (the negative gradient of squared loss); the production libraries optimize this exact loop.\n",
    "> - Bagging cuts variance, boosting cuts bias; early stopping guards boosting against variance creeping back.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a4d446a",
   "metadata": {},
   "source": [
    "## Part 7 — Stacking, broken on purpose then fixed\n",
    "\n",
    "> **Objectives.** Build a stacking ensemble the wrong way, watch its meta-learner hit 100% training accuracy while learning nothing useful, diagnose the target leakage, and fix it with out-of-fold predictions.\n",
    "\n",
    "**Stacking** trains a *meta-learner* on the base models' outputs instead of hard-coding a vote. The setup: fit `N` base models, collect their predictions, and train a final model (usually a logistic regression) to combine them.\n",
    "\n",
    "The trap is *which* predictions feed the meta-learner. If you fit the base models on the training set and then ask them to predict that *same* training set, an overfitting base learner (a deep tree, a 1-nearest-neighbor) returns a near-perfect copy of the labels. The meta-learner sees labels disguised as features and learns \"trust the memorizer unconditionally\". Watch it happen.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00aed6f8",
   "metadata": {},
   "source": [
    "> **Predict:** we will fit the meta-learner on base predictions made on the training data the base models were trained on. What training accuracy will the meta-learner report, and will it generalize? <details><summary>Answer</summary>It reports 1.00 training accuracy because one base learner is a 1-NN that memorizes the training set, so its in-sample prediction equals the label. It generalizes no better than the base models, and its inflated train score is a leakage fingerprint, not skill.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "c5fb582a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.403275Z",
     "iopub.status.busy": "2026-06-10T18:50:52.403190Z",
     "iopub.status.idle": "2026-06-10T18:50:52.410886Z",
     "shell.execute_reply": "2026-06-10T18:50:52.410610Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LEAKY stacking: meta train accuracy 1.000 · test accuracy 0.880\n",
      "train-test gap: 0.120  (a leakage fingerprint)\n"
     ]
    }
   ],
   "source": [
    "# THE BROKEN RUN: meta-learner trained on IN-SAMPLE base predictions (target leakage).\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "def fresh_bases():\n",
    "    # include a 1-NN: it memorizes the training set, so its in-sample proba copies the labels\n",
    "    return [DecisionTreeClassifier(max_depth=None, random_state=SEED),\n",
    "            KNeighborsClassifier(n_neighbors=1),\n",
    "            LogisticRegression(random_state=SEED)]\n",
    "\n",
    "bases = fresh_bases()\n",
    "for b in bases:\n",
    "    b.fit(X_train, y_train)\n",
    "\n",
    "# leak: base predictions ON THE TRAINING SET the bases just trained on\n",
    "Z_train_leaky = np.column_stack([b.predict_proba(X_train)[:, 1] for b in bases])\n",
    "Z_test        = np.column_stack([b.predict_proba(X_test)[:, 1]  for b in bases])\n",
    "meta_leaky = LogisticRegression(random_state=SEED).fit(Z_train_leaky, y_train)\n",
    "\n",
    "leaky_train = meta_leaky.score(Z_train_leaky, y_train)\n",
    "leaky_test  = meta_leaky.score(Z_test, y_test)\n",
    "print(f\"LEAKY stacking: meta train accuracy {leaky_train:.3f} · test accuracy {leaky_test:.3f}\")\n",
    "print(f\"train-test gap: {leaky_train - leaky_test:.3f}  (a leakage fingerprint)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1e64191",
   "metadata": {},
   "source": [
    "> **Interpretation.** The meta-learner reports a perfect (or near-perfect) training accuracy and a far lower test accuracy. That gap is the tell. The 1-NN base learner's in-sample predictions *are* the training labels, so the meta-learner learned to parrot it. Nothing about the held-out world improved.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab359b7e",
   "metadata": {},
   "source": [
    "The fix is to feed the meta-learner **out-of-fold** predictions: for each row, use a base-model copy that did *not* train on that row. `cross_val_predict` does exactly this. The base models are then refit on the full training set for inference. Now the meta-learner sees honest, out-of-sample base predictions.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "6ec9938c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.411869Z",
     "iopub.status.busy": "2026-06-10T18:50:52.411785Z",
     "iopub.status.idle": "2026-06-10T18:50:52.433912Z",
     "shell.execute_reply": "2026-06-10T18:50:52.433613Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FIXED stacking: meta train accuracy 0.877 · test accuracy 0.880\n",
      "train-test gap: -0.003  (closed)\n"
     ]
    }
   ],
   "source": [
    "# THE FIX: out-of-fold base predictions for the meta-features.\n",
    "from sklearn.model_selection import cross_val_predict\n",
    "\n",
    "bases2 = fresh_bases()\n",
    "Z_train_oof = np.column_stack([\n",
    "    cross_val_predict(b, X_train, y_train, cv=5, method=\"predict_proba\")[:, 1]\n",
    "    for b in bases2\n",
    "])\n",
    "for b in bases2:                      # refit on full train for inference-time predictions\n",
    "    b.fit(X_train, y_train)\n",
    "Z_test2 = np.column_stack([b.predict_proba(X_test)[:, 1] for b in bases2])\n",
    "meta_ok = LogisticRegression(random_state=SEED).fit(Z_train_oof, y_train)\n",
    "\n",
    "ok_train = meta_ok.score(Z_train_oof, y_train)\n",
    "ok_test  = meta_ok.score(Z_test2, y_test)\n",
    "print(f\"FIXED stacking: meta train accuracy {ok_train:.3f} · test accuracy {ok_test:.3f}\")\n",
    "print(f\"train-test gap: {ok_train - ok_test:.3f}  (closed)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5eee824",
   "metadata": {},
   "source": [
    "The repair is assertable: the proper, out-of-fold stack has a far smaller train-test gap than the leaky one. That gap, not the test accuracy, is what diagnoses the bug.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "835ff92b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.434902Z",
     "iopub.status.busy": "2026-06-10T18:50:52.434818Z",
     "iopub.status.idle": "2026-06-10T18:50:52.439235Z",
     "shell.execute_reply": "2026-06-10T18:50:52.438757Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] 7.1 out-of-fold stacking closes the leakage gap\n"
     ]
    }
   ],
   "source": [
    "_ = check(\"7.1 out-of-fold stacking closes the leakage gap\",\n",
    "      lambda: None if (leaky_train - leaky_test) > (ok_train - ok_test) + 0.05\n",
    "              else (_ for _ in ()).throw(AssertionError(\n",
    "                  f\"the leaky gap {leaky_train - leaky_test:.3f} should far exceed the \"\n",
    "                  f\"fixed gap {ok_train - ok_test:.3f}; if not, the leakage demo did not trigger\")))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df3888ca",
   "metadata": {},
   "source": [
    "sklearn's `StackingClassifier` builds the out-of-fold features for you when you pass `cv=`. Use it and never hand-roll the leakage.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "964ef704",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.440040Z",
     "iopub.status.busy": "2026-06-10T18:50:52.439960Z",
     "iopub.status.idle": "2026-06-10T18:50:52.464677Z",
     "shell.execute_reply": "2026-06-10T18:50:52.464314Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sklearn StackingClassifier test accuracy 0.880\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import StackingClassifier\n",
    "\n",
    "stack = StackingClassifier(\n",
    "    estimators=[(\"dt\", DecisionTreeClassifier(max_depth=None, random_state=SEED)),\n",
    "                (\"knn\", KNeighborsClassifier(n_neighbors=1)),\n",
    "                (\"lr\", LogisticRegression(random_state=SEED))],\n",
    "    final_estimator=LogisticRegression(random_state=SEED),\n",
    "    cv=5,    # the cross-validation that prevents the leakage above\n",
    ").fit(X_train, y_train)\n",
    "print(f\"sklearn StackingClassifier test accuracy {stack.score(X_test, y_test):.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "277e601a",
   "metadata": {},
   "source": [
    "> **Key takeaways.**\n",
    "> - Stacking trains a meta-learner on base-model outputs; the danger is feeding it in-sample predictions.\n",
    "> - A memorizing base learner turns its in-sample predictions into the labels, so the meta-learner learns to parrot it and overfits.\n",
    "> - The fix is out-of-fold predictions (`cv=` in `StackingClassifier`). The rule generalizes: any prediction passed between models must be made on data the producer did not train on.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "753738b8",
   "metadata": {},
   "source": [
    "## Safety lens\n",
    "\n",
    "Ensembles inherit their base learners' failure modes and add a few. The sharp one for shipping a forest: a single tree is auditable (print it, read the splits), but the average of 300 trees is not. If the base trees latch onto a shortcut feature that proxies a protected attribute, the forest amplifies it through averaging, and `feature_importances_` is too weak and too cardinality-biased to surface it.\n",
    "\n",
    "The deployment-grade tool is **permutation importance**: shuffle one feature's column, measure the accuracy drop. It does not have the cardinality bias of impurity importance and it measures predictive reliance, not split counts. Here the two methods are compared on the same forest.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "50393c3d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.465675Z",
     "iopub.status.busy": "2026-06-10T18:50:52.465598Z",
     "iopub.status.idle": "2026-06-10T18:50:52.716895Z",
     "shell.execute_reply": "2026-06-10T18:50:52.716471Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                   impurity  permutation\n",
      "petal length (cm)     0.438        0.236\n",
      "petal width (cm)      0.434        0.223\n",
      "sepal length (cm)     0.100        0.019\n",
      "sepal width (cm)      0.028        0.015\n"
     ]
    }
   ],
   "source": [
    "# permutation importance vs impurity importance on the same Iris forest\n",
    "from sklearn.inspection import permutation_importance\n",
    "\n",
    "perm = permutation_importance(rf_iris, iris.data, iris.target,\n",
    "                              n_repeats=10, random_state=SEED)\n",
    "comp = pd.DataFrame({\n",
    "    \"impurity\": rf_iris.feature_importances_,\n",
    "    \"permutation\": perm.importances_mean,\n",
    "}, index=iris.feature_names).sort_values(\"permutation\", ascending=False)\n",
    "print(comp.to_string(float_format=lambda v: f\"{v:.3f}\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "471167c9",
   "metadata": {},
   "source": [
    "> **Interpretation.** Both methods agree that the petal features dominate Iris, which is the easy case. The methods *disagree* exactly when a feature has many split points but little real signal: impurity importance inflates it, permutation importance does not. When the two disagree, trust permutation importance and investigate the gap, because that gap is your cardinality-bias or shortcut-feature signal. For any model that gates a real decision, report permutation importance plus a per-subgroup error breakdown (`pandas.groupby` on the protected attribute), never impurity importance alone.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3438d0a0",
   "metadata": {},
   "source": [
    "## Going further\n",
    "- Géron, *Hands-On ML* 3e, Ch 7 — the voting/bagging/boosting/stacking arc this notebook compresses, with the same moons figures.\n",
    "- Chen and Guestrin, *XGBoost: A Scalable Tree Boosting System* (2016) — Section 2 (the regularized objective) is the core; the rest is engineering. XGBoost, LightGBM, and CatBoost optimize the exact residual-fitting loop from Part 6, adding second-order steps, histogram split-finding, and regularized leaf values. They are not in Colab's default image (a `pip install` there can force a runtime restart), so they are omitted from the runnable path here.\n",
    "- scikit-learn user guide, *Ensemble methods* — the canonical reference for every estimator used above, including `permutation_importance`.\n",
    "- Friedman, *Greedy Function Approximation: A Gradient Boosting Machine* (2001) — the function-space-gradient-descent view that unifies Part 6.\n",
    "\n",
    "## What this enables\n",
    "- **Ch 07 — Dimensionality Reduction**: PCA feeding a random forest is a standard production pipeline for wide tabular data; you now have the second half.\n",
    "- **Ch 23 — Eval Science**: the `1/sqrt(N)` variance argument from Part 1 is the same machinery behind bootstrap confidence intervals on evaluation metrics.\n",
    "- **Ch 19 — RL + RLHF**: the variance-reduction-by-averaging idea recurs in policy-gradient baselines and target networks.\n",
    "\n",
    "We stopped at sklearn's gradient booster. The gap: on real tabular benchmarks a tuned XGBoost or LightGBM, with early stopping and row/column subsampling, will beat the `GradientBoostingRegressor` here by a meaningful margin while training faster. The algorithm is identical to Part 6; the difference is entirely engineering. That gap is the subject of the production-ML chapters.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba5cc416",
   "metadata": {},
   "source": [
    "## Test yourself\n",
    "\n",
    "Three parts: concept self-checks with folded answers, auto-checked problems, and a capstone with a rubric and a folded reference. Every answer is in this notebook; if unsure, re-run that section.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00dd4f17",
   "metadata": {},
   "source": [
    "### Part A — Concepts\n",
    "\n",
    "1. Bagging reduces variance but not bias; boosting reduces bias but not much variance. How can both reach similar test accuracy? <details><summary>Answer</summary>They use opposite base learners. Bagging averages low-bias, high-variance deep trees, so the variance cancels and the low bias survives. Boosting stacks high-bias, low-variance shallow stumps, so the bias falls additively while the variance was small to begin with. Both land at low-bias, low-variance.</details>\n",
    "2. Why does a bootstrap sample cover about 63% of the rows? <details><summary>Answer</summary>The chance a given row is missed in one draw is `1 - 1/m`; over `m` independent draws that is `(1 - 1/m)^m -> 1/e ≈ 0.368`. So ~63.2% of rows appear at least once. See Exercise 6.2.</details>\n",
    "3. A forest's `feature_importances_` lists a high-cardinality ID column as most important. Should you trust it? <details><summary>Answer</summary>No. Impurity importance is biased toward features with many candidate split points, which is exactly what a high-cardinality column has. Compute permutation importance (Safety lens); if it disagrees, trust permutation importance and treat the gap as a warning.</details>\n",
    "4. Look at the gradient-boosting diminishing-returns plot in Part 6. Why is the curve steep then flat? <details><summary>Answer</summary>Each tree fits the current residual. The early residuals are large and structured, so the first trees remove most of the error; the later residuals are small and noisy, so each later tree barely moves the test MSE. Pushing too far re-introduces variance, which is why early stopping exists.</details>\n",
    "5. In Part 7 the leaky meta-learner hit 1.00 training accuracy. Which base learner caused it, and why? <details><summary>Answer</summary>The 1-nearest-neighbor. On the data it trained on, 1-NN returns each point's own label, so its in-sample `predict_proba` equals the target. The meta-learner reads that as a feature and learns to copy it, inflating train accuracy with zero generalization gain.</details>\n",
    "6. Why does soft voting not always beat hard voting? <details><summary>Answer</summary>Soft voting trusts the base models' probability estimates. If those are poorly calibrated (over- or under-confident), averaging them can do worse than a plain label vote. Soft voting wins *when the probabilities are calibrated*, which is a condition, not a law. See Part 2.</details>\n",
    "7. You set `n_estimators=5` for a random forest and the OOB score is much lower than the test score. Why? <details><summary>Answer</summary>With only 5 trees, each row is judged by the ~37% of trees that excluded it, which is one or two trees. That OOB committee is tiny and noisy, biasing the estimate low. OOB needs a reasonably large forest to be reliable (Part 5 caveat).</details>\n",
    "8. One sentence: what is the single word the entire `1/sqrt(N)` argument depends on, and what breaks it? <details><summary>Answer</summary>\"Independent.\" Correlated base learners share error that averaging cannot cancel; at full correlation `N` learners equal one (Exercise 6.1). Every method here is a different trick for reducing that correlation.</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c3b43b0",
   "metadata": {},
   "source": [
    "### Part B — Auto-checked problems\n",
    "\n",
    "Two problems. Write the body; the check asserts the property; the solution is folded below.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96d694a9",
   "metadata": {},
   "source": [
    "#### Problem B1 — Weighted classification error\n",
    "`Difficulty 2/5 · ~8 min`\n",
    "\n",
    "AdaBoost's per-round error is the *weighted* misclassification rate. Implement `weighted_error(y_true, y_pred, w)` returning `sum(w_i * 1[y_pred_i != y_true_i]) / sum(w_i)`. With uniform weights it must equal the plain error rate; concentrating weight on the mistakes must raise it.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "cad1e3e6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.717940Z",
     "iopub.status.busy": "2026-06-10T18:50:52.717871Z",
     "iopub.status.idle": "2026-06-10T18:50:52.720422Z",
     "shell.execute_reply": "2026-06-10T18:50:52.720162Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] B1 weighted error: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    }
   ],
   "source": [
    "def weighted_error(y_true, y_pred, w):\n",
    "    # TODO: weighted fraction misclassified; guard against sum(w) == 0\n",
    "    result = None\n",
    "    attempted(result)\n",
    "    return result\n",
    "\n",
    "def _b1():\n",
    "    yt = np.array([1, 1, 0, 0]); yp = np.array([1, 0, 0, 1])  # 2 of 4 wrong (indices 1 and 3)\n",
    "    uniform = np.ones(4) / 4\n",
    "    check_close(weighted_error(yt, yp, uniform), 0.5, msg=\"uniform weights -> plain error rate 0.5\")\n",
    "    heavy = np.array([0.1, 0.7, 0.1, 0.1])  # most weight on a wrong example (index 1)\n",
    "    assert weighted_error(yt, yp, heavy) > 0.5, \"piling weight on mistakes must raise weighted error\"\n",
    "\n",
    "_ = check(\"B1 weighted error\", _b1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e807908",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1</summary>`(y_pred != y_true)` is a boolean mask; weight it by `w`, sum, divide by `w.sum()`.</details>\n",
    "<details><summary>Solution</summary>\n",
    "\n",
    "```python\n",
    "def weighted_error(y_true, y_pred, w):\n",
    "    w = np.asarray(w, dtype=float)\n",
    "    s = w.sum()\n",
    "    if s == 0:\n",
    "        return 0.0\n",
    "    return float((w * (np.asarray(y_pred) != np.asarray(y_true))).sum() / s)\n",
    "```\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "9e140a39",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.721321Z",
     "iopub.status.busy": "2026-06-10T18:50:52.721256Z",
     "iopub.status.idle": "2026-06-10T18:50:52.723886Z",
     "shell.execute_reply": "2026-06-10T18:50:52.723635Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] B1 weighted error\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines weighted_error; the check re-verifies it.\n",
    "def weighted_error(y_true, y_pred, w):\n",
    "    w = np.asarray(w, dtype=float)\n",
    "    s = w.sum()\n",
    "    return float((w * (np.asarray(y_pred) != np.asarray(y_true))).sum() / s) if s else 0.0\n",
    "\n",
    "_ = check(\"B1 weighted error\", _b1, required=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4a77af3",
   "metadata": {},
   "source": [
    "#### Problem B2 — Soft-vote aggregation\n",
    "`Difficulty 2/5 · ~10 min`\n",
    "\n",
    "Implement `soft_vote(proba_list)` where `proba_list` is a list of `(n_samples, n_classes)` probability arrays from several models. Return the averaged-probability argmax label per row. Property to satisfy: averaging a confident-correct model with a hesitant-wrong one should recover the confident model's answer.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "3683756c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.724609Z",
     "iopub.status.busy": "2026-06-10T18:50:52.724543Z",
     "iopub.status.idle": "2026-06-10T18:50:52.727319Z",
     "shell.execute_reply": "2026-06-10T18:50:52.727002Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] B2 soft vote: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    }
   ],
   "source": [
    "def soft_vote(proba_list):\n",
    "    # TODO 1: average the probability matrices elementwise\n",
    "    avg = None\n",
    "    attempted(avg)\n",
    "    # TODO 2: argmax along the class axis\n",
    "    out = None\n",
    "    attempted(out)\n",
    "    return np.asarray(out).astype(int)\n",
    "\n",
    "def _b2():\n",
    "    # model 1 is confident class-1 on row 0, class-0 on row 1\n",
    "    p1 = np.array([[0.05, 0.95], [0.90, 0.10]])\n",
    "    # model 2 is hesitant and points the other way, but only barely\n",
    "    p2 = np.array([[0.55, 0.45], [0.45, 0.55]])\n",
    "    out = soft_vote([p1, p2])\n",
    "    assert out.tolist() == [1, 0], f\"confident model should dominate the hesitant one; got {out.tolist()}\"\n",
    "    # shape check on random input\n",
    "    rng_ = np.random.default_rng(SEED)\n",
    "    ps = [rng_.random((7, 3)) for _ in range(4)]\n",
    "    assert soft_vote(ps).shape == (7,), \"one label per sample\"\n",
    "\n",
    "_ = check(\"B2 soft vote\", _b2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dedc6981",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1</summary>`np.mean(proba_list, axis=0)` averages across models (the list axis); then `argmax(axis=1)` over classes.</details>\n",
    "<details><summary>Solution</summary>\n",
    "\n",
    "```python\n",
    "def soft_vote(proba_list):\n",
    "    avg = np.mean(proba_list, axis=0)\n",
    "    return avg.argmax(axis=1).astype(int)\n",
    "```\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "81aa4aff",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.728314Z",
     "iopub.status.busy": "2026-06-10T18:50:52.728195Z",
     "iopub.status.idle": "2026-06-10T18:50:52.730731Z",
     "shell.execute_reply": "2026-06-10T18:50:52.730229Z"
    },
    "collapsed": true,
    "jupyter": {
     "source_hidden": true
    },
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] B2 soft vote\n"
     ]
    }
   ],
   "source": [
    "#@title Solution { display-mode: \"form\" }\n",
    "# solution: redefines soft_vote; the check re-verifies it.\n",
    "def soft_vote(proba_list):\n",
    "    avg = np.mean(proba_list, axis=0)\n",
    "    return avg.argmax(axis=1).astype(int)\n",
    "\n",
    "_ = check(\"B2 soft vote\", _b2, required=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "501489c4",
   "metadata": {},
   "source": [
    "### Part C — Capstone: build `MyRandomForest` from scratch and beat a single tree\n",
    "\n",
    "Assemble the pieces from Parts 3-5 into one class, then verify it against sklearn. This is the from-scratch lab in one cell.\n",
    "\n",
    "**Deliverables.**\n",
    "1. A `MyRandomForest` class with `fit` (bagging: each tree trains on a bootstrap of the rows, with *per-split* feature subsampling delegated to `DecisionTreeClassifier(max_features=\"sqrt\")`, which is exactly how a real forest decorrelates trees from Part 4), `predict` (majority vote), and `oob_score` (Part 5 logic).\n",
    "2. On moons, `MyRandomForest` test accuracy within 5 points of `sklearn.ensemble.RandomForestClassifier`.\n",
    "3. `oob_score` within 8 points of the held-out test accuracy.\n",
    "\n",
    "**Self-assessment (pass / partial / fail).**\n",
    "- (a) `fit` stores one tree and one bootstrap mask per estimator, and passes `max_features` into each tree.\n",
    "- (b) `predict` majority-votes the trees on the full feature matrix.\n",
    "- (c) test accuracy beats a single `DecisionTreeClassifier` and tracks sklearn within 5 points.\n",
    "- (d) `oob_score` uses only the trees that excluded each row and lands within 8 points of test.\n",
    "- (e) the notebook still runs top-to-bottom.\n",
    "\n",
    "> **Common confusion:** there are two ways to subsample features. Selecting a fixed column subset *once per tree* (the \"random subspace\" method) is too aggressive on a 2-feature dataset like moons, where it can leave a tree with one feature and cripple it. A real forest subsamples features *per split* (Part 4), so a tree can still touch both features across its depth. We do the latter by handing `max_features=\"sqrt\"` to each `DecisionTreeClassifier`, which is what sklearn does internally.\n",
    "\n",
    "Write your attempt, run the check, then compare against the folded reference.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "ae92b419",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.731621Z",
     "iopub.status.busy": "2026-06-10T18:50:52.731548Z",
     "iopub.status.idle": "2026-06-10T18:50:52.735143Z",
     "shell.execute_reply": "2026-06-10T18:50:52.734705Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ -- ] C MyRandomForest vs sklearn: not attempted yet — fill in the TODO above, then re-run.\n"
     ]
    }
   ],
   "source": [
    "class MyRandomForest:\n",
    "    \"\"\"Random forest from scratch: bagging + per-split feature subsampling\n",
    "    (delegated to each tree's own max_features, as a real forest does).\"\"\"\n",
    "    def __init__(self, n_estimators=50, max_features=\"sqrt\", max_depth=None, random_state=SEED):\n",
    "        self.n_estimators = n_estimators\n",
    "        self.max_features = max_features\n",
    "        self.max_depth = max_depth\n",
    "        self.random_state = random_state\n",
    "        self.trees, self.in_bag = [], []\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        # TODO 1: rng = np.random.default_rng(self.random_state); m = len(y)\n",
    "        # TODO 2: per estimator: bootstrap rows; fit a DecisionTreeClassifier with\n",
    "        #         max_features=self.max_features and a distinct random_state on X[rows], y[rows]\n",
    "        # TODO 3: store the tree and a length-m in-bag boolean mask (True where the row appeared)\n",
    "        raise NotImplementedError\n",
    "        return self\n",
    "\n",
    "    def predict(self, X):\n",
    "        # TODO 4: majority vote of every tree's prediction on the full X\n",
    "        raise NotImplementedError\n",
    "\n",
    "    def oob_score(self, X, y):\n",
    "        # TODO 5: for each row, vote only the trees whose in_bag mask is False there;\n",
    "        #         return accuracy over the rows that had at least one such tree\n",
    "        raise NotImplementedError\n",
    "\n",
    "def _capstone():\n",
    "    rf_mine = MyRandomForest(n_estimators=N_TREES, max_features=\"sqrt\", max_depth=None,\n",
    "                             random_state=SEED).fit(X_train, y_train)\n",
    "    acc = (rf_mine.predict(X_test) == y_test).mean()\n",
    "    single = DecisionTreeClassifier(random_state=SEED).fit(X_train, y_train).score(X_test, y_test)\n",
    "    assert acc > single - 1e-9, f\"forest {acc:.3f} should not trail a single tree {single:.3f}\"\n",
    "    sk = RandomForestClassifier(n_estimators=N_TREES, random_state=SEED).fit(X_train, y_train)\n",
    "    assert abs(acc - sk.score(X_test, y_test)) < 0.05, \\\n",
    "        f\"MyRandomForest {acc:.3f} should track sklearn {sk.score(X_test, y_test):.3f} within 5 points\"\n",
    "    oob = rf_mine.oob_score(X_train, y_train)\n",
    "    assert abs(oob - acc) < 0.08, f\"OOB {oob:.3f} should track test {acc:.3f} within 8 points\"\n",
    "\n",
    "_ = check(\"C MyRandomForest vs sklearn\", _capstone)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60bd82b8",
   "metadata": {},
   "source": [
    "<details><summary>Hint 1 (conceptual)</summary>You already wrote every piece: bootstrap indices (6.2), bagging fit/predict (6.3), and OOB voting (6.5). The feature subsampling is one keyword: pass `max_features=\"sqrt\"` to each `DecisionTreeClassifier` so it samples features *per split* (Part 4), and then `predict` and `oob_score` work on the full feature matrix.</details>\n",
    "\n",
    "<details><summary>Hint 2 (pseudocode)</summary>\n",
    "\n",
    "```python\n",
    "# fit\n",
    "rng = np.random.default_rng(self.random_state); m = len(y)\n",
    "for _ in range(self.n_estimators):\n",
    "    rows = rng.integers(0, m, size=m)\n",
    "    tree = DecisionTreeClassifier(max_depth=self.max_depth, max_features=self.max_features,\n",
    "                                  random_state=int(rng.integers(0, 2**31 - 1)))\n",
    "    tree.fit(X[rows], y[rows])\n",
    "    mask = np.zeros(m, dtype=bool); mask[rows] = True\n",
    "    self.trees.append(tree); self.in_bag.append(mask)\n",
    "\n",
    "# predict\n",
    "votes = np.array([t.predict(X) for t in self.trees]).astype(int)\n",
    "return np.apply_along_axis(lambda c: np.bincount(c).argmax(), 0, votes)\n",
    "```\n",
    "</details>\n",
    "\n",
    "<details><summary>Help — OOB much lower than test, or forest trails the single tree</summary>If the forest underperforms a single tree, you may have pre-selected a fixed column subset per tree (the over-aggressive route in the confusion box). Hand `max_features` to the tree instead and predict on all columns. For OOB, select per row the trees with `~in_bag[t][i]`; if you accidentally vote with all trees, the estimate stops being out-of-sample.</details>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "7fcfc783",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:52.735876Z",
     "iopub.status.busy": "2026-06-10T18:50:52.735806Z",
     "iopub.status.idle": "2026-06-10T18:50:53.222652Z",
     "shell.execute_reply": "2026-06-10T18:50:53.222174Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ ok ] C MyRandomForest vs sklearn\n",
      "MyRandomForest test 0.872 · OOB 0.885\n"
     ]
    }
   ],
   "source": [
    "# reference solution: redefines MyRandomForest; the check re-verifies it.\n",
    "class MyRandomForest:\n",
    "    def __init__(self, n_estimators=50, max_features=\"sqrt\", max_depth=None, random_state=SEED):\n",
    "        self.n_estimators = n_estimators\n",
    "        self.max_features = max_features\n",
    "        self.max_depth = max_depth\n",
    "        self.random_state = random_state\n",
    "        self.trees, self.in_bag = [], []\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        rng = np.random.default_rng(self.random_state)\n",
    "        m = len(y)\n",
    "        self.trees, self.in_bag = [], []\n",
    "        for _ in range(self.n_estimators):\n",
    "            rows = rng.integers(0, m, size=m)\n",
    "            # per-SPLIT feature subsampling via the tree's own max_features (a real forest)\n",
    "            tree = DecisionTreeClassifier(max_depth=self.max_depth, max_features=self.max_features,\n",
    "                                          random_state=int(rng.integers(0, 2**31 - 1)))\n",
    "            tree.fit(X[rows], y[rows])\n",
    "            mask = np.zeros(m, dtype=bool); mask[rows] = True\n",
    "            self.trees.append(tree); self.in_bag.append(mask)\n",
    "        return self\n",
    "\n",
    "    def predict(self, X):\n",
    "        votes = np.array([t.predict(X) for t in self.trees]).astype(int)\n",
    "        return np.apply_along_axis(lambda c: np.bincount(c).argmax(), 0, votes)\n",
    "\n",
    "    def oob_score(self, X, y):\n",
    "        m = len(y)\n",
    "        in_bag = np.array(self.in_bag)              # (n_trees, m)\n",
    "        preds = np.array([t.predict(X) for t in self.trees]).astype(int)\n",
    "        correct, total = 0, 0\n",
    "        for i in range(m):\n",
    "            oob = ~in_bag[:, i]\n",
    "            if oob.sum() == 0:\n",
    "                continue\n",
    "            vote = np.bincount(preds[oob, i]).argmax()\n",
    "            total += 1\n",
    "            correct += int(vote == int(y[i]))\n",
    "        return correct / total if total else 0.0\n",
    "\n",
    "_ = check(\"C MyRandomForest vs sklearn\", _capstone, required=True)\n",
    "rf_mine = MyRandomForest(n_estimators=N_TREES, random_state=SEED).fit(X_train, y_train)\n",
    "print(f\"MyRandomForest test {(rf_mine.predict(X_test) == y_test).mean():.3f} · \"\n",
    "      f\"OOB {rf_mine.oob_score(X_train, y_train):.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "294e858f",
   "metadata": {},
   "source": [
    "## Reflection\n",
    "\n",
    "Write ~150 words, for yourself, in the cell below. Nobody grades it; writing it is the point. What was the dumbest bug you hit in this notebook, and how did you find it? Candidates: voting down the rows instead of the columns in `predict_bag`; inverting the in-bag mask so OOB voted with the wrong trees; letting `F` inherit an int dtype in `fit_gbm` and getting NaNs; sampling without replacement and seeing 100% bootstrap coverage; pre-selecting one fixed feature per tree in `MyRandomForest` and watching the forest trail a single tree. Name the symptom you saw first, the hypothesis you formed, and the one print or assert that confirmed the fix. Debugging literacy is the transferable skill; the ensemble math is just the setting it happened in.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "bbb2747f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-06-10T18:50:53.223777Z",
     "iopub.status.busy": "2026-06-10T18:50:53.223694Z",
     "iopub.status.idle": "2026-06-10T18:50:53.225721Z",
     "shell.execute_reply": "2026-06-10T18:50:53.225454Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "reflection length: 21 words (add more once you fill it in)\n"
     ]
    }
   ],
   "source": [
    "reflection = \"\"\"\n",
    "(replace this with ~150 words: the dumbest bug you hit, the symptom, your\n",
    "hypothesis, and the check that confirmed the fix)\n",
    "\"\"\"\n",
    "print(f\"reflection length: {len(reflection.split())} words \"\n",
    "      f\"({'looks complete' if len(reflection.split()) > 100 else 'add more once you fill it in'})\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16765904",
   "metadata": {},
   "source": [
    "---\n",
    "*Built top-to-bottom. If every check above printed `[ ok ]`, you reproduced the chapter from voting to a from-scratch random forest.*\n",
    "\n",
    "Total running time: written by CI · Verified on: numpy 2.x, sklearn 1.7, Python 3.12 · 2026-06-11\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "obvix-nb",
   "language": "python",
   "name": "obvix-nb"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "obvix": {
   "title": "Ch 06 — Ensemble Methods"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}