{
"cells": [
{
"cell_type": "markdown",
"id": "e6a096e2",
"metadata": {},
"source": [
"# Ch 09: Intro to Neural Networks (notebook)\n",
"\n",
"`[<- 08 unsupervised-learning]` · **this notebook** · `[10 pytorch-foundations ->]`\n",
"\n",
"Runs top-to-bottom in ~3 min on free Colab CPU. Last verified 2026-06-11.\n",
"\n",
"**What you'll build**\n",
"- A scalar autograd engine (`Value` with `_backward` closures and a topological sort), checked element-for-element against `torch.autograd`.\n",
"- `Neuron` -> `Layer` -> `MLP` stacked on that engine, trained on a 4-point toy until its loss collapses.\n",
"- A forward and backward pass for a 2-layer MLP in pure NumPy, every gradient verified by finite differences, then trained on a FashionMNIST subset to a real accuracy.\n",
"- A working version of the canonical micrograd bug (gradients that overwrite instead of accumulate), experienced and then fixed.\n",
"\n",
"**How this notebook works.** Code cells with a `# TODO` are yours to fill in. Run the cell to grade yourself: `[ ok ]` passed, `[FAIL]` shows what went wrong, `[ -- ]` means not attempted yet. Every exercise has a hint ladder (open only as many as you need) and a folded solution below it. The notebook runs top-to-bottom even if you fill in nothing: the solution cells redefine the pieces so the later cells work. See Ch 00 for the full protocol.\n"
]
},
{
"cell_type": "markdown",
"id": "a65346e0",
"metadata": {},
"source": [
"## Before you start\n",
"\n",
"1. You stack two linear layers with no activation between them: $O = (X W_1 + b_1) W_2 + b_2$. How many distinct functions can this express that one linear layer cannot? AnswerNone. Multiply it out: $O = X (W_1 W_2) + (b_1 W_2 + b_2) = X W' + b'$, a single affine map. The non-linearity is the only thing that buys you expressive power. Part 1 makes you prove this with an assert.\n",
"2. A node $a$ feeds into two downstream operations. When you run backprop, do you overwrite $a$'s gradient or add to it? AnswerAdd. The chain rule for a branching graph sums the contributions from every path out of $a$. Using `=` instead of `+=` in the backward pass is *the* canonical micrograd bug, and you will trigger it on purpose in Part 3.\n",
"3. Predict before you run: a 10-class classifier that has learned nothing outputs a uniform distribution. What is its cross-entropy loss at initialization? Answer$\\ln 10 \\approx 2.30$. \"Verify the loss at init is $\\ln K$\" is Karpathy's 30-second check that catches a whole class of bugs; you will use it on the FashionMNIST model in Part 5.\n"
]
},
{
"cell_type": "markdown",
"id": "f81b576b",
"metadata": {},
"source": [
"## Setup\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "fffdfced",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:37.102990Z",
"iopub.status.busy": "2026-06-10T19:14:37.102908Z",
"iopub.status.idle": "2026-06-10T19:14:41.735929Z",
"shell.execute_reply": "2026-06-10T19:14:41.735182Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"numpy 2.2.6 · torch 2.12.0+cpu\n",
"device cpu (this notebook is CPU-canonical; any GPU section prints-and-skips)\n"
]
}
],
"source": [
"import numpy as np\n",
"import torch\n",
"import matplotlib.pyplot as plt\n",
"torch.set_num_threads(1) # deterministic, polite on shared CPU; not a correctness knob\n",
"print(f\"numpy {np.__version__} · torch {torch.__version__}\")\n",
"if np.__version__ < \"2.0\":\n",
" print(\"WARN: written for NumPy 2.x; np.trapezoid etc., older versions may differ\")\n",
"device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
"print(f\"device {device} (this notebook is CPU-canonical; any GPU section prints-and-skips)\")"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "fa78d420",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:41.737138Z",
"iopub.status.busy": "2026-06-10T19:14:41.736981Z",
"iopub.status.idle": "2026-06-10T19:14:41.750589Z",
"shell.execute_reply": "2026-06-10T19:14:41.750263Z"
}
},
"outputs": [],
"source": [
"import os, random\n",
"SEED = 0\n",
"FAST = bool(os.environ.get('NB_FAST')) # CI smoke mode: ~10x fewer steps, same code paths\n",
"rng = np.random.default_rng(SEED) # the one numpy RNG we thread through the notebook\n",
"torch.manual_seed(SEED); random.seed(SEED)\n",
"\n",
"# ── house self-check harness (identical across all chapter notebooks) ──\n",
"import numpy as _np\n",
"\n",
"def check(label, test_fn, required=False):\n",
" \"\"\"Run one self-check. test_fn raises AssertionError (with a teaching\n",
" message) on failure, NotImplementedError if the stub is unfilled.\n",
" required=True is used only in solution cells; it is what CI grades.\"\"\"\n",
" try:\n",
" test_fn()\n",
" except NotImplementedError:\n",
" if required:\n",
" raise AssertionError(f\"{label}: reference solution incomplete\")\n",
" print(f\"[ -- ] {label}: not attempted yet — fill in the TODO above, then re-run.\")\n",
" return False\n",
" except AssertionError as e:\n",
" if required:\n",
" raise\n",
" print(f\"[FAIL] {label}: {e}\")\n",
" return False\n",
" print(f\"[ ok ] {label}\")\n",
" return True\n",
"\n",
"def attempted(*vals):\n",
" \"\"\"Treat None placeholders as 'not attempted'.\"\"\"\n",
" if any(v is None for v in vals):\n",
" raise NotImplementedError\n",
"\n",
"def check_shape(x, want):\n",
" assert tuple(x.shape) == tuple(want), \\\n",
" f\"shape {tuple(x.shape)}, expected {tuple(want)} — check your reshape/transpose order\"\n",
"\n",
"def check_close(got, want, atol=1e-5, rtol=1e-4, msg=\"\"):\n",
" g, w = _np.asarray(got, dtype=float), _np.asarray(want, dtype=float)\n",
" assert g.shape == w.shape, f\"shape {g.shape} vs expected {w.shape}. {msg}\"\n",
" bad = ~_np.isclose(g, w, atol=atol, rtol=rtol)\n",
" assert not bad.any(), \\\n",
" f\"{bad.mean():.2%} of values wrong (max diff {abs(g - w).max():.3g}). {msg}\""
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "684a1d3e",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:41.752129Z",
"iopub.status.busy": "2026-06-10T19:14:41.751714Z",
"iopub.status.idle": "2026-06-10T19:14:41.758791Z",
"shell.execute_reply": "2026-06-10T19:14:41.758495Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"helpers ready: grad_check, rel_err\n"
]
}
],
"source": [
"# a few house helpers (defined once, never imported), each <= 15 lines\n",
"def grad_check(f, x, eps=1e-5):\n",
" 'Central finite-difference gradient of scalar f at numpy array x (returns same shape).'\n",
" x = np.asarray(x, dtype=float)\n",
" g = np.zeros_like(x)\n",
" it = np.nditer(x, flags=[\"multi_index\"])\n",
" while not it.finished:\n",
" i = it.multi_index\n",
" old = x[i]\n",
" x[i] = old + eps; fp = f(x)\n",
" x[i] = old - eps; fm = f(x)\n",
" x[i] = old\n",
" g[i] = (fp - fm) / (2 * eps)\n",
" it.iternext()\n",
" return g\n",
"\n",
"def rel_err(a, b):\n",
" 'Relative error used to compare an analytic gradient against a numeric one.'\n",
" a, b = np.asarray(a, float), np.asarray(b, float)\n",
" denom = np.maximum(1e-12, np.abs(a) + np.abs(b))\n",
" return float(np.max(np.abs(a - b) / denom))\n",
"\n",
"print(\"helpers ready: grad_check, rel_err\")"
]
},
{
"cell_type": "markdown",
"id": "6f241e7d",
"metadata": {},
"source": [
"> **Note:** seeds make this notebook's printed numbers reproduce on CPU. Library versions and BLAS threading can shift the last digit or two; quoted numbers hold for the pinned environment. If your loss is 0.541 and the page says 0.542, you did nothing wrong. We re-seed at the top of every stochastic cell so a mid-notebook re-run reproduces the same output.\n"
]
},
{
"cell_type": "markdown",
"id": "720f98da",
"metadata": {},
"source": [
"## The map\n",
"\n",
"> **Part 1: One neuron, and why it isn't enough.** A single neuron is a hyperplane. Watch it fail on XOR, then prove algebraically (with an assert) that stacking affine layers without a non-linearity buys nothing.\n",
"> **Part 2: Activations and the forward pass.** Plot the activation zoo and its derivatives, see where the gradient vanishes, then run a 2-layer forward pass with strict shape discipline.\n",
"> **Part 3: Backprop from scratch: the `Value` engine.** Build micrograd's scalar autograd, verify every local rule by finite differences, cross-check a whole graph against `torch.autograd`, and trigger the `=` vs `+=` bug on purpose.\n",
"> **Part 4: Neuron -> Layer -> MLP.** Stack the engine into a network and train it on a 4-point toy until the loss collapses.\n",
"> **Part 5: Backprop on tensors + the NumPy MLP.** Derive the matmul and softmax-cross-entropy gradients, finite-difference-check them, and train a pure-NumPy MLP on a FashionMNIST subset. One run is deliberately broken, then fixed.\n"
]
},
{
"cell_type": "markdown",
"id": "6bbfa7cd",
"metadata": {},
"source": [
"## Part 1: One neuron, and why it isn't enough\n",
"\n",
"> **Objectives.** Build one neuron, watch it solve OR and fail XOR, and prove that affine layers without a non-linearity collapse to a single affine layer.\n",
"\n",
"A neuron takes a vector $x \\in \\mathbb{R}^d$, computes $z = w \\cdot x + b$, and passes $z$ through a non-linearity $\\sigma$ to produce $a = \\sigma(z)$. With a logistic $\\sigma$ this is logistic regression; its decision boundary $\\{x : w \\cdot x + b = 0\\}$ is a hyperplane. A single neuron can solve linearly separable problems and nothing else. The canonical counterexample is XOR.\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "0262d16b",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:41.760114Z",
"iopub.status.busy": "2026-06-10T19:14:41.760029Z",
"iopub.status.idle": "2026-06-10T19:14:41.777095Z",
"shell.execute_reply": "2026-06-10T19:14:41.776828Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"OR truth [0 1 1 1] neuron [0 1 1 1]\n"
]
}
],
"source": [
"def sigmoid(z):\n",
" return 1.0 / (1.0 + np.exp(-z))\n",
"\n",
"def neuron(X, w, b):\n",
" return sigmoid(X @ w + b) # X:(n,d) w:(d,) b:scalar -> (n,)\n",
"\n",
"X4 = np.array([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=float)\n",
"y_or = np.array([0, 1, 1, 1]) # OR is linearly separable\n",
"y_xor = np.array([0, 1, 1, 0]) # XOR is not\n",
"w, b = np.array([6.0, 6.0]), -3.0 # hand-chosen line that fires when either input is 1\n",
"pred_or = (neuron(X4, w, b) > 0.5).astype(int)\n",
"print(\"OR truth\", y_or, \"neuron\", pred_or)"
]
},
{
"cell_type": "markdown",
"id": "ec91527a",
"metadata": {},
"source": [
"> **Predict:** can *any* single line separate the XOR points $(0,1),(1,0)$ from $(0,0),(1,1)$? AnswerNo. The two positive points sit on opposite corners; the two negatives sit on the other diagonal. No straight line puts both positives on one side. Below, sklearn's logistic regression confirms it lands at 50% (a coin flip).\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "bbd13ff3",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:41.781190Z",
"iopub.status.busy": "2026-06-10T19:14:41.781099Z",
"iopub.status.idle": "2026-06-10T19:14:44.741513Z",
"shell.execute_reply": "2026-06-10T19:14:44.741178Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"logistic regression on XOR: accuracy 0.50 (chance is 0.50)\n",
"[ ok ] one linear boundary cannot carve XOR, Part 1's whole point\n"
]
}
],
"source": [
"# library path: even a fitted logistic regression can only reach chance on XOR\n",
"from sklearn.linear_model import LogisticRegression\n",
"clf = LogisticRegression().fit(X4, y_xor)\n",
"acc_xor = clf.score(X4, y_xor)\n",
"print(f\"logistic regression on XOR: accuracy {acc_xor:.2f} (chance is 0.50)\")\n",
"assert acc_xor <= 0.5 + 1e-9, \"a linear model cannot beat chance on XOR; if it did, the data is wrong\"\n",
"print(\"[ ok ] one linear boundary cannot carve XOR, Part 1's whole point\")"
]
},
{
"cell_type": "markdown",
"id": "affbbd9b",
"metadata": {},
"source": [
"> **Interpretation.** OR is solved with one hand-picked line; XOR defeats the best line sklearn can fit. The fix is to compose neurons: a hidden layer of two neurons plus a non-linearity bends the boundary into the XOR shape. But composition only helps *if* there is a non-linearity between the layers, which is what the next exercise proves.\n"
]
},
{
"cell_type": "markdown",
"id": "0744a3a4",
"metadata": {},
"source": [
"### Exercise 9.1: Two affine layers collapse to one\n",
"`Difficulty 2/5 · ~10 min`\n",
"\n",
"The claim from the draft: without an activation, $O = (X W_1 + b_1) W_2 + b_2$ equals a single affine map $X W' + b'$. Find the equivalent single-layer weights `W_eq` and bias `b_eq` so that the composed two-layer output matches `X @ W_eq + b_eq` exactly. This is the algebra behind \"the non-linearity is load-bearing\".\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "d912a2a3",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:44.743825Z",
"iopub.status.busy": "2026-06-10T19:14:44.743664Z",
"iopub.status.idle": "2026-06-10T19:14:44.765265Z",
"shell.execute_reply": "2026-06-10T19:14:44.762173Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] 9.1 affine collapse: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def collapse_affine(W1, b1, W2, b2):\n",
" \"\"\"Two stacked LINEAR layers -> equivalent single (W_eq, b_eq).\n",
" Layer 1: H = X @ W1 + b1 (W1:(d,h) b1:(h,))\n",
" Layer 2: O = H @ W2 + b2 (W2:(h,q) b2:(q,))\n",
" Want W_eq:(d,q), b_eq:(q,) with X @ W_eq + b_eq == O for all X.\n",
" \"\"\"\n",
" # TODO 1: W_eq is the product of the two weight matrices (mind the order)\n",
" W_eq = None\n",
" # TODO 2: b_eq folds b1 through the second layer, then adds b2\n",
" b_eq = None\n",
" attempted(W_eq, b_eq)\n",
" return W_eq, b_eq\n",
"\n",
"# self-check (run this cell): random layers, then compare composed vs collapsed on random X\n",
"def _check_collapse(fn):\n",
" g = np.random.default_rng(1)\n",
" d, h, q, n = 3, 5, 2, 7\n",
" W1, b1 = g.standard_normal((d, h)), g.standard_normal(h)\n",
" W2, b2 = g.standard_normal((h, q)), g.standard_normal(q)\n",
" X = g.standard_normal((n, d))\n",
" composed = (X @ W1 + b1) @ W2 + b2\n",
" W_eq, b_eq = fn(W1, b1, W2, b2)\n",
" check_shape(W_eq, (d, q)); check_shape(b_eq, (q,))\n",
" check_close(X @ W_eq + b_eq, composed,\n",
" msg=\"collapsed map must reproduce the two-layer output for every X\")\n",
"\n",
"check(\"9.1 affine collapse\", lambda: _check_collapse(collapse_affine))"
]
},
{
"cell_type": "markdown",
"id": "2415598a",
"metadata": {},
"source": [
"Hint 1 (conceptual)Substitute layer 1 into layer 2 and expand: $(XW_1+b_1)W_2+b_2 = X(W_1 W_2) + (b_1 W_2 + b_2)$. Read off $W'$ and $b'$ from that.\n",
"\n",
"Hint 2 (pseudocode)\n",
"\n",
"```python\n",
"W_eq = W1 @ W2 # (d,h) @ (h,q) -> (d,q)\n",
"b_eq = b1 @ W2 + b2 # (h,) @ (h,q) -> (q,), then + (q,)\n",
"```\n",
"\n",
"\n",
"Help: \"shapes (h,) and (q,) not aligned\"You probably added `b2` before pushing `b1` through `W2`. `b1` lives in the hidden space (shape `(h,)`); it must be multiplied by `W2` to land in output space before `b2` is added.\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "a7c0a7fb",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:44.774372Z",
"iopub.status.busy": "2026-06-10T19:14:44.774289Z",
"iopub.status.idle": "2026-06-10T19:14:44.782507Z",
"shell.execute_reply": "2026-06-10T19:14:44.782174Z"
},
"collapsed": true,
"jupyter": {
"source_hidden": true
},
"tags": [
"hide-input"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] 9.1 affine collapse\n",
"Two boring layers compose to one boring layer. Add a non-linearity and they do not.\n"
]
}
],
"source": [
"#@title Solution { display-mode: \"form\" }\n",
"# solution: redefines collapse_affine; the check below re-verifies the reference.\n",
"def collapse_affine(W1, b1, W2, b2):\n",
" W_eq = W1 @ W2 # (d,h) @ (h,q) -> (d,q)\n",
" b_eq = b1 @ W2 + b2 # (h,) @ (h,q) -> (q,), then + b2\n",
" return W_eq, b_eq\n",
"\n",
"check(\"9.1 affine collapse\", lambda: _check_collapse(collapse_affine), required=True)\n",
"print(\"Two boring layers compose to one boring layer. Add a non-linearity and they do not.\")"
]
},
{
"cell_type": "markdown",
"id": "b4038acc",
"metadata": {},
"source": [
"> **Key takeaways**\n",
"> - One neuron is a hyperplane; it solves OR and is stuck at chance on XOR.\n",
"> - Stacking affine layers without a non-linearity gives you exactly one affine layer back (you just proved it with an assert).\n",
"> - The activation between layers is the only source of expressive power; everything else in the chapter is about computing its gradient.\n"
]
},
{
"cell_type": "markdown",
"id": "0ab4f59a",
"metadata": {},
"source": [
"## Part 2: Activations and the forward pass\n",
"\n",
"> **Objectives.** Plot the common activations and their derivatives, see exactly where the gradient vanishes, and run a 2-layer forward pass with per-line shape comments.\n",
"\n",
"Five activations matter historically. **Sigmoid** $\\sigma(x)=1/(1+e^{-x})$ saturates: its derivative $\\sigma'(x)=\\sigma(x)(1-\\sigma(x))$ peaks at $0.25$ and decays to $0$ for $|x|>4$. **tanh** is sigmoid shifted to be zero-centered. **ReLU** $\\max(0,x)$ has derivative exactly $1$ on the positive side and $0$ on the negative side: no saturation when active, but a neuron stuck negative is dead. **GELU** $x\\,\\Phi(x)$ is a smooth ReLU. We plot the first three and their derivatives, then shade the saturation zones.\n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "43e16a4c",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:44.785969Z",
"iopub.status.busy": "2026-06-10T19:14:44.785892Z",
"iopub.status.idle": "2026-06-10T19:14:45.446191Z",
"shell.execute_reply": "2026-06-10T19:14:45.445819Z"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAArIAAAHqCAYAAAD4TK2HAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAApD9JREFUeJzs3Xd8U1UbwPFfkjbdi5YOSqHsXWjLEBBfRJAhIuBAQbYbVIZVcYA4QEGGIqKCCxcOQFGWyJIlo2VD2RtaSilt6W5y3z/ShpYWKG2am7TP9/OJubk5954nqaRPT557jkZRFAUhhBBCCCHsjFbtAIQQQgghhCgNSWSFEEIIIYRdkkRWCCGEEELYJUlkhRBCCCGEXZJEVgghhBBC2CVJZIUQQgghhF2SRFYIIYQQQtglSWSFEEIIIYRdclA7AGszGo2cP38eDw8PNBqN2uEIIYQQQogCFEUhNTWVatWqodXefMy10iWy58+fJyQkRO0whBBCCCHETZw5c4bq1avftE2lS2Q9PDwA05vj6empcjRCCCGEEKKglJQUQkJCzDnbzVS6RDa/nMDT01MSWSGEEEIIG1WSElC52EsIIYQQQtglSWSFEEIIIYRdsstE9ty5czz++OP4+vri4uJCs2bN2LFjh9phCSGEEEIIK7K7GtmkpCTat2/P3XffzfLly6latSpHjhzBx8fHov0YDAZycnIsek5RNo6Ojuh0OrXDEEIIIYSNsLtE9oMPPiAkJISvv/7avK9WrVoWO7+iKMTFxXHlyhWLnVNYjre3N4GBgTIHsBBCCGEliqKQZcjC2cFZ7VCKsLtEdsmSJXTt2pWHH36Y9evXExwczHPPPceTTz5ZbPusrCyysrLMj1NSUm56/vwk1t/fH1dXV0mYbISiKKSnp3Px4kUAgoKCVI5ICCGEqPhiL8fy4fYPqeJchSn/m6J2OEXYXSJ7/Phx5syZw5gxY3jttdfYvn07L7zwAnq9nsGDBxdpP3nyZCZOnFiicxsMBnMS6+vra+nQRRm5uLgAcPHiRfz9/aXMQAghhCgnCekJzNo5i9+P/o6CgpPOifi0eALcAtQOrRCNoiiK2kHcDr1eT8uWLdm8ebN53wsvvMD27dvZsmVLkfbFjciGhISQnJxcZB7ZzMxMTpw4QWhoqDlpErYlIyODkydPUqtWLZydbe8rDiGEEMKeZeRmMH//fL7c9yUZuRkAdAvtxqjIUQS7B1slhpSUFLy8vIrN1a5ndyOyQUFBNG7cuNC+Ro0asXDhwmLbOzk54eTkdFt9SDmB7ZKfjRBCCGF5RsXIshPLmBk9k/j0eADC/MKIahVFC/8W6gZ3E3aXyLZv355Dhw4V2nf48GFq1qypUkRCCCGEEPYrJj6Gqdunsi9xHwBBbkGMjhxNt9BuNj+AZHfzyI4ePZr//vuPSZMmcfToUX788Ue++OILRowYoXZoNmnIkCH07t1b7TAACA0NZebMmTdto9Fo+P33360SjxBCCFGZnUk9w5h1Yxi8YjD7Evfh6uDKixEvsqT3ErrX6m7zSSzY4Yhsq1atWLx4MePGjePtt9+mVq1azJw5kwEDBqgdmk366KOPsJUy6O3bt+Pm5qZ2GEIIIUSllpqdytw9c/n+4PfkGHPQarT0qduHkeEj8XPxUzu822J3iSxAz5496dmzp9ph2AUvLy+1QzCrWrWq2iEIIYQQlVauMZffDv/Gp7s+JSkrCYA7gu7gpZYv0aBKA5WjKx27Ky0Qxfvtt99o1qwZLi4u+Pr60rlzZ9LS0oqUFqSmpjJgwADc3NwICgpixowZdOzYkVGjRpnbhIaG8u677zJo0CDc3d2pWbMmS5YsISEhgQceeAB3d3fCwsKKLAu8cOFCmjRpgpOTE6GhoUybNq3Q89eXFhw5coS77roLZ2dnGjduzKpVq8rjrRFCCCEqvQ1nN/Dgkgd5b+t7JGUlUcurFrPvmc0XXb6w2yQW7HRE1poURSEjx6BK3y6OuhLVp1y4cIHHHnuMKVOm0KdPH1JTU9mwYUOxJQVjxoxh06ZNLFmyhICAAMaPH09MTAwtWrQo1G7GjBlMmjSJN998kxkzZjBw4EDatWvHsGHDmDp1Kq+88gqDBg1i//79aDQaoqOjeeSRR3jrrbfo168fmzdv5rnnnsPX15chQ4YUicNoNNK3b18CAgLYunUrycnJhZJpIYQQQpTdkaQjTNsxjU3nNwHg7eTNcy2e46H6D+GodVQ5urKTRPYWMnIMNB6/UpW+D7zdFVf9rX9EFy5cIDc3l759+5pnb2jWrFmRdqmpqXz77bf8+OOP3HPPPQB8/fXXVKtWrUjbHj168PTTTwMwfvx45syZQ6tWrXj44YcBeOWVV2jbti3x8fEEBgYyffp07rnnHt58800A6tevz4EDB5g6dWqxiew///xDbGwsK1euNPc/adIkunfvXoJ3RgghhBA3cynjEp/u+pSFRxZiVIw4aB0Y0HAATzV/Ck/9zedmtSdSWlABNG/enHvuuYdmzZrx8MMPM3fuXJKSkoq0O378ODk5ObRu3dq8z8vLiwYNin6lEBYWZt4OCDCt4lEwOc7fl79k7MGDB2nfvn2hc7Rv354jR45gMBQd0T548CAhISGFkui2bduW6PUKIYQQonhZhizm7Z1Hz8U9+fXwrxgVI11qdmHJA0t4qdVLFSqJBRmRvSUXRx0H3u6qWt8lodPpWLVqFZs3b+bvv/9m1qxZvP7662zdurXUfTs6Xvu6Ib+8obh9RqOx1H0IIYQQwjIURWHlyZXMiJ7B+bTzADT2bUxUyyhaBrZUObryI4nsLWg0mhJ9va82jUZD+/btad++PePHj6dmzZosXry4UJvatWvj6OjI9u3bqVGjBgDJyckcPnyYu+66q0z9N2rUiE2bNhXat2nTJurXr49OVzQhb9SoEWfOnOHChQsEBQUB8N9//5UpBiGEEKIy2pOwhynbp7A7YTcA/q7+jIoYxX2170Orqdhfvtt+hiZuaevWraxevZp7770Xf39/tm7dSkJCAo0aNWLPnj3mdh4eHgwePJioqCiqVKmCv78/EyZMQKvVlnnS47Fjx9KqVSveeecd+vXrx5YtW/jkk0/49NNPi23fuXNn6tevz+DBg5k6dSopKSm8/vrrZYpBCCGEqEzOXz3PzJiZLD+xHAAXBxeGNh3K4MaDcXV0VTk666jYaXol4enpyb///kuPHj2oX78+b7zxBtOmTSv2wqnp06fTtm1bevbsSefOnWnfvj2NGjXC2dm5TDFERETwyy+/sGDBApo2bcr48eN5++23i73QC0Cr1bJ48WIyMjJo3bo1TzzxBO+9916ZYhBCCCEqg7ScND6O+Zhev/di+YnlaNDQu25v/urzF882f7bSJLEAGsVWln2ykpSUFLy8vEhOTsbTs3DBc2ZmJidOnKBWrVplTuzsRVpaGsHBwUybNo3hw4erHc4tVcafkRBCCAFgMBpYfHQxs3bO4nLmZQBaBbYiqmUUjXwbqRyd5dwsV7uelBZUMjt37iQ2NpbWrVuTnJzM22+/DcADDzygcmRCCCGEuJEt57cwdcdUjiQdAaCGRw3GthzL3SF3l7k80J5JIlsJffjhhxw6dAi9Xk9kZCQbNmzAz8++1lYWQgghKoPjV44zLXoa/579FwAPvQfPNn+WRxs8iqPO/hc0KCtJZCuZ8PBwoqOj1Q5DCCGEEDeRlJnEnN1z+OXQLxgUAw4aBx5t+ChPhz2Nt7O32uHZDElkhRBCCCFsRLYhm59if+Lz3Z+TmpMKQMeQjoyNHEuoV6i6wdkgSWSFEEIIIVSmKAqrT69mevR0zqSeAaCBTwOiWkXRJqiNytHZLklkhRBCCCFUtP/SfqZsn0LMxRgA/Fz8eCH8BXrV6YVOW7JVPisrSWSFEEIIIVQQlxbHrJ2zWHJsCQBOOieGNBnCsKbDKtVcsGUhiawQQgghhBWl56Tz9f6v+WbfN2QaMgHoWbsnL0a8SKBboMrR2RdJZMVNhYaGMmrUKEaNGqV2KEIIIYRdMypGlhxbwscxH5OQkQBAhH8EUa2iaOrXVOXo7JMkshVEx44dadGiBTNnzlQ7FCGEEEJcZ3vcdqZun8rBywcBCHYPZkzkGLrU7FKpFzQoK0lkhRBCCCHKyamUU0zfMZ01Z9YA4O7oztNhT9O/UX/0Or3K0dk/rdoBiLIbMmQI69ev56OPPkKj0aDRaDh27BjDhw+nVq1auLi40KBBAz766KMix/Xu3ZsPP/yQoKAgfH19GTFiBDk5OYXapaenM2zYMDw8PKhRowZffPGFNV+eEEIIYXeSs5L5YNsH9P69N2vOrEGn0dGvQT+W9l3KkKZDJIm1EBmRrQA++ugjDh8+TNOmTXn77bcB8PHxoXr16vz666/4+vqyefNmnnrqKYKCgnjkkUfMx65du5agoCDWrl3L0aNH6devHy1atODJJ580t5k2bRrvvPMOr732Gr/99hvPPvss//vf/2jQoIHVX6sQQghhy3KMOfxy6Bfm7J5DclYyAB2COzC25VjqeNdRObqKRxLZW1EUyElXp29HVyhB3YyXlxd6vR5XV1cCA69d7Thx4kTzdq1atdiyZQu//PJLoUTWx8eHTz75BJ1OR8OGDbnvvvtYvXp1oUS2R48ePPfccwC88sorzJgxg7Vr10oiK4QQQuRRFIX1Z9czbcc0TqacBKCud12iWkbRLridusFVYJLI3kpOOkyqpk7fr50HvVupD589ezZfffUVp0+fJiMjg+zsbFq0aFGoTZMmTdDprk22HBQUxN69ewu1CQsLM29rNBoCAwO5ePFiqeMSQgghKpLYy7F8uP1DtsZtBaCKcxVGho+kT90+OGgl1SpP8u5WUAsWLOCll15i2rRptG3bFg8PD6ZOncrWrVsLtXN0dCz0WKPRYDQab7uNEEIIUdkkpCcwa+csfj/6OwoKeq2egY0H8kSzJ3DXu6sdXqUgieytOLqaRkbV6ruE9Ho9BoPB/HjTpk20a9fOXBIAcOzYMYuGJ4QQQlRGGbkZzN8/ny/3fUlGbgYA3UK7MSpyFMHuwSpHV7lIInsrGk2Zvt63ltDQULZu3crJkydxd3enXr16zJ8/n5UrV1KrVi2+++47tm/fTq1atdQOVQghhLBLRsXIshPLmBk9k/j0eADC/MKIahVFC/8W6gZXSUkiW0G89NJLDB48mMaNG5ORkUFsbCw7d+6kX79+aDQaHnvsMZ577jmWL1+udqhCCCGE3YmJj2Hq9qnsS9wHQJBbEKMjR9MttJssaKAijaIoitpBWFNKSgpeXl4kJyfj6elZ6LnMzExOnDhBrVq1cHZ2VilCcTPyMxJCCGFNZ1LPMDN6Jn+f+hsAVwdXngx7kscbPY6zg/weKg83y9WuJyOyQgghhBDXSc1OZe6euXx/8HtyjDloNVr61O3DyPCR+Ln4qR2eyCOJrBBCCCFEnlxjLr8d/o1Pd31KUlYSAHcE3cFLLV+iQRWZP93WSCIrhBBCCAFsOLuBD3d8yPHk4wDU8qrFSy1fokNwB6mDtVGSyAohhBCiUjuSdIRpO6ax6fwmALydvHmuxXM8VP8hHLWOtzhaqEkSWSGEEEJUSpcyLvHprk9ZeGQhRsWIg9aBAQ0H8FTzp/DU3/wiI2EbJJEVQgghRKWSZcjiuwPfMW/vPNJy0gDoUrMLoyNGE+IZonJ04nZIIiuEEEKISkFRFFaeXMmM6BmcTzOt2tnYtzFRLaNoGdhS5ehEaUgiK4QQQogKb0/CHqZsn8LuhN0A+Lv6MypiFPfVvg+tRqtydKK0JJEVQgghRIV1/up5ZsbMZPkJ08qWLg4uDG06lMGNB+Pq6KpydKKsJJGtxDp27EiLFi2YOXOm2qEIIYQQFpWWk8aXe79k/oH5ZBmy0KDhgboP8Hz48/i7+qsdnrAQSWSFEEIIUWEYjAYWH13MJzs/ITEzEYBWga2IahlFI99GKkcnLE0S2QoqOzsbvV6vdhhCCCGE1Ww5v4WpO6ZyJOkIADU8ajC25VjuDrlbFjSooKS6uYLo2LEjI0eOZNSoUfj5+dG1a1f27dtH9+7dcXd3JyAggIEDB3Lp0qUbnkOj0fD7778X2uft7c0333xTvsELIYQQZXD8ynFGrB7BU6ue4kjSETz0Hrzc6mV+f+B3OtXoJElsBWbXiez777+PRqNh1KhRaodiE7799lv0ej2bNm3i/fffp1OnToSHh7Njxw5WrFhBfHw8jzzyiNphCiGEEBaRlJnEpK2T6LukL/+e/RcHjQOPN3qcZX2WMbDxQBx1sipXRWe3pQXbt2/n888/JywsrFz7URSFjNyMcu3jRlwcXG7rr8h69eoxZcoUAN59913Cw8OZNGmS+fmvvvqKkJAQDh8+TP369S0erxBCCGEN2YZsfor9ic93f05qTioAHUM6MjZyLKFeoeoGJ6zKLhPZq1evMmDAAObOncu7775brn1l5GbQ5sc25drHjWztv/W2pgaJjIw0b+/evZu1a9fi7u5epN2xY8ckkRVCCGF3FEVh9enVTI+ezpnUMwA08GlAVKso2gSp87taqMsuE9kRI0Zw33330blz51smsllZWWRlZZkfp6SklHd4qnFzczNvX716lfvvv58PPvigSLugoKBij9doNCiKUmhfTk6OZYMUQgghSmH/pf1M2T6FmIsxAPi5+PFC+Av0qtMLnVancnRCLXaXyC5YsICYmBi2b99eovaTJ09m4sSJpe7PxcGFrf23lvr4snBxcCn1sRERESxcuJDQ0FAcHEr2Y65atSoXLlwwPz5y5Ajp6emljkEIIYQoq7i0OGbtnMWSY0sAcNI5MaTJEIY1HSYLGgj7SmTPnDnDiy++yKpVq3B2di7RMePGjWPMmDHmxykpKYSEhJS4T41GY5f/UEaMGMHcuXN57LHHePnll6lSpQpHjx5lwYIFzJs3D52u6F+vnTp14pNPPqFt27YYDAZeeeUVHB2lUF4IIYT1peek8/X+r/lm3zdkGjIB6Fm7Jy9GvEigW6DK0QlbYVeJbHR0NBcvXiQiIsK8z2Aw8O+///LJJ5+QlZVVJEFzcnLCycnJ2qGqrlq1amzatIlXXnmFe++9l6ysLGrWrEm3bt3QaoufrGLatGkMHTqUDh06UK1aNT766COio6OtHLkQQojKzKgYWXJsCR/HfExCRgIAEf4RRLWKoqlfU5WjE7ZGo1xfFGnDUlNTOXXqVKF9Q4cOpWHDhrzyyis0bXrr/8FTUlLw8vIiOTkZT0/PQs9lZmZy4sQJatWqVeIRX2Fd8jMSQoiKa3vcdqZun8rBywcBCHYPZkzkGLrU7CJzwVYiN8vVrmdXI7IeHh5FklU3Nzd8fX1LlMQKIYQQwvacSjnF9B3TWXNmDQDuju48HfY0/Rv1R6+TVSrFjdlVIiuEEEKIiiM5K5nPdn/GgtgF5Cq56DQ6Hqr/EM+1eI4qzlXUDk/YAbtPZNetW6d2CEIIIYS4DTnGHH459Atzds8hOSsZgA7BHRjbcix1vOuoHJ2wJ3afyAohhBDCPiiKwvqz65m2YxonU04CUNe7LlEto2gX3E7d4IRdsloiGx8fz0svvcTq1au5ePFikYn3DQaDtUIRQgghhJXFXo7lw+0fsjXONDd7FecqjAwfSZ+6fXDQyriaKB2r/Z8zZMgQTp8+zZtvvklQUJBNX31oRxM5VDrysxFCCPuSkJ7ArJ2z+P3o7ygo6LV6BjYeyBPNnsBdX3QZdSFuh9US2Y0bN7JhwwZatGhhrS5vW/7k/+np6bi4lH5VLVF+8lcak4UahBDCtmXkZjB//3y+3PclGbkZAHQL7caoyFEEuwerHJ2oKKyWyIaEhNj8aJpOp8Pb25uLFy8C4OrqatMjx5WJoiikp6dz8eJFvL29i12ZTAghhPqMipFlJ5YxM3om8enxAIT5hRHVKooW/i3UDU5UOFZLZGfOnMmrr77K559/TmhoqLW6vW2BgaZl7/KTWWFbvL29zT8jIYQQtiUmPoap26eyL3EfAEFuQYyOHE230G4yMCTKhdVW9vLx8SE9PZ3c3FxcXV2LfDV8+fJla4RR4tUiDAYDOTk5VolJlIyjo6OMxAohhA06k3qGmdEz+fvU3wC4OrjyZNiTPN7ocZwdZBVGcXtscmWvmTNnWqsri9DpdJI0CSGEEDeRmp3K3D1z+f7g9+QYc9BqtPSp24eR4SPxc/FTOzxRCVgtkR08eLC1uhJCCCFEOco15vLb4d/4dNenJGUlAXBH0B281PIlGlRpoHJ0ojKx6sRtBoOB33//nYMHDwLQpEkTevXqJSOfQgghhJ3YcHYDH+74kOPJxwGo5VWLl1q+RIfgDlIHK6zOaons0aNH6dGjB+fOnaNBA9Nfa5MnTyYkJISlS5dSp44sSSeEEELYqiNJR5i2Yxqbzm8CwNvJm+daPMdD9R/CUStTIgp1WO1irx49eqAoCj/88ANVqlQBIDExkccffxytVsvSpUutEcZtFRALIYQQlV1iRiKzd81m4ZGFGBUjDloHBjQcwFPNn8JTL79HheXZ5MVe69ev57///jMnsQC+vr68//77tG/f3lphCCGEEKIEsgxZfH/ge+bunUtaThoAXWp2YXTEaEI8Q1SOTggTqyWyTk5OpKamFtl/9epV9Hq9tcIQQgghxE0oisLKkyuZET2D82nnAWjs25iollG0DGypcnRCFGa1RLZnz5489dRTfPnll7Ru3RqArVu38swzz9CrVy9rhSGEEEKIG9iTsIcp26ewO2E3AP6u/oyKGMV9te9Dq9GqHJ0QRVktkf34448ZPHgwbdu2NS+GkJubS69evfjoo4+sFYYQQgghrnP+6nlmxsxk+YnlALg4uDCs6TAGNxmMi4OLytEJcWNWS2S9vb35448/OHLkCLGxsQA0atSIunXrWisEIYQQQhSQlpPGl3u/ZP6B+WQZstCg4YG6D/B8+PP4u/qrHZ4Qt2TVeWQB6tWrR7169azdrRBCCCHyGIwGFh9dzCc7PyExMxGAVoGtiGoZRSPfRipHJ0TJlWsiO2bMGN555x3c3NwYM2bMTdtOnz69PEMRQgghBLDl/Bam7pjKkaQjANTwqMHYlmO5O+RuWdBA2J1yTWR37txJTk6OeVsIIYQQ6jh+5TjToqfx79l/AfDUe/Js82fp16AfjjpZ0EDYJ6stiGArZEEEIYQQlUlSZhJzds/hl0O/YFAMOGgceLThozzT/Bm8nLzUDk+IIm4nV7PaXBrDhg0rdh7ZtLQ0hg0bZq0whBBCiEoh25DNt/u/5b5F9/FT7E8YFAMdQzqy+IHFvNL6FUliRYVgtRFZnU7HhQsX8PcvfBXkpUuXCAwMJDc31xphyIisEEKICk1RFFafXs306OmcST0DQAOfBkS1iqJNUBuVoxPi1mxqidqUlBQURUFRFFJTU3F2djY/ZzAYWLZsWZHkVgghhBC3b3/ifqZun0p0fDQAfi5+vBD+Ar3q9EKn1akcnRCWV+6JrLe3NxqNBo1GQ/369Ys8r9FomDhxYnmHIYQQQlRYcWlxzNo5iyXHlgDgrHNmcJPBDGs6DFdHV5WjE6L8lHsiu3btWhRFoVOnTixcuJAqVaqYn9Pr9dSsWZNq1aqVdxhCCCFEhZOek87X+7/mm33fkGnIBKBn7Z68GPEigW6BKkcnRPkr90T2f//7HwAnTpwgJCQErVbWahZCCCHKwqgYWXJsCR/HfExCRgIAEf4RRLWKoqlfU5WjE8J6rLayV82aNQFIT0/n9OnTZGdnF3o+LCzMWqEIIYQQdmt73Hambp/KwcsHAQh2D2ZM5Bi61OwiCxqISsdqiWxCQgJDhw5l+fLlxT5vMBisFYoQQghhd06lnGL6jumsObMGAHdHd54Oe5r+jfqj1+lVjk4IdVgtkR01ahRXrlxh69atdOzYkcWLFxMfH8+7777LtGnTrBWGEEIIYVeSs5L5bPdnLIhdQK6Si06j46H6D/Fci+eo4lzl1icQogKzWiK7Zs0a/vjjD1q2bIlWq6VmzZp06dIFT09PJk+ezH333WetUIQQQgibl2PM4ZdDvzBn9xySs5IB6BDcgbEtx1LHu47K0QlhG6yWyKalpZnni/Xx8SEhIYH69evTrFkzYmJirBWGEEIIYdMURWH92fVM2zGNkyknAajrXZeollG0C26nbnBC2BirJbINGjTg0KFDhIaG0rx5cz7//HNCQ0P57LPPCAoKslYYQgghhM2KvRzLh9s/ZGvcVgCqOFdhZPhI+tTtg4PWar+yhbAbVvtX8eKLL3LhwgUAJkyYQLdu3fjhhx/Q6/V888031gpDCCGEsDkJ6Ql8susTFh9ZjIKCXqtnYOOBPNHsCdz17mqHJ4TN0iiKoqjRcXp6OrGxsdSoUQM/Pz+r9Xs76/cKIYQQ5SkjN4P5++fz5b4vycjNAKB7aHdejHyRYPdglaMTQh23k6tZbUR248aN3HnnnebHrq6uREREWKt7IYQQwmYYFSPLTixjZvRM4tPjAQjzCyOqVRQt/FuoG5wQdsRqiWynTp0IDg7mscce4/HHH6dx48bW6loIIYSwGTHxMUzdPpV9ifsACHILYnTkaLqFdpMFDYS4TVZLZM+fP8+CBQv46aefeP/99wkLC2PAgAE89thjVK9e3VphCCGEEKo4k3qGmdEz+fvU3wC4OrjyZNiTPN7ocZwdnFWOTgj7pEqN7IkTJ/jxxx/56aefiI2N5a677mLNmjVW6VtqZIUQQlhTanYqc/fM5fuD35NjzEGr0dKnbh9Gho/Ez8V614gIYS9uJ1dT7WIvg8HA8uXLefPNN9mzZ4/VlqiVRFYIIYQ15Bpz+e3wb3y661OSspIAuCPoDl5q+RINqjRQOTohbJdNXuyVb9OmTfzwww/89ttvZGZm8sADDzB58mRrhyGEEEKUmw1nN/Dhjg85nnwcgFpetXip5Ut0CO4gdbBCWJDVEtlx48axYMECzp8/T5cuXfjoo4944IEHcHV1va3zTJ48mUWLFhEbG4uLiwvt2rXjgw8+oEED+etWCCGEuo4kHWHajmlsOr8JAG8nb55r8RwP1X8IR62jytEJUfFYLZH9999/iYqK4pFHHinTvLHr169nxIgRtGrVitzcXF577TXuvfdeDhw4gJubmwUjFkIIIUomMSOR2btms/DIQoyKEQetAwMaDuCp5k/hqZcyNiHKi2o1spaSkJCAv78/69ev56677rple6mRFUIIYSlZhiy+P/A9c/fOJS0nDYAuNbswOmI0IZ4hKkcnhH2ymRrZJUuW0L17dxwdHVmyZMlN2/bq1atUfSQnJwNQpUqVUh0vhBBC3C5FUVh5ciUzomdwPu08AE18mxDVKorIgEiVoxOi8ijXEVmtVktcXBz+/v5otdobB6HRlGrWAqPRSK9evbhy5QobN24stk1WVhZZWVnmxykpKYSEhMiIrBBCiFLZk7CHKdunsDthNwD+rv6MihjFfbXvQ6u58e86IUTJ2MyIrNFoLHbbUkaMGMG+fftumMSC6eKwiRMnWrxvIYQQlcv5q+eZGTOT5SeWA+Di4MKwpsMY3GQwLg4uKkcnROVktT8d58+fX2hkNF92djbz58+/7fONHDmSv/76i7Vr1950ZbBx48aRnJxsvp05c+a2+xJCCFF5peWk8XHMx/T6vRfLTyxHg4bedXvzV5+/eKb5M5LECqEiq13spdPpuHDhAv7+/oX2JyYm4u/vX+LSAkVReP7551m8eDHr1q2jXr16txWHXOwlhBCiJAxGA78f/Z1ZO2eRmJkIQKvAVkS1jKKRbyOVoxOi4rKZ0oKCFEUpdhLos2fP4uXlVeLzjBgxgh9//JE//vgDDw8P4uLiAPDy8sLFRf4qFkIIUXZbzm9h6o6pHEk6AkBNz5qMiRzD3SF3y4IGQtiQck9kw8PD0Wg0aDQa7rnnHhwcrnVpMBg4ceIE3bp1K/H55syZA0DHjh0L7f/6668ZMmSIJUIWQghRSR2/cpxp0dP49+y/AHjqPXm2+bP0a9APR50saCCErSn3RLZ3794A7Nq1i65du+Lu7m5+Tq/XExoayoMPPlji89n5tLdCCCFsUFJmEnN2z+GXQ79gUAw4aBx4tOGjPNP8GbycSv6toRDCuso9kZ0wYQIAoaGhPProozg5OZV3l0IIIUSJZBuy+Sn2Jz7f/TmpOakAdAzpyNjIsYR6haobnBDilqxWI9u4cWN27dpFmzZtCu3funUrOp2Oli1bWisUIYQQlZyiKKw+vZrp0dM5k2qazaaBTwOiWkXRJqjNLY4WQtgKq02/NWLEiGKnvjp37hwjRoywVhhCCCEquf2J+xm6ciij143mTOoZ/Fz8eLvd2/zc82dJYoWwM1YbkT1w4AARERFF9oeHh3PgwAFrhSGEEKKSikuLY9bOWSw5Zloy3VnnzOAmgxnWdBiujq4qRyeEKA2rJbJOTk7Ex8dTu3btQvsvXLhQaCYDIYQQwpLSc9L5ev/XfLPvGzINmQD0rN2TFyNeJNAtUOXohBBlYbUM8t5772XcuHH88ccf5nljr1y5wmuvvUaXLl2sFYYQQohKwqgYWXJsCR/HfExCRgIAEf4RRLWKoqlfU5WjE0JYgtUS2Q8//JC77rqLmjVrEh4eDpim5AoICOC7776zVhhCCCEqge1x25m6fSoHLx8EINg9mDGRY+hSs4ssaCBEBWK1RDY4OJg9e/bwww8/sHv3blxcXBg6dCiPPfYYjo4yybQQQoiyO5Vyiuk7prPmzBoA3B3deTrsafo36o9ep1c5OiGEpVm1ONXNzY0777yTGjVqkJ2dDcDy5csB6NWrlzVDEUIIUYEkZyXz+Z7P+Sn2J3KNueg0Oh6q/xDPtXiOKs5V1A5PCFFOrJbIHj9+nD59+rB37140Gg2KohT6esdgMFgrFCGEEBVEjjGHXw79wpzdc0jOSgagQ3AHxrYcSx3vOipHJ4Qob1abR/bFF1+kVq1aXLx4EVdXV/bt28f69etp2bIl69ats1YYQgghKgBFUVh3Zh19/+jL+9veJzkrmbredfm88+d82vlTSWKFqCSsNiK7ZcsW1qxZg5+fH1qtFp1Ox5133snkyZN54YUX2Llzp7VCEUIIYcdiL8fy4fYP2Rq3FYAqzlUYGT6SPnX74KCV6RyFqEys9i/eYDDg4eEBgJ+fH+fPn6dBgwbUrFmTQ4cOWSsMIYQQdiohPYFPdn3C4iOLUVDQa/UMbDyQJ5o9gbveXe3whBAqsFoi27RpU3bv3k2tWrVo06YNU6ZMQa/X88UXXxRZJEEIIYTIl5Gbwfz98/ly35dk5GYA0D20Oy9Gvkiwe7DK0Qkh1GS1RPaNN94gLS0NgLfffpuePXvSoUMHfH19+fnnn60VhhBCCDthVIwsO7GMmdEziU+PByDML4yoVlG08G+hbnBCCJugURRFUavzy5cv4+PjY9XJqVNSUvDy8iI5ORlPT0+r9SuEEKLkYuJjmLp9KvsS9wEQ5BbE6MjRdAvtJgsaCFHB3U6upmpVfJUqMrefEEKIa86knmFm9Ez+PvU3AK4OrjwZ9iSPN3ocZwdnlaMTQtgaubxTCCGE6lKzU5m7Zy7fH/yeHGMOWo2WvvX6MqLFCPxc/NQOTwhhoySRFUIIoZpcYy4LDy9k9q7ZJGUlAdA2qC0vtXqJ+j71VY5OCGHrJJEVQgihig1nN/Dhjg85nnwcgFpetXip5Ut0CO5QMepgFQUMOWDIgty8W8FtowGMuYVvivG6fcW0KbhPUUzHKEZAyXusFHhsLKaN8Vq7W7a57rwohV/ftQdFX3sxzymAYjSa7hVQFKPpHtMiF4qiFPOckvdYMT8mrx1K/tmv7St46Y+S18i8S1FQCsaT17n5HPl9ARrlWkulUJtrL08peJ7iXvENrkK6fnfh9krx7ZRiNwuf50ZxFGpzg4MLvTMUeF9MDI7uNH/x1xsdrBpJZIUQQljVkaQjTNsxjU3nNwHg7eTNcy2e46H6D+GodbR+QIoCWamQcRkykiAzBbKvQnaaaX922o0fZ6cVTVALbt8w5aicNHk3YX+SsM0L5CWRFUIIYRWJGYnM3jWbhUcWYlSMOGgdGNBwAE81fwpPvYV/SeZmw9U4SI2H1AuQGme6vxoP6ZevJa35N2OuZfsvhqJ1xKjVY9Q5YdA6YkSHQaPDgI5ctKZ7RUuuoiMHLTmKNu+xaTtH0ZKdf2/UkGPUYkCDUdGgoMGIJm9MUYsCGAvdF2xjuqfAMUa01/Yphdteu5HXJu/1XJeSKoWeK6iE7TQatBoNGjDda02PtZoCz+XdtKZdoNGiyd/G9Bwa0OadL39gX4OWvKdAA5qCxxXYzv8mwPT42vlM74zpfNr8NpqC7Qq/zmsxmV9agXdCU2R/wb0FT3X9NxP5MRT3zhbqQ6Mp0qbQeQsdd308haPMf1rn6Myd2B5JZIUQQpSrLEMW3x/4nrl755KWY5pPvEvNLoyOGE2IZ0gpT3oVrpyCpFOQdPLaLeWcKWFNT7z9czq4gIsPOHuhOLljcHAlR+dGts6FTK0LGbiQgTNXFWdSFSdSjE6k5DqSkutAcraW5BwtV7I1XMnW5t00ZCqOZOFINg4opvSq3OgdtDg5aHF21BW51+u0ODlqcdBq0TtocNBqcdQV3nbUafLutTjoNOjz9jnotOjz9uU/f31bnVaDQ17iad4usM9Bp0GnKbCv4HNa0zEVopxEWJ0kskIIIcqFoiisPLmSGdEzOJ92HoAmvk2IahVFZEBkyU6SfhkSYvNuh033lw6bEtZb0TqCRxCKRwDZLv5kOFUl1dGPFK0nVxR3Eo3uXDS4EJ/txvksZxIytSSlZ5OUlENyRjY5BsuVBbg46nBz0uGi1+Gmd8BVr8PNySFv//WPdbjoHXB11OHkqMXZIe++mAS1YKKq1UoiKCofSWSFEEJY3J6EPUzZPoXdCbsB8Hf1Z1TEKO6rfR9azQ1GJq8mwIVdcH6X6f7Cbkg+c8M+jE7eZHqEkOpSncv6asRrAzir+HEmx4sTmR6cSHfiUmo2SfE5t4g2M+9WlF6nxcPZAU8XRzycHUzbzvnbjkUee7qYHrs5OeDmpMNVb0pOdZJkClEuJJEVQghhMeevnmdmzEyWn1gOgIuDC8OaDmNwk8G4OLhca6gocOkInN4MpzbDqS2QfLrYc17RB3LesSbHCSY2txq7swLYkxlAcqY7JN8okty8m4lGA57Ojvi4OuLtqsfH1REfV71529ut4D7TvY+rHhe9zjJvjBCiXEgiK4QQoszSctL4cu+XzD8wnyxDFho0PFD3AZ4Pfx5/V38Asi6dIHXvcjTH1+IWvwPn7MuFzmFEw3FjEPuUUPYaa7FfqcV+Yyipma7F9umg1eDrrsfP3Qk/dyeqejjlbesLbJsee7vqZVRUiApIElkhhBClZjAa+P3o78zaOYvETNMFVnU9m3OnzzCMV3z59cffCUncRLOM7dTiHE4Fjs1UHNml1GWrsSHbjQ3ZaaxLGi44aDUEeDoT4OnEXV4uBHg6E+jlRICnM1U9nKial6B6uThKXagQlZwkskIIIUokLSuX05fTOZuUwbmkdHbEb2V76rekcxYAY5YfhotdqJqeST3dFO7RxuCmyTIfn6to2UV9djtFctarJel+zajq7UmAlzODPZ151cuZAE9nfN30kqAKIUpEElkhhBCAaZaBhKtZnE5M5/TldE6Z79M4fTmdS1ezAdDqL+LkvwwHj1jTcQYXal2qx5DURLrpPsJNf+3CqTS9H5er/Q9jnXvwaNSFSN+qtJRploQQFiKJrBBCVCI5BiPnr2RwKjGdU5fTOZ2YZk5YT19OJz3bcMNjNbo0PALXoHhsAY0RLVq6Gf2JijuEX/YhyL8uyisEGj8ATfrgFhyJmySuQohyIomsEEJUMGlZuQWS07QCI6vpnLuSgcF44/lRtRoI8nKhpq8rNX1dqVHFjWAfBw6kLeePk99yNecqAB0NDow9f5rQ3JOmAz2DoUkf0y04svCSRUIIUU4kkRVCCDt0JT2bk4mmr/1PJaZzMjGN04npnExM59LVrJse6+SgpUaVa4lqTV9Xavi6UrOKK9V9XNE7mOZ5VRSF1adXMz16OmdSTfO5NsjOISrxMm0ys0CnNyWu4QOhdkfQylRVQgjrkkRWCCFskKIoJKRmFUpWT+XVq568lEZKZu5Nj/dxdaSGrxs1zQmrKzV9TUlrVXenW15MtT9xP1O3TSX6YjQAfrkGXki6Qq+raej8m0DHgRDWD1yrWOw1CyHE7ZJEVgghVGIwKgXqVfOS1fykNTGdjJwb16sCBHg6UdPXjVDfa0lqzSpu1PB1xcvFsVQxxaXFMWvHDJacXAaAs9HI4ORUhqVcxbXh/XDHsxDSRkoHhBA2QRJZIYQoRymZOZzJm7Iq/z4/WT2TlE6O4eb1qsE+LoQWSFJr5iWtNaq4WnTVqfScdL7e/TnfHPiWTMWUQPe8msaLaQYCWwyG1k+Cdw2L9SeEEJYgiawQQpRBenZuoSTVfJ9kuk/OyLnp8XqdlpAq+cmqm/kiq5q+bgR7u5jrVcuLUTGy5OACPo6ZSYIhA4CIzEyisp1p2mYcNH8MnNzLNQYhhCgtSWSFEOIGFEUhJSOXCykZXLiSybkrGYWS1LOX00lMy77leXzd9FT3caF6FVdCfFwLJauBns6qLZ26/dRapm4az8GcKwAE5+QyJseZLne8hSbsEdCVrjxBCCGsRRJZIUSldTUrlwtXMjifnGm+j0vO4EJyJuevmO5vNq9qPk9nB6r7uBJSxYUQH1eq+7gQkjcDQHUfF9ycbOuj9lRiLNPXjmVN2mkA3I1Gns5xpv8db6Bv+pDMPiCEsBu29ekqhBAWcDUrl4TULC6mZHIxNcu0nZrFxdTMvP1ZnE/OIPUWV/7n83F1JNDLhWpezkWS1JAqpb+wytqS0y7x+ZrR/JS4k1yNBp2i8FCOI8+1fpkqzfqBtnzLGIQQwtIkkRVC2DxFUUjJzCUpLZvEtGyS0rK5nJbNpTRTUppgTlZNiWtJRlHzeTo7UM3bhUAvZ4LyktUgbxeCvJzzbi4WvahKDTm5Wfyy7nXmnFlJshbQaOiQA2ObPU2dVs/JCKwQwm5JIiuEsCqDUeFqZi7JGTkkZ+SQkplDUropMb3RLSk9+6ZX9xfHTa+jqocT/h7OVPV0wj9/28O0Xc3blKTa2tf+lqQoCutjPmPans85qTWAFurmGomq1Yd2d40HB73aIQohRJnY7Sf47NmzmTp1KnFxcTRv3pxZs2bRunVrtcMSokIzGhXScwykZeVyNSu3wL1pX2pmfnKaS3J6TqFkNX/7alYuyu3lpGaueh0+rnp83fVUcTPdCian/h5O+Hs64+/hVKET1JKIPb6KDzeOZ6tyFbRQxWBkpH87+nSehoOzp9rhCSGERdjlJ/3PP//MmDFj+Oyzz2jTpg0zZ86ka9euHDp0CH9/f7XDE0JVBqNCZo6BjBwDmeabkYwcAxnZhuueM5q307MN5uT0+gTVvP82vrK/FRdHHZ4uDni5OOLtYkpKfdz0+LpdS1Kvvzk7ylfgt5KQeIRP/nmBxRlnUDQa9IrCQJdQnrh3Fu4+tdQOTwghLEqjKKUdG1FPmzZtaNWqFZ988gkARqORkJAQnn/+eV599dWbHpuSkoKXlxfJycl4esqohLg1o1HBoCgYjAq5RgWDQSHXaCT3usfm5833RnIN1x7nGo1k5ypkG4xk5xrJybvPzjWa92UbjORc9zi/TY6h4H4lb7+BHINCVm5+kmpqU960GnBzcsDdyQFXvQ53JwfcnBzwcDYlpvk3z+vvnfMfO+DkIEmpJWVmJvPtP6P4MmE7GXnTeXXXePDiXZMJDv2fytEJIUTJ3U6uZncjstnZ2URHRzNu3DjzPq1WS+fOndmyZYuKkRXv9/07OJp4zvxYybuR9+eDouTvK/D3hHLtsVKgHXntCv3lkX98/k5FudaHua9rR1zrj+v2KYV2Kopy3TmKxlToNRV4/vqYrh2noChgND+voCgKhryAjHl9GvOCNJLX3mg6hzHv+PxzG83nU66du2A/eccrxgLbipJ3LtN+g6KYE1VjXrJqzNtvMJr2Ge3lTz2d6VYwPXTUaXBy0OKo0+LkoEPvoMXRQYOTToveQYdep0XvoEXvYGrn7KjDxVGHs/mmNW8X3K/XadAUWaLUkHfLKja8DCAjB+JzgJTyeAMqp/NXz7Mp9lf+SzpImgbQaggz6IgKf4EW4cPUDk8IIcqV3SWyly5dwmAwEBAQUGh/QEAAsbGxRdpnZWWRlXXtF2tKinV/g34SM5d442ar9lkpaPJu5SQvJ6wQjJhSy2LTSyOQnXcT9k0D1QwKo2r1ottdb6PR2d3HuxBC3LYK/0k3efJkJk6cqFr/we7VSL5Sw5x03Sz3KvKc5mYPb/boVsfeoO9r/7lxmxvsKNn5NYXbaq5tawrsKHguTaE+NEX2Fz5XgecLPKkpdEzhNubzaDSFH6Mp8FzeuTUF8ufr+hPCqnKzIPUCZCbjbjTSJtvAnQ0epFHH8WidPNSOTgghrMbuElk/Pz90Oh3x8fGF9sfHxxMYGFik/bhx4xgzZoz5cUpKCiEhIeUeZ75v+0ywWl9CiAou/TL8+yHsmguGbEAD4Y9DpzfAo+jnnxBCVHR2l8jq9XoiIyNZvXo1vXv3BkwXe61evZqRI0cWae/k5ISTk5OVoxRCCAvKyYCtn8GGGZCVbNpX+264910IbKpubEIIoSK7S2QBxowZw+DBg2nZsiWtW7dm5syZpKWlMXToULVDE0IIyzEaYPcCWPsepORdNBrQDLq8BXU7qxqaEELYArtMZPv160dCQgLjx48nLi6OFi1asGLFiiIXgAkhhF1SFDiyCv55Cy7uN+3zCjGVEDR7BLRaVcMTQghbYZfzyJaFzCMrhLBp52Jg1Xg4ucH02NkLOrwErZ8CR2d1YxNCCCuo0PPICiFEhXRhN6x7Hw4tMz3WOUGbp+DOMeBaRd3YhBDCRkkiK4QQaorbB+smQ+xfpscaLYT1g7tfB2/rzbAihBD2SBJZIYRQQ9w++HcKHPgjb4cGmj0E/3sF/OqpGpoQQtgLSWSFEMJaFAVOboRNM+HoP3k7NdCkjymB9W+oZnRCCGF3JJEVQojyZjTAwT9h00dwPsa0T6OFxr3hrigIaKxqeEIIYa8kkRVCiPKSkQS7foLtc+HycdM+B2fTalxtR0CV2urGJ4QQdk4SWSGEsLTzO2H7PNi7EHIzTPtcfExTaLV+Ctz81I1PCCEqCElkhRDCEjJTTBduRX8N56Kv7fdvAq2Gm2YicHJXLz4hhKiAJJEVQojSMuTA0dWw52fT/K+5mab9Wkdo0htaPQEhbUCjUTVMIYSoqCSRFUKI22E0wJmtsH8x7FsI6YnXnvOrD80fg/CB4F5VvRiFEKKSkERWCCFuJTsdjq0xjboeXlE4eXXzN83/GvYIBLWQ0VchhLAiSWSFEOJ6igIJh+DEeji2Fo6vvVY2AODsDfW7QbOHoXZH0MlHqRBCqEE+fYUQQlEg6aRpsYIT/5puV+MKt/GuAQ3ug4Y9oEZb0DmqEqoQQohrJJEVQlQ+WalwLgbOboezO0z36ZcKt3Fwhhp3QK27oN69ENBUygaEEMLGSCIrhKi4jEa4cgri95tuF/PuE48BSuG2Wkeo1gJq/Q9q/w+qtwZHZzWiFkIIUUKSyAoh7JvRaBpNTTppSlATj5pul49B4nHISSv+OK8aUL0lVG9lug8Mk8RVCCHsjCSyQgjblZsFaZcgLQFSzkPKOdMtOf/+LKReAEP2jc+h00PVhhDQpMCtKbj7W+91CCGEKBeSyAohypchx1STmpViWv0qK8X02LydAhlJkJZomtYq/ZIpeU1PhOyrJexEA57VwLcOVKkDvnWvbVepJRdmCSFEBSWJrBC2RlFMk+4rxgK3go+Le75gG8V0f6NzGI1gzDGNYhpyTffGHFPCaci+dm/MLfA457pjsiAnA3LS8+4L3tILbxtzyvZ+aB3A1Rc8gsCrOngGg1ew6T5/2yNIklUhhKiEJJEtbytfh9il1+1UirZTitl3fbvimpT2XJZup0aft9VOjT5L2u66hLSicnABZ09w8gQnj2vbzp6meVldfcHNz3Tv6pe3XcX0nMwWIIQQohiSyJa3qxch6YTaUYgKSQMabeGbVpe3nf+crpg2WlPdqNbRNIqpczQ91jnm7dObJvjX6fPaOVx73txWD44u4Oiad5+37eBcYF+B55w8ZMRUCCGExUkiW946vgqtnyzmiWJGmIoddSquXXEdleV8ZWmnZt/l8VrKud9i22oKJKA3uRVpI6OUQgghKjdJZMubbx3TTQghhBBCWJRW7QCEEEIIIYQoDUlkhRBCCCGEXZJEVgghhBBC2CVJZIUQQgghhF2SRFYIIYQQQtilSjdrgZI3SX1KSorKkQghhBBCiOvl52hKsQsLFVbpEtnU1FQAQkJCVI5ECCGEEELcSGpqKl5eXjdto1FKku5WIEajkfPnz+Ph4YHGChPKp6SkEBISwpkzZ/D09Cz3/oSJvO/qkPddHfK+W5+85+qQ910d1n7fFUUhNTWVatWqodXevAq20o3IarVaqlevbvV+PT095R+dCuR9V4e87+qQ99365D1Xh7zv6rDm+36rkdh8crGXEEIIIYSwS5LICiGEEEIIuySJbDlzcnJiwoQJODk5qR1KpSLvuzrkfVeHvO/WJ++5OuR9V4ctv++V7mIvIYQQQghRMciIrBBCCCGEsEuSyAohhBBCCLskiawQQgghhLBLksgKIYQQQgi7JImsEEIIIYSwS5LICiGEEEIIuySJrBBCCCGEsEuSyAohhBBCCLskiawQQgghhLBLksgKIYQQQgi7JImsEEIIIYSwS5LICiGEEEIIu+SgdgDWZjQaOX/+PB4eHmg0GrXDEUIIIYQQBSiKQmpqKtWqVUOrvfmYa6VLZM+fP09ISIjaYQghhBBCiJs4c+YM1atXv2mbSpfIenh4AKY3x9PTU+VohBBCCCFEQSkpKYSEhJhztpupdIlsfjmBp6enJLJCCCGEEDaqJCWgcrGXEEIIIYSwS5LICiGEEEIIu6RqIvvvv/9y//33U61aNTQaDb///vstj1m3bh0RERE4OTlRt25dvvnmm3KPUwghhBBC2B5Va2TT0tJo3rw5w4YNo2/fvrdsf+LECe677z6eeeYZfvjhB1avXs0TTzxBUFAQXbt2tWhsBoOBnJwci55TCGE5jo6O6HQ6tcMQQgihIlUT2e7du9O9e/cSt//ss8+oVasW06ZNA6BRo0Zs3LiRGTNmWCyRVRSFuLg4rly5YpHzCSHKj7e3N4GBgTIntBBCVFJ2NWvBli1b6Ny5c6F9Xbt2ZdSoURbrIz+J9ff3x9XVVX5BCmGDFEUhPT2dixcvAhAUFKRyRCq5cgbO7QBFYXvqSbblXAbPSvpeCCHKlbPOmeHNhqsdRhF2lcjGxcUREBBQaF9AQAApKSlkZGTg4uJS5JisrCyysrLMj1NSUm54foPBYE5ifX19LRa3oigWO5eovOSPqsLy/71fvHgRf3//ylVmcPEgbJwJe38FxQDAizWqk6qT63eFEOXDx8lHElk1TJ48mYkTJ5aobX5NrKurq8X6VxSF3NxcSWZFmWg0GhwcHCSZvU7+v9WcnJzKkcgqCvwzATZ9dG1fUHOMendStWcA6JN6Fb2igEcQ1O0CDnqVghW2RFEUMjIy5HeRKDVXB1cMBoPNfdbaVSIbGBhIfHx8oX3x8fF4enoWOxoLMG7cOMaMGWN+nL9axM1YOlnI/+CQJESUhqIo8svnBirVvymjAf58AXZ+b3rc6H64czQER2Iw5sB3EQC8VPcRPKO/g8QDkKmDxxeBu7+KgQtbkJubS3x8PBqN5pZr1wtxPaPRaLO/i+wqkW3bti3Lli0rtG/VqlW0bdv2hsc4OTnh5ORU3qHdUqX6hSssSqPR2OSHh7Ci3GxY9AQc+AM0Wug1C8IfNz9tMBrM2w73TIAWj8N3fSBuL3zVDQb9Ad43/wNeVA5ardbmRtSEfTAYDLdupAJV/yy7evUqu3btYteuXYBpeq1du3Zx+vRpwDSaOmjQIHP7Z555huPHj/Pyyy8TGxvLp59+yi+//MLo0aPVCN8uDB8+nAcffFDtMACoV68eH3/88U3b6PV6/vjjDytFJISdWP6yKYnV6eHhbwslsQAG5dovGJ1WB4HNYNhK8KoBl4/BDw9Bdrq1oxZCiHKn6ojsjh07uPvuu82P80sABg8ezDfffMOFCxfMSS1ArVq1WLp0KaNHj+ajjz6ievXqzJs3z+JzyFYk06dPt5nRvM2bN+Pm5qZ2GELYlwN/QPTXpu1HvoMG3Yo0yTXmmrd1mrzRNt86MGwFzL0bEmJh5Ti4/6MixwohhD1TNZHt2LHjTZOs4lbt6tixIzt37izHqCoWLy8vtUMwq1q1qtohCGFfkk7BH8+bttuPKjaJhetGZDUFvjb2CoY+n5vKDKK/gdodoUmfcgtXCCGsTSq+K4iFCxcSHh6Op6cngYGBdOvWjbS0tCKlBampqQwaNAhvb29q1KjBRx99ROfOnRk7dqy5Tb169Zg0aRJDhw7Fx8eHunXr8ueff5KQkEDfvn3x8fEhIiKC6OjoQjEsWrSI5s2b4+7uTr169ZgxY0ah568vLThy5AidOnXCw8ODsLAw/vnnn3J6d4SwQ4ZcWPgEZCVDcEvo9MaNm+bVyOo0uqL1+HXuNl0UBrDkRUg6WU4BCyGE9UkiewuKopCenVvGm6FUx5W0JODChQsMHDiQIUOGsGfPHlatWkXv3r2LPT4qKootW7awaNEili1bxsaNG4sd4f74449p164d27Zto3v37gwdOpRhw4bRv39/tm7dSu3atRk6dKi5j5iYGPr3788jjzxCTEwMb775Jm+99Rbz588vNmaj0cgjjzyCXq9n48aNzJ49m9dee+02fjJCVHDb58LZbeDkBQ99CTrHGzbNH5EtNBpb0N2vQfXWpqR4WVR5RCuEEKqwq1kL1JCRY6Dx+JWq9L37zU646m/9I4qLiyM3N5fevXtTs2ZNAJo1a1akXWpqKt999x3z58+nU6dOAMybN898TEHdunXjySefBOD111/n888/JzIykoceeggwJcQdOnQgPj6ewMBAZs6cSadOnXj99dcBqF+/PgcOHGDatGmFLtjLt3r1ag4dOsTSpUupVq0aAO+88w73339/Sd4aISq29Muw7n3TdpeJ4BN60+b5NbI67Q0SWZ0j9PkMZreBI3/D0X+gbufi2wohhB2REdkKICwsjE6dOhEREcGjjz7Kl19+SVJSUpF2x48fJycnh1atWpn3eXl5Ub9+/SJtCybC+aupNW3a1LzP3980L2X+EqGxsbFFpkFr164dR48eLXbKjtjYWEJCQsxJLMAdd9xRotcrRIW37n3IvAIBTSGi6B+C17vliCyYLv5q/ZRpe+XrptIFIYSwczIiewsujjoOvF36WREURSEnJ5fSTCPr4liyuf50Oh3Lly9ny5YtrFq1itmzZzN+/Hg2btx4+53mcXS89jVmfs1dcfuMRmOp+xBCFCPhMGyfZ9ru+h7caJS1AHON7K3a/i8Kdv9kmsUg5hto9UQZgxVCCHXJiOwtaDQaXPUOZbzpSnXc7SyioNFoaNeuHRMmTGD79u3Fzsdau3ZtHB0d2bFjh3lfcnIyR44cKfP71LBhQ7Zs2VJo3+bNm6lXr16xk283bNiQM2fOcOHCBfO+rVu3ljkOIeze32+AYoAGPUyzDJRAiUZkAVx8TPWyAGsnQcaV0scphBA2QBLZCmDbtm28//77REdHc/r0aRYvXkxCQgINGzYs1M7Dw4OBAwcybtw41q1bx/79+3nqqafQarVlXnls9OjRrFmzhvfee4/Dhw8zf/585syZU2h54ILuuece6tWrx/Dhw9m9ezcbN25k/PjxZYpBCLt3eiscWQlaB+jyTokPy09kHTQl+JItcij4NYD0RPjv09JGKoQQNkES2QrAw8ODjRs30qtXL5o0acKECROYMmUK3boVnXNy6tSptGnTht69e9O9e3fatWtHw4YNy7yMb3h4OD/++CO//PIL4eHhvP3220yYMKHYC73AtEzir7/+SkZGBu3bt+fpp5/m7bffLlMMQti9DdNM9y36g1/dEh9W4tICAJ0D3D3OtL31c8hKvd0ohRDCZmgUW1n2yUpSUlLw8vIiOTkZT0/PQs9lZmZy4sQJatWqhbOzs0X6M9XI5pR5xLO8pKWlERoaypQpUxg6dKja4YgbUBQFR0dHm/3/SC3l8W9WNXF74bM7QaOFkTtMF2eV0O6E3Ty+7HGqu1dn+YPLb32A0QCzW0PiUbj3XWj3fBkCF/YgNzeX+Ph4dDpdseVeQtyMwWDAYDAQEBCAg0P5X151s1ztejIiW8ns3LmTBQsWcOzYMXbu3GkeMZVpr4RQ2ca8BUQa976tJBaujcg6aEv4C0arM60UBrD5E8jNuq3+hBDCVkgiWwnNmDGDli1b0q1bN9LT01mzZg1+fn5qhyVE5ZV4DPYvNm3nr8J1G0p8sVdBYf3AMxiuxsGuH2+7TyGEsAUy/VYlEx4eLrMDCGFrNn8MihHqdoGgsNs+/JYLIhTHQW8qKVjxKmz6CMIHmupnhRDCjsiIrBBCqCktEXb9ZNruUPwsH7dSqhFZMC224OoLSSfg0NJS9S2EEGqSRFYIIdS0cz4YsiCoBdRoe8vmxbntGtl8ejeIHGLazl+EQQgh7IgkskIIoRajAXZ8Zdpu9QSlWgIQyFVMpQVaTSk+0iOHmmZKOPEvJBwqVf9CCKEWSWSFEEItR1bBldPg7A1NHyz1aczzyN5uaQGAdwjU727a3v5lqWMQQgg1SCIrhBBqyf86P/xx0LuW+jRGxQiUorQgX6vhpvvdP0HW1VLHIYQQ1iaJrBBCqOHycTj6j2m75bAynSq/tKBUI7IAte+GKrUhKwX2/lKmWIQQwpokkRU3Va9ePT7++GO1wxCi4tnxFaBA3c63vQDC9W5ridriaLWmGl2AbfOgci34KISwY5LIVhCdO3dm7NixaochhCiJ3OxrixC0HF7m0+VPv+WgKcM8sC36g4MzXNwP52LKHJMQQliDJLJCCGFtR1ZCeiK4B0C9e8t8ulItiHA9Fx9olLdU9a7vyxyTEEJYgySyFcDw4cP5999/mTVrFnq9Hr1ez7Fjx3jqqaeoX78+np6eNGnShFmzZhU57sEHH2T69OnUqFGDwMBAXnjhBXJycgq1S09P58knn6RKlSrUqVOHefNkvkkhyiR/NDasn0VW0yr1ggjXazHAdL93IeRkljEqIYQof7Ie4a0oCuSkl/H4nNLND+noWqLjpk+fzpEjR2jSpAkTJkwAwMfHh+DgYH766SeqVKnCli1beO655wgMDOThhx82H7t+/XqCgoL4+++/OXbsGAMGDKB58+YMH37t686ZM2fy1ltv8corr7Bo0SJGjhxJhw4daNCgwe2/JiEqu6sX4fBK03b44xY5ZZlrZPPV+h94hUDyGYj9C5o9ZIHohBCi/Egieys56TCpWqkP1wD60nYddcq08s4teHl5odfrcXV1JTAw0Lw/P6kFqFWrFlu3buW3334rlMj6+Pjw0UcfodPpaNiwId27d2fNmjWFEtlu3brxzDPPABAVFcXHH3/M+vXrJZEVojT2/AyKAYJbQlXL/BuySI0smC76av4Y/DsFdv0giawQwuZJaUEFNmfOHNq0aUO1atXw8fFh3rx5nDlzplCbxo0bo9NdG8UJCgoiISGhUJtmzZqZtzUaDQEBAVy8eLF8gxeiIlIU2PmDaTt8gMVOa5Ea2Xwt+pvuj62F5LNlP58QQpQjGZG9FUdXeO18qQ9XFIWcnBw0pS0tKKWff/6ZV155hSlTptCmTRs8PDyYPn0627ZtK9TOwaHw/wIajQaj0Vg4DEfHW7YRQpTA+RhIOGiaHaBJX4ud1mI1sgBVakHNO+HURtMCCXdFlf2cQghRTlQfkZ09ezahoaE4OzvTpk2bIonW9WbOnEmDBg1wcXEhJCSE0aNHk5lZjhclaDSmr/fVuN1G8qvX6zEYDObHW7ZsoW3btjzzzDOEh4dTt25djh8/Xh7vkBCipHb9ZLpv2BNcvC122jItUVuc/FHZXT/KnLJCCJumaiL7888/M2bMGCZMmEBMTAzNmzena9euN/za+scff+TVV19lwoQJHDx4kC+//JKff/6Z1157zcqR256aNWuybds2Tp48yaVLl6hbty7R0dH8/fffHD58mAkTJrBjxw61wxSi8jLkwP5Fpu0Wj1n01OaVvSxRWgDQ+AHTN0KXj5tGkYUQwkapmshOnz6dJ598kqFDh9K4cWM+++wzXF1d+eqrr4ptv3nzZtq3b0///v0JDQ3l3nvv5bHHHrvlKG5lMHr0aHQ6Hc2bN6datWp06dKF3r17M2DAAO68804uX77M008/rXaYQlRex9aa5o51qwq1Olr01EbFVOpjsRFZJ3do0MO0vedXy5xTCCHKgWo1stnZ2URHRzNu3DjzPq1WS+fOndmyZUuxx7Rr147vv/+ebdu20bp1a44fP86yZcsYOHCgtcK2WfXr12fDhg2F9s2bN6/InK/vvfeeefvLL78scp5p06YVenzkyJEibWRkV4hS2PuL6b5JX4vMHVtQfmmBg9aC5232MOz7DfYthHvftXjMQghhCap9Ml26dAmDwUBAQECh/QEBAcTGxhZ7TP/+/bl06RJ33nkniqKQm5vLM888c9PSgqysLLKyssyPU1JSLPMChBCipLLTIHapaTvsEYuf3lxaYKkRWYC694BLFUi7CCf/hTqdLHduIYSwENUv9rod69atY9KkSXz66afExMSwaNEili5dyjvvvHPDYyZPnoyXl5f5FhISYsWIhRACiF1mmpPapxYER1r89BZbEKEgnSM06WPalvICIYSNUi2R9fPzQ6fTER8fX2h/fHx8oUn9C3rzzTcZOHAgTzzxBM2aNaNPnz5MmjSJyZMn33A6qHHjxpGcnGy+XT+PqhBClLu9eYlgs4dLt8rfLVhsQYTrNctbPOXgn5CTYdlzCyGEBaiWyOr1eiIjI1m9erV5n9FoZPXq1bRt27bYY9LT09FqC4ecP5m/coMpYpycnPD09Cx0E0IIq0lLhGN5n3PlUFYAFl4QoaCQNuBVA7JT4fAKy55bCCEsQNXSgjFjxjB37ly+/fZbDh48yLPPPktaWhpDhw4FYNCgQYUuBrv//vuZM2cOCxYs4MSJE6xatYo333yT+++/v9DqVEIIYTMOLAZjLgS1AL965dKFRRdEKEirhWYPmrb3/mbZcwshhAWoehlqv379SEhIYPz48cTFxdGiRQtWrFhhvgDs9OnThUZg33jjDTQaDW+88Qbnzp2jatWq3H///YWuxBdCCJuy/3fTfdMHy62Lcpm1IF/Th2DjDDiyCjJTwFm+1RJC2A7V51MZOXIkI0eOLPa5devWFXrs4ODAhAkTmDBhghUiE0KIMrp6EU5tMm03fqDcuimXWQvyBTQB33qQeMRUXlBO5RFCCFEadjVrgRBC2JWDS0AxmmYq8KlZbt3kj8hqNeXwka7RXJu9YP9iy59fCCHKQBJZIYQoL/llBY17l2s35lkLyqO0AK4lskf/MZUXCCGEjZBEthLr3LkzY8eOVTsMISomK5UVQIF5ZMujtADAvxH4NQBDNhxaXj59CCFEKUgiK4QQ5SG/rKBaRLmWFUCBWQssPf1WPikvEELYKElkK6js7Gy1QxCicssvK8hPAMtRuU2/VVCT3qb7Y6sh40r59SOEELdBEtkKonPnzrz44ouMHTuWoKAg7rvvPvbt28f999+Pj48P1atXZ8iQIVy6dOmG59Dr9fzxxx+F9lWtWpX58+eXd/hCVCxWLCuAcp5+K59/I6jaUMoLhBA2RRLZW1AUhfSc9DLdMnIzSnW70WplN/Ldd9/h6OjIunXreO+99+jatSvNmzdny5Yt/Pnnn1y8eJH+/fuX0zslhDCzYlkBlPP0WwVJeYEQwsaoPo+srcvIzaDNj21U6XvTI5twcXApcfu6devy/vvvAzBp0iRatGjBu+++a37+iy++oHbt2hw+fJj69etbPF4hRB5zWUFvq3RnvtirvGpk8zXuDesmw7E1kJEELj7l258QQtyCjMhWIBEREebtPXv2sG7dOnx8fMy3Zs2aAXD8+HG1QhSi4ruaUKCsoLdVujRPv6Up57EJ/4bg3xiMORC7rHz7EkKIEpAR2VtwcXBha/+tpT5eURRyc3PRaDS3fayzzvm22ru5uZm309LSuO+++5g0aVKRdkFBQcUer9FoipQz5OTk3FYMQlR6Vi4rAMg15pUWlPeILJjKCy4eMJUXhA8o//6EEOImJJG9BY1Gg6uja6mPVxSFHHJKlciWRYsWLVi8eDGhoaE4OJTsx1y1alXi4uLMj48cOUJ6enp5hShExZRfP2qlsgKw0qwF+Rr3hrXvwfG1Ul4ghFCdlBZUUM8++yxJSUk8/vjj7Nixg2PHjvH333/zxBNPYDAYij2mY8eOfPrpp+zcuZPo6GhGjhyJo6OjlSMXwo6pUFYAVpq1IF/V+uDfBIy5ELu0/PsTQoibkES2gqpWrRrr1q3DYDDQo0cPIiIiGDt2LN7e3mi1xf/Yp0yZQkhICJ06dWLgwIGMHj0aV9fSj0YLUemoUFYA10ZktRorfaTL7AVCCBshpQUVxD///FNkX7169fj1119LfEy1atVYurTwCEtCQoJlAhSiMlChrAAK1Mhao7QATK9v7btwfB2kXwbXKtbpVwghriMjskIIYQmFygrKfxGEgsyzFlijtADArx4ENDOVFxz80zp9CiFEMSSRFUIISyhUVhBq1a6terFXviZ5yfqB363XpxBCXKdMiezRo0dZuXIlGRkZALe9EpUQQlQY+QmdlcsKwIoLIhTUOK9O9vh6U3mBEEKooFSJbGJiIp07d6Z+/fr06NGDCxcuADB8+HDGjh1r0QCFEMLmXU2AkxtN21YuKwArLohQkF9dU3mBYoDYv6zXrxBCFFCqRHb06NE4ODhw+vTpQle19+vXjxUrVlgsOCGEsAvmsoJwq5cVgEojsnCtvEBmLxBCqKRUiezff//NBx98QPXq1Qvtr1evHqdOnbJIYEIIYTfMZQV9VOk+V7HyrAX5pLxACKGyUiWyaWlpxc4vevnyZZycnMoclBBC2A2VywrAygsiFCTlBUIIlZUqke3QoQPz5883P9ZoNBiNRqZMmcLdd99tseCEEMLmqVxWACrNWpAv/+I2KS8QQqigVH++T5kyhXvuuYcdO3aQnZ3Nyy+/zP79+7l8+TKbNm2ydIxCCGG78ssKrLgkbUGKolxLZK1dIwumcoo171wrL5DFEYQQVlSqEdmmTZty+PBh7rzzTh544AHS0tLo27cvO3fupE6dOpaO0SYYDAZyc3OtdjMYDBaLffjw4Tz44IMWO19Z1KtXj48//vimbfR6PX/88YeVIro9iYmJBAcHc/LkSfO+zp07F/qGojiW+hm89dZbtGjR4raOCQ0NZd26dWXu29o+++wz7r//frXDuLmCZQUqTLsF10ZjQaURWd86Ul4ghFBNqQuqvLy8eP311y0Zi80yGAwkJCSUOrkszXEODg5UqVIFna7sv5imT59uM3P8bt68GTc3N1Vj0Ov1/PrrrzzwwO3XM77//vvcf//9hIaGWj6wEnjppZd4/vnnrdrn5cuXef755/nzzz/RarU8+OCDfPTRR7i7u9/wmMzMTMaOHcuCBQvIysqia9eufPrppwQEBJjbaDSaIsf99NNPPProowAMGzaMd955hw0bNtChQwfLvzBLsKGyAlApkQVTEh+/11ReEDFInRiEEJVSqUZk69aty1tvvcWRI0csHY9NUhQFg8GARqNBp9OV+02r1ZKbm2ux5NPLywtvb2+LnKusqlatWuyFgvYgPT2dr7/+mqFDh1q9b0VRyM3Nxd3dHV9f33LtKykpiatXr5ofDxgwgP3797Nq1Sr++usv/v33X5566qmbnmP06NH8+eef/Prrr6xfv57z58/Tt2/fIu2+/vprLly4YL717t3b/Jxer6d///63HMFXlcplBXDtQi9QqbQArs3WILMXCCGsrFSJ7IgRI1i6dCkNGjSgVatWfPTRR8TFxZUqgNmzZxMaGoqzszNt2rRh27ZtN21/5coVRowYQVBQEE5OTtSvX59ly5aVqu/bpdVqrZLIFjdSdSsLFy4kPDwcT09PAgMD6datG2lpaUDRr7VTU1MZNGgQ3t7e1KhRg48++ojOnTsXWsyiXr16TJo0iaFDh+Lj40PdunX5888/SUhIoG/fvvj4+BAREUF0dHShOBYtWkTz5s1xd3enXr16zJgxo9Dz15cWHDlyhE6dOuHh4UFYWBj//PNPmV7rjh076N69O0FBQfj5+XHPPfewc+fOQv0DPPzww+j1evPjkli+fDlOTk60adPmpu0MBgNRUVFUrVqVwMBAXn311SJ/lBiNRj744APq16+Pp6cnkZGRLFy40Pz8+vXr0ev1rFixgjZt2uDh4cHGjRsLlRb8/fffODs7c+XKlULnfvHFF+nUqVOJXxdAbm4uS5cu5eGHHyYoKIhjx44BcPDgQVasWMG8efNo06YNd955J7NmzWLBggWcP3++2HMlJyfz5ZdfMn36dDp16kRkZCRff/01mzdv5r///ivU1tvbm8DAQPPN2dm50PP3338/S5YsMa8eaFNsoKwACo/IWnVBhIKkvEAIoZJSL4iwfft2YmNj6dGjB7NnzyYkJIR77733lrWCBf3888+MGTOGCRMmEBMTQ/PmzenatSsXL14stn12djZdunTh5MmT/Pbbbxw6dIi5c+cSHBxcmpdRYVy4cIGBAwcyZMgQ9uzZw6pVq+jdu/cNR3SjoqLYsmULixYtYtmyZWzcuLFQspfv448/pl27dmzbto3u3bszdOhQhg0bRv/+/dm6dSu1a9dm6NCh5n5iYmLo378/jzzyCDExMbz55pu89dZbN/x/wmg08sgjj6DX69m4cSOzZ8/mtddeK9NrTU1N5fHHH2ft2rVs2LCBunXr0qtXL1JTUwFTaQPAvHnzOH36tPlxSWzatImIiIhbtpsxYwbz58/niy++YO3atSQlJRWp+f3ggw/4/vvv+eSTT9i1axcvvvgiQ4YM4d9//y3U7vXXX+e9995j9+7dhIWFFXrunnvuwdvbu1ACbDAY+PnnnxkwYECJXtPevXsZO3Ys1atXZ9CgQVStWpW1a9fSvHlzALZs2YK3tzctW7Y0H9O5c2e0Wi1bt24t9pzR0dHk5OTQuXNn876GDRtSo0YNtmzZUqjtiBEj8PPzo3Xr1nz11VdF/p9t2bIlubm5N+xLVbF/ql5WADYyIgsye4EQQhVl+vO9fv36TJw4kYkTJ/Lff//x7LPPMnToUAYNKlmN1PTp03nyySfNX9V+9tlnLF26lK+++opXX321SPuvvvqKy5cvs3nzZhwdHQFUq1W0JXFxceTm5tK7d29q1qwJQLNmzYptm5qaynfffcf8+fPNo3bz5s0zH1dQt27dePLJJwFTQvX5558TGRnJQw89BJgS4g4dOhAfH09gYCAzZ86kU6dO5trp+vXrc+DAAaZNm1bs/xOrV6/m0KFDLF26lGrVqgHwzjvv3PQCn1u91uunf5szZw5Vq1bl33//5b777qNq1aqAqdwiMDDwhv0U59SpUwQFBd2y3axZs3j55Zfp08f0devs2bNZtWqV+fmsrCw++OADVqxYwR133AFA7dq12bRpE3PnzuWuu+4yt50wYQKdO3dGURTz//P5dDodjz76KD/++CPDhw8HTO/plStXbnphWWJiIt9//z3ffvst+/fvp0ePHnz66af07NkTvV5fqG1cXBz+/v6F9uXXb9/oW5i4uDj0en2RcpaAgIBCx7z99tt06tQJV1dX/v77b5577jmuXr3KCy+8YG7j6uqKl5eXbS60kp+wqVhWANcWQ9CgQasp1diEZcjsBUIIFZT5U2/btm2MGjWKPn36cPjwYR5++OESHZednU10dHShURutVkvnzp2LjNrkW7JkCW3btmXEiBEEBATQtGlTJk2aZNEr/O1RWFgYnTp1IiIigkcffZQvv/ySpKSkYtseP36cnJwcWrVqZd7n5eVF/fr1i7QtmCDmX6TTtGlT8778BCd/BD02Npa2bdsWOke7du04evRosT+j2NhYQkJCzEksYE7sSvta4+PjeeaZZ2jcuDF+fn74+vpy9epVzpw5c9PzlkRmZmaRr76vl5yczIULF2jdurV5n4ODQ6GR3KNHj5Kenk737t3x8fEx377//nuOHz9e6HyRkZE37W/AgAGsW7fO/DX/Dz/8wH333XfTmuhZs2YxatQo3N3dOXr0KIsXL6Zv375Fktjy9uabb9K+fXvCw8N55ZVXePnll5k6dWqRdi4uLqSnp1s1tluykbICUHF52uv51oHAvPKCg3+qG4sQotIoVSJ7+PBhJkyYQP369Wnfvj0HDx7kgw8+ID4+ngULFpToHJcuXcJgMBS6ihmKjtoUdPz4cX777TcMBgPLli3jzTffZNq0abz77rs37CcrK4uUlJRCt4pGp9OxfPly/vzzTxo1asTs2bNp2rQpJ06cKNN5C44A5tftFrfPaDSWqZ/bcavXOnz4cHbv3s20adNYv34927dvx9fXl+zs7DL37evre8M/EG5Hfj3vH3/8wfbt28233bt3F/n3c6sZHlq1akWdOnVYsGABGRkZLF68+JZlBU899RTvvPMOcXFxNGnShKFDh7JmzZpif46BgYFFSn1yc3O5fPnyDUe0AwMDyc7OLlK7mz9yfyNt2rTh7NmzZGVlFdp/+fJl80i6zbCRsgK4ViOrWn1sQfmj0/kXwQkhRDkrVSLbsGFDVqxYwYgRIzh79iwrV65k0KBBN52OxxKMRiP+/v588cUXREZG0q9fP15//XU+++yzGx4zefJkvLy8zLeQkJByjVEtGo2Gdu3aMWHCBLZv337DuVhr166No6MjO3bsMO9LTk62yAwUDRs2LDKavnnzZurVq1fsNGINGzbkzJkzXLhwwbyvJLWQN3utmzdvZsSIEXTv3p0mTZrg5OTEpUuXCh3v6OhYqlH8Fi1acPDgwZu28fLyIigoqNBFi7m5uYVqkBs1aoSTkxOnT5+mbt26hW6l+f9zwIAB/PDDD+bpse67776btq9WrRpvvPEGhw8fZsWKFej1evr27UvNmjV59dVX2b9/v7lt27ZtuXLlSqGL+vKT3htd9BYZGYmjoyOrV6827zt06BCnT58uMmJf0K5du/Dx8Sm0zPWxY8fIzMwkPDz8lu+DVdlIWQHY0IgsyOwFQgirK1Uie+jQIbZu3cqLL75YZES1pPz8/NDpdMTHxxfaf7NRm6CgIOrXr18oKWrUqBFxcXE3HHEbN24cycnJ5pslvmK2Ndu2beP9998nOjqa06dPs3jxYhISEmjYsGGRth4eHgwcOJBx48axbt069u/fz1NPPYVWqy3VbAkFjR49mjVr1vDee+9x+PBh5s+fz5w5cxgzZkyx7e+55x7q1atnHkXduHEj48ePL9NrrVu3Lj/++CMHDx5k27ZtDB48GBcXl0LnqFmzJmvXriUuLs48wrp9+3aaNm3KuXPnbth3ly5dOHDgwC1HZUeOHMnUqVP5448/iI2N5fnnny80Ounh4cHo0aOJiopi/vz5HDt2jJ07dzJ79uzbulgy34ABA4iJieG9997joYceKpQI3kq7du34/PPPiYuLY+rUqezatYvmzZuzd+9ewPTvK79Wetu2bWzatImRI0fy6KOPmktCzp07R8OGDc3Ju5eXF8OHD2fMmDGsXbuW6Ohohg4dStu2bc2lI3/++Sfz5s1j3759HD16lDlz5jBp0qQic+Ru2LCB2rVr29ZCK6lxcGKDaVvlsgK4ViOr2hyyBUl5gRDCykqVyN7OlEU3otfriYyMLDRqYzQaWb169Q1Hbdq3b8/Ro0cLfQV6+PBhgoKCbljf5+TkhKenZ6FbaRmNRgwGQ7nfbnf+2PypmXr16kWTJk2YMGECU6ZMoVu3bsW2nzp1Km3atKF37950796ddu3a0bBhw9tKgIoTHh7Ojz/+yC+//EJ4eDhvv/02EyZMuOHFf1qtll9//ZWMjAzat2/P008/zdtvv12m1/r555+TlJREmzZtGDJkCCNGjChysdKUKVNYvXo1tWvXNteypqenc/jwYXJycm7Yd7NmzQgPD+e33367aYyjR49mwIABDB8+nLvuugsPD48iiy9MnDiR1157jSlTphAWFkbPnj1Zvnw5tWrVuum5i1O3bl1at27Nnj17SjxbwfWcnZ159NFHWbFiBadPny508d8PP/xAw4YNueeee+jRowd33nknX3zxhfn5nJwcDh06VKiOdcaMGfTs2ZMHH3yQu+66i8DAQBYtWmR+3tHRkdmzZ9O2bVtatGjB559/zvTp05kwYUKhuH766SfzBYc248AfgALVW6leVgDXRmQdtDZQWgBSXiCEsCqNUsKsqUqVKhw+fBg/Pz98fHxuOnp3+XLJvlL6+eefGTx4MJ9//jmtW7dm5syZ/PLLL8TGxhIQEMCgQYMIDg5m8uTJAJw5c4YmTZowePBgnn/+eY4cOcKwYcN44YUXSrzKWEpKCl5eXiQnJxdJajMzMzlx4gS1atUqdFGPva/sdStpaWmEhoYyZcoUVSb7tyfLli3j1VdfZdeuXWi1pr8DO3fuzKBBg0o8W0dp5M9aUJpR89DQUL755hs6duxo+cDK0f79++nUqROHDx/Gy8ur2DY3+jdbrr68F85shW7vwx3PWqfPmzh0+RAP/fkQVV2qsuaRNWqHA4nHYFYEaHQQdVRmL7ARubm5xMfHm+crF+J25A+0BQQE4OBQ/n803yxXu16Jo5kxYwYeHh7m7bJ+DQ3Qr18/EhISGD9+PHFxcbRo0YIVK1aYyxVOnz5tThYAQkJCWLlyJaNHjyYsLIzg4GBefPFFXnnllTLHcjM6nY6qVauWaqUtRVHIycm57fcrfxWx8rBz504OHTpEq1atSElJMV8sZ/Pr2tuAHj16cPToUc6dO1dh661txYULF5g/f/4Nk1hVXDltSmLR2ER9LBQoLbCFGlm4Vl4Qt9dUXhA5WO2IhBAVWIkT2cGDr30YDRkyxGIBjBw5kpEjRxb73Lp164rsa9u2bZHVgayhtEmloigoimKRxN+SZsyYweHDh9Hr9URERLBmzRr8/PzUDssuFJznVJSfglPz2Yz8i7xC7wTPW88pbA3mi71soUY2X+PepkT2wO+SyAohylWpxod1Oh0XLlwoUnuYmJiIv79/pZ/X1daFh4fb5kpJdmrQoEFFVt2yJaNGjZKFQyxlb159dNO+6sZRQP70WzaVyMriCEIIKylVInujr9izsrKsPqm6EGorz9pYSxg1apTaIVQMl45A3B7QOkCjB27d3kpsavqtfFJeIISwkttKZD/++GPAVL85b968QvPGGgwG/v3332KnfBJCCLu3L2/Whdp3g5uvurEUYJMjsnCtvGD/YklkhRDl5rYS2RkzZgCmEdnPPvusUN2oXq8nNDT0posT2IvSXNQlhLA+q/1bVRTYl19W8KB1+iwhm5t+K1/TvqbyghPrITUePEo357gQQtzMbX3y5S8Devfdd7No0SJ8fHzKJSi15C+/mp6eXmQSfSGE7cmfu7bg0snlIn4/XDoMOido2KN8+7pNNrUgQkFVakNwSzi3A/YvsompyoQQFU+p/oRfu3atpeOwCTqdDm9vb/Pa8q6urmWebaC0028JUZCiKBgMBvn/KI+iKKSnp3Px4kW8vb3Lf17MfQtN9/W6gLMNTQeGjdbI5gvrZ0pk9/wiiawQolyU+ruos2fPsmTJEk6fPl1kedjp06eXOTC15C+Pm5/MlpUkIMISFEVBp9PJ/0fX8fb2vuGS1hajKNcSWRsrK4BrNbIOGhsrLQDT7AUrXoXzMXDpKPjVVTsiIUQFU6pPvtWrV9OrVy9q165NbGwsTZs25eTJkyiKQkREhKVjtCqNRkNQUBD+/v43Xa60pHJzc0lMTESr1cpqKqJUDAYDRqMRX19fq6yoYi8cHR2t82/qXDRcOQWOblC/+GWf1WRzCyIU5F4V6t4DR/6Gvb/A3a+pHZEQooIp1W/FcePG8dJLLzFx4kQ8PDxYuHAh/v7+DBgwwLzmvb2z1DJ+ubm5ODg4yLKAotS0Wi0GgwFnZ2dJZNWQPxrbsAfoXdWNpRg2uSBCQc0eMSWye36GjuNAvlUQQliQ9tZNijp48KB57kwHBwcyMjJwd3fn7bff5oMPPrBogEIIoRqj4dq0WzZYVgAFpt+yxRFZMP0B4OgGSSfh7A61oxFCVDClSmTd3NzMdbFBQUEcO3bM/NylS5csE5kQQqjt1Ga4Gme6wKtOJ7WjKZbNj8jq3aBRT9P2np/VjUUIUeGUKpG944472LhxIwA9evRg7NixvPfeewwbNow77rjDogEKIYRq8hOvRveDg5O6sdyAzU6/VVCzR0z3+xeBoezXHgghRL5SFdxNnz6dq1evAjBx4kSuXr3Kzz//TL169ex6xgIhhDDLyYADf5i2wx5VN5absNkFEQqq3RHcqkJaAhxbC/XvVTsiIUQFUapPvtq1a5u33dzcKsRqXkIIUcih5ZCVAl4hULO92tHckM0uUVuQzsFUY7z1M9MotySyQggLKVVpwRNPPMG6dessHIoQQtiQ3QtM92GPgLZUH5VWYdMLIhQUlldeELsUslLVjUUIUWGU6tM5ISGBbt26ERISQlRUFLt377Z0XEIIoZ6rCXD0H9O2DZcVgJ2MyAJUi4AqdSA3w5TMCiGEBZQqkf3jjz+4cOECb775Jtu3byciIoImTZowadIkTp48aeEQhRDCyvb9BorBlHxVra92NDdlXtnLlmtkwTR/bP6o7J5f1I1FCFFhlPr7Mh8fH5566inWrVvHqVOnGDJkCN999x1168oShEIIO5dfVtDctkdjAXKNdjBrQb5mD5vuj6+F1Hh1YxFCVAhlLvzKyclhx44dbN26lZMnTxIQEGCJuIQQQh0XY+HCLtA62OwiCAXZTWkBgG8dCG4JivHaimlCCFEGpU5k165dy5NPPklAQABDhgzB09OTv/76i7Nnz1oyPiGEsK5d35vu690Lbn7qxlICdjH9VkH5o9w7vwdFUTcWIYTdK9UnX3BwMJcvX6Zbt2588cUX3H///Tg52eZk4UIIUWKGnGtlBeED1Y2lhOxiQYSCmj0EK1+Hi/tNI9/VwtWOSAhhx0qVyL711ls8/PDDeHt7WzgcIYRQ0eGVpkn73fyhXhe1oykRu5l+K5+Lj2mltH2/Qcx3ksgKIcqkVKUFTz75pCSxQoiKZ2deWUHzR0HnqG4sJWRXNbL5wh833e/9zbSCmhBClFKJR2T79u3LN998g6enJ3379r1p20WLFpU5MCGEsKrUODjyt2nbTsoKoMCsBfYyIgtQ63/gVQOST8PBvyDsYbUjEkLYqRKPyHp5eaHRaMzbN7sJIYTd2b3ANHdsSBubnzu2ILsckdVqIXyAaXvnd+rGIoSwayUekf36668BUBSFiRMnUrVqVVxcXMotMCGEsBpFuVZWkP+1t50wKkbAjmYtyNeiP6x7H06sh6ST4BOqdkRCCDt02zWyiqJQt25di06zNXv2bEJDQ3F2dqZNmzZs27atRMctWLAAjUZD7969LRaLEKISOrUZEo+Aoxs06aN2NLfFrhZEKMi7BtTuaNqOkVFZIUTp3HYiq9VqqVevHomJiRYJ4Oeff2bMmDFMmDCBmJgYmjdvTteuXbl48eJNjzt58iQvvfQSHTp0sEgcQohKbMdXpvuwh8HJQ91YbpNdlhbkaznUdB8z3zT1mRBC3KZSzVrw/vvvExUVxb59+8ocwPTp03nyyScZOnQojRs35rPPPsPV1ZWvvvrqhscYDAYGDBjAxIkTqV27dpljEEJUYlcT4MAfpu2Ww9SNpRTsbvqtghr0APcASLsIsUvVjkYIYYdKlcgOGjSIbdu20bx5c1xcXKhSpUqhW0llZ2cTHR1N586drwWk1dK5c2e2bNlyw+Pefvtt/P39GT58eGnCF0KIa3b9AMYcqBYBQc3Vjua25Y/IOmjsrEYWTFOc5c8QsePGgxdCCHEjpfrkmzlzpkU6v3TpEgaDgYCAgEL7AwICiI2NLfaYjRs38uWXX7Jr164S9ZGVlUVWVpb5cUpKSqnjFUJUMEYjRJsuZLXH0Viw0+m3CoocDBummS76unQU/OqqHZEQwo6UKpEdPHiwpeMokdTUVAYOHMjcuXPx8yvZGuiTJ09m4sSJ5RyZEMIuHV9rumLeyQua3nx+bFtl1zWyYLroq969cGSl6Y+Kru+pHZEQwo6UqrQA4NixY7zxxhs89thj5guzli9fzv79+0t8Dj8/P3Q6HfHx8YX2x8fHExgYWGyfJ0+e5P7778fBwQEHBwfmz5/PkiVLcHBw4NixY0WOGTduHMnJyebbmTNnbvOVCiEqrPyvs5s/Cno3dWMppfwaWbubfqug/NHwXT/ISl9CiNtSqkR2/fr1NGvWjK1bt7Jo0SKuXr0KwO7du5kwYUKJz6PX64mMjGT16tXmfUajkdWrV9O2bdsi7Rs2bMjevXvZtWuX+darVy/uvvtudu3aRUhISJFjnJyc8PT0LHQTQgiSTsGhZabt/Kvn7VCuYqfTbxVUrwt4hUBGkmnZWiGEKKFSJbKvvvoq7777LqtWrUKv15v3d+rUif/++++2zjVmzBjmzp3Lt99+y8GDB3n22WdJS0tj6FDTL5ZBgwYxbtw4AJydnWnatGmhm7e3Nx4eHjRt2rRQLEIIcVPb54FiNC2X6t9I7WhKLX9EVqst9Rds6tPqoNUTpu2tn5sWqBBCiBIo1XdRe/fu5ccffyyy39/fn0uXLt3Wufr160dCQgLjx48nLi6OFi1asGLFCvMFYKdPn7bvD2ghhO3JToOYb03bdzyrbixlZNezFhQUMci00lf8Xji1CULvVDsiIYQdKNUnn7e3NxcuXKBWrVqF9u/cuZPg4ODbPt/IkSMZOXJksc+tW7fupsd+8803t92fEKKS270AMpPBpxbU66p2NGVinkfWnksLAFyrQIvHTHXL/82RRFYIUSKlGup89NFHeeWVV4iLi0Oj0WA0Gtm0aRMvvfQSgwYNsnSMQghhOYpi+voaoM0zYOff+JhnLbDX6bcKavOM6f7QMlMNsxBC3EKpPsEnTZpEw4YNCQkJ4erVqzRu3JgOHTrQrl073njjDUvHKIQQlnNsDVw6BHoPaNFf7WjKrMKUFgBUbQB1Oplql7d9oXY0Qgg7UKpEVq/XM3fuXI4fP85ff/3F999/z6FDh/juu+/Q6SrAqIAQouL671PTffjj4Gz/s5jY/YII12uTV7Mc8x1kygI2QoibK/Gf8GPGjLnp8wVnK5g+fXrpIxJCiPIStxeO/gMaLbR5Wu1oLMLuF0S4Xt3O4NfANGoe/TW0f1HtiIQQNqzEiezOnTsLPY6JiSE3N5cGDRoAcPjwYXQ6HZGRkZaNUAghLGXTR6b7Jn2gSq2bt7UTFWJBhIK0Wmj/AvwxArZ8aqqbdXBSOyohhI0q8Sff2rVrzdvTp0/Hw8ODb7/9Fh8fHwCSkpIYOnQoHTp0sHyUQghRVkknYd9C03YFGuWrcCOyAM0egTXvQep52POzaWouIYQoRqlqZKdNm8bkyZPNSSyAj48P7777LtOmTbNYcEIIYTGbPzFdRFSnEwQ1Vzsai6lwNbIADnpo+5xpe9PHYDSqG48QwmaVKpFNSUkhISGhyP6EhARSU1PLHJQQQlhU2iXY+b1pu/0oVUOxtAo5IgsQOQScvSDxCBxaqnY0QggbVapEtk+fPgwdOpRFixZx9uxZzp49y8KFCxk+fDh9+/a1dIxCCFE2/82B3AyoFg617lI7GouqcDWy+Zw8ri1bu2GaLFsrhChWqRLZzz77jO7du9O/f39q1qxJzZo16d+/P926dePTTz+1dIxCCFF66ZevLYDQYSxoNOrGY2G5iqm0QKux74UditXmWXB0hfM74cjfakcjhLBBpfrkc3V15dNPPyUxMZGdO3eyc+dOLl++zKeffoqbm5ulYxRCiNLb8glkp0JgM2jYU+1oLM48IlsRFkS4nnvVa6Oy6ybLqKwQoogy/Qnv5uZGWFgYYWFhksAKIWxPwdHY/71a4UZjAYyK6UKoCnWxV0HtXrg2Knt4pdrRCCFsTAX8LkoIIfJsngXZVyEwDBrep3Y05SK/tKDCXeyVz70qtH7StC2jskKI60giK4SomNISYdsXpu2O4yrkaCxcKy2osCOykDcq6wYXdsHhFWpHI4SwIZLICiEqpg3TTKOxQc2hQXe1oykXRsWIgmmEskLWyOZz87s2Krv6HchL3oUQQhJZIUTFk3QKts81bd8zocKPxkIFH5EF02pszl5wcb9ptS8hhEASWSFERbTmXTBkQ+2OUPcetaMpN/n1sVCBa2TzuVYxTZ8GpuVrczLUjUcIYRMkkRVCVCznd8HeX0zbXd5WNZTyVnBEtsItiFCc1k+DVwiknL02G4UQolKTRFYIUXEoCqwab9pu9oipPrYCy1+eFirBiCyAozPc/bppe8N00/RqQohKTRJZIUTFcXglnFgPOj10ekPtaMpdrvFaaUGFXNmrOGGPQEBTyEo2TcclhKjUKsF3UUKISiEnE1a8Ytq+41nwqaluPGWQmWMgITWL+JRM4lOyiEvJJCE1i7SsXNKyc8nINpCebSA5+xK4AIqWfp//h1YLDlotOq0GNycd3q56fFwd8XHV4+2qp4qbI9W8XQjxccXNyU4//rU66DoJ5veC7fMgYjAENlU7KiGESuz0k0wIIa6zeRYknQSPILjrZbWjuSVFUTh3JYNjCWkcvXiVYwlXOXrxKscTrnLpanaJzqFxuIJ7PVAULdtO3t7X7L5ueqpXcaVGFVdCfV1pGOhJoyAPQn3d0GptfJaH2v+Dxr3hwO+wLAqGLquwM1MIIW5OElkhhP27cto0byzAve+Ck7u68RQjLjmT3WevsOfsFfacTWbP2WSSM3Ju2F7voCXQ05kATyf8PZ3x93DCw9kRV70ON70OF70DmcpFPtgPTg4OTB0QQa5RwWBUyDUqXM3MISk9hyvp2VzJMG0nXs3ibFIGyRk5JKZlk5iWze4zVwr16+Koo0GgB42CPGle3YuWoT7UqeqOxtYSxXvfhSN/w+nNsPc3CHtY7YiEECqQRFYIYf9Wvga5GVDzTmj6oNrRoCgKpxLT2XI8kS3HEtl6IpH4lKwi7Rx1GkJ93ajr706dqu7U8XejTlV3Qnxc8XZ1vGXyeCrFYEpkdY50bxZU4viSM3I4czmds0npnLmcwbGEqxy8kEJsXCoZOQZ2nbnCrjNX+Gmbqb23qyORNXyIDPWhVWgVmlf3Ru+gck2ud4hpOq4178Dfb0D9ruDsqW5MQgirk0RWCGHfYpfCwT9Bo4MeU1T7ivnS1SzWH0pg09FLbDmeyIXkzELPazVQP8CD5tW9CQvxonl1bxoEeuCoK31CWNrlab1cHPEK9qJpsNd151M4cSmNgxdSOHAhhZhTSew+e4Ur6Tmsjr3I6tiLALjpdbSt40uHelXpUM+PWn5u6ozYtnsedv0Al4/D6olw3zTrxyCEUJUkskII+5VxBZbmTZLf/gUIaGK1ro1GhX3nk1kTe5G1hxLYc/YKinLteUedhvAQH+6o48sdtavQIsQbV71lP3LzF0Sw1NRbOq2Guv7u1PV35/7m1QDIzjWy/3wy0aeS2HEyiW0nL3M5LZt/Dl7kn4OmxDbY24W76lelS2N/2tXxw9nRSlOBOThBz5nXLvxq+iDUbGedvoUQNsEmEtnZs2czdepU4uLiaN68ObNmzaJ169bFtp07dy7z589n3759AERGRjJp0qQbthdCVGCrxkPqBfCtC/97pdy7y841svnYJVbuj2PVgYtculq4XKBpsCf/q1+VtrX9iKzpg4u+fBO60o7I3g69g5bwGj6E1/DhiQ6mBP7AhRQ2HLnEhiMJ7DiZxLkrGfy07TQ/bTuNq17HXfWq0qVxAJ0a+uPjpi+32ADThV8RgyBmPix5Hp7ZZJpvVghRKaieyP7888+MGTOGzz77jDZt2jBz5ky6du3KoUOH8Pf3L9J+3bp1PPbYY7Rr1w5nZ2c++OAD7r33Xvbv309wcLAKr0AIoYrj6yHmW9N2r1ng6FIu3WTmGPj3cAIr9sXxz8F4UjKvzd3qptfRoV5V7m5Ylbsb+OPvad0EKn9BBAeN9T7KtVoNTfPKEp7tWIf07Fy2Hr/MmtiLrDoQT1xKJiv2x7Fifxw6rYY7alfh/rBqdGsaiLdrOSW1Xd6Bw39D4lFY/wF0nlA+/QghbI5GUQp+GWZ9bdq0oVWrVnzyyScAGI1GQkJCeP7553n11VdvebzBYMDHx4dPPvmEQYMG3bJ9SkoKXl5eJCcn4+lZ/hcG5ObmEh8fj06nQ6erBCvvCIszGAwYDAYCAgJwcFD9b0/bkJkMn91pmq2g1RMWr400GhX+O5HIwuhzrNh3gbTsayto+bk70bVJAN2aBtKmlq+qFz3turiLgcsHEuIRwrK+y1SLI5+iKOw7l8KqA3H8fSCe2LhU83MOWg0d6vlxf/NqdGkcgIezo2U7j10KC/qbaqWHrYSQVpY9v52T30WiLKz9e+h2cjVVfytmZ2cTHR3NuHHjzPu0Wi2dO3dmy5YtJTpHeno6OTk5VKlSpdjns7KyyMq69vVfSkpK2YIWQqhvWZQpifWuCfdYbvTteMJVFsWcY/HOc5y7kmHeX83LmW5Ng+jeLJCIGj7obGSe1fwRWVtZnlaj0dCsuhfNqnsx5t4GnE5M56+95/lr9wUOXEhh7aEE1h5KQO+g5e4GVbm/eTU6NfS3TO1ww/ug2cOw91dY9AQ8sxGcPMp+XiGETVM1kb106ZI5wy8oICCA2NjYEp3jlVdeoVq1anTu3LnY5ydPnszEiRPLHKsQwkbs+RX2/GwaeXtwXpmnXLqSns2fey6wKOYsO09fMe/3cHagZ1g1HowIJrKmj+3No8q1GlkHrW2O1NfwdeW5jnV5rmNdjl68yl97zvPn7vMcS0hj5f54Vu6Px1Wvo1vTQB6KqM4dtX3LthhDjw/h9FbTwhjLXoY+cyz2WoQQtsk2P/1K6P3332fBggWsW7cOZ+fia9PGjRvHmDFjzI9TUlIICQmxVohCCEtKOgVL8/49/+9lCCndRZ45BiPrDyWwaOdZ/jlwkWyDETBdtX9XPT8ejKxO50YB1rv6vpQsPWtBearr786ozvV58Z56xMal8ufu8/y15wKnL6ezKOYci2LOEeztQp/wYPpGBFO7aikWtXDxhr5fwDc9YPePUK+zTcwrLIQoP6omsn5+fuh0OuLj4wvtj4+PJzAw8KbHfvjhh7z//vv8888/hIWF3bCdk5MTTk5OFolXCKGi3Cz4bShkpUBIG+jw0m0drigK+8+nsDDmLEt2nScx7doysA0DPXgosjq9WlTD38N+rnjPH5HValRenOA2aDQaGgV50ijIk6iuDYg5fYWFMWf5c/d5zl3J4JO1R/lk7VHCa3jzYER17g+rhpfrbdTT1mxr+n/j3ynw5ygIagG+dcrr5QghVKZqIqvX64mMjGT16tX07t0bMF3stXr1akaOHHnD46ZMmcJ7773HypUradmypZWiFUKoavkrcC4anL2h71zQlezj62JKJr/vOsfC6HMcir928ZGfu54HWgTzYER1GlezzxWhzLMW2Ghpwa1oNBoia/oQWdOH8T0b88/BeBbFnGP94QR2nr7CztNXePvPA3Ru7M+DEdW5q37Vki0g8b+X4eQGOL0Ffn4cnvgH9G7l/4KEEFan+qffmDFjGDx4MC1btqR169bMnDmTtLQ0hg4dCsCgQYMIDg5m8uTJAHzwwQeMHz+eH3/8kdDQUOLi4gBwd3fH3d321lcXQljAzu8h+mtAAw9+CT41b9o8M8fA3wfiWRh9lg1HEjDmzc2i12np0jiAByOD6VCvhEmRDTPPI2sHpQW34uyoo2dYNXqGVeNiaiZLdp3nt+izxMalsmxvHMv2xuHnrqd3i2AejKxOo6Cb/PGhc4SHv4HP74KLB0zzyz74pWqrvgkhyo/qiWy/fv1ISEhg/PjxxMXF0aJFC1asWGG+AOz06dNotdd+2cyZM4fs7GweeuihQueZMGECb731ljVDF0JYw7kY+CuvLvbu10x1j8VQFIXtJ5NYFHOWpXsukJp1bb7XiBrePBhZnZ7NbvNrahtnnrWgHBdEUIO/hzNPdKjNEx1qs/98MotizvHHrnNcuprNvI0nmLfxBE2qefJQZHUeaBFMleIWXfAIhIe/hW97wr6FUC0C2t34mz4hhH1SfR5Za5N5ZIW9qdTzyF45DfM6w9V4qN8NHv0JtIVHUU8nprNo51kWxZzj9OV08/5gbxf6RgTTN6I6tfwq5tfKy08s5+V/X6ZNYBvmdZ2ndjjlKsdg5N/DCfwWfZZ/DsaTYzD96nLUaejU0FR6cHdD/6Kj7P99BiteATTw6A+maboqIfldJMpC5pEVQojblZkMPzxiSmL9m5jqYvOS2JTMHJbtucCimHNsO3nZfIibXkf3ZkE8GFGdNrWqlG0qJzuQa8ybtaCCjcgWx1Gn5Z5GAdzTKICktGz+3GMqPdhzNtk8lZevm6nu+aHIAnXPbZ6GhIMQ/Q0sfAKGLIXgCFVfixDCciSRFULYntxs+GWQKQHxCIIBv2DQe7Dh0EUWxZxj5f44snJNU2ZpNNC+jh8PRgbTtUmgZSbXtxO2tiCCtfi46RnUNpRBbUM5FJfKwhjTiPylq1l8tekEX206QaOg/NKDavj1mAZXzsCx1fBjP9PFX7eosxZC2IfK84kvhLAPhlzTykzH14GjGye6fsVPm1L5fedqLqZeW6WvTlU3HoysTp/wYIK8XNSLV0Xmi70qwYjsjTQI9OC1Ho14uWsDNhy5xG/RZ1l1IJ6DF1J4568DTF52kLsb+vNIsynckzoI7cX98F1vGLrcVEcrhLBrksgKIWyH0Qh/jIADf2DQODBe/zI/fJ8EJAHg7epIr+bVeDCiOmHVvWxytS1rMk+/pZGPcgedlrsb+nN3Q3/Tam27TaUHu88ms+pAPKsOxFPf5Xl+1k/E5/JxlPkPoBmyFNz81A5dCFEG8uknhLAJqRlZxC8YSd1Tv5CraHku+3n+zqiHo07D3Q386RtRnU4N/dE72PeUWZZUmWpkb4e3q56BbUMZ2DaUI/Gp/JZXenA41ZNemS/zi/4dghJiuTSnB8ZBS/D3l5FZIeyVJLJCCNXkX4n+e8xpOh56lwe16zAqGsbkPMeVGl2ZFB5Mj2aBeLsWM72SqLQ1srejXoAH47o3IureBmw4aio9GLr/db53mIjf1UMc+ORexlf7gLsjm9CtSVCFmp5NiMpAElkhhFUZjQo7zyTxx67z/LXnAqlp6cx0/IT7dNswoGVNw/FEdX2akCquaodq8/JrZO11ZS9rctBpubuBP3c38OdKelM2bArlzi1P0JhTRJ0fzcCT43jzd3/+16AqD7Soxj0NA3DRyx8IQtg6+fQTQpS7/OR16Z64/7d37+FVlPeix78z65rryj0hECBAJCBIuCXcClhRbLGV51jkWLXK00NrDyqW7qp4o3X3kbbUyq56RNu93W6qlaMeaYugIm6RChblZgC5Ewi53+9Zt5nzx6ysJCTcw5qs5Pd5nnlm5p131vqtIYv5rXfeeYeN+SWU1rcCEE0zr0W8wDR9L5pqR/3ev3Pj6O+aHG348OlG1wJVke4WlyIu0s53bpwDE7bg+8/vMrzhDOsjnuau1p+z+aDG5oNlRNkt3HRtGt8ZN4DpI5JwWCWpFaI3kkRWCHFVGMlrLRvzS9iYX0JJXWtwW7TDyu0jNH5W+Rui6o6CLRJ14Z9hxA0mRhx++tIjak2ROBzr//oA/ms+KVVH2RT9r/y/4b/i304N4UxNC+/uKeLdPUXEOKzcMCqFm8cMYPbIZJw2Od5C9BaSyAoheozPr7HrVA0fHixjU34JxWclr3NGpTDvunRmRpzA8fbd0FQB0Wlwx19kkPrLEBy1QLoWXD7XIPjhh7DuLiynPmPB4WV87+Zfszt1AX//qoRN+0soq3ezfm8x6/cWE2Gz8M3sFG4ek8Y3s1OIcsixF8JM8g0UQlyRJrePbUcr+PBgGf99qJyaZm9wW5TdwpzRqcwbO4CZ1yTjtKrwzzXw9pOgeSFtLNyxDlwDTfwE4Utu9uohkQlw93rY8BDsfR1l08NMHPslE295jqduGc2ewlo25ZewaX8pRbUtvJdfwnv5JTisKjOvSWbOKGPYr5QYp9mfRIh+RxJZIcQlK69v5aOvy9l8sJTPjlfhCTxlC4yxXr85MoWbrk3rfBm2tQ7efQAO/tVYH30r3Pp/wBFtwifoG+SBCD3IaodbX4TkbPjoF5D/f6FkH+rt/8XEIdlMHBLP4/NGkV9Ux6b9pWzKL6Ggqjk4Ri3AuEEuvpmdyg2jUrg2Pbbfj3MsRChIIiuEuCCvX2PP6Vq2Hinn0yOV5BfVddo+OCGSG0encuPoVCYNicdqOevmo5Ofwvr/DXWFoNrgpl9B3o+N58uKyyYPROhhigLTH4RBk+CtRVB5GF6ZBTesgLz7UFSV6wbFcd2gOB6eO5JDpQ18eKCMjw+Vse9MXXB67qMjpMY6jKQ2O4VpIxL71aOThQgl+WYJIbpVWN3Mp0cr2Hq4gh3Hq2hw+zptH5cRx02jU5kzKpVrUqO7b33yNMGWp43uBADxQ+G2fzcSBXHF5IEIV8mQaXDfP+DdH8PxLfDBcji8EW59wfgbBhRFYdSAWEYNiGXpnCzKG1r570PlbPm6nH8cq6Ss3s1fdp7mLztPY7eoTBgSxzeykpkxIokxA11YVPkRJ0RPkERWCAFAdZOHnSer+PxENZ8ereBERVOn7QlRdmaMSGLmNcnMzEoiJfYC/QEPvQebHjFaYQEmLjJaYqUrQY+RPrJXUXQy3PUOfPkf8OGTULANXpwCM/8Fpj1odEXoICXGycLJg1k4eTCtXj//PFnNlq/L2PJ1OUW1LXx+oprPT1Sz6oPDuCJsTBueyIysJL4xIpnBiTJmshCXSxJZIfqpqkY3O09W8/mJKv55sppDpQ2dtltUhYmD45l5jZG8jkl3oV5MK1LlUfjwCTjyvrHuGgy3PAdZc67Cp+jfpI/sVaYoMPmHMPx6+NuDRjL78b/CV+tg7jMwYk633WOcNguzrklm1jXJ/PK7OgVVzfzjaAXbjlay43gVdS1eo5/t/lIAMhIiyB2aSG5mPLmZiQxNjJT+tUJcJElkhegHdF3nTE0Lu0/XsOtUDZ+fqOJIWWOXeiNTY8gblsC04UlMG5FIrPMSHtfZWA6f/Bp2/SfofqMv7LQHjBYse1TPfRgRJH1kQyRhGNzzd8h/Cz54HCqPwOvfg8xZcOPTkJ5zzl0VRSEzKYrMpCjunjoUn1/jq6I6/nG0kn8crWT36RoKq1sorD7DO7vPAJAU7SA3M57JQxPIzUwgOy1WuiIIcQ7yv58QfVCzx8dXZ+rYfbqGPadr2XO6lspGd5d62Wkx5GUmMGVYIrmZCSRGOy79zRrKYPsfjEuw3maj7JpvwY2/hOSRV/hJxPlIH9kQUhS47nbIugm2/Q7++TKc3GrcDJZ9C8z8+XkT2jZWi8qEwfFMGBzPgzdk0eT28UVBtTGdrGHvGeO7ujG/lI35RottjNNKTkYcORnGjWbjMlwy1JcQAZLIChHmWr1+Dpc2cKC4nv3Fdew9Xcvhsgb8mt6pns2iMDrdxfiMOKYMSyA3M5GEKPs5XvUiVB6Df74Ee/4MvsCDDwZONFqohs64gk8kLpb0kTVBRJzR13vyYqObQf7bcGiDMWXdBFOXGC21F9k1IMphZfbIFGaPTAGM7/NXZ+r4oqCanSer2XWqhoZWH9uOVrLtaGVwv3SXk3EZccY0KI6xg1xEy8MZRD8kf/VChJGGVi8Hi+vZX1zPgeI6DhbXc7S8sUvSCjDA5WT84DgmDI5n/OA4rk13XfmjNTU/HP8Ydv4Rjn7QXj4oF2Y9fM4+g+LqkEfUmih+CNz2J/jGv8C2Z2H/23D0Q2NKGW0MLzfmNnDEXNLLOm0WcjONLgVLrjeelneotIG9hbXsK6zlqzN1HClvoLiuleK69n62AEMSIxmVFhsYTSGG0emxDIyLkP62ok+TRFaIXqjZ4+dkdQsnKls4XtnMicoWTtcdoqi2tdv6CVF2rk2P5dp0F9cNcjF+cBwDXBE9F1DVceMGlz1/hvqiQKEC19wMU34CmTMlgTWBT5euBaZLyYbb/gizHzWGmdvzOpQfhL8vhfcfgzH/A8bdAYOnwGX8O1ktKmMGuhgz0MVdU4YA0Oj2sb+ojn2Ftew7U8u+wjqKals4VdXMqapm3j/QntzGOq1kD4glOzWa1AiNYYmRDE+OIj7yEvq/C9GLSSIrhEk0Xaei0UthTSuna1sprGnlZFULJ6taKK73nHO/gXERjE6P5dr0WMaku7h2YCxpsc6eb3WpOg6HN8H+d6B4d3t5RDxc9z8hdzEkDu/Z9xSXRNONJ6pJi2wvkDgcvr0Krn8c9qyFXa9B1VFjec9aiE6FUd8xnmg3eBpYLv/0G+2wMmVYIlOGJQbLqhrdfF3SwNcl9XxdUs/BknqOVzRS3+pj50mjm0JHLqeVzEQnQxMiGJrgZGhiBEMTIkiNsWOVG8tEGJFEVoiryOPTKGvwUFLvpqTew+maVgoDSWthrRt3h0e7ni0h0sqwxMBJJt7BxBHpZA9wEX8l/VrPx+eB09vhyIdGt4GqY+3bFAsMmwU5dxo3ttjkRpPeoK1rgVWV/8p7jYg4Y7SOqffD6c+NJPbQBmgsgy/+ZEyRSZA9z+iKM3QGRCZc8dsmRjuYkeVgRlZSsMzj0zhW3mgktsV1HDhTxakaN6X1HupafewtamRvUefRSyyqQnqsnUFxTgbFORgU52Sgy8GgOAcDXU6cNvXstxbCVPK/nxBXoNHtp7TBODGU1HecG4lrVZOXrr1X21kUSHc5yIh3khHnZGiCk2FJEQxLiCAucOnP7/fj9/tJTY3Hau3Br6zPDSX74PQO44R7cht4Oowlq1ph8FSjBWn0fGOAeNGrBLsWSIts76MoMGSqMflWG49pPrjeSGqbK2H3a8aEAgPGGT8UM2ca37keGq7OblUZnR7L6PRYbh2XRllZGRaLBa8Gp2paKahq5WR1CwXVLRRUGz+wPX6dwlo3hbVdRzkB4wd2aoyD1Bg7abF2Yx5YT42xkxhlk6HCREhJIivEWXRdp9Htp7LJG5g8VDYay1VNHqMssN7k8V/w9RxWlbQYOwNcDjLijKR1cJyTjHgn6bF2rJYQtHBoGtSchNJ8KNlrJK5Fu8F/1skqKtm48zrrJmMQeKfr6scmLps8ECFMWO3GA0Gy5hgPBynYZnTbObEVKg8b38mSvfDZvxlXP1JGQfp4GDjBmKdc2+VJYlfCabMwMiWKkSmdE2ZN1ylv8FBU56awppUzdW6Kat2cqTWuIDV5/FQ3+6hu9vF1WVO3r21RFVKibaTFOEiOtpEYZSMh0pgHp0gbcZE26cIgeoQksqLP82s6Da0+alt81LT4qG3xUtvi6zB1WG/2UtXsO+8l/7PFOi2kxThIi7UzILbrPD7CGrq7hjUN6s8Y/VurjkHZASjbD2UHwdvNiScyybgJJSMPhk6HAeNBlUuH4UIeiBCGLDYY/k1jAqgvMVprT241Etv6M4Hv7H6jWwKAxQHJ10BytjE2c3K2McVnXlFf27OpikJarIO0WAcTM2I7bdN1nboWH6UNHsoCU2m925gH1isbPfg1nZJ6DyXn6ecPoADxkdZgopsQacMVYcXltAbncRGdl6941BXRJ/WK//1efPFFVq1aRWlpKePGjeP5558nNzf3nPXfeustnnzySQoKCsjKyuI3v/kN3/72t0MYsQgVXddp9Wk0ezSaPX4aPX4aWn00uv00uv00uH00BJYbA8sNgeXGwHKT23/ey/vnEmW3kBRlIynaZsyj7J2Xo2wkRduJdoTwP1dNg6YK42RXVwR1Z4xRBGoKjMS1+mTXVtY2VqfR0pM21khcM6YYN6jIaANhSx6I0AfEDoBxC41J16G+2Li5smg3FO8xllvrjKsppfmd91Vt4BpkDAUWNwTiBkP8UHBlQEyqcYOZrWdGL1EUhbhAS2p2avddH3yaTmVje6JrXMUyrmRVNRmNBFVNXmqajS5Xba270HJRMTisSjDRjbZbiXJYiHZYiLYH5g4L0Q4r0XaLsa1TuYVIuwVV/r/rc0xPZNetW8eyZctYs2YNeXl5rF69mrlz53L48GFSUlK61N++fTt33HEHK1eu5JZbbuGNN95g/vz57N69mzFjxpjwCfovn6bj8Wm4O006bp+Gx6/R6tM6bNc71WvxGolpi9dPs9dYb/G0Lftp9hjzVq92WUlod2IcFuIirMRF2AJza6d1V2A9McpIVq/6r39dB78bxd2A6qk35q21KC1VqMGpGqW5EqWlCou71uhb5z9/SweqzTiZJQ43Wm3SxkLqGEgc0aOtN8J88kCEPkZRwDXQmEZ9xyjTdaNbUPkhqDgEFYeNeeUR40l6NSeN6VwcLohOQY1OJd7qQo9KRo9KQXfGoTlc6I5YY+6MQ3fEotuiLvvHrVVtb9E9H5+mU9vipbrJR1WzkeRWN/moa/VR19I+r+2w7td03D6d8kYv5Y3ey4pPAZw2lQibitNmwWk1ltvW25YjbBYcHbZF2CzB/SJsFuxWBbtFDUwKdmvnuc2iSj/hEFJ0Xe+pPOGy5OXlMXnyZF544QUANE0jIyODBx54gEcffbRL/YULF9LU1MSGDRuCZVOmTCEnJ4c1a9Zc8P3q6+txuVzU1dURGxt7wfpXyu3xUlxaiqJYQFXxazqapuPTdTTN6JPk03Q0nU7lft2o59d1/J3qGduDy7px6dyo176v12/U9/l1vJoWXPf6dXx+DW9wW2Du14LbjbIO62fVc/s03H6920H4rxYFiLCpRDusxDgtxHT49d22HOOwBn95xwTKowLLLqflwn1RNT9oXhTNC/62uc+YdyzTfCh+T2DuRvG1onhbwNdiLHeY421bb0HxtqB4GlA8DajuehRPo/F6l3wwVIhOM052sQONFpm4wUbimjDcaI2RhLVfuGPDHeyv2s+LN7zIzEEzzQ5HhJKmGVdjak9D7SmoOdU+ry8yRknwdT/u9PnoqhXdHmNMtsjAFIFujWxft0Z02YbFjm6xGXM1MLfYwWJDV+3Bdd1iB9UW3IZiuWDirOs6TR4tkOB6qetwVa7RbVypawpciWvyaMErco2e9jq+EJ6vwOgr7AgktQ6rMQ8mvBa1QzKsYLUoWFUFi2rMg5NFwaK0b7eqKhaVDtvVwH4EtwfXA/taVAVVUVAVo46qGssWpeuyRTG6l6hqoEwxXsuiKuiaBprGwPS0nr3p+BwuJVcz9Wzn8XjYtWsXy5cvD5apqsqcOXPYsWNHt/vs2LGDZcuWdSqbO3cu69ev77a+2+3G7W6/1FpfX3/lgV+CFa8tobjpEABKN22LF/rN1t0+F9re+TX1Lhu6224LTBEdt6uByXb2ft2/p6oYc0UxLkMpdFzvsNxWN1DH2E9BUdrKjbdV0APLeuC9daN1gsCkA14NxQM06LTqOq3oVALoWnuds/cD0HUUXQvUM76gIfv9bAEigcjotk/V/p+/1YFudbZPFieaxShzRMeh2iPBFtVNP9YaqP7SmES/UdxUDEiLbL+kqhCXYUxM77pd18FdDw1l0FiGv66YxpJjWFoqsbRWobTWobrrAleD6lHddSh+D4rmQ2mtgdaakH0UXVEDCa2KrloCyxajPLCeqFgC24wyPVAHVW1fVlRjsiroNoDACQXFaPTRFfw6+HXQOix3LPdrBJe9mhIs8+ng67DNp3Xc36ijn30W0UH3KeDrVNT183dz9un+zN9dvfYyDeiuY1n3r39x79mxnt8awT1P/Ee3kZnJ1ES2srIyMKxQaqfy1NRUDh061O0+paWl3dYvLS3ttv7KlSv55S9/2TMBX4YqfT974hsuXFFcRW0pdTjwGpPWAGf3IKg1IRwRFmLsl/YYVNEPKIox6ojTZdwo5vfTklaB33+OkVZ0HcXfGkhq61G9jcZVJG8ziq8ZxdeC6m0OXF1qDl5lUgPb8HtQAletjCtWxvzs5e6uQrU3KoBy4YFgepe200s/uEe2hphe+bjjPn/9cfny5Z1acOvr68nIyAjZ+88f+S1GV3wFtF09CbRUdvPHoF/0H8g56nW7fzdlPV2vU/EF9jlvvQ7lgV/R7XOMX9qdyjrMuyvjfNvVQHnbL3y1w3Lg131vSn6V7v9mhEiPTmds0lizwxC9nMViITk5mVD3JtTPmqPrRlctnxv8XtD9oPmMrhK63+ji1Vama53XA3UUrcP2TvsEsmBda78K12V+9rZATMGreB2v+p1dTud9u+xz9oc/u+xi6lxKvct7LeUi6um60X1RD3Rh1HUdhy0Si6X3Xf0xNZFNSkrCYrFQVlbWqbysrIy0tLRu90lLS7uk+g6HA4fj/B3Pr6Z5s55knmnvLoQQQhh6TxJiA0ek2UGIC+gtfy0XYmpjuN1uZ+LEiWzZsiVYpmkaW7ZsYerUqd3uM3Xq1E71ATZv3nzO+kIIIYQQom8yvWvBsmXLuOeee5g0aRK5ubmsXr2apqYmFi1aBMAPfvADBg4cyMqVKwFYunQps2bN4tlnn2XevHm8+eabfPnll7zyyitmfgwhhBBCCBFipieyCxcupKKigqeeeorS0lJycnJ4//33gzd0nT59GrXDHdrTpk3jjTfe4IknnuCxxx4jKyuL9evXyxiyQgghhBD9jOnjyIZaqMeRFUIIIYQQF+9ScrV+MGCEEEIIIYToiySRFUIIIYQQYcn0PrKh1taTItRP+BJCCCGEEBfWlqNdTO/XfpfINjQYT9kK5UMRhBBCCCHEpWloaMDlcp23Tr+72UvTNIqLi4mJCc2j1tqeJFZYWCg3l4WQHHdzyHE3hxz30JNjbg457uYI9XHXdZ2GhgbS09M7jVzVnX7XIquqKoMGDQr5+8bGxsqXzgRy3M0hx90cctxDT465OeS4myOUx/1CLbFt5GYvIYQQQggRliSRFUIIIYQQYUkS2avM4XCwYsUKHA6H2aH0K3LczSHH3Rxy3ENPjrk55Libozcf9353s5cQQgghhOgbpEVWCCGEEEKEJUlkhRBCCCFEWJJEVgghhBBChCVJZE3w3nvvkZeXR0REBPHx8cyfP9/skPoFt9tNTk4OiqKwd+9es8Pp0woKCvjhD39IZmYmERERDB8+nBUrVuDxeMwOrc958cUXGTp0KE6nk7y8PHbu3Gl2SH3aypUrmTx5MjExMaSkpDB//nwOHz5sdlj9yq9//WsUReGhhx4yO5Q+r6ioiLvuuovExEQiIiIYO3YsX375pdlhdSKJbIi988473H333SxatIh9+/bx2Wef8f3vf9/ssPqFhx9+mPT0dLPD6BcOHTqEpmm8/PLLHDhwgOeee441a9bw2GOPmR1an7Ju3TqWLVvGihUr2L17N+PGjWPu3LmUl5ebHVqftXXrVpYsWcLnn3/O5s2b8Xq93HTTTTQ1NZkdWr/wxRdf8PLLL3PdddeZHUqfV1NTw/Tp07HZbGzatImDBw/y7LPPEh8fb3ZonekiZLxerz5w4ED9T3/6k9mh9DsbN27Us7Oz9QMHDuiAvmfPHrND6nd++9vf6pmZmWaH0afk5ubqS5YsCa77/X49PT1dX7lypYlR9S/l5eU6oG/dutXsUPq8hoYGPSsrS9+8ebM+a9YsfenSpWaH1Kc98sgj+owZM8wO44KkRTaEdu/eTVFREaqqMn78eAYMGMC3vvUt9u/fb3ZofVpZWRmLFy9m7dq1REZGmh1Ov1VXV0dCQoLZYfQZHo+HXbt2MWfOnGCZqqrMmTOHHTt2mBhZ/1JXVwcgf9shsGTJEubNm9fpb15cPX/729+YNGkSCxYsICUlhfHjx/PHP/7R7LC6kEQ2hE6cOAHAL37xC5544gk2bNhAfHw8s2fPprq62uTo+iZd17n33nu57777mDRpktnh9FvHjh3j+eef58c//rHZofQZlZWV+P1+UlNTO5WnpqZSWlpqUlT9i6ZpPPTQQ0yfPp0xY8aYHU6f9uabb7J7925Wrlxpdij9xokTJ3jppZfIysrigw8+4Cc/+QkPPvggr732mtmhdSKJbA949NFHURTlvFNbn0GAxx9/nNtuu42JEyfy6quvoigKb731lsmfIrxc7DF//vnnaWhoYPny5WaH3Cdc7HHvqKioiJtvvpkFCxawePFikyIXouctWbKE/fv38+abb5odSp9WWFjI0qVLef3113E6nWaH029omsaECRN45plnGD9+PD/60Y9YvHgxa9asMTu0TqxmB9AX/OxnP+Pee+89b51hw4ZRUlICwOjRo4PlDoeDYcOGcfr06asZYp9zscf8448/ZseOHV0eqzdp0iTuvPPOXvfLsre72OPepri4mOuvv55p06bxyiuvXOXo+pekpCQsFgtlZWWdysvKykhLSzMpqv7j/vvvZ8OGDXz66acMGjTI7HD6tF27dlFeXs6ECROCZX6/n08//ZQXXngBt9uNxWIxMcK+acCAAZ3yFYBRo0bxzjvvmBRR9ySR7QHJyckkJydfsN7EiRNxOBwcPnyYGTNmAOD1eikoKGDIkCFXO8w+5WKP+R/+8Ad+9atfBdeLi4uZO3cu69atIy8v72qG2Cdd7HEHoyX2+uuvD155UFW5ANST7HY7EydOZMuWLcEh/DRNY8uWLdx///3mBteH6brOAw88wLvvvssnn3xCZmam2SH1eTfccAP5+fmdyhYtWkR2djaPPPKIJLFXyfTp07sMLXfkyJFel69IIhtCsbGx3HfffaxYsYKMjAyGDBnCqlWrAFiwYIHJ0fVNgwcP7rQeHR0NwPDhw6UV5SoqKipi9uzZDBkyhN/97ndUVFQEt0lrYc9ZtmwZ99xzD5MmTSI3N5fVq1fT1NTEokWLzA6tz1qyZAlvvPEGf/3rX4mJiQn2R3a5XERERJgcXd8UExPTpQ9yVFQUiYmJ0jf5KvrpT3/KtGnTeOaZZ7j99tvZuXMnr7zySq+7uiaJbIitWrUKq9XK3XffTUtLC3l5eXz88ce9b1w2Ia7A5s2bOXbsGMeOHevyg0HXdZOi6nsWLlxIRUUFTz31FKWlpeTk5PD+++93uQFM9JyXXnoJgNmzZ3cqf/XVVy/Y7UaIcDJ58mTeffddli9fztNPP01mZiarV6/mzjvvNDu0ThRdzipCCCGEECIMSac1IYQQQggRliSRFUIIIYQQYUkSWSGEEEIIEZYkkRVCCCGEEGFJElkhhBBCCBGWJJEVQgghhBBhSRJZIYQQQggRliSRFUIIIYQQYUkSWSGEEEIIEZYkkRVCCCGEEGFJElkhhBBCCBGWJJEVQogwVFFRQVpaGs8880ywbPv27djtdrZs2WJiZEIIETqKruu62UEIIYS4dBs3bmT+/Pls376dkSNHkpOTw6233srvf/97s0MTQoiQkERWCCHC2JIlS/joo4+YNGkS+fn5fPHFFzgcDrPDEkKIkJBEVgghwlhLSwtjxoyhsLCQXbt2MXbsWLNDEkKIkJE+skIIEcaOHz9OcXExmqZRUFBgdjhCCBFS0iIrhBBhyuPxkJubS05ODiNHjmT16tXk5+eTkpJidmhCCBESksgKIUSY+vnPf87bb7/Nvn37iI6OZtasWbhcLjZs2GB2aEIIERLStUAIIcLQJ598wurVq1m7di2xsbGoqsratWvZtm0bL730ktnhCSFESEiLrBBCCCGECEvSIiuEEEIIIcKSJLJCCCGEECIsSSIrhBBCCCHCkiSyQgghhBAiLEkiK4QQQgghwpIkskIIIYQQIixJIiuEEEIIIcKSJLJCCCGEECIsSSIrhBBCCCHCkiSyQgghhBAiLEkiK4QQQgghwpIkskIIIYQQIiz9f5PAq57zmae+AAAAAElFTkSuQmCC",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# viz: activations (top) and their derivatives (bottom), saturation shaded\n",
"def d_sigmoid(x):\n",
" s = sigmoid(x); return s * (1 - s)\n",
"def relu(x):\n",
" return np.maximum(0.0, x)\n",
"def d_relu(x):\n",
" return (x > 0).astype(float)\n",
"def d_tanh(x):\n",
" return 1 - np.tanh(x) ** 2\n",
"\n",
"xs = np.linspace(-6, 6, 200)\n",
"acts = [(\"sigmoid\", sigmoid, d_sigmoid), (\"tanh\", np.tanh, d_tanh), (\"relu\", relu, d_relu)]\n",
"fig, ax = plt.subplots(2, 1, figsize=(7, 5), sharex=True)\n",
"for name, f, df in acts:\n",
" ax[0].plot(xs, f(xs), label=name)\n",
" ax[1].plot(xs, df(xs), label=name)\n",
"# shade where every smooth activation's derivative is < 0.05 (the \"vanishing\" band for sigmoid)\n",
"sat = np.abs(d_sigmoid(xs)) < 0.05\n",
"ax[1].fill_between(xs, 0, 1, where=sat, color=\"#cccccc\", alpha=0.4, label=\"sigmoid sat. (|deriv|<0.05)\")\n",
"ax[0].set_ylabel(\"activation\"); ax[1].set_ylabel(\"derivative\"); ax[1].set_xlabel(\"x\")\n",
"ax[0].legend(loc=\"upper left\"); ax[1].legend(loc=\"upper left\")\n",
"plt.tight_layout(); plt.show()"
]
},
{
"cell_type": "markdown",
"id": "e620294c",
"metadata": {},
"source": [
"> **Interpretation.** Read the bottom plot. Sigmoid's derivative is a low hill capped at 0.25 and flat-zero past $\\pm 4$: stack a few sigmoid layers and the input gradient is a product of small numbers, which is the vanishing-gradient story. ReLU's derivative is a clean step: 1 where active, 0 where dead, no decay. This single plot is why ReLU replaced sigmoid in deep nets around 2010.\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "a6b62681",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:45.447015Z",
"iopub.status.busy": "2026-06-10T19:14:45.446934Z",
"iopub.status.idle": "2026-06-10T19:14:45.466243Z",
"shell.execute_reply": "2026-06-10T19:14:45.461858Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"sigmoid analytic-deriv vs finite-diff: rel err 1.29e-08\n",
"tanh analytic-deriv vs finite-diff: rel err 1.76e-08\n",
"relu analytic-deriv vs finite-diff: rel err 1.59e-10\n",
"[ ok ] every activation derivative matches finite differences\n"
]
}
],
"source": [
"# falsification check: the derivative identities must match finite differences everywhere\n",
"xs_chk = np.linspace(-5, 5, 50)\n",
"for name, f, df in acts:\n",
" num = grad_check(lambda z: float(f(z)) if np.ndim(z) == 0 else f(z).sum(), xs_chk)\n",
" # ReLU is non-differentiable exactly at 0; grad_check straddles it, so exclude a tiny band\n",
" keep = np.abs(xs_chk) > 1e-3\n",
" err = rel_err(df(xs_chk)[keep], num[keep])\n",
" print(f\"{name:8} analytic-deriv vs finite-diff: rel err {err:.2e}\")\n",
" assert err < 1e-4, f\"{name} derivative disagrees with finite differences (rel err {err:.1e})\"\n",
"print(\"[ ok ] every activation derivative matches finite differences\")"
]
},
{
"cell_type": "markdown",
"id": "1b1ecc1d",
"metadata": {},
"source": [
"Now the forward pass. A 2-layer MLP on a minibatch $X \\in \\mathbb{R}^{n\\times d}$ is one matmul, one element-wise non-linearity, one matmul. The shapes are the only part that bites, so we comment every line.\n"
]
},
{
"cell_type": "markdown",
"id": "4e4786a6",
"metadata": {},
"source": [
"> **Predict:** with `X` of shape `(8, 4)`, `W1` of `(4, 16)`, `W2` of `(16, 3)`, what is the shape of the logits? Answer`(8, 3)`: the batch dimension 8 is carried through, the feature dim goes 4 -> 16 -> 3. The hidden activation `H` is `(8, 16)`.\n"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "d4683b0b",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:45.467308Z",
"iopub.status.busy": "2026-06-10T19:14:45.467213Z",
"iopub.status.idle": "2026-06-10T19:14:45.484969Z",
"shell.execute_reply": "2026-06-10T19:14:45.484186Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"logits shape (8, 3) | hidden shape (8, 16)\n",
"[ ok ] forward-pass shapes\n"
]
}
],
"source": [
"def mlp_forward(X, W1, b1, W2, b2):\n",
" Z1 = X @ W1 + b1 # (n,d)@(d,h) + (h,) -> (n,h) b1 broadcasts over the batch\n",
" H = relu(Z1) # -> (n,h)\n",
" Z2 = H @ W2 + b2 # (n,h)@(h,q) + (q,) -> (n,q) these are logits (no softmax yet)\n",
" return Z2, (X, Z1, H) # cache X, Z1, H: the backward pass needs all three\n",
"\n",
"# randn smoke test after the module, before anything real (lucidrains habit)\n",
"g = np.random.default_rng(SEED)\n",
"d, h, q, n = 4, 16, 3, 8\n",
"W1, b1 = g.standard_normal((d, h)) * np.sqrt(2/d), np.zeros(h) # Kaiming init for ReLU\n",
"W2, b2 = g.standard_normal((h, q)) * np.sqrt(2/h), np.zeros(q)\n",
"logits, cache = mlp_forward(g.standard_normal((n, d)), W1, b1, W2, b2)\n",
"print(\"logits shape\", logits.shape, \"| hidden shape\", cache[2].shape)\n",
"assert logits.shape == (n, q) and cache[2].shape == (n, h), \"forward-pass shapes are wrong\"\n",
"print(\"[ ok ] forward-pass shapes\")"
]
},
{
"cell_type": "markdown",
"id": "b20ef33a",
"metadata": {},
"source": [
"> **Key takeaways**\n",
"> - Sigmoid and tanh saturate; their derivative vanishes for large $|x|$. ReLU's derivative is 1 (active) or 0 (dead), with no decay.\n",
"> - We checked every activation derivative against finite differences, so the backward pass in Part 5 can trust them.\n",
"> - The forward pass is easy and rarely buggy; the hard direction is backward, and it needs the cached `Z1`, `H`.\n"
]
},
{
"cell_type": "markdown",
"id": "ad772d6f",
"metadata": {},
"source": [
"## Part 3: Backprop from scratch: the `Value` engine\n",
"\n",
"> **Objectives.** Build micrograd's scalar autograd, verify each local rule against finite differences, cross-check a whole expression graph against `torch.autograd`, and trigger the `=` vs `+=` bug on purpose.\n",
"\n",
"A computational graph is a DAG whose nodes are values and whose edges are operations. The forward pass builds it; the backward pass walks it in topological-reverse order, calling a `_backward` closure at each node that uses the chain rule to push gradients to its inputs. The only local rules you need:\n",
"\n",
"$$c = a + b \\;\\Rightarrow\\; \\frac{\\partial L}{\\partial a}=\\frac{\\partial L}{\\partial c},\\quad \\frac{\\partial L}{\\partial b}=\\frac{\\partial L}{\\partial c}$$\n",
"$$c = a \\cdot b \\;\\Rightarrow\\; \\frac{\\partial L}{\\partial a}=b\\,\\frac{\\partial L}{\\partial c},\\quad \\frac{\\partial L}{\\partial b}=a\\,\\frac{\\partial L}{\\partial c}$$\n",
"$$c = \\max(0,a) \\;\\Rightarrow\\; \\frac{\\partial L}{\\partial a}=\\mathbb{1}[a>0]\\,\\frac{\\partial L}{\\partial c}$$\n",
"\n",
"> **Common confusion:** when a node feeds several downstream operations, its gradient is the *sum* over those paths. That is why each `_backward` uses `+=`, not `=`. Forgetting this is the canonical micrograd bug, and you will see it bite in Exercise 9.3.\n"
]
},
{
"cell_type": "markdown",
"id": "038d44c3",
"metadata": {},
"source": [
"### Exercise 9.2: Implement the `Value` autograd engine\n",
"`Difficulty 4/5 · ~30 min`\n",
"\n",
"This is the heart of the chapter. Fill in `__add__`, `__mul__`, `__pow__`, `relu`, and `backward` on the `Value` class. Each operation returns a new `Value` and attaches a `_backward` closure that captures its inputs. `backward()` does a topological sort from the output, seeds the output gradient to 1, and runs the closures in reverse. Use the hints freely; this is one of the harder sections, and all the stones need to be in place at once.\n"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "5fd9e934",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:45.485999Z",
"iopub.status.busy": "2026-06-10T19:14:45.485914Z",
"iopub.status.idle": "2026-06-10T19:14:45.530047Z",
"shell.execute_reply": "2026-06-10T19:14:45.529325Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] 9.2 hand-computed grads: not attempted yet — fill in the TODO above, then re-run.\n",
"[ -- ] 9.2 vs torch.autograd: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class Value:\n",
" \"\"\"A scalar and its gradient, plus the graph edges that produced it.\"\"\"\n",
" def __init__(self, data, _children=(), _op=\"\"):\n",
" self.data = float(data)\n",
" self.grad = 0.0\n",
" self._backward = lambda: None # closure set by whichever op created this Value\n",
" self._prev = set(_children)\n",
" self._op = _op\n",
"\n",
" def __add__(self, other):\n",
" other = other if isinstance(other, Value) else Value(other)\n",
" out = Value(self.data + other.data, (self, other), \"+\")\n",
" def _backward():\n",
" # TODO 1: d(a+b)/da = 1 and d(a+b)/db = 1, both times out.grad. ACCUMULATE with +=.\n",
" raise NotImplementedError\n",
" out._backward = _backward\n",
" return out\n",
"\n",
" def __mul__(self, other):\n",
" other = other if isinstance(other, Value) else Value(other)\n",
" out = Value(self.data * other.data, (self, other), \"*\")\n",
" def _backward():\n",
" # TODO 2: d(a*b)/da = b, d(a*b)/db = a, each times out.grad. Use += .\n",
" raise NotImplementedError\n",
" out._backward = _backward\n",
" return out\n",
"\n",
" def __pow__(self, other):\n",
" assert isinstance(other, (int, float)), \"only int/float powers\"\n",
" out = Value(self.data ** other, (self,), f\"**{other}\")\n",
" def _backward():\n",
" # TODO 3: d(a**k)/da = k * a**(k-1), times out.grad. Use += .\n",
" raise NotImplementedError\n",
" out._backward = _backward\n",
" return out\n",
"\n",
" def relu(self):\n",
" out = Value(0.0 if self.data < 0 else self.data, (self,), \"ReLU\")\n",
" def _backward():\n",
" # TODO 4: gradient flows only where out.data > 0\n",
" raise NotImplementedError\n",
" out._backward = _backward\n",
" return out\n",
"\n",
" def backward(self):\n",
" # TODO 5: build a topo order (children before parents) via DFS over ._prev,\n",
" # then set self.grad = 1.0 and call v._backward() for v in reversed(topo).\n",
" raise NotImplementedError\n",
"\n",
" # free convenience ops (these only use the five above; do not edit)\n",
" def __neg__(self): return self * -1\n",
" def __radd__(self, other): return self + other\n",
" def __sub__(self, other): return self + (-other)\n",
" def __rsub__(self, other): return other + (-self)\n",
" def __rmul__(self, other): return self * other\n",
" def __truediv__(self, other): return self * other ** -1\n",
" def __repr__(self): return f\"Value(data={self.data:.4f}, grad={self.grad:.4f})\"\n",
"\n",
"# self-checks (run this cell): a hand-computed value, then a torch.autograd cross-check\n",
"def _check_value_handcomp(V):\n",
" # y = (a*b + c)**2 with a=2,b=-3,c=10 -> ab+c=4, y=16\n",
" # dy/da = 2*(ab+c)*b = -24 ; dy/db = 2*(ab+c)*a = 16 ; dy/dc = 2*(ab+c) = 8\n",
" a, b, c = V(2.0), V(-3.0), V(10.0)\n",
" y = (a * b + c) ** 2\n",
" y.backward()\n",
" check_close(y.data, 16.0, msg=\"forward: (2*-3+10)**2 = 16\")\n",
" check_close([a.grad, b.grad, c.grad], [-24.0, 16.0, 8.0],\n",
" msg=\"hand-computed grads dy/da,dy/db,dy/dc = -24,16,8\")\n",
"\n",
"def _check_value_vs_torch(V):\n",
" g = np.random.default_rng(7)\n",
" av, bv, cv = (float(x) for x in g.standard_normal(3))\n",
" a, b, c = V(av), V(bv), V(cv)\n",
" y = (a * b + c).relu() * c + a ** 3 # an arbitrary graph with a reused node (c, a)\n",
" y.backward()\n",
" ta = torch.tensor(av, requires_grad=True)\n",
" tb = torch.tensor(bv, requires_grad=True)\n",
" tc = torch.tensor(cv, requires_grad=True)\n",
" ty = (ta * tb + tc).relu() * tc + ta ** 3\n",
" ty.backward()\n",
" check_close([a.grad, b.grad, c.grad],\n",
" [ta.grad.item(), tb.grad.item(), tc.grad.item()],\n",
" msg=\"Value grads must match torch.autograd on the same graph\")\n",
"\n",
"check(\"9.2 hand-computed grads\", lambda: _check_value_handcomp(Value))\n",
"check(\"9.2 vs torch.autograd\", lambda: _check_value_vs_torch(Value))"
]
},
{
"cell_type": "markdown",
"id": "9d2c9114",
"metadata": {},
"source": [
"Hint 1 (conceptual)Inside each op, `_backward` is a closure: it captures `self`, `other`, and `out` from the enclosing scope. When `backward()` later calls `out._backward()`, those references are still live, so it can do `self.grad += * out.grad`.\n",
"\n",
"Hint 2 (pseudocode for the five bodies)\n",
"\n",
"```python\n",
"# __add__: self.grad += out.grad; other.grad += out.grad\n",
"# __mul__: self.grad += other.data * out.grad; other.grad += self.data * out.grad\n",
"# __pow__: self.grad += (other * self.data ** (other - 1)) * out.grad\n",
"# relu: self.grad += (out.data > 0) * out.grad\n",
"# backward:\n",
"topo, visited = [], set()\n",
"def build(v):\n",
" if v not in visited:\n",
" visited.add(v)\n",
" for ch in v._prev: build(ch)\n",
" topo.append(v) # node appended AFTER its children -> children first\n",
"build(self)\n",
"self.grad = 1.0\n",
"for v in reversed(topo): v._backward()\n",
"```\n",
"\n",
"\n",
"Help: \"my grads are right for a*b+c but wrong when a node is reused\"You almost certainly wrote `self.grad = ...` somewhere instead of `self.grad += ...`. A reused node receives a contribution from every downstream path; `=` keeps only the last one. This is the exact bug Exercise 9.3 dissects.\n",
"\n",
"Help: \"RecursionError\" or \"the topo order looks wrong\"Recurse into `v._prev` (the children) *before* appending `v`. Appending first gives parents-before-children, and reversing that runs `_backward` in the wrong order so gradients are read before they are written.\n"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "ab7e0535",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:45.531153Z",
"iopub.status.busy": "2026-06-10T19:14:45.531049Z",
"iopub.status.idle": "2026-06-10T19:14:45.564274Z",
"shell.execute_reply": "2026-06-10T19:14:45.563635Z"
},
"collapsed": true,
"jupyter": {
"source_hidden": true
},
"tags": [
"hide-input"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] 9.2 hand-computed grads\n",
"[ ok ] 9.2 vs torch.autograd\n"
]
},
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#@title Solution { display-mode: \"form\" }\n",
"# solution: redefines Value; the checks below re-verify against torch.autograd.\n",
"class Value:\n",
" \"\"\"A scalar and its gradient, plus the graph edges that produced it.\"\"\"\n",
" def __init__(self, data, _children=(), _op=\"\"):\n",
" self.data = float(data)\n",
" self.grad = 0.0\n",
" self._backward = lambda: None\n",
" self._prev = set(_children)\n",
" self._op = _op\n",
"\n",
" def __add__(self, other):\n",
" other = other if isinstance(other, Value) else Value(other)\n",
" out = Value(self.data + other.data, (self, other), \"+\")\n",
" def _backward():\n",
" self.grad += out.grad\n",
" other.grad += out.grad\n",
" out._backward = _backward\n",
" return out\n",
"\n",
" def __mul__(self, other):\n",
" other = other if isinstance(other, Value) else Value(other)\n",
" out = Value(self.data * other.data, (self, other), \"*\")\n",
" def _backward():\n",
" self.grad += other.data * out.grad\n",
" other.grad += self.data * out.grad\n",
" out._backward = _backward\n",
" return out\n",
"\n",
" def __pow__(self, other):\n",
" assert isinstance(other, (int, float)), \"only int/float powers\"\n",
" out = Value(self.data ** other, (self,), f\"**{other}\")\n",
" def _backward():\n",
" self.grad += (other * self.data ** (other - 1)) * out.grad\n",
" out._backward = _backward\n",
" return out\n",
"\n",
" def relu(self):\n",
" out = Value(0.0 if self.data < 0 else self.data, (self,), \"ReLU\")\n",
" def _backward():\n",
" self.grad += (out.data > 0) * out.grad\n",
" out._backward = _backward\n",
" return out\n",
"\n",
" def backward(self):\n",
" topo, visited = [], set()\n",
" def build(v):\n",
" if v not in visited:\n",
" visited.add(v)\n",
" for ch in v._prev:\n",
" build(ch)\n",
" topo.append(v)\n",
" build(self)\n",
" self.grad = 1.0\n",
" for v in reversed(topo):\n",
" v._backward()\n",
"\n",
" def __neg__(self): return self * -1\n",
" def __radd__(self, other): return self + other\n",
" def __sub__(self, other): return self + (-other)\n",
" def __rsub__(self, other): return other + (-self)\n",
" def __rmul__(self, other): return self * other\n",
" def __truediv__(self, other): return self * other ** -1\n",
" def __repr__(self): return f\"Value(data={self.data:.4f}, grad={self.grad:.4f})\"\n",
"\n",
"check(\"9.2 hand-computed grads\", lambda: _check_value_handcomp(Value), required=True)\n",
"check(\"9.2 vs torch.autograd\", lambda: _check_value_vs_torch(Value), required=True)"
]
},
{
"cell_type": "markdown",
"id": "06c99b99",
"metadata": {},
"source": [
"> **Interpretation.** Eighty lines of pure Python reproduce `torch.autograd` on an arbitrary scalar graph, to floating-point agreement. There is no symbolic differentiation and no compiler in here. The library is doing a generalized, tensor-valued version of exactly these closures.\n"
]
},
{
"cell_type": "markdown",
"id": "949cc198",
"metadata": {},
"source": [
"A node used twice must accumulate. The cleanest demonstration: $y = a \\cdot a$ at $a=3$. Analytically $dy/da = 2a = 6$, and the only way to get $6$ is for the two uses of `a` to each contribute $a$ and *add*.\n"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "5db4d9db",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:45.565604Z",
"iopub.status.busy": "2026-06-10T19:14:45.565206Z",
"iopub.status.idle": "2026-06-10T19:14:45.578425Z",
"shell.execute_reply": "2026-06-10T19:14:45.578179Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"y = a*a at a=3 -> y=9.0, da=6.0 (analytic 2a = 6)\n",
"[ ok ] multi-use accumulation works\n"
]
}
],
"source": [
"a = Value(3.0)\n",
"y = a * a # a is a child of y twice over\n",
"y.backward()\n",
"print(f\"y = a*a at a=3 -> y={y.data}, da={a.grad} (analytic 2a = 6)\")\n",
"assert abs(a.grad - 6.0) < 1e-9, \"a*a should give da=6; if you got 3, the second use overwrote the first\"\n",
"print(\"[ ok ] multi-use accumulation works\")"
]
},
{
"cell_type": "markdown",
"id": "0583d36e",
"metadata": {},
"source": [
"> **Notice that** the gradient is `6`, not `3`. If `_backward` had used `=` instead of `+=`, the second contribution would clobber the first and you would read `3`. We make that failure explicit next.\n"
]
},
{
"cell_type": "markdown",
"id": "58f91011",
"metadata": {},
"source": [
"### Exercise 9.3: Break it on purpose, then read the fix\n",
"`Difficulty 3/5 · ~12 min`\n",
"\n",
"This is the chapter's deliberate failure demo. Below is a `BrokenValue` whose `_backward` closures use `=` instead of `+=`. Your job is not to fix it yet, it is to *predict and confirm* exactly how it fails, because recognizing this signature in the wild saves hours. Compute `da` for `y = a*a` at `a=3` with the broken engine and explain the number you get.\n"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "790db791",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:45.579653Z",
"iopub.status.busy": "2026-06-10T19:14:45.579565Z",
"iopub.status.idle": "2026-06-10T19:14:45.593489Z",
"shell.execute_reply": "2026-06-10T19:14:45.593189Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] 9.3 reproduce the overwrite bug: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class BrokenValue(Value):\n",
" \"\"\"Same engine, but the multiply backward OVERWRITES (=) instead of accumulating (+=).\"\"\"\n",
" def __mul__(self, other):\n",
" other = other if isinstance(other, BrokenValue) else BrokenValue(other)\n",
" out = BrokenValue(self.data * other.data, (self, other), \"*\")\n",
" def _backward():\n",
" self.grad = other.data * out.grad # BUG: should be +=\n",
" other.grad = self.data * out.grad # BUG: should be +=\n",
" out._backward = _backward\n",
" return out\n",
"\n",
"def broken_da_for_a_squared():\n",
" # TODO 1: build a = BrokenValue(3.0); y = a * a; y.backward()\n",
" # TODO 2: return a.grad (predict it FIRST, then run to confirm)\n",
" result = None\n",
" attempted(result)\n",
" return result\n",
"\n",
"def _check_broken(fn):\n",
" got = fn()\n",
" # The two writes to a.grad clobber each other; only the last (self.data*out.grad = 3) survives.\n",
" assert abs(got - 3.0) < 1e-9, \\\n",
" f\"the broken engine returns {got}; the overwrite bug should leave exactly 3.0 (not the correct 6.0)\"\n",
"\n",
"check(\"9.3 reproduce the overwrite bug\", lambda: _check_broken(broken_da_for_a_squared))"
]
},
{
"cell_type": "markdown",
"id": "3a6692d3",
"metadata": {},
"source": [
"Hint 1 (conceptual)When `y = a * a` runs `_backward`, it writes to `a.grad` twice. With `=`, the second write replaces the first. Each write stores `other.data * out.grad`; for `a*a` that is `3 * 1 = 3`. So the surviving value is `3`, not the correct `6`.\n",
"\n",
"Hint 2 (the two lines)\n",
"\n",
"```python\n",
"a = BrokenValue(3.0)\n",
"y = a * a; y.backward()\n",
"result = a.grad # this will be 3.0, half of the correct 6.0\n",
"```\n",
"\n",
"\n",
"Help: \"why is half-the-gradient so dangerous?\"Because nothing crashes. The loss still goes down, just along the wrong direction and at the wrong scale. Multi-use nodes are everywhere (shared weights, residual connections, reused activations), so the bug silently corrupts training. The only defense is the finite-difference / torch cross-check you ran in 9.2; the correct engine passed it, the broken one would not.\n"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "6d1b50e2",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:45.594936Z",
"iopub.status.busy": "2026-06-10T19:14:45.594813Z",
"iopub.status.idle": "2026-06-10T19:14:45.613475Z",
"shell.execute_reply": "2026-06-10T19:14:45.613181Z"
},
"collapsed": true,
"jupyter": {
"source_hidden": true
},
"tags": [
"hide-input"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] 9.3 reproduce the overwrite bug\n",
"broken engine da=3.0 -> fixed engine da=6.0 (analytic 6.0)\n",
"[ ok ] bug reproduced and the += fix recovers the correct gradient\n"
]
}
],
"source": [
"#@title Solution { display-mode: \"form\" }\n",
"# solution: the \"fix\" is to use the correct (accumulating) Value; the check confirms the bug exists,\n",
"# then we show the fix recovers the right answer.\n",
"def broken_da_for_a_squared():\n",
" a = BrokenValue(3.0)\n",
" y = a * a\n",
" y.backward()\n",
" return a.grad\n",
"\n",
"check(\"9.3 reproduce the overwrite bug\", lambda: _check_broken(broken_da_for_a_squared), required=True)\n",
"\n",
"# now the fix: the accumulating Value from 9.2 gives the correct 6.0\n",
"a = Value(3.0); y = a * a; y.backward()\n",
"print(f\"broken engine da={broken_da_for_a_squared()} -> fixed engine da={a.grad} (analytic 6.0)\")\n",
"assert abs(a.grad - 6.0) < 1e-9, \"the fixed engine must recover the correct gradient\"\n",
"print(\"[ ok ] bug reproduced and the += fix recovers the correct gradient\")"
]
},
{
"cell_type": "markdown",
"id": "0a3e9611",
"metadata": {},
"source": [
"> **Key takeaways**\n",
"> - The `Value` engine is closures plus a topological sort; nothing more.\n",
"> - It agrees with `torch.autograd` to floating point on arbitrary graphs.\n",
"> - `=` vs `+=` in the backward pass is the canonical micrograd bug: it halves (or worse) the gradient of any reused node, fails silently, and is only caught by a finite-difference or library cross-check.\n"
]
},
{
"cell_type": "markdown",
"id": "7f87cd58",
"metadata": {},
"source": [
"## Part 4: Neuron -> Layer -> MLP\n",
"\n",
"> **Objectives.** Stack the `Value` engine into Karpathy's `Neuron` / `Layer` / `MLP`, count its parameters, and train it on a 4-point toy until the loss collapses.\n",
"\n",
"`micrograd.nn` is another ~50 lines on top of the engine. A `Neuron` holds a list of weight `Value`s and a bias `Value`; calling it computes $\\sum_i w_i x_i + b$ and optionally a ReLU. A `Layer` is a list of `Neuron`s. An `MLP` is a list of `Layer`s, every layer non-linear except the last.\n"
]
},
{
"cell_type": "markdown",
"id": "043310ef",
"metadata": {},
"source": [
"### Exercise 9.4: Build `Neuron`, `Layer`, `MLP`\n",
"`Difficulty 3/5 · ~20 min`\n",
"\n",
"Fill in the three classes. A `Neuron(n_in)` initializes `n_in` weight `Value`s in $[-1,1]$ and a bias `Value(0.0)`; calling it returns `act.relu()` when `nonlin` else the raw `act`. A `Layer` is `n_out` neurons. An `MLP(n_in, [h1, ..., q])` makes every layer non-linear except the last. `parameters()` must return every weight and bias `Value` so training can update them.\n"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "8fb04c68",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:45.614874Z",
"iopub.status.busy": "2026-06-10T19:14:45.614781Z",
"iopub.status.idle": "2026-06-10T19:14:45.656514Z",
"shell.execute_reply": "2026-06-10T19:14:45.656211Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] 9.4 parameter count: not attempted yet — fill in the TODO above, then re-run.\n",
"[ -- ] 9.4 forward returns Value: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class Module:\n",
" def zero_grad(self):\n",
" for p in self.parameters():\n",
" p.grad = 0.0\n",
" def parameters(self):\n",
" return []\n",
"\n",
"class Neuron(Module):\n",
" def __init__(self, n_in, nonlin=True):\n",
" # TODO 1: self.w = list of Value(random.uniform(-1,1)) for each of n_in inputs\n",
" # TODO 2: self.b = Value(0.0)\n",
" self.w = None\n",
" self.b = None\n",
" self.nonlin = nonlin\n",
" def __call__(self, x):\n",
" attempted(self.w, self.b) # not attempted -> the check prints [ -- ], it does not crash\n",
" # TODO 3: act = sum(wi*xi for wi,xi in zip(self.w, x)) + self.b (seed the sum with self.b)\n",
" # TODO 4: return act.relu() if self.nonlin else act\n",
" raise NotImplementedError\n",
" def parameters(self):\n",
" attempted(self.w, self.b)\n",
" return self.w + [self.b]\n",
"\n",
"class Layer(Module):\n",
" def __init__(self, n_in, n_out, **kwargs):\n",
" # TODO 5: self.neurons = [Neuron(n_in, **kwargs) for _ in range(n_out)]\n",
" self.neurons = None\n",
" def __call__(self, x):\n",
" attempted(self.neurons)\n",
" out = [n(x) for n in self.neurons]\n",
" return out[0] if len(out) == 1 else out\n",
" def parameters(self):\n",
" attempted(self.neurons)\n",
" return [p for n in self.neurons for p in n.parameters()]\n",
"\n",
"class MLP(Module):\n",
" def __init__(self, n_in, n_outs):\n",
" self.n_in, self.n_outs = n_in, list(n_outs)\n",
" # TODO 6: sizes = [n_in] + n_outs; build self.layers, layer i non-linear iff i < len(n_outs)-1\n",
" self.layers = None\n",
" def __call__(self, x):\n",
" attempted(self.layers)\n",
" for layer in self.layers:\n",
" x = layer(x)\n",
" return x\n",
" def parameters(self):\n",
" attempted(self.layers)\n",
" return [p for layer in self.layers for p in layer.parameters()]\n",
"\n",
"# self-checks (run this cell): exact parameter count, then a forward returns a Value\n",
"import random as _random\n",
"def _check_paramcount(MLPcls):\n",
" _random.seed(0)\n",
" m = MLPcls(2, [16, 16, 1])\n",
" # params = (2*16+16) + (16*16+16) + (16*1+1) = 48 + 272 + 17 = 337\n",
" n = len(m.parameters())\n",
" assert n == 337, f\"expected 337 parameters for MLP(2,[16,16,1]), got {n}\"\n",
"\n",
"def _check_forward_value(MLPcls):\n",
" _random.seed(0)\n",
" m = MLPcls(2, [4, 4, 1])\n",
" out = m([Value(1.0), Value(-2.0)])\n",
" assert isinstance(out, Value), \"a single-output MLP should return one Value, not a list\"\n",
"\n",
"check(\"9.4 parameter count\", lambda: _check_paramcount(MLP))\n",
"check(\"9.4 forward returns Value\", lambda: _check_forward_value(MLP))"
]
},
{
"cell_type": "markdown",
"id": "dceff028",
"metadata": {},
"source": [
"Hint 1 (conceptual)The load-bearing detail: weights are `Value` objects, not floats. Store raw floats and there are no gradients. `import random` is already aliased as `_random` in the check; in your bodies use `random.uniform(-1, 1)` (the module is imported below in the solution; for your attempt, `import random` at the top of the cell).\n",
"\n",
"Hint 2 (pseudocode)\n",
"\n",
"```python\n",
"# Neuron.__init__\n",
"self.w = [Value(random.uniform(-1, 1)) for _ in range(n_in)]\n",
"self.b = Value(0.0)\n",
"# Neuron.__call__\n",
"act = sum((wi * xi for wi, xi in zip(self.w, x)), start=self.b)\n",
"return act.relu() if self.nonlin else act\n",
"# MLP.__init__\n",
"self.layers = [Layer(sizes[i], sizes[i+1], nonlin=(i < len(n_outs)-1))\n",
" for i in range(len(n_outs))]\n",
"```\n",
"\n",
"\n",
"Help: \"parameter count is off by a few\"Each neuron has `n_in` weights *plus one bias*. For `MLP(2,[16,16,1])`: layer 1 = 16*(2+1)=48, layer 2 = 16*(16+1)=272, layer 3 = 1*(16+1)=17, total 337. If you forgot the bias you get 320.\n"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "2b0ffd5d",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:45.661172Z",
"iopub.status.busy": "2026-06-10T19:14:45.661081Z",
"iopub.status.idle": "2026-06-10T19:14:45.689249Z",
"shell.execute_reply": "2026-06-10T19:14:45.685181Z"
},
"collapsed": true,
"jupyter": {
"source_hidden": true
},
"tags": [
"hide-input"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] 9.4 parameter count\n",
"[ ok ] 9.4 forward returns Value\n",
"MLP(2,[16,16,1]) has 337 parameters\n"
]
}
],
"source": [
"#@title Solution { display-mode: \"form\" }\n",
"# solution: redefines the three classes; checks re-verify.\n",
"import random\n",
"\n",
"class Neuron(Module):\n",
" def __init__(self, n_in, nonlin=True):\n",
" self.w = [Value(random.uniform(-1, 1)) for _ in range(n_in)]\n",
" self.b = Value(0.0)\n",
" self.nonlin = nonlin\n",
" def __call__(self, x):\n",
" act = sum((wi * xi for wi, xi in zip(self.w, x)), start=self.b)\n",
" return act.relu() if self.nonlin else act\n",
" def parameters(self):\n",
" return self.w + [self.b]\n",
"\n",
"class Layer(Module):\n",
" def __init__(self, n_in, n_out, **kwargs):\n",
" self.neurons = [Neuron(n_in, **kwargs) for _ in range(n_out)]\n",
" def __call__(self, x):\n",
" out = [n(x) for n in self.neurons]\n",
" return out[0] if len(out) == 1 else out\n",
" def parameters(self):\n",
" return [p for n in self.neurons for p in n.parameters()]\n",
"\n",
"class MLP(Module):\n",
" def __init__(self, n_in, n_outs):\n",
" sizes = [n_in] + list(n_outs)\n",
" self.layers = [Layer(sizes[i], sizes[i + 1], nonlin=(i < len(n_outs) - 1))\n",
" for i in range(len(n_outs))]\n",
" def __call__(self, x):\n",
" for layer in self.layers:\n",
" x = layer(x)\n",
" return x\n",
" def parameters(self):\n",
" return [p for layer in self.layers for p in layer.parameters()]\n",
"\n",
"check(\"9.4 parameter count\", lambda: _check_paramcount(MLP), required=True)\n",
"check(\"9.4 forward returns Value\", lambda: _check_forward_value(MLP), required=True)\n",
"random.seed(SEED)\n",
"_demo = MLP(2, [16, 16, 1])\n",
"print(f\"MLP(2,[16,16,1]) has {len(_demo.parameters())} parameters\")"
]
},
{
"cell_type": "markdown",
"id": "79583831",
"metadata": {},
"source": [
"Now train it. The 4-point toy (two positives on one diagonal, two negatives on the other) is XOR-shaped, so a linear model cannot fit it but a 2-hidden-layer MLP can. We use a hinge-style loss $\\sum_i \\max(0, 1 - y_i s_i)$ with labels in $\\{-1, +1\\}$, full-batch gradient descent, and re-seed first so the run reproduces.\n"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "755c664d",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:45.694471Z",
"iopub.status.busy": "2026-06-10T19:14:45.694367Z",
"iopub.status.idle": "2026-06-10T19:14:56.315560Z",
"shell.execute_reply": "2026-06-10T19:14:56.315182Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"first loss 11.4100 -> last loss 0.0000 (150 steps)\n"
]
}
],
"source": [
"# the 4-point toy (salvaged from the chapter lab); labels in {-1,+1}\n",
"xs_toy = [[2.0, 3.0], [3.0, -1.0], [0.5, 1.0], [1.0, 1.0]]\n",
"ys_toy = [1.0, -1.0, -1.0, 1.0]\n",
"\n",
"def hinge_loss(model, xs, ys):\n",
" scores = [model(x) for x in xs]\n",
" losses = [(1 + -yi * si).relu() for yi, si in zip(ys, scores)] # max(0, 1 - y*s)\n",
" return sum(losses, start=Value(0.0))\n",
"\n",
"random.seed(SEED) # re-seed: identical init on every re-run\n",
"toy = MLP(2, [8, 8, 1])\n",
"STEPS_TOY = 20 if FAST else 150 # FAST cuts ~10x; full run easily collapses the loss\n",
"lr = 0.05 # small full-batch step; larger diverges on this tiny set\n",
"loss_log = []\n",
"for step in range(STEPS_TOY):\n",
" loss = hinge_loss(toy, xs_toy, ys_toy)\n",
" toy.zero_grad()\n",
" loss.backward()\n",
" for p in toy.parameters():\n",
" p.data -= lr * p.grad # SGD update on .data (never reassign p itself)\n",
" loss_log.append(loss.data)\n",
"print(f\"first loss {loss_log[0]:.4f} -> last loss {loss_log[-1]:.4f} ({STEPS_TOY} steps)\")"
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "e65625e3",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:56.323444Z",
"iopub.status.busy": "2026-06-10T19:14:56.323326Z",
"iopub.status.idle": "2026-06-10T19:14:56.985462Z",
"shell.execute_reply": "2026-06-10T19:14:56.984816Z"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA3wAAAE1CAYAAAC83OKvAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAZRJJREFUeJzt3XdUU/f/BvAnAwLIRqYioFhluFEKOHCvVqmtVmsVR9U66vrWVn+tWrWK2rqqVqxtXXXUuqq2DtyjigriHqgsUURFhmzI/f0BRCOgAQIJ8LzOyTnm5ubmyQW5eeezRIIgCCAiIiIiIqIqR6zpAERERERERFQ+WPARERERERFVUSz4iIiIiIiIqigWfERERERERFUUCz4iIiIiIqIqigUfERERERFRFcWCj4iIiIiIqIpiwUdERERERFRFseAjIiIiIiKqoljwEWmIo6MjhgwZUqrn+vr6wtfXV615VFWW3EREmnThwgV4e3ujRo0aEIlECAsLw3fffQeRSFSur3v8+HGIRCIcP368XF+nomnqeiCXy+Hu7o65c+dW+GtT2R04cACGhoZ48uSJpqNUGyz4iIrx33//4bvvvkNiYqKmoxARURllZ2ejb9++SEhIwJIlS7Bx40Y4ODhoOhaVwpYtWxATE4Nx48ap5XiZmZn4+uuvYWdnB319fXh6eiIoKEgtx67Ofv75Z6xbt67Q9m7dusHZ2RkBAQEVH6qaEgmCIGg6BJE2+vHHHzFlyhRERETA0dFR7cfPzMyEWCyGjo5OiZ+blZUFANDV1VV3rLdydHSEr69vkX/EiYi01a1bt+Di4oI1a9bgs88+U2zPyclBTk4O9PT0yu21jx8/jvbt2+PYsWMa651RHspyHSuLpk2bwtPTE6tXr1bL8QYMGIDt27dj4sSJqF+/PtatW4cLFy7g2LFjaN26tVpeozpyd3dHzZo1i2zZXrVqFb788kvExcXByMio4sNVM2zhI1IDuVyOjIyMEj1HJpOV+iKpq6urkWKPiKiyio+PBwCYmpoqbZdKpeVa7FUWOTk5ii8TVVWW61hpXbp0CZcvX0a/fv2Utj99+hSPHj0q8fHOnz+PrVu3IiAgAD/88ANGjhyJo0ePwsHBAV999ZW6YmuF1NTUYh9LS0urwCTAhx9+iMzMTPz1118V+rrVFQs+oiJ89913mDJlCgDAyckJIpEIIpEIkZGRAACRSIRx48Zh06ZNcHNzg0wmw4EDBwDktQx6e3vDwsIC+vr6aNGiBbZv317oNV4f+7Bu3TqIRCKcOXMGkydPhqWlJWrUqIEPPvigUD/318fwFYwP2bZtG+bOnYvatWtDT08PHTt2xN27dwu99sqVK1G3bl3o6+ujVatWOHXqVJnGBd6/fx99+/aFubk5DAwM8O677+Kff/4ptN/y5cvh5uYGAwMDmJmZwcPDA5s3b1Y8npKSgokTJ8LR0REymQxWVlbo3LkzQkNDS5WLiAgAhgwZgnbt2gEA+vbtC5FIpPh7V9QYvoK/8bt374a7uztkMhnc3NwUf+cLREVFYcyYMWjQoAH09fVhYWGBvn37Kq4VJVWQ5c6dO/j0009hYmICS0tLTJ8+HYIgICYmBr1794axsTFsbGywaNEipednZWVhxowZaNGiBUxMTFCjRg20adMGx44dU9ovMjISIpEIP/74I5YuXYp69epBJpPhxo0bAPKuKR4eHtDT00O9evWwevXqIs9TWa5jFy9eRNeuXVGzZk3o6+vDyckJw4YNe+s52r17N3R1ddG2bVul7deuXUOdOnXQu3dv7NmzBzk5OW89FgBs374dEokEI0eOVGzT09PD8OHDcfbsWcTExKh0HHW5desW+vXrB0tLS+jr66NBgwb45ptvlPa5dOkSunfvDmNjYxgaGqJjx444d+6c0j4FP4sTJ05gzJgxsLKyQu3atQHkfYZwd3dHSEgI2rZtCwMDA/zf//0fgLxW25kzZ8LZ2RkymQz29vb46quvkJmZWSjrH3/8gVatWimu6W3btsWhQ4cA5P1uXL9+HSdOnFB8hnr1M4aVlRUaN26Mv//+W52nj4oh1XQAIm3Up08f3LlzB1u2bMGSJUtQs2ZNAIClpaVin6NHj2Lbtm0YN24catasqej2uWzZMvTq1QsDBw5EVlYWtm7dir59+2Lfvn3o2bPnW1/7iy++gJmZGWbOnInIyEgsXboU48aNw59//vnW586fPx9isRhffvklkpKSsHDhQgwcOBDBwcGKfVatWoVx48ahTZs2mDRpEiIjI+Hn5wczMzPFxaAkHj9+DG9vb6SlpWH8+PGwsLDA+vXr0atXL2zfvh0ffPABAGDNmjUYP348PvroI0yYMAEZGRm4cuUKgoOD8cknnwAAPv/8c2zfvh3jxo2Dq6srnj17htOnT+PmzZto3rx5ibMREQHAqFGjUKtWLcybNw/jx49Hy5YtYW1t/cbnnD59Gjt37sSYMWNgZGSEn376CR9++CGio6NhYWEBIG8SmP/++w/9+/dH7dq1ERkZiVWrVsHX1xc3btyAgYFBqfJ+/PHHcHFxwfz58/HPP//g+++/h7m5OVavXo0OHTpgwYIF2LRpE7788ku0bNlSUfwkJyfj119/xYABAzBixAikpKTgt99+Q9euXXH+/Hk0bdpU6XXWrl2LjIwMjBw5EjKZDObm5rh06RK6desGW1tbzJo1C7m5uZg9e7bS9e9t3nYdi4+PR5cuXWBpaYmpU6fC1NQUkZGR2Llz51uP/d9//8Hd3b1Qy2LTpk0xffp0rFu3Dr1794atrS38/f0xbNgw1K9fv9jjXbp0Ce+88w6MjY2Vtrdq1QoAEBYWBnt7+2Kfn5aWplLrmEQigZmZ2Rv3uXLlCtq0aQMdHR2MHDkSjo6OuHfvHvbu3auYoOb69eto06YNjI2N8dVXX0FHRwerV6+Gr68vTpw4AU9PT6VjjhkzBpaWlpgxY4ZSC9+zZ8/QvXt39O/fH59++imsra0hl8vRq1cvnD59GiNHjoSLiwuuXr2KJUuW4M6dO9i9e7fi+bNmzcJ3330Hb29vzJ49G7q6uggODsbRo0fRpUsXLF26FF988QUMDQ0VBevr/+datGihdEwqRwIRFemHH34QAAgRERGFHgMgiMVi4fr164UeS0tLU7qflZUluLu7Cx06dFDa7uDgIPj7+yvur127VgAgdOrUSZDL5YrtkyZNEiQSiZCYmKjY1q5dO6Fdu3aK+8eOHRMACC4uLkJmZqZi+7JlywQAwtWrVwVBEITMzEzBwsJCaNmypZCdna3Yb926dQIApWMW5/XcEydOFAAIp06dUmxLSUkRnJycBEdHRyE3N1cQBEHo3bu34Obm9sZjm5iYCGPHjn1rBiKikir4O/nXX38pbZ85c6bw+schAIKurq5w9+5dxbbLly8LAITly5crtr3+914QBOHs2bMCAGHDhg2FXvvYsWNvzFiQZeTIkYptOTk5Qu3atQWRSCTMnz9fsf358+eCvr6+0t/jnJwcpWtAwX7W1tbCsGHDFNsiIiIEAIKxsbEQHx+vtP/7778vGBgYCLGxsYpt4eHhglQqLXSeSnsd27VrlwBAuHDhwhvPR1Fq164tfPjhh8U+LpfLhaNHjwqffvqpoK+vLwAQ2rZtK6xfv77In5ebm1uh67MgCML169cFAEJgYOAb8xT8zN52c3BweOt7a9u2rWBkZCRERUUVek8F/Pz8BF1dXeHevXuKbQ8fPhSMjIyEtm3bKrYV/Cxat24t5OTkKB2vXbt2Rb63jRs3CmKxWOl6LgiCEBgYKAAQzpw5IwhC3u+DWCwWPvjgA8U1vqisbm5ub/xcMW/ePAGA8Pjx42L3IfVgl06iUmrXrh1cXV0LbdfX11f8+/nz50hKSkKbNm1U7pY4cuRIpW4zbdq0QW5uLqKiot763KFDhyqN7WvTpg2AvC6XQF4XmmfPnmHEiBGQSl828A8cOPCt3zwW599//0WrVq2UBrYbGhpi5MiRiIyMVHQRMjU1xYMHD3DhwoVij2Vqaorg4GA8fPiwVFmIiNSlU6dOqFevnuJ+48aNYWxsrPh7Cij/vc/OzsazZ8/g7OwMU1PTMnVFf3VSGYlEAg8PDwiCgOHDhyu2m5qaokGDBkp5JBKJ4hogl8uRkJCAnJwceHh4FJnnww8/VGq5y83NxeHDh+Hn5wc7OzvFdmdnZ3Tv3l3l/G+7jhWMo9y3bx+ys7NVPi6Q1zL1puuVSCRC+/btsXHjRsTFxSEwMBCZmZnw9/eHra0tRo8ejefPnyv2T09Ph0wmK3ScgnGd6enpb8wzePBgBAUFvfW2adOmNx7nyZMnOHnyJIYNG4Y6deoUek9A3s/n0KFD8PPzQ926dRWP29ra4pNPPsHp06eRnJys9NwRI0ZAIpEUej2ZTIahQ4cqbfvrr7/g4uKChg0b4unTp4pbhw4dAEDRNXj37t2Qy+WYMWMGxGLlUqIkS5wU/ByfPn2q8nOodNilk6iUnJycity+b98+fP/99wgLC1Pq867qH8HX/9AX/EF89QJV2ucWXGydnZ2V9pNKpaWeiTQqKqpQFxIAcHFxUTzu7u6Or7/+GocPH0arVq3g7OyMLl264JNPPoGPj4/iOQsXLoS/vz/s7e3RokUL9OjRA4MHD1a6sBERVYTX/54CeX9TXy8WAgICsHbtWsTGxkJ4ZeLzpKQktb22iYkJ9PT0FMMLXt3+7NkzpW3r16/HokWLcOvWLaViqqhr1uvb4uPjkZ6eXugaARS+bpQk/+vXonbt2uHDDz/ErFmzsGTJEvj6+sLPzw+ffPJJkcXX6wQVJ5g3NjbGqFGj4O/vj7lz52Lu3LkIDAzEqFGjFJn09fWLHJ9WMBHbq0V9UerWrauWa1RB4e7u7l7sPk+ePEFaWhoaNGhQ6DEXFxfI5XLExMTAzc1Nsb24zyq1atUqNPlbeHg4bt68WWz33YKJj+7duwexWFzkl94lUfBzLO91MIkFH1GpFXUROHXqFHr16oW2bdvi559/hq2tLXR0dLB27VqlyUnepKhv4gDVLnBleW55c3Fxwe3bt7Fv3z4cOHAAO3bswM8//4wZM2Zg1qxZAIB+/fqhTZs22LVrFw4dOoQffvgBCxYswM6dO0v07TIRUVmp8vf0iy++wNq1azFx4kR4eXnBxMQEIpEI/fv3h1wuV+trq5Lnjz/+wJAhQ+Dn54cpU6bAysoKEokEAQEBuHfvXqHnvq2YKa23ZRWJRNi+fTvOnTuHvXv34uDBgxg2bBgWLVqEc+fOwdDQsNhjW1hYqPQFKJA3xvL333/H1q1bkZiYCE9PTwwfPlzxhSSQ1zoWGxtb6LkFM36+2tJZlBcvXuDFixdvzSKRSEo0DlJdivsZF7VdLpejUaNGWLx4cZHPedNYxtIo+Dm+/kUGqR8LPqJilOYbpx07dkBPTw8HDx5U+pZy7dq16oxWagWLDN+9exft27dXbM/JyUFkZCQaN25cqmPevn270PZbt24pvSYA1KhRAx9//DE+/vhjZGVloU+fPpg7dy6mTZum6D5ja2uLMWPGYMyYMYiPj0fz5s0xd+5cFnxEpHW2b98Of39/pdkyMzIykJiYqLE8devWxc6dO5WuYTNnzlTp+VZWVtDT0ytydueitpXVu+++i3fffRdz587F5s2bMXDgQGzdulWpS+vrGjZsiIiIiGIfj4+Px8aNG7F27Vpcv34dFhYWGDJkCIYPH15k61nTpk1x7NgxJCcnK03cUjDZ2esT3bzuxx9/VHxp+SYODg5vnL21oJXw2rVrxe5jaWkJAwODYq+5YrG4TEVZvXr1cPnyZXTs2PGNn4Hq1asHuVyOGzduvPH8vO1zVEREBGrWrKmRQri64Rg+omLUqFEDAEp04ZZIJBCJRMjNzVVsi4yM1JpZqDw8PGBhYYE1a9YoTVm9adMmlb8xfV2PHj1w/vx5nD17VrEtNTUVv/zyCxwdHRVdPl7vdqSrqwtXV1cIgoDs7Gzk5uYW6gJlZWUFOzu7IrvbEBFpmkQiKdSDYvny5UrXgIrOAyi3+gUHByv9fX7b8zt16oTdu3crjaW+e/cu9u/fr7acz58/L3TeCgqHt/299/LywrVr1wrtFxMTAz8/P9SqVQtTpkyBra0ttm7diocPH2LJkiXFdpX86KOPkJubi19++UWxLTMzE2vXroWnp+dbCyh1jeGztLRE27Zt8fvvvyM6OlrpsYJzJZFI0KVLF/z9999KxePjx4+xefNmtG7dutBsoyXRr18/xMbGYs2aNYUeS09PV8zy6efnB7FYjNmzZxdqyX7151qjRo03foYKCQmBl5dXqfOS6tjCR1SMFi1aAAC++eYb9O/fHzo6Onj//fcVhWBRevbsicWLF6Nbt2745JNPEB8fj5UrV8LZ2RlXrlypqOjF0tXVxXfffYcvvvgCHTp0QL9+/RAZGYl169ahXr16pWrVnDp1KrZs2YLu3btj/PjxMDc3x/r16xEREYEdO3YoBnR36dIFNjY28PHxgbW1NW7evIkVK1agZ8+eMDIyQmJiImrXro2PPvoITZo0gaGhIQ4fPowLFy4UWmuKiEgbvPfee9i4cSNMTEzg6uqKs2fP4vDhw4plGzSRZ+fOnfjggw/Qs2dPREREIDAwEK6urip1OwTy1gI8dOgQfHx8MHr0aOTm5mLFihVwd3dHWFiYWnKuX78eP//8Mz744APUq1cPKSkpWLNmDYyNjdGjR483Prd3796YM2cOTpw4gS5duii237t3D6GhoZg2bRqGDRum8rh0T09P9O3bF9OmTUN8fDycnZ2xfv16REZG4rfffnvr89U1hg8AfvrpJ7Ru3RrNmzfHyJEj4eTkhMjISPzzzz+Kc//9998jKCgIrVu3xpgxYyCVSrF69WpkZmZi4cKFZXr9QYMGYdu2bfj8889x7Ngx+Pj4IDc3F7du3cK2bdtw8OBBeHh4wNnZGd988w3mzJmDNm3aoE+fPpDJZLhw4QLs7OwQEBAAIO9z1KpVq/D999/D2dkZVlZWiglg4uPjceXKFYwdO7ZMmUk1LPiIitGyZUvMmTMHgYGBOHDgAORyOSIiIt5Y8HXo0AG//fYb5s+fj4kTJ8LJyQkLFixAZGSkVhR8ADBu3DgIgoBFixbhyy+/RJMmTbBnzx6MHz9e0a2yJKytrfHff//h66+/xvLly5GRkYHGjRtj7969SusOjho1Cps2bcLixYvx4sUL1K5dG+PHj8e3334LADAwMMCYMWNw6NAh7Ny5E3K5HM7Ozvj5558xevRotb1/IiJ1WbZsGSQSCTZt2oSMjAz4+Pjg8OHD6Nq1q0byDBkyBHFxcVi9ejUOHjwIV1dX/PHHH/jrr79w/PhxlY7RokUL7N+/H19++SWmT58Oe3t7zJ49Gzdv3lR01S+rdu3a4fz589i6dSseP34MExMTtGrVCps2bSp2kpFX8zVu3Bjbtm1TKvg8PT0RGRlZaNZIVWzYsAHTp0/Hxo0b8fz5czRu3Bj79u0rtLh7eWvSpAnOnTuH6dOnY9WqVcjIyICDgwP69eun2MfNzQ2nTp3CtGnTEBAQALlcDk9PT/zxxx9FTqBWEmKxGLt378aSJUuwYcMG7Nq1CwYGBqhbty4mTJiAd955R7Hv7Nmz4eTkhOXLl+Obb76BgYEBGjdujEGDBin2mTFjBqKiorBw4UKkpKSgXbt2ioJv586dkMlkSu+Nyo9I0IbZHIhIo+RyOSwtLdGnT58iu3IQEVH15ufnh+vXryM8PFzTUbBx40aMHTsW0dHRiiUeqHJp1qwZfH19sWTJEk1HqRY4ho+omsnIyCg0dmLDhg1ISEiAr6+vZkIREZHWeH3tufDwcPz7779ac40YOHAg6tSpg5UrV2o6CpXCgQMHEB4ejmnTpmk6SrXBFj6iaub48eOYNGkS+vbtCwsLC4SGhuK3336Di4sLQkJCCq3LQ0RE1YutrS2GDBmCunXrIioqCqtWrUJmZiYuXbqE+vXrazoeEZUQx/ARVTOOjo6wt7fHTz/9hISEBJibm2Pw4MGYP38+iz0iIkK3bt2wZcsWxMXFQSaTwcvLC/PmzWOxR1RJsYWPiIiIiIioiuIYPiIiIiIioiqqynfplMvlePjwIYyMjEq1xhgREVVegiAgJSUFdnZ2pZquXRvxukZEVH2V5rpW5Qu+hw8fwt7eXtMxiIhIg2JiYlC7dm1Nx1ALXteIiKgk17UqX/AZGRkByDspxsbGGk5DREQVKTk5Gfb29oprQVXA6xoRUfVVmutalS/4Crq7GBsb88JIRFRNVaWuj7yuERFRSa5rVWNAAxERERERERXCgo+IiIiIiKiKYsFHRERERERURbHgIyIiIiIiqqJY8BEREREREVVRLPiIiIiIiIiqqCq/LIM67LvyELcepaCDixWa1zHTdBwiIiIiIiKVsOBTwf5rcfjnyiPUNNRlwUdERERERJUGCz4V6Eryer5m5woaTkJERKQ+Du01nUD7RR3TdAIiorLhGD4VSMV5K9lny+UaTkJERERERKQ6FnwqkOa38OWwhY+IiIiIiCoRFnwq0JXkt/DlsoWPiIiIiIgqDxZ8KpByDB8REREREVVCLPhUIM1v4cthCx8REREREVUiLPhUoCPOH8MnZwsfERERERFVHiz4VKCT36Uziy18RERERERUibDgUwG7dBIRERERUWXEgk8FOoqCj106iYiIiIio8mDBpwJ26SQiIiIiosqIBZ8KuPA6ERERERFVRiz4VKAjzu/SKWcLHxERERERVR4s+FTwsksnW/iIiIiIiKjyYMGnAs7SSURERERElRELPhXocAwfERERERFVQiz4VCDNH8OXzTF8RERERERUibDgU4GONO80ZbNLJxERERERVSIs+FSgI2aXTiIiIiIiqnxY8KmgYNIWtvAREREREVFlwoJPBQWTtmSzhY+IiIiIiCoRjRZ8J0+exPvvvw87OzuIRCLs3r1b6XFBEDBjxgzY2tpCX18fnTp1Qnh4eIXn1OGyDEREREREVAlptOBLTU1FkyZNsHLlyiIfX7hwIX766ScEBgYiODgYNWrUQNeuXZGRkVGhOaX5Y/iy5WzhIyIiIiKiykOqyRfv3r07unfvXuRjgiBg6dKl+Pbbb9G7d28AwIYNG2BtbY3du3ejf//+FZZTV8oxfEREREREVPlo7Ri+iIgIxMXFoVOnToptJiYm8PT0xNmzZ4t9XmZmJpKTk5VuZSXlLJ1ERERERFQJaW3BFxcXBwCwtrZW2m5tba14rCgBAQEwMTFR3Ozt7cuchbN0EhERERFRZaS1BV9pTZs2DUlJSYpbTExMmY9ZMEtnDsfwERERERFRJaK1BZ+NjQ0A4PHjx0rbHz9+rHisKDKZDMbGxkq3sioo+HLlAuQs+oiIiIiIqJLQ2oLPyckJNjY2OHLkiGJbcnIygoOD4eXlVaFZCrp0AkC2nN06iYiIiIioctBowffixQuEhYUhLCwMQN5ELWFhYYiOjoZIJMLEiRPx/fffY8+ePbh69SoGDx4MOzs7+Pn5VWhOHfHL08SJW4iIqCxyc3Mxffp0ODk5QV9fH/Xq1cOcOXMgCLy+EBGR+ml0WYaLFy+iffv2ivuTJ08GAPj7+2PdunX46quvkJqaipEjRyIxMRGtW7fGgQMHoKenV6E5dV5t4ePELUREVAYLFizAqlWrsH79eri5ueHixYsYOnQoTExMMH78eE3HIyKiKkajBZ+vr+8bv9EUiUSYPXs2Zs+eXYGpCpOIXy34+A0sERGV3n///YfevXujZ8+eAABHR0ds2bIF58+f13AyIiKqirR2DJ82EYlEila+HI7hIyKiMvD29saRI0dw584dAMDly5dx+vRpdO/evcj9y2N9WSIiqj402sJXmUjFYmTn5nIMHxERlcnUqVORnJyMhg0bQiKRIDc3F3PnzsXAgQOL3D8gIACzZs2q4JRERFRVsIVPRQUtfFkcw0dERGWwbds2bNq0CZs3b0ZoaCjWr1+PH3/8EevXry9y//JYX5aIiKoPtvCpSLH4Olv4iIioDKZMmYKpU6eif//+AIBGjRohKioKAQEB8Pf3L7S/TCaDTCar6JhERFRFsIVPRQVr8XGWTiIiKou0tDSIxcqXX4lEAjnHiBMRUTlgC5+KClr4WPAREVFZvP/++5g7dy7q1KkDNzc3XLp0CYsXL8awYcM0HY2IiKogFnwqUnTplLNLJxERld7y5csxffp0jBkzBvHx8bCzs8OoUaMwY8YMTUcjIqIqiAWfiqRidukkIqKyMzIywtKlS7F06VJNRyEiomqAY/hU9LJLJ1v4iIiIiIiocmDBpyLFwuts4SMiIiIiokqCBZ+KpGzhIyIiIiKiSoYFn4oKxvDlcNpsIiIiIiKqJFjwqUhXymUZiIiIiIiocmHBp6KXs3SySycREREREVUOLPhUVDCGL4cFHxERERERVRIs+FSkK2GXTiIiIiIiqlxY8KlIKuHC60REREREVLmw4FORVJzfpVPOLp1ERERERFQ5sOBTka40v4Uvhy18RERERERUObDgU1FBC182W/iIiIiIiKiSYMGnooIxfDkcw0dERERERJUECz4V6Ug4ho+IiIiIiCoXFnwq0slv4cviGD4iIiIiIqokWPCp6OUsnSz4iIiIiIiocmDBpyIdxRg+dukkIiIiIqLKQasLvtzcXEyfPh1OTk7Q19dHvXr1MGfOHAhCxRddBWP4sjhpCxERERERVRJSTQd4kwULFmDVqlVYv3493NzccPHiRQwdOhQmJiYYP358hWaRFkzawhY+IiIiIiKqJLS64Pvvv//Qu3dv9OzZEwDg6OiILVu24Pz588U+JzMzE5mZmYr7ycnJasmi6NLJMXxERERERFRJaHWXTm9vbxw5cgR37twBAFy+fBmnT59G9+7di31OQEAATExMFDd7e3u1ZFF06cxhCx8REREREVUOWt3CN3XqVCQnJ6Nhw4aQSCTIzc3F3LlzMXDgwGKfM23aNEyePFlxPzk5WS1Fn1TMFj4iIiIiIqpctLrg27ZtGzZt2oTNmzfDzc0NYWFhmDhxIuzs7ODv71/kc2QyGWQymdqz6HAMHxERERERVTJaXfBNmTIFU6dORf/+/QEAjRo1QlRUFAICAoot+MqLNH8MXzZn6SQiIiIiokpCq8fwpaWlQSxWjiiRSCDXQLfKghY+FnxERERERFRZaHUL3/vvv4+5c+eiTp06cHNzw6VLl7B48WIMGzaswrO8nKWTXTqJiIiIiKhy0OqCb/ny5Zg+fTrGjBmD+Ph42NnZYdSoUZgxY0aFZ5GKC1r4WPAREREREVHloNUFn5GREZYuXYqlS5dqOgq7dBIRERERUaWjljF8iYmJ6jiMVlN06WTBR0RERERElUSJC74FCxbgzz//VNzv168fLCwsUKtWLVy+fFmt4bSJVMIunUREREREVLmUuOALDAxULGQeFBSEoKAg7N+/H927d8eUKVPUHlBbFCy8zi6dRERERERUWZR4DF9cXJyi4Nu3bx/69euHLl26wNHREZ6enmoPqC10pfkLr3OWTiIiIiIiqiRK3MJnZmaGmJgYAMCBAwfQqVMnAIAgCMjNzVVvOi3CFj4iIiIiIqpsSlzw9enTB5988gk6d+6MZ8+eoXv37gCAS5cuwdnZWe0BtUXBLJ05HMNHRFQtJSYm4tdff8W0adOQkJAAAAgNDUVsbKyGkxERERWvxF06lyxZAkdHR8TExGDhwoUwNDQEADx69AhjxoxRe0BtwWUZiIiqrytXrqBTp04wMTFBZGQkRowYAXNzc+zcuRPR0dHYsGGDpiMSEREVqcQtfDo6Ovjyyy+xbNkyNGvWTLF90qRJ+Oyzz9QaTptIC5ZlkAsQBLbyERFVJ5MnT8aQIUMQHh4OPT09xfYePXrg5MmTJT5ebGwsPv30U1hYWEBfXx+NGjXCxYsX1RmZiIgIQCkKvvXr1+Off/5R3P/qq69gamoKb29vREVFqTWcNtERvzxVnLiFiKh6uXDhAkaNGlVoe61atRAXF1eiYz1//hw+Pj7Q0dHB/v37cePGDSxatAhmZmbqiktERKRQ4oJv3rx50NfXBwCcPXsWK1euxMKFC1GzZk1MmjRJ7QG1hY5UpPg3u3USEVUvMpkMycnJhbbfuXMHlpaWJTrWggULYG9vj7Vr16JVq1ZwcnJCly5dUK9evSL3z8zMRHJystKNiIhIVSUu+GJiYhSTs+zevRsffvghRo4ciYCAAJw6dUrtAbWF9JUWPi6+TkRUvfTq1QuzZ89GdnY2AEAkEiE6Ohpff/01PvzwwxIda8+ePfDw8EDfvn1hZWWFZs2aYc2aNcXuHxAQABMTE8WtYGkkIiIiVZS44DM0NMSzZ88AAIcOHULnzp0BAHp6ekhPT1dvOi2iI3nZwpfDFj4iompl0aJFePHiBaysrJCeno527drB2dkZRkZGmDt3bomOdf/+faxatQr169fHwYMHMXr0aIwfPx7r168vcv9p06YhKSlJcStYGomIiEgVJZ6ls3Pnzvjss8/QrFkz3LlzBz169AAAXL9+HY6OjurOpzVEIhEkYhFy5QJb+IiIqhkTExMEBQXhzJkzuHz5Ml68eIHmzZsr1qItCblcDg8PD8ybNw8A0KxZM1y7dg2BgYHw9/cvtL9MJoNMJivzeyAiouqpxAXfypUr8e233yImJgY7duyAhYUFACAkJAQDBgxQe0BtoiMpKPjYwkdEVF1kZ2dDX18fYWFh8PHxgY+PT5mOZ2trC1dXV6VtLi4u2LFjR5mOS0REVJQSF3ympqZYsWJFoe2zZs1SSyBtpiMWIwNyztJJRFSN6OjooE6dOsjNzVXL8Xx8fHD79m2lbXfu3IGDg4Najk9ERPSqEhd8AJCYmIjffvsNN2/eBAC4ublh2LBhMDExUWs4baNYi48tfERE1co333yD//u//8PGjRthbm5epmNNmjQJ3t7emDdvHvr164fz58/jl19+wS+//KKmtERERC+VuOC7ePEiunbtCn19fbRq1QoAsHjxYsydOxeHDh1C8+bN1R5SW+hI8ua4yWLBR0RUraxYsQJ3796FnZ0dHBwcUKNGDaXHQ0NDVT5Wy5YtsWvXLkybNg2zZ8+Gk5MTli5dioEDB6o7NhERUckLvkmTJqFXr15Ys2YNpNK8p+fk5OCzzz7DxIkTcfLkSbWH1BYFBV8OJ20hIqpW/Pz81Hq89957D++9955aj0lERFSUUrXwvVrsAYBUKsVXX30FDw8PtYbTNoounXK28BERVSczZ87UdAQiIqJSKXHBZ2xsjOjoaDRs2FBpe0xMDIyMjNQWTBspunTmsIWPiKg6CgkJURq/3qxZMw0nIiIierMSF3wff/wxhg8fjh9//BHe3t4AgDNnzmDKlClVflkGqZgtfERE1VF8fDz69++P48ePw9TUFEDeBGbt27fH1q1bYWlpqdmARERExShxwffjjz9CJBJh8ODByMnJAZA3ZfXo0aMxf/58tQfUJhzDR0RUPX3xxRdISUnB9evX4eLiAgC4ceMG/P39MX78eGzZskXDCYmIiIpW4oJPV1cXy5YtQ0BAAO7duwcAqFevHgwMDNQeTtsUjOHjLJ1ERNXLgQMHcPjwYUWxBwCurq5YuXIlunTposFkREREb1aqdfgAwMDAAI0aNVJnFq3HFj4ioupJLpdDR0en0HYdHR3I2c2fiIi0mEoFX58+fVQ+4M6dO0sdRtvpcJZOIqJqqUOHDpgwYQK2bNkCOzs7AEBsbCwmTZqEjh07ajgdERFR8cSq7GRiYqLyTd1iY2Px6aefwsLCAvr6+mjUqBEuXryo9tdRhVScd7qy2cJHRFStrFixAsnJyXB0dES9evVQr149ODk5ITk5GcuXL9d0PCIiomKp1MK3du3a8s5RpOfPn8PHxwft27fH/v37YWlpifDwcJiZmWkkT0GXzmyO4SMiqlbs7e0RGhqKw4cP49atWwAAFxcXdOrUScPJiIiI3qzUY/gqwoIFC2Bvb69UcDo5Ob3xOZmZmcjMzFTcT05OVlseRZdOFnxERNWOSCRC586d0blzZ01HISIiUplKXTo1Zc+ePfDw8EDfvn1hZWWFZs2aYc2aNW98TkBAgFIXU3t7e7XlkUrYpZOIqDoaP348fvrpp0LbV6xYgYkTJ1Z8ICIiIhVpdcF3//59rFq1CvXr18fBgwcxevRojB8/HuvXry/2OdOmTUNSUpLiFhMTo7Y8OvkLr7NLJxFR9bJjxw74+PgU2u7t7Y3t27drIBEREZFqtLpLp1wuh4eHB+bNmwcAaNasGa5du4bAwED4+/sX+RyZTAaZTFYueRTLMsjZwkdEVJ08e/asyInJjI2N8fTpUw0kIiIiUk2ZWvgyMjLUlaNItra2cHV1Vdrm4uKC6Ojocn3d4hQsvM4WPiKi6sXZ2RkHDhwotH3//v2oW7euBhIRERGppsQtfHK5HHPnzkVgYCAeP36MO3fuoG7dupg+fTocHR0xfPhwtYXz8fHB7du3lbbduXMHDg4OanuNkuAsnURE1dPkyZMxbtw4PHnyBB06dAAAHDlyBIsWLcLSpUs1G46IiOgNStzC9/3332PdunVYuHAhdHV1Fdvd3d3x66+/qjXcpEmTcO7cOcybNw93797F5s2b8csvv2Ds2LFqfR1VvZylk106iYiqk2HDhmHRokX47bff0L59e7Rv3x5//PEHVq1ahREjRmg6HhERUbFKXPBt2LABv/zyCwYOHAiJRKLY3qRJE8XaROrSsmVL7Nq1C1u2bIG7uzvmzJmDpUuXYuDAgWp9HVVxlk4ioupr9OjRePDgAR4/fozk5GTcv38fgwcP1nQsIiKiNypxl87Y2Fg4OzsX2i6Xy5Gdna2WUK9677338N5776n9uKVRMEtnjpxdOomIqpP09HQIggADAwNYWloiKioKv/76K1xdXdGlSxdNxyMiIipWiVv4XF1dcerUqULbt2/fjmbNmqkllLbiGD4iouqpd+/e2LBhAwAgMTERrVq1wqJFi9C7d2+sWrVKw+mIiIiKV+IWvhkzZsDf3x+xsbGQy+XYuXMnbt++jQ0bNmDfvn3lkVFrsEsnEVH1FBoaiiVLlgDI+4LTxsYGly5dwo4dOzBjxgyMHj1awwmJiIiKVuIWvt69e2Pv3r04fPgwatSogRkzZuDmzZvYu3cvOnfuXB4ZtcbLSVvYwkdEVJ2kpaXByMgIAHDo0CH06dMHYrEY7777LqKiojScjoiIqHilWni9TZs2CAoKUncWrScVF6zDxxY+IqLqxNnZGbt378YHH3yAgwcPYtKkSQCA+Ph4GBsbazgdERFR8cq08Hp1oyPlGD4ioupoxowZ+PLLL+Ho6AhPT094eXkByGvtq+rj14mIqHIrcQufmZkZRCJRoe0ikQh6enpwdnbGkCFDMHToULUE1CY64ryCL0fOFj4iourko48+QuvWrfHo0SM0adJEsb1jx4744IMPNJiMiIjozUo1acvcuXPRvXt3tGrVCgBw/vx5HDhwAGPHjkVERARGjx6NnJycKrcYrVRS0KWTLXxERNWNjY0NbGxslLYVXAeJiIi0VYkLvtOnT+P777/H559/rrR99erVOHToEHbs2IHGjRvjp59+qnIFH5dlICIiIiKiyqTEY/gOHjyITp06FdresWNHHDx4EADQo0cP3L9/v+zptIyOhJO2EBERERFR5VHigs/c3Bx79+4ttH3v3r0wNzcHAKSmpiqmr65KTA10AQAJqVkaTkJERERERPR2Je7SOX36dIwePRrHjh1TjF24cOEC/v33XwQGBgIAgoKC0K5dO/Um1QK1TPUBAA8T0yEIQpGT1xARUdWSm5sLkUgEsVgMQRAgl8shkUg0HYuIiEglJS74RowYAVdXV6xYsQI7d+4EADRo0AAnTpyAt7c3AOB///ufelNqCWtjPYhEQGaOHAmpWbAwlGk6EhERlbNly5ZBJBJh0qRJWLFiBbKzszF58mRNxyIiIlJJqRZe9/HxgY+Pj7qzaD1dqRiWhjLEp2TiYWIGCz4iomrgiy++QMeOHdGhQwds374dR44c0XQkIiIilZWq4JPL5bh79y7i4+MhlyvPWNm2bVu1BNNWdqb6iE/JRGxiOhrVNtF0HCIiKkezZs2CSCSCjY0NWrdujR49emDevHkA8pYpIiIi0nYlLvjOnTuHTz75BFFRURAE5dkqRSIRcnNz1RZOG9mZ6iEsBniUlK7pKEREVM58fX0BAAkJCbC3t4ednV2VHKNORERVV4ln6fz888/h4eGBa9euISEhAc+fP1fcEhISyiOjVrEzeTlxCxERVW3t2rWDq6srzp8/j3PnziE4OBhubm4s+oiIqNIoccEXHh6OefPmwcXFBaampjAxMVG6VXV2ipk6MzSchIiIKsLOnTvx7bffwtjYGDNnzsSOHTvUevz58+dDJBJh4sSJaj0uERERUIounZ6enrh79y6cnZ3LI4/WszPVAwA8ZJdOIqJqYdSoUYp/d+3aVa3HvnDhAlavXo3GjRur9bhEREQFSlzwffHFF/jf//6HuLg4NGrUCDo6OkqPV/WLlp0pu3QSEVHZvXjxAgMHDsSaNWvw/fffF7tfZmYmMjMzFfeTk5MrIh4REVURJS74PvzwQwDAsGHDFNtEIpFiIfKqP2lLXsEXn5KJrBw5dKUl7hVLRESEsWPHomfPnujUqdMbC76AgADMmjWrApMREVFVUuKCLyIiojxyVBoWNXShKxUjK0eOx8kZsDc30HQkIiKqZLZu3YrQ0FBcuHDhrftOmzZNaaH35ORk2Nvbl2c8IiKqQkpc8Dk4OJRHjkpDJBLBzkQPkc/S8DAxnQUfERGVSExMDCZMmICgoCDo6em9dX+ZTAaZTFYByYiIqCpSqeDbs2cPunfvDh0dHezZs+eN+/bq1UstwbSZnal+XsHHiVuIiKoFf39/DB8+HG3bti3zsUJCQhAfH4/mzZsrtuXm5uLkyZNYsWIFMjMzIZFIyvw6REREgIoFn5+fH+Li4mBlZQU/P79i96sOY/gALs1ARFTdJCUloVOnTnBwcMDQoUPh7++PWrVqlepYHTt2xNWrV5W2DR06FA0bNsTXX3/NYo+IiNRKpRlH5HI5rKysFP8u7lYdij0AsDPJX5qBM3USEVULu3fvRmxsLEaPHo0///wTjo6O6N69O7Zv347s7OwSHcvIyAju7u5Ktxo1asDCwgLu7u7l9A6IiKi6qlRTTGrL4rRcmoGIqPqxtLTE5MmTcfnyZQQHB8PZ2RmDBg2CnZ0dJk2ahPDwcE1HJCIiKqTEk7YAwJEjR3DkyBHEx8dDLpcrPfb777+rJdjrtGlxWnbpJCKqvh49eoSgoCAEBQVBIpGgR48euHr1KlxdXbFw4UJMmjSpxMc8fvy4+oMSERGhFC18s2bNQpcuXXDkyBE8ffoUz58/V7qVh1cXpzUzM3vjvpmZmUhOTla6qZudaX6XTk7aQkRULWRnZ2PHjh1477334ODggL/++gsTJ07Ew4cPsX79ehw+fBjbtm3D7NmzNR2ViIhISYlb+AIDA7Fu3ToMGjSoPPIUSdXFaYGKWaDW1iSvhS8lIwfJGdkw1tMp19cjIiLNsrW1hVwux4ABA3D+/Hk0bdq00D7t27eHqalphWcjIiJ6kxIXfFlZWfD29i6PLEUqyeK0QMUsUFtDJoWVkQzxKZm4HZeClo7maj0+ERFplyVLlqBv375vXDfP1NQUERERFZiKiIjo7UrcpfOzzz7D5s2byyNLIQWL027atEmlxWmBvAVqjY2NlW7loYVDXtfSkKjy6cZKRETaY9CgQSpfh4iIiLSJSi18r7aYyeVy/PLLLzh8+DAaN24MHR3l7oyLFy9WWzhtXpy2hYMZ9l+LY8FHRERERERaS6WC79KlS0r3C8YuXLt2TWm7SCRST6p82rw4bfP8Fr7QqOcQBEHt752IiIiIiKisVCr4jh07Vt45ilSwOO2rtGVxWjc7Y+hKxXiWmoXIZ2lwqllDo3mIiIiIiIheV6kWXtcmMqkEjWuZAOA4PiIiIiIi0k6lWnhdk7RpcdoWDma4GPUcIVHP8VGL2pqOQ0REREREpIQtfGXw6jg+IiIiIiIibcOCrwya18kr+O7EpyApPVvDaYiIiIiIiJSx4CsDSyMZHC0MIAhAWEyipuMQEREREREpYcFXRi0czAEAx2/HazgJERERERGRMhZ8ZdSjkQ0AYE/YQ2TnyjWchoiIiIiI6CUWfGXU9h1L1DTUxbPULJy880TTcYiIiIiIiBRY8JWRjkSM3k1rAQB2hD7QcBoiIiIiIqKXWPCpwYfN89bgO3wjHolpWRpOQ0RERERElIcFnxq42hmjoY0RsnLl2HflkabjEBERERERAWDBpzYftchr5fvjXBRy5YKG0xAREREREbHgU5s+zWvDSCbFrbgUjuUjIiIiIiKtwIJPTcxr6GJ8x/oAgB8O3saLzBwNJyIiIiIiouqOBZ8a+Xs7wtHCAE9SMvHzsbuajkNERERERNUcCz410pWK8X89XAAAv56KwH/3nmo4ERERERERVWcs+NSss6s1urhaIytXjmHrLuDMXRZ9RERERESkGSz41EwkEuGnAc3QvoElMrLzir7jt+M1HYuIiIiIiKohFnzlQE9HgsBBLdCxoRUyc+T4bP1F/B0Wq+lYRERERERUzbDgKycyqQSrPm2B95vYIUcuYOKfYVj/X6SmYxERERERUTXCgq8c6UrFWPZxUwz2coAgADP3XMeSoDsQBC7MTkRERERE5U+q6QBVnVgswqxebjCvoYulh8Ox7Eg4ElKz8F0vN0jEIk3HIyIiogri0F7TCbRf1DFNJyCqetjCVwFEIhEmdnoHc3q7QSQCNp6LwoStl5CVI9d0NConSenZiHiaqukYRERERFTNseCrQIO8HPFT/2bQkYiw78ojDF9/AamZOZqOReVg5IaL6LDoOA5ce6TpKERERERUjbHgq2DvN7HDb/4toa8jwanwpxj4azCep2ZpOhapkSAIuPwgEYIA/G/bZYQ/TtF0JCIiIiKqpljwaUDbdyyxeYQnTA10EBaTiL6rz+JRUrqmY5GaPHmRiYzsvO66qVm5GLkxBEnp2RpORURERETVkVYXfAEBAWjZsiWMjIxgZWUFPz8/3L59W9Ox1KJZHTP8NcoLtiZ6uBv/Au8vP42jtx5rOhapQUxCGgCgpqEuapnqI+JpKgJP3NNwKiIiIiKqjrS64Dtx4gTGjh2Lc+fOISgoCNnZ2ejSpQtSU6vGZBj1rY2wfbQ3Glgb4emLLAxbdxHf7r6K7FxO5lKZxSTktdY6WxliUud3AADnIxI0GYmItEhV/jKTiIi0j1YXfAcOHMCQIUPg5uaGJk2aYN26dYiOjkZISIimo6lNLVN9/D3OB8N8nAAAf5yLxlfbr0Au51p9lVVBC5+9mQGa1TEFAFyLTWIhT0QAqv6XmUREpF0q1Tp8SUlJAABzc/Ni98nMzERmZqbifnJycrnnKis9HQlmvO8Kz7rmGLMpFLsuxcJEXwcz33eFSMS1+iqb6IKCz9wAThY1YKQnRUpGDm7HpcC9lomG0xGRph04cEDp/rp162BlZYWQkBC0bdtWQ6mIiKiq0uoWvlfJ5XJMnDgRPj4+cHd3L3a/gIAAmJiYKG729vYVmLJsurrZYFHfJgCAdf9F4oeDtyEIbOmrbGKe5xV8dcwNIBaL0NTeFAAQFpOouVBEpLXe9mVmZmYmkpOTlW5ERESqqjQF39ixY3Ht2jVs3br1jftNmzYNSUlJiltMTEwFJVQPv2a1MKuXGwDg5+P3MHvfDRZ9lUzBGD57c30AYMFHRMVS5cvMyvxFJhERaV6lKPjGjRuHffv24dixY6hdu/Yb95XJZDA2Nla6VTb+3o6Y0zuv6Ft7JhJjN4ci8mn1G9vx/b4baDb7kGJMXGWQnStXLLFhb2YAAGhS2xQAcJkFHxG9RpUvMyv7F5lERKRZWl3wCYKAcePGYdeuXTh69CicnJw0HanCDPJyxA8fNYZYBPx7NQ4dFh3H+C2XcOxWPDJzcjUdr9wJgoBdl2LxPC0bh29WnuUqHiamQy4AMqkYlkYyAECT/Ba+u09eICWD6/ERUR5Vv8ysCl9kEhGR5mj1pC1jx47F5s2b8ffff8PIyAhxcXEAABMTE+jr62s4Xfnr62GP+tZGWHb4Do7dfoI9lx9iz+WHMJJJ0dHFCt3cbeHbwBJ6OhIAQEZ2Lk7eeQK5IKCGTIoGNkawMtIr9eunZ+VCX1eirrdTInHJGXiWmgWg/LtCyuUCcgUBOpKyf//xsjungWLCHUsjGWqZ6iM2MR1XHyTB27lmmV+HiCovQRDwxRdfYNeuXTh+/Hi1+jKTiIgqnlYXfKtWrQIA+Pr6Km1fu3YthgwZUvGBNKCpvSnWDm2Fqw+S8FdIDA5ci0N8SiZ2hz3E7rCHMJRJ0c3dBnam+th0LkpRJAGArlSMEW2cMMbXGTVkJftRLzp0G8uP3oWnkzk+8ayDbu42kEkrrvi7FvtyUoLyLPgEQcBHgf/hyYtMBE1qpyieS0sxQ6eZ8hcSTeuYIjYxHWEPElnwEVVz1f3LTCIiqlhaXfBxspKXGtU2QaPaJvjufTeERj/H/mtxOHAtDrGJ6dge8kCxXy1TfdiY6CEhNQsRT1Ox8tg9bLv4AMN8nPBJqzowMdBR7BuTkIYLkQno6GINE/2X268/TMLKY3cBAMERCQiOSIC1sQwj29bDJ63qVEir37XYJMW/o56lISE1C+Y1dNX+Oo+TMxEanQgAuPfkBdzsyrZswqszdL6qaW1T/HPlEcLyX4uIqi9+mUlERBVJqws+KkwsFsHD0Rwejub4pocLQqKfY2doLB4lpeODZrXQs5EtpBIxBEHAoRuPMfefm4hOSMOCA7fw05FwdG9kg/ca2+JSdCJWn7yPrBw5DGVSfOJZB8N8nGBlJMP03dcgF4BOLtZoVMsEW85HIy45A3P23cDPx+5ieBsnDHrXAUZ6Om8PXErXHyYp3b8ck4j2Da3K9XViEtLLXvC9sgbfq5rmL8B+5UHS608homqGX2YSEVFFYsFXiYnFIrR0NEdLx8JrN4lEInR1s4FvA0vsCXuI305H4FZcCnaGxmJnaKxiP4sauniWmoVfTt7H2jMRaOFghtDoRNTQleB7P3fYmOjhc9+62Bkai5+P30VMQjoWHriNwOP38EWH+vD3doSuVP1z/xR06XSwMEDUszRcKreC72XX0QfPyz4baEHBV9tMueBraGMEIG9sYlJ6tlKLKhERERFRedHqWTqp7GRSCfp62GP/hDbY/rkX/L0cYG0sQz3LGgj8tDkufNMJa4e0RCsnc2TnCjh3PwEAMKnzO7Ax0VMcY0CrOjj2P18s7tcE9SxrIDkjB3P/vYmuS0/i8I3HRX5j/fRFJrJy5CXO/CQlE3HJGRCJgP4t6wAovyUNXu06qo7lH2Ke503a8nqXTiM9HdgY553Pu/Evyvw6RERERESqYAtfNSESvewKOqu38uK+7RtaoX1DK4RGP8faM5GQScXw93YsdAypRIw+zWujd9Na2BH6AAsP3EbE01R8tuEi2tSvienvueId67yWrAPX4vDFllDUNjPAnyPfhZWx6rOFFnSzdKpZAz7OFgCAyw8SIQiCYuZLdXm1ha+gWCutF5k5SMifNKdg0fVX1bc2RFxyBu7Fv0ALB7MyvRYRERERkSrYwkcKzeuYYfmAZvixb5M3LlEgEYvQz8Mex75sh8/b1YOuRIxT4U/RY9kpzN9/C4eux2H8lkvIzhUQ8TQVn/4WrCiEVFFQhLnbmaChjTF0pWIkpmUj8pl6F2BPTMtCbOLLIq+sLXyRT1MBAKYGOkWOb3S2MgQAhMenlOl1iIiIiIhUxYKPSs1ITwdTuzdE0OS26OxqjRy5gMAT9zByYwiycuXo0NAK1sYy3Hn8AoN/D0ZcUkahY1yMTMC6MxHIyX3Z9bOghc+9Vl6x526Xt8hwWMxzteYvKCz1dPL+Gzx4nl6oa+qGs5GYvfcG5PK3T7JwKvwpgLylNIrysuAr3KUzJ1eudA4qyt9hsfCcdxghUQkV/tpEREREVP5Y8FGZOVjUwJrBHlgz2AO2+eP+2r5jiVWfNsemzzxhXkMX12KT0W3ZSRy8Hqd43s1HyRj023l8t/cGVp+8r9heMGGLe/6MmU3yC6it52OQkpGtttwFhWVrZ0uIREB6di6evnjZEpmVI8ecfTfw+5kIXH6Q+NbjHb31GADQ0cW6yMfrW+V1d319DN+LzBy0++E4+v9yrtxn73v9+P9ceYTHyZk4eiu+XF+XiIiIiDSDBR+pTWdXawRNbocNw1rh18EekEklcLYywo7R3mhUywSJadkYtTEEk/8Mw/0nLzBqYwjSs3MBAMsOh+N2XAouRT9XLF5esESCX9Na0JWIERyRAL+VZ3DvSeknPbnzOAU/HryNhNQsRQtfszqmiglVYl6ZqTPiaSqyc/MKpFcndynK89QshETltUB2KGY20YIWvgfP05GWlaPYfjEyAbGJ6bgY9fK9l4cHz9PgFXAUSw/fUWwr6NIal5RZbq9LRERERJrDgo/UylAmRdt3LJWWanCqWQM7RntjZNu6EImAnZdi0WHRCUQnpKG2mT7a1K+JrFw5Rm28iP6/nAMA+DhbKBaJb2Jvim2fe8HGWA/3nqTCb+UZnLv/TOl1c+UCvtl1FbP2Xn9jK9mcfTew4thdDF13AWH5M3+62hnDPn8ZhVfH8d2Kezmhy9X8gk8uF/Drqfs4H6HcBfL4nXjIhbzlF2qZFp6wBQDMa+jCIn/x+HvxqYrtYa/MQFrwvgRBwIXIBGTkF8SlkZUjV5ol9fjtJ4hLzsCeyw8V2x7kT1QTn1K4uy0RERERVX4s+KhC6ErF+L8eLtg52huutnlj8vR0xFg9qAV+7NsExnpSRD5LQ2ZO3ti/1YM8lJ7f1N4Ue77wQQsHM6Rk5GDwb+ex95XC5Y9zUdgUHI21ZyJx6MbjIjNkZOciOL9QuxyTiKj8SWDc7UxQO39WzQevzNR5O+7l5CpX87uZHrsdj+//uYkxm0KUxtwduZnXJbJTMd05CxS08t198vLYrxZ8wfnLYmy7GIO+gWcxbefVNx6vOLlyAb1XnkHnJScUReOdx3mv+SAhHblyASkZ2UhKz+siW9T4SiIiIiKq/FjwUYVqVscMe8b54KcBzbBtlBfc7ExgbayHhR81hpmBDka0ccKawR4wlBVeMcTKSA+bPvNENzcbZOXK8cWWS/jl5D08TEzHwgO3FPv9cPB2kROgnI9IQFaOHCb6OtDNn4XUykgGSyNZkS18BQUSAIQ/TkFGdi5O3nkCAHj6IkuxZmF2rhwn8rd3cHnz4vCKiVse53VLFQRBaY3Bc/efQRAEbL0QAyBvUpVoFWYnTUrLxpGbL9dDvBabhJuPkhH1LA03H+UVqwUFbFauHHHJGUozlMYls+AjIiIiqopY8FGFk0rE6NXEDo1rmyq2dXO3Rej0zvimpysk4uLX2tPTkWDlwOYY6uMIAJj37y188PMZpGblokltE5ga6OBu/AvsvBRb6LmnwvOKsi6u1ljUrwl0JCLFeDv7/IXSXx3Dd+uVFr4cuYBbcSmKmTgBYN+VvBbGi5HPkZKRA4saumjyynsqSv2CFr78iVuinqXheVo2dCViSMUiPEzKwH/3nuFSdCIAQC4Av53Om9Am8mkqNp6LKjTpi1wuYMi68xi+/iJ2hMYqvVcgr/gTBEGpgI16lorYV1ozUzJylMYVEhEREVHVwIKPtIaqi6pLxCLMfN8N3/Z0AQA8Ts6EVCzCwo+aYKyvMwBgadCdQuPfCoq1Nu9Y4v0mdrj4TWfM+6ARAMDeLK9LZ0xCXhH0IjNH0b2zSe28yWMOXIvD/acvx94duB6H7Fw5Np+PBgD4NrB6Y7EKAM6vzdRZ0J3TrZaxYjbSGX9fAwBYG8sAANsuPsDp8Kfw+/kMpu++hk6LT6DDj8ex/+ojAMCfF2MUBeKuSw+U3iuQN+vp0xdZeJ72cobT6GdpSt1XC84jEREREVUtLPio0vqsTV2s+KQZ7M318W1PFzSwMcIgLwfYmejhYVIGZv79cgKX+JQMRYudTz0LAICJgQ7E+QVaQQvfw8S88W0FrWFWRjK0rl8TQN44QSBvVs+ahjIkpmVjzr4b2Hv5IcQiwN/b4a2Z61vntfBFPktFZk6uouBram+Kd+uaAwDuPckrKv/XpQFcbY2Rnp2LT38LRmJaNmyM9aAjEeH+01SM2RyKwBP3sOCV7qxn7z1DTEIaQqNfrll4NTZJqXUPAKIS0pS6dAIcx0dERERUFbHgo0rtvcZ2OPVVBwzxcQKQ1+Vz7geNIBbltXz9fPweAODM3bwWL/daxrAwlBU6jrWxHnQlYuTIBTxKSleMd2tgY4RGtfJa+F5k5nV59H3HCj0b2QAANpzNKwKHt3ZS6qJaHCsjGYz0pJALwI2Hybj0SsHn6WSh2E9XKkY3dxuMaldXsc3DwQxBk9sidHpnDPSsA0EA5u+/hcS0bDS0MYJ7LWPIBWDW3uvIzhVgpJc3DvLO4xTFLKMFop+lKXXpBIDHHMdHREREVOWw4KMqp31DK8zq5QYgbwKXgH9v4u+wvPF2rZ0ti3yORCxCrVe6dRYUfHmFlInSvq3r18R7TewU9+uYG2By5wYqZROJRPB0ymvJm/RnGG4WrAVob4YWDmaQ5rc4dnKxgrGeDno0skVnV2t0d7fB+mGtYKSnAyM9HXzv565UDM7xc0ev/EyH82cMfa+xLcwMdJAjFxQzmja0yetSGpWQigf54xWN8ifIYcFHREREVPUUngqRqAoY5OWI6IQ0rDkVgdUn7yu2t8nvnlmU2mb6iHiaijuPUxQF3zvWeevqmRno4HlaNoz0pGhS2wRikQhONWsg8lkqAvo0gr6uROVs8z5ohJuP/kNk/uybFjV0YW+un1cM1jXHmbvP8FGL2gAAHYkYawZ7FDqGSCTC1G4N0czeFBKxGC0dzWFrood5/77s3tna2RIPnqfjVPhTxSLzXVytcSsuBVHP0hQzlTatY4pT4U85UycREVUpDu01naByiDqmnuPwfKtGXee7JNjCR1XWtO4uWD6gGbq6WUNPRwynmjXQwsGs2P0LHltw4BauPEgEADS0MYZIJEKj/O6aPvVqQioRQywWYfMIT/zzRRv4OBdfRBbFylgP64e1gol+3sLyTe1NFRPWLPm4KbaMeBcdGr55PT8gr+jr5p7XAggAtc0M0KyOaf5jgHc9i0Ktkx3z1wlMycjBs9QsAEDzOnnvmy18RERERFUPW/ioyhKLRXi/iR3eb2KHrBw5pGKRYpKWooz2rYeLkc9xOn+8n0j0cpKVPs1q4XzEM3ziWUexv62JPmxN9EuVzdnKEGuHtsQPB25jeBsnxXYrIz1YGemV6phA3pjGS9GJaFzbFGY1dBXjDwFAKhbBxdYYNsZ6itY8Q5kUDfK7eXLSFiIiIqKqhwUfVQu60rc3ZsukEgQOaoFP1pzDlQdJcLKoAT2dvK6afs1qwa9ZLbVmal7HDFtGvqvWY376bh2kZGSjU35Lnrvdy4LPqWYN6ErFqGNhoCj4apnqw9o4r8DksgxEREREVQ+7dBK9wlAmxdohLdHPoza+6tZQ03FKTCaVYGKndxRdOe3N9WGcP1vnO/kteQ75S1AAeeMWbUzyCr74lAzI5UIFJyYiIiKi8sSCj+g1FoYyLPyoCbq522g6SpmJRCJF8fdO/qLvDhYvC75aZvqwzF+mIjtXQEJaVsWHJCIiIqJywy6dRFXc2PbO0JGI8XFLewBAHYsaisdqmepDVypGTUNdPH2RhbikDNQsYp1CIiIiIqqcWPARVXE+zjWVZhJV7tKZ929rYz08fZGF+JQMACavH4KIiIiIKil26SSqZl7v0gkANvkTt8QlceIWIiIioqqkUhR8K1euhKOjI/T09ODp6Ynz589rOhJRpWVqoAtnK0MY6UlRzzKve6dVQcHHtfiIiIiIqhStL/j+/PNPTJ48GTNnzkRoaCiaNGmCrl27Ij4+XtPRiCqt3WN9cPxLXxjp5S3+XtDC95hr8RERERFVKVo/hm/x4sUYMWIEhg4dCgAIDAzEP//8g99//x1Tp07VcDqiyslQJoWh7OV/fxuTvIlazkcm4IeDtzQVi6iQse2dYaCr9ZcqIiIiraXVV9GsrCyEhIRg2rRpim1isRidOnXC2bNni3xOZmYmMjNfjkNKTk4u95xElZ19/kQuEU9TsfLYPQ2nIXppeOu6MNDVdAoiIqLKS6sLvqdPnyI3NxfW1tZK262trXHrVtGtEAEBAZg1a1ZFxCOqMt51ssC3PV0Qm5iu6ShESmRSrR95QEREpNW0uuArjWnTpmHy5MmK+8nJybC3t9dgIiLtJxaL8FmbupqOQURERERqptUFX82aNSGRSPD48WOl7Y8fP4aNjU2Rz5HJZJDJuHA0ERERERGRVveV0dXVRYsWLXDkyBHFNrlcjiNHjsDLy0uDyYiIiIiIiLSfVrfwAcDkyZPh7+8PDw8PtGrVCkuXLkVqaqpi1k4iIiIiIiIqmtYXfB9//DGePHmCGTNmIC4uDk2bNsWBAwcKTeRCREREREREyrS6S2eBcePGISoqCpmZmQgODoanp6emIxEREZXJypUr4ejoCD09PXh6euL8+fOajkRERFVQpSj4iIiIqpI///wTkydPxsyZMxEaGoomTZqga9euiI+P13Q0IiKqYljwERERVbDFixdjxIgRGDp0KFxdXREYGAgDAwP8/vvvmo5GRERVjNaP4SsrQRAA5K3HR0RE1UvB3/6Ca4E2yMrKQkhICKZNm6bYJhaL0alTJ5w9e7bQ/pmZmcjMzFTcT0pKAgDExsaW+dqWk1Gmp1cLDx6o71g832/H813x1HXOeb5VU9bznZKSAqBk17UqX/AVnBQuvk5EVH2lpKTAxMRE0zEAAE+fPkVubm6hycesra1x69atQvsHBARg1qxZhba7urqWW0Z6iR8fKhbPd8XjOa9Y6jrfJbmuVfmCz87ODjExMTAyMoJIJCrVMZKTk2Fvb4+YmBgYGxurOWH5YnbNYHbNYHbN0ObsgiAgJSUFdnZ2mo5SatOmTcPkyZMV9+VyORISEmBhYVHsdU2bfyZvwtwVi7krFnNXrKqauzTXtSpf8InFYtSuXVstxzI2Nq5UvzCvYnbNYHbNYHbN0Nbs2tKyV6BmzZqQSCR4/Pix0vbHjx/Dxsam0P4ymQwymUxpm6mpqUqvpa0/k7dh7orF3BWLuStWVcxd0usaJ20hIiKqQLq6umjRogWOHDmi2CaXy3HkyBF4eXlpMBkREVVFVb6Fj4iISNtMnjwZ/v7+8PDwQKtWrbB06VKkpqZi6NChmo5GRERVDAs+FchkMsycObNQl5rKgNk1g9k1g9k1ozJn15SPP/4YT548wYwZMxAXF4emTZviwIEDhSZyKa3K+jNh7orF3BWLuSsWc78kErRprmoiIiIiIiJSG47hIyIiIiIiqqJY8BEREREREVVRLPiIiIiIiIiqKBZ8REREREREVRQLPhWsXLkSjo6O0NPTg6enJ86fP6/pSIUEBASgZcuWMDIygpWVFfz8/HD79m2lfTIyMjB27FhYWFjA0NAQH374YaGFfzVt/vz5EIlEmDhxomKbNueOjY3Fp59+CgsLC+jr66NRo0a4ePGi4nFBEDBjxgzY2tpCX18fnTp1Qnh4uAYT58nNzcX06dPh5OQEfX191KtXD3PmzMGrczhpS/aTJ0/i/fffh52dHUQiEXbv3q30uCo5ExISMHDgQBgbG8PU1BTDhw/HixcvNJo9OzsbX3/9NRo1aoQaNWrAzs4OgwcPxsOHD7U+++s+//xziEQiLF26VGm7prJXR6U5176+vhCJREq3zz//vNyzlvSa+tdff6Fhw4bQ09NDo0aN8O+//5Z7xqKUJPe6desKnVs9Pb0KTFuy/8MFjh8/jubNm0Mmk8HZ2Rnr1q0r95yvK2nu48ePFzrXIpEIcXFxFRM4nyqfw4qi6d/v0uTWht/vVatWoXHjxorFyb28vLB///43PkfT5xooeW51nWsWfG/x559/YvLkyZg5cyZCQ0PRpEkTdO3aFfHx8ZqOpuTEiRMYO3Yszp07h6CgIGRnZ6NLly5ITU1V7DNp0iTs3bsXf/31F06cOIGHDx+iT58+Gkyt7MKFC1i9ejUaN26stF1bcz9//hw+Pj7Q0dHB/v37cePGDSxatAhmZmaKfRYuXIiffvoJgYGBCA4ORo0aNdC1a1dkZGRoMDmwYMECrFq1CitWrMDNmzexYMECLFy4EMuXL1fsoy3ZU1NT0aRJE6xcubLIx1XJOXDgQFy/fh1BQUHYt28fTp48iZEjR2o0e1paGkJDQzF9+nSEhoZi586duH37Nnr16qW0nzZmf9WuXbtw7tw52NnZFXpMU9mro9Ke6xEjRuDRo0eK28KFC8s1Z0mvqf/99x8GDBiA4cOH49KlS/Dz84Ofnx+uXbtWrjnLmhsAjI2Nlc5tVFRUBSZW/f9wgYiICPTs2RPt27dHWFgYJk6ciM8++wwHDx4s56TKSpq7wO3bt5XOt5WVVTklLJoqn8Nepw2/36XJDWj+97t27dqYP38+QkJCcPHiRXTo0AG9e/fG9evXi9xfG851aXIDajrXAr1Rq1athLFjxyru5+bmCnZ2dkJAQIAGU71dfHy8AEA4ceKEIAiCkJiYKOjo6Ah//fWXYp+bN28KAISzZ89qKqZCSkqKUL9+fSEoKEho166dMGHCBEEQtDv3119/LbRu3brYx+VyuWBjYyP88MMPim2JiYmCTCYTtmzZUhERi9WzZ09h2LBhStv69OkjDBw4UBAE7c0OQNi1a5fivio5b9y4IQAQLly4oNhn//79gkgkEmJjYzWWvSjnz58XAAhRUVGCIGh/9gcPHgi1atUSrl27Jjg4OAhLlixRPKYt2auD0p7rV//WVpSSXlP79esn9OzZU2mbp6enMGrUqHLN+bqS5l67dq1gYmJSQeneTpW/P1999ZXg5uamtO3jjz8WunbtWo7J3kyV3MeOHRMACM+fP6+QTKp6/XNYUbTl9/tVquTWtt/vAmZmZsKvv/5a5GPaeK4LvCm3us41W/jeICsrCyEhIejUqZNim1gsRqdOnXD27FkNJnu7pKQkAIC5uTkAICQkBNnZ2UrvpWHDhqhTp45WvJexY8eiZ8+eSvkA7c69Z88eeHh4oG/fvrCyskKzZs2wZs0axeMRERGIi4tTym5iYgJPT0+NZ/f29saRI0dw584dAMDly5dx+vRpdO/eHYB2Z3+VKjnPnj0LU1NTeHh4KPbp1KkTxGIxgoODKzzzmyQlJUEkEsHU1BSAdmeXy+UYNGgQpkyZAjc3t0KPa3P2qqYs53rTpk2oWbMm3N3dMW3aNKSlpZVbztJcU8+ePVvoutC1a9cK/TtU2s8CL168gIODA+zt7d/6Db420IZzXRZNmzaFra0tOnfujDNnzmg6TqHPYUXRxnOuSm5Au36/c3NzsXXrVqSmpsLLy6vIfbTxXKuSG1DPuZaWJWhV9/TpU+Tm5sLa2lppu7W1NW7duqWhVG8nl8sxceJE+Pj4wN3dHQAQFxcHXV1dxQfJAtbW1hXez/11W7duRWhoKC5cuFDoMW3Off/+faxatQqTJ0/G//3f/+HChQsYP348dHV14e/vr8hX1O+PprNPnToVycnJaNiwISQSCXJzczF37lwMHDgQALQ6+6tUyRkXF1eoa49UKoW5ublWvZeMjAx8/fXXGDBgAIyNjQFod/YFCxZAKpVi/PjxRT6uzdmrmtKe608++QQODg6ws7PDlStX8PXXX+P27dvYuXNnueQszTU1Li5O43+HSpO7QYMG+P3339G4cWMkJSXhxx9/hLe3N65fv47atWtXROwSK+5cJycnIz09Hfr6+hpK9ma2trYIDAyEh4cHMjMz8euvv8LX1xfBwcFo3ry5RjIV9TmsKNrw+/0qVXNry+/31atX4eXlhYyMDBgaGmLXrl1wdXUtcl9tOtclya2uc82CrwoaO3Ysrl27htOnT2s6ylvFxMRgwoQJCAoKqvABv2Ull8vh4eGBefPmAQCaNWuGa9euITAwEP7+/hpO92bbtm3Dpk2bsHnzZri5uSnGa9jZ2Wl99qooOzsb/fr1gyAIWLVqlabjvFVISAiWLVuG0NBQiEQiTcepsqZOnYoFCxa8cZ+bN2+W+vivjvFr1KgRbG1t0bFjR9y7dw/16tUr9XEJ8PLyUvrG3tvbGy4uLli9ejXmzJmjwWRVT4MGDdCgQQPFfW9vb9y7dw9LlizBxo0bNZKpMn0Oe5WqubXl97tBgwYICwtDUlIStm/fDn9/f5w4caLY4klblCS3us41u3S+Qc2aNSGRSArNCPn48WPY2NhoKNWbjRs3Dvv27cOxY8eUKn8bGxtkZWUhMTFRaX9Nv5eQkBDEx8ejefPmkEqlkEqlOHHiBH766SdIpVJYW1trZW4g71vF1/9zuri4IDo6GgAU+bTx92fKlCmYOnUq+vfvj0aNGmHQoEGYNGkSAgICAGh39lepktPGxqbQxAo5OTlISEjQivdSUOxFRUUhKChI0boHaG/2U6dOIT4+HnXq1FH8v42KisL//vc/ODo6AtDe7JXJ//73P9y8efONt7p166rtXHt6egIA7t69q9b3UaA011QbGxuN/x1Sx2cBHR0dNGvWrNzOrToUd66NjY21tnWvOK1atdLYuS7uc1hRtOH3u0BJcr9OU7/furq6cHZ2RosWLRAQEIAmTZpg2bJlRe6rTee6JLlfV9pzzYLvDXR1ddGiRQscOXJEsU0ul+PIkSNv7GurCYIgYNy4cdi1axeOHj0KJycnpcdbtGgBHR0dpfdy+/ZtREdHa/S9dOzYEVevXkVYWJji5uHhgYEDByr+rY25AcDHx6fQ1MV37tyBg4MDAMDJyQk2NjZK2ZOTkxEcHKzx7GlpaRCLlf/7SyQSyOVyANqd/VWq5PTy8kJiYiJCQkIU+xw9ehRyuVzxAVdTCoq98PBwHD58GBYWFkqPa2v2QYMG4cqVK0r/b+3s7DBlyhTFjH7amr0ysbS0RMOGDd9409XVVdu5DgsLA5D3ZVZ5KM011cvLS2l/AAgKCqrQv0Pq+CyQm5uLq1evltu5VQdtONfqEhYWVuHn+m2fw4qiDee8NLlfpy2/33K5HJmZmUU+pg3nujhvyv26Up/rMk/7UsVt3bpVkMlkwrp164QbN24II0eOFExNTYW4uDhNR1MyevRowcTERDh+/Ljw6NEjxS0tLU2xz+effy7UqVNHOHr0qHDx4kXBy8tL8PLy0mDqor0+c5y25j5//rwglUqFuXPnCuHh4cKmTZsEAwMD4Y8//lDsM3/+fMHU1FT4+++/hStXrgi9e/cWnJychPT0dA0mFwR/f3+hVq1awr59+4SIiAhh586dQs2aNYWvvvpKsY+2ZE9JSREuXbokXLp0SQAgLF68WLh06ZJiJktVcnbr1k1o1qyZEBwcLJw+fVqoX7++MGDAAI1mz8rKEnr16iXUrl1bCAsLU/p/m5mZqdXZi/L6LJ2azF4dve1cP3jwQGjQoIEQHBwsCIIg3L17V5g9e7Zw8eJFISIiQvj777+FunXrCm3bti3XnG+7pg4aNEiYOnWqYv8zZ84IUqlU+PHHH4WbN28KM2fOFHR0dISrV6+Wa86y5p41a5Zw8OBB4d69e0JISIjQv39/QU9PT7h+/XqFZX7b/+GpU6cKgwYNUux///59wcDAQJgyZYpw8+ZNYeXKlYJEIhEOHDhQYZlLk3vJkiXC7t27hfDwcOHq1avChAkTBLFYLBw+fLhCc6vyOUwbf79Lk1sbfr+nTp0qnDhxQoiIiBCuXLkiTJ06VRCJRMKhQ4eKzKwN57o0udV1rlnwqWD58uVCnTp1BF1dXaFVq1bCuXPnNB2pEABF3tauXavYJz09XRgzZoxgZmYmGBgYCB988IHw6NEjzYUuxusFnzbn3rt3r+Du7i7IZDKhYcOGwi+//KL0uFwuF6ZPny5YW1sLMplM6Nixo3D79m0NpX0pOTlZmDBhglCnTh1BT09PqFu3rvDNN98oFRrakr1gyu3Xb/7+/irnfPbsmTBgwADB0NBQMDY2FoYOHSqkpKRoNHtERESx/2+PHTum1dmLUlTBp6ns1dHbznXB71vB71Z0dLTQtm1bwdzcXJDJZIKzs7MwZcoUISkpqdyzvuma2q5du0K/Y9u2bRPeeecdQVdXV3BzcxP++eefcs9YlJLknjhxomJfa2troUePHkJoaGiF5n3b/2F/f3+hXbt2hZ7TtGlTQVdXV6hbt67SZwhtzb1gwQKhXr16gp6enmBubi74+voKR48erfDcqnwO08bf79Lk1obf72HDhgkODg6Crq6uYGlpKXTs2FFRNBWVWRA0f64FoeS51XWuRYIgCCVrEyQiIiIiIqLKgGP4iIiIiIiIqigWfERERERERFUUCz4iIiIiIqIqigUfERERERFRFcWCj4iIiIiIqIpiwUdERERERFRFseAjIiIiIiKqoljwERERERERVVEs+IiIiIio0li3bh1MTU01HYOo0hAJgiBoOgQRlc2QIUOQmJiI3bt3azoKERFRuUpPT0dKSgqsrKxUfo6vry+aNm2KpUuXll8wIi0l1XQAIiIiIiJV6evrQ19fX9MxiCoNdukkqkS2b9+ORo0aQV9fHxYWFujUqROmTJmC9evX4++//4ZIJIJIJMLx48cBADExMejXrx9MTU1hbm6O3r17IzIyUnG8IUOGwM/PD7NmzYKlpSWMjY3x+eefIysrSzNvkIiIqjxfX1+MGzcO48aNg4mJCWrWrInp06ejoNPZ8+fPMXjwYJiZmcHAwADdu3dHeHi44vmvd+n87rvv0LRpU2zcuBGOjo4wMTFB//79kZKSAiDvWnfixAksW7ZMcZ189VpIVNWx4COqJB49eoQBAwZg2LBhuHnzJo4fP44+ffpg5syZ6NevH7p164ZHjx7h0aNH8Pb2RnZ2Nrp27QojIyOcOnUKZ86cgaGhIbp166ZU0B05ckRxvC1btmDnzp2YNWuWBt8pERFVdevXr4dUKsX58+exbNkyLF68GL/++iuAvALt4sWL2LNnD86ePQtBENCjRw9kZ2cXe7x79+5h9+7d2LdvH/bt24cTJ05g/vz5AIBly5bBy8sLI0aMUFwn7e3tK+R9EmkDdukkqiQePXqEnJwc9OnTBw4ODgCARo0aAcjr3pKZmQkbGxvF/n/88Qfkcjl+/fVXiEQiAMDatWthamqK48ePo0uXLgAAXV1d/P777zAwMICbmxtmz56NKVOmYM6cORCL+Z0QERGpn729PZYsWQKRSIQGDRrg6tWrWLJkCXx9fbFnzx6cOXMG3t7eAIBNmzbB3t4eu3fvRt++fYs8nlwux7p162BkZAQAGDRoEI4cOYK5c+fCxMQEurq6MDAwULpOElUX/DRHVEk0adIEHTt2RKNGjdC3b1+sWbMGz58/L3b/y5cv4+7duzAyMoKhoSEMDQ1hbm6OjIwM3Lt3T+m4BgYGivteXl548eIFYmJiyvX9EBFR9fXuu+8qvowE8q494eHhuHHjBqRSKTw9PRWPWVhYoEGDBrh582axx3N0dFQUewBga2uL+Pj48glPVMmwhY+okpBIJAgKCsJ///2HQ4cOYfny5fjmm28QHBxc5P4vXrxAixYtsGnTpkKPWVpalndcIiKiCqOjo6N0XyQSQS6XaygNkXZhCx9RJSISieDj44NZs2bh0qVL0NXVxa5du6Crq4vc3FylfZs3b47w8HBYWVnB2dlZ6WZiYqLY7/Lly0hPT1fcP3fuHAwNDTm+gYiIys3rX1aeO3cO9evXh6urK3JycpQef/bsGW7fvg1XV9dSv15R10mi6oIFH1ElERwcjHnz5uHixYuIjo7Gzp078eTJE7i4uMDR0RFXrlzB7du38fTpU2RnZ2PgwIGoWbMmevfujVOnTiEiIgLHjx/H+PHj8eDBA8Vxs7KyMHz4cNy4cQP//vsvZs6ciXHjxnH8HhERlZvo6GhMnjwZt2/fxpYtW7B8+XJMmDAB9evXR+/evTFixAicPn0aly9fxqeffopatWqhd+/epX49R0dHBAcHIzIyEk+fPmXrH1Ur/ERHVEkYGxvj5MmT6NGjB9555x18++23WLRoEbp3744RI0agQYMG8PDwgKWlJc6cOQMDAwOcPHkSderUQZ8+feDi4oLhw4cjIyMDxsbGiuN27NgR9evXR9u2bfHxxx+jV69e+O677zT3RomIqMobPHgw0tPT0apVK4wdOxYTJkzAyJEjAeRNMNaiRQu899578PLygiAI+Pfffwt12yyJL7/8EhKJBK6urrC0tER0dLS63gqR1hMJBYueEFG1M2TIECQmJmL37t2ajkJERNWEr68vmjZtiqVLl2o6ClG1wBY+IiIiIiKiKooFHxERERERURXFLp1ERERERERVFFv4iIiIiIiIqigWfERERERERFUUCz4iIiIiIqIqigUfERERERFRFcWCj4iIiIiIqIpiwUdERERERFRFseAjIiIiIiKqoljwERERERERVVH/D6pIaYTNoJhuAAAAAElFTkSuQmCC",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"correctly classified: 4/4\n",
"[ ok ] the from-scratch MLP trained on the XOR-shaped toy\n"
]
}
],
"source": [
"# viz: the loss curve and the final per-point margins\n",
"fig, ax = plt.subplots(1, 2, figsize=(9, 3.2))\n",
"ax[0].plot(loss_log); ax[0].set_xlabel(\"step\"); ax[0].set_ylabel(\"hinge loss\"); ax[0].set_title(\"training loss\")\n",
"margins = [ys_toy[i] * toy(xs_toy[i]).data for i in range(len(xs_toy))]\n",
"ax[1].bar(range(len(margins)), margins, color=[\"#1E40FF\" if m > 0 else \"#d33\" for m in margins])\n",
"ax[1].axhline(0, color=\"k\", lw=0.8); ax[1].set_xlabel(\"point\"); ax[1].set_ylabel(\"y * score\")\n",
"ax[1].set_title(\"final margins (>0 = correct)\")\n",
"plt.tight_layout(); plt.show()\n",
"correct = sum(m > 0 for m in margins)\n",
"print(f\"correctly classified: {correct}/4\")\n",
"assert loss_log[-1] < loss_log[0] * 0.5, \"the MLP should at least halve the loss; if not, init or lr is off\"\n",
"print(\"[ ok ] the from-scratch MLP trained on the XOR-shaped toy\")"
]
},
{
"cell_type": "markdown",
"id": "d4e96c71",
"metadata": {},
"source": [
"> **Interpretation.** The loss falls and all four margins go positive: the network the engine built carves a boundary no single neuron could. Every gradient in that training loop came from the closures you wrote in 9.2, walked in topological order. That is the whole idea of the chapter, working end to end on scalars.\n",
"\n",
"> **Key takeaways**\n",
"> - `Neuron`/`Layer`/`MLP` is ~50 lines over the engine; `parameters()` is what lets the training loop reach the weights.\n",
"> - A 2-hidden-layer MLP fits the XOR-shaped toy that defeated one neuron in Part 1.\n",
"> - The training loop is forward -> `zero_grad` -> `backward` -> update `p.data`; forgetting `zero_grad` would accumulate stale gradients (the next chapter's recurring footgun).\n"
]
},
{
"cell_type": "markdown",
"id": "49c58229",
"metadata": {},
"source": [
"## Part 5: Backprop on tensors + the NumPy MLP\n",
"\n",
"> **Objectives.** Derive the matmul and softmax-cross-entropy gradients, finite-difference-check every one, then train a pure-NumPy 2-layer MLP on a FashionMNIST subset, including one deliberately broken run.\n",
"\n",
"Scalars do not scale. PyTorch runs autograd on tensors, and the local rules generalize cleanly. For $C = AB$:\n",
"$$\\frac{\\partial L}{\\partial A} = \\frac{\\partial L}{\\partial C}\\,B^\\top,\\qquad \\frac{\\partial L}{\\partial B} = A^\\top\\,\\frac{\\partial L}{\\partial C}.$$\n",
"A bias $b\\in\\mathbb{R}^h$ broadcast over a batch gets its gradient by summing over the batch axis. And the softmax-cross-entropy gradient with respect to the logits is famously clean: with $p=\\mathrm{softmax}(z)$ and one-hot target $y$,\n",
"$$\\frac{\\partial L}{\\partial z} = p - y.$$\n",
"That \"predicted minus actual\" is why cross-entropy on logits is the universal classification loss: no division by tiny probabilities, no $\\log 0$.\n"
]
},
{
"cell_type": "markdown",
"id": "4dc9c4c0",
"metadata": {},
"source": [
"### Exercise 9.5: Stable softmax-cross-entropy and its gradient\n",
"`Difficulty 3/5 · ~18 min`\n",
"\n",
"Implement `softmax_xent(logits, y)` returning `(loss, grad)` where `loss` is the mean cross-entropy over the batch and `grad` is $\\partial L / \\partial \\text{logits}$, shape `(n, q)`. Use the max-subtraction trick for numerical stability. The gradient is `(probs - one_hot(y)) / n`. The check finite-difference-verifies your gradient, the strongest tier available.\n"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "a90257ac",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:56.986746Z",
"iopub.status.busy": "2026-06-10T19:14:56.986380Z",
"iopub.status.idle": "2026-06-10T19:14:57.018199Z",
"shell.execute_reply": "2026-06-10T19:14:57.009180Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] 9.5 loss at init = ln(q): not attempted yet — fill in the TODO above, then re-run.\n",
"[ -- ] 9.5 grad vs finite diff: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def softmax_xent(logits, y):\n",
" \"\"\"logits:(n,q) y:(n,) int class indices. Returns (mean_loss, grad) with grad:(n,q).\"\"\"\n",
" logits = np.asarray(logits, dtype=float)\n",
" n = logits.shape[0]\n",
" # TODO 1: stable shift z = logits - rowwise max (keepdims so it broadcasts)\n",
" z = None\n",
" # TODO 2: probs = exp(z) / rowwise sum of exp(z)\n",
" probs = None\n",
" attempted(z, probs)\n",
" # TODO 3: mean cross-entropy = mean over batch of -log(prob of the true class)\n",
" loss = -np.log(probs[np.arange(n), y] + 1e-12).mean()\n",
" # TODO 4: grad = copy of probs; subtract 1 at the true class; divide by n\n",
" grad = None\n",
" attempted(grad)\n",
" return loss, grad\n",
"\n",
"# self-checks (run this cell): loss at init ~ ln(q), and grad vs finite differences\n",
"def _check_xent_init(fn):\n",
" g = np.random.default_rng(0)\n",
" q = 5\n",
" logits = np.zeros((100, q)) # all-equal logits -> uniform probs\n",
" y = g.integers(0, q, size=100)\n",
" loss, _ = fn(logits, y)\n",
" check_close(loss, np.log(q), atol=1e-9, msg=\"uniform logits give loss = ln(q) (Karpathy's init check)\")\n",
"\n",
"def _check_xent_grad(fn):\n",
" g = np.random.default_rng(1)\n",
" n, q = 4, 3\n",
" logits = g.standard_normal((n, q))\n",
" y = g.integers(0, q, size=n)\n",
" _, grad = fn(logits, y)\n",
" num = grad_check(lambda L: fn(L.reshape(n, q), y)[0], logits.ravel()).reshape(n, q)\n",
" err = rel_err(grad, num)\n",
" assert err < 1e-4, f\"softmax-xent grad disagrees with finite diff (rel err {err:.1e})\"\n",
"\n",
"check(\"9.5 loss at init = ln(q)\", lambda: _check_xent_init(softmax_xent))\n",
"check(\"9.5 grad vs finite diff\", lambda: _check_xent_grad(softmax_xent))"
]
},
{
"cell_type": "markdown",
"id": "ca95a2d5",
"metadata": {},
"source": [
"Hint 1 (conceptual)Subtract the per-row max from the logits before `exp` so the largest exponent is 0 and nothing overflows. This shift does not change the softmax (it cancels in the ratio).\n",
"\n",
"Hint 2 (pseudocode)\n",
"\n",
"```python\n",
"z = logits - logits.max(axis=1, keepdims=True)\n",
"expz = np.exp(z)\n",
"probs = expz / expz.sum(axis=1, keepdims=True)\n",
"grad = probs.copy()\n",
"grad[np.arange(n), y] -= 1.0\n",
"grad /= n\n",
"```\n",
"\n",
"\n",
"Help: \"loss at init is not ln(q)\"If it is wildly off, your `probs` rows do not sum to 1: check that the `sum` in the denominator uses `axis=1, keepdims=True`. If it is close but not exact, you forgot the mean (you summed instead of averaging over the batch).\n"
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "c0d81f5d",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:57.022621Z",
"iopub.status.busy": "2026-06-10T19:14:57.022505Z",
"iopub.status.idle": "2026-06-10T19:14:57.048452Z",
"shell.execute_reply": "2026-06-10T19:14:57.048177Z"
},
"collapsed": true,
"jupyter": {
"source_hidden": true
},
"tags": [
"hide-input"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] 9.5 loss at init = ln(q)\n",
"[ ok ] 9.5 grad vs finite diff\n"
]
},
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#@title Solution { display-mode: \"form\" }\n",
"# solution: redefines softmax_xent; checks re-verify against finite differences.\n",
"def softmax_xent(logits, y):\n",
" logits = np.asarray(logits, dtype=float)\n",
" n = logits.shape[0]\n",
" z = logits - logits.max(axis=1, keepdims=True)\n",
" expz = np.exp(z)\n",
" probs = expz / expz.sum(axis=1, keepdims=True)\n",
" loss = -np.log(probs[np.arange(n), y] + 1e-12).mean()\n",
" grad = probs.copy()\n",
" grad[np.arange(n), y] -= 1.0\n",
" grad /= n\n",
" return loss, grad\n",
"\n",
"check(\"9.5 loss at init = ln(q)\", lambda: _check_xent_init(softmax_xent), required=True)\n",
"check(\"9.5 grad vs finite diff\", lambda: _check_xent_grad(softmax_xent), required=True)"
]
},
{
"cell_type": "markdown",
"id": "3086faba",
"metadata": {},
"source": [
"> **Interpretation.** The gradient passes a finite-difference check to better than $10^{-4}$ relative error, and the loss at uniform logits is exactly $\\ln q$. Both facts will be reused: the init-loss identity is the first sanity check on the FashionMNIST model below.\n"
]
},
{
"cell_type": "markdown",
"id": "df06d62d",
"metadata": {},
"source": [
"### Exercise 9.6: The 2-layer MLP backward pass\n",
"`Difficulty 4/5 · ~25 min`\n",
"\n",
"Implement `mlp_backward(grad_logits, cache, W2)` returning gradients for `W1, b1, W2, b2`. The forward was `Z1 = X@W1+b1; H = relu(Z1); Z2 = H@W2+b2`, with `cache = (X, Z1, H)`. Walk it backward using the matmul rule, the broadcast-bias sum, and the ReLU mask. Every gradient is finite-difference-checked against the full forward loss.\n"
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "2cae26ed",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:57.049752Z",
"iopub.status.busy": "2026-06-10T19:14:57.049661Z",
"iopub.status.idle": "2026-06-10T19:14:57.078449Z",
"shell.execute_reply": "2026-06-10T19:14:57.078177Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] 9.6 all grads vs finite diff: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def mlp_backward(grad_logits, cache, W2):\n",
" \"\"\"grad_logits = dL/dZ2 : (n,q). cache = (X, Z1, H). Returns dict of grads.\"\"\"\n",
" X, Z1, H = cache\n",
" # Z2 = H @ W2 + b2\n",
" # TODO 1: dW2 = H^T @ grad_logits -> (h,q)\n",
" dW2 = None\n",
" # TODO 2: db2 = sum of grad_logits over the batch axis -> (q,)\n",
" db2 = None\n",
" # TODO 3: dH = grad_logits @ W2^T -> (n,h)\n",
" dH = None\n",
" attempted(dW2, db2, dH)\n",
" # H = relu(Z1) -> dZ1 = dH * (Z1 > 0)\n",
" # TODO 4: dZ1 (apply the ReLU mask) -> (n,h)\n",
" dZ1 = None\n",
" # Z1 = X @ W1 + b1\n",
" # TODO 5: dW1 = X^T @ dZ1 -> (d,h)\n",
" dW1 = None\n",
" # TODO 6: db1 = sum of dZ1 over the batch axis -> (h,)\n",
" db1 = None\n",
" attempted(dZ1, dW1, db1)\n",
" return {\"W1\": dW1, \"b1\": db1, \"W2\": dW2, \"b2\": db2}\n",
"\n",
"# self-check (run this cell): every gradient vs finite differences through the full forward+loss\n",
"def _check_mlp_backward(fwd, back, xent):\n",
" g = np.random.default_rng(3)\n",
" d, h, q, n = 4, 6, 3, 5\n",
" P = {\"W1\": g.standard_normal((d, h)) * np.sqrt(2/d), \"b1\": g.standard_normal(h),\n",
" \"W2\": g.standard_normal((h, q)) * np.sqrt(2/h), \"b2\": g.standard_normal(q)}\n",
" X = g.standard_normal((n, d)); y = g.integers(0, q, size=n)\n",
" logits, cache = fwd(X, P[\"W1\"], P[\"b1\"], P[\"W2\"], P[\"b2\"])\n",
" _, dlogits = xent(logits, y)\n",
" grads = back(dlogits, cache, P[\"W2\"])\n",
" def loss_of(name):\n",
" def f(flat):\n",
" P2 = dict(P); P2[name] = flat.reshape(P[name].shape)\n",
" lg, _ = fwd(X, P2[\"W1\"], P2[\"b1\"], P2[\"W2\"], P2[\"b2\"])\n",
" return xent(lg, y)[0]\n",
" return f\n",
" for name in [\"W1\", \"b1\", \"W2\", \"b2\"]:\n",
" num = grad_check(loss_of(name), P[name].ravel()).reshape(P[name].shape)\n",
" err = rel_err(grads[name], num)\n",
" assert err < 1e-4, f\"d{name} disagrees with finite diff (rel err {err:.1e})\"\n",
"\n",
"check(\"9.6 all grads vs finite diff\", lambda: _check_mlp_backward(mlp_forward, mlp_backward, softmax_xent))"
]
},
{
"cell_type": "markdown",
"id": "816162a1",
"metadata": {},
"source": [
"Hint 1 (conceptual)Go layer by layer from the loss backward. At each matmul `C = A @ B`, `dA = dC @ B.T` and `dB = A.T @ dC`. A bias added with broadcasting gets `db = dZ.sum(axis=0)`. ReLU multiplies the incoming gradient by `(Z1 > 0)`.\n",
"\n",
"Hint 2 (pseudocode)\n",
"\n",
"```python\n",
"dW2 = H.T @ grad_logits\n",
"db2 = grad_logits.sum(axis=0)\n",
"dH = grad_logits @ W2.T\n",
"dZ1 = dH * (Z1 > 0)\n",
"dW1 = X.T @ dZ1\n",
"db1 = dZ1.sum(axis=0)\n",
"```\n",
"\n",
"\n",
"Help: \"dW1 has the wrong shape\"`dW1` must match `W1`, i.e. `(d, h)`. That comes from `X.T @ dZ1` = `(d,n)@(n,h)`. If you wrote `dZ1 @ X.T` you get `(n,n)` or an error; the order of the matmul rule matters.\n"
]
},
{
"cell_type": "code",
"execution_count": 23,
"id": "1a667787",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:57.082660Z",
"iopub.status.busy": "2026-06-10T19:14:57.082542Z",
"iopub.status.idle": "2026-06-10T19:14:57.107314Z",
"shell.execute_reply": "2026-06-10T19:14:57.106604Z"
},
"collapsed": true,
"jupyter": {
"source_hidden": true
},
"tags": [
"hide-input"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] 9.6 all grads vs finite diff\n"
]
},
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#@title Solution { display-mode: \"form\" }\n",
"# solution: redefines mlp_backward; the check re-verifies against finite differences.\n",
"def mlp_backward(grad_logits, cache, W2):\n",
" X, Z1, H = cache\n",
" dW2 = H.T @ grad_logits\n",
" db2 = grad_logits.sum(axis=0)\n",
" dH = grad_logits @ W2.T\n",
" dZ1 = dH * (Z1 > 0)\n",
" dW1 = X.T @ dZ1\n",
" db1 = dZ1.sum(axis=0)\n",
" return {\"W1\": dW1, \"b1\": db1, \"W2\": dW2, \"b2\": db2}\n",
"\n",
"check(\"9.6 all grads vs finite diff\", lambda: _check_mlp_backward(mlp_forward, mlp_backward, softmax_xent), required=True)"
]
},
{
"cell_type": "markdown",
"id": "2d33431b",
"metadata": {},
"source": [
"> **Interpretation.** Six lines reproduce what `loss.backward()` does for this graph, and finite differences confirm all four gradients. PyTorch generalizes exactly this to arbitrary graphs. You now understand the thing the next chapter hides behind `nn.Module`.\n"
]
},
{
"cell_type": "markdown",
"id": "3d3b2114",
"metadata": {},
"source": [
"### The anchor dataset: a FashionMNIST subset\n",
"\n",
"FashionMNIST is 70,000 grayscale 28x28 images in 10 balanced classes (the MNIST drop-in that is not trivially easy). We load it via torchvision into `root=\"data\"` (idempotent: the second run is a no-op), take a subset so the whole notebook stays under the CPU budget, flatten to 784 features, and standardize.\n"
]
},
{
"cell_type": "code",
"execution_count": 24,
"id": "bbca53b2",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:14:57.108496Z",
"iopub.status.busy": "2026-06-10T19:14:57.108378Z",
"iopub.status.idle": "2026-06-10T19:15:00.740513Z",
"shell.execute_reply": "2026-06-10T19:15:00.740221Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"full train (60000, 784), test (10000, 784), 10 classes\n"
]
}
],
"source": [
"# load FashionMNIST once; download=True is a no-op if the cache already exists\n",
"from torchvision import datasets\n",
"_train = datasets.FashionMNIST(root=\"data\", train=True, download=True)\n",
"_test = datasets.FashionMNIST(root=\"data\", train=False, download=True)\n",
"CLASSES = _train.classes\n",
"# .data/.targets are uint8 tensors; go straight to numpy (avoids a per-image PIL conversion)\n",
"Xtr_full = _train.data.numpy().reshape(-1, 784).astype(np.float64)\n",
"ytr_full = _train.targets.numpy().astype(np.int64)\n",
"Xte_full = _test.data.numpy().reshape(-1, 784).astype(np.float64)\n",
"yte_full = _test.targets.numpy().astype(np.int64)\n",
"print(f\"full train {Xtr_full.shape}, test {Xte_full.shape}, {len(CLASSES)} classes\")"
]
},
{
"cell_type": "code",
"execution_count": 25,
"id": "1e8790eb",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:15:00.748480Z",
"iopub.status.busy": "2026-06-10T19:15:00.748309Z",
"iopub.status.idle": "2026-06-10T19:15:01.765501Z",
"shell.execute_reply": "2026-06-10T19:15:01.765185Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"subset train (12000, 784), test (2000, 784) | standardized mean~3.67e-18 std~1.00\n"
]
}
],
"source": [
"# subset + standardize using TRAIN statistics only (never peek at test)\n",
"N_TR = 4000 if FAST else 12000 # subset size; full notebook stays < a few minutes on CPU\n",
"N_TE = 2000\n",
"sub = np.random.default_rng(SEED) # local RNG so the subset is reproducible in isolation\n",
"tr_idx = sub.choice(len(Xtr_full), size=N_TR, replace=False)\n",
"te_idx = sub.choice(len(Xte_full), size=N_TE, replace=False)\n",
"mu, sd = Xtr_full[tr_idx].mean(), Xtr_full[tr_idx].std() + 1e-8\n",
"Xtr = (Xtr_full[tr_idx] - mu) / sd; ytr = ytr_full[tr_idx]\n",
"Xte = (Xte_full[te_idx] - mu) / sd; yte = yte_full[te_idx]\n",
"print(f\"subset train {Xtr.shape}, test {Xte.shape} | standardized mean~{Xtr.mean():.2e} std~{Xtr.std():.2f}\")"
]
},
{
"cell_type": "code",
"execution_count": 26,
"id": "4822b13e",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:15:01.772255Z",
"iopub.status.busy": "2026-06-10T19:15:01.772125Z",
"iopub.status.idle": "2026-06-10T19:15:03.358496Z",
"shell.execute_reply": "2026-06-10T19:15:03.358172Z"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAABEEAAACcCAYAAACQjrU2AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAPu9JREFUeJzt3XeYFWWaNvDbEclRWjJNFGiguwlNVJIKBpaBQXEVRV3TmkZHx3FlvRxdLx0dxXVcdeQadHRwFBUDCmYUVEyAEkVyRkFyEASF+v7ws4fnfh+6Di0dTtf9++85nrdO9am33qou+7k5KoqiCCIiIiIiIiIiZdyvSnoHRERERERERESKgx6CiIiIiIiIiEgi6CGIiIiIiIiIiCSCHoKIiIiIiIiISCLoIYiIiIiIiIiIJIIegoiIiIiIiIhIIughiIiIiIiIiIgkgh6CiIiIiIiIiEgi6CGIiIiIiIiIiCRCYh6ClCtXDh07dkTbtm3RuXNnjBkzpqR3SYpZuXLl0KFDB7Rv3x7Dhg3D7t27C3x/RkYGAGDq1Kk466yzimMXpZS744470K5dO2RnZyMvLw8rVqzInyfs0ksvxbJly9z/du+99xblbkoR8+bBL5HKGnPRRRdh0qRJv+hzpOSsXbsWQ4cORYsWLZCXl4dhw4Zhw4YNh7WNqVOnYvr06UW0h1KSdG1JriN9PTlY3759MX/+/CO2PSl++/btQ4cOHdChQwfUq1cPjRo1QocOHdCzZ8+S3rW0V66kd6C41KxZE7NmzQIArF69GkOGDEEURbj88svN+/bv34+jjz66JHZRiljNmjUxe/ZsAMB5552H0aNH44YbbijZnYLmXLr4+OOPMWXKFMyePRvHHHMM1q5diypVqhzy/Y899pj7+oEDB3DvvffipptuKqpdlSJ0uPNAJIoiDB48GFdddRVeeuklAMCHH36IjRs3om7duilvZ+rUqcjIyEDXrl2LalelBOjaklyl9Xqi+9LSo3z58vm/u9x+++3IyMjANddck//fi/NYlbV5kZi/BDlYZmYm7r//fvz1r38F8NOkuvDCC9GzZ09ce+21WLZsGU499VTk5eXhpJNOwsqVKwEAf/nLX9C6dWvk5ubiyiuvBAA8++yzyMrKQm5uLgYPHlxSP5Icpl69emHp0qXB/4E966yzMHXq1EOO27RpEwYNGoScnBz07dsXK1euxPbt23H88cfnv2fVqlXo0KEDAGDmzJno06cPOnfujEGDBmHLli0AgKZNm+Lmm29Gx44d8d577xXJzyhH1vr165GRkYFjjjkGANCoUSPUqlULAHDjjTciOzsbJ598Mr777jsA9v/A1K5dG9dccw2ys7MxbNgwbNu2DR06dMAVV1xRMj+MFNqh5sHll1+Ozp07o127dhg1alT++zMyMtz5MX36dLRv3x4dOnTA+PHj898/YcIEdO3aFR07dsTAgQOxbdu2Yv355Mh79913UbVqVVxyySX5r/Xq1QstWrTAiBEjkJOTg65du+bf6H766afo0aMHOnXqhD59+mDVqlVYs2YNRo8ejXvuuQcdOnTIf6+kP11bkutQx/5Q141D/X4yevRodOnSBbm5uRg+fDh++OEH8zn79u3D0KFDMXr0aHz33Xe46KKL0KVLF3Tu3BnvvPMOgPB3ISm9LrroIlx55ZXo2rUr7rnnHrz99tv5f+l+ww03IIoiADB/Tfbwww/j9ttvB+D/Prtx40YMHToUeXl56NGjR/4fDvBnlSlRQtSuXdvU27ZtiypWrBhFURTddtttUc+ePaO9e/dGURRFAwYMiFasWBFFURS9++670VlnnRVFURTVqlUr2rVrV/74KIqi9u3bR4sXLzavSen08xz44Ycfol//+tfRX//612jKlCnRmWeemf+eM888M5oyZYp5/8Hvufrqq6M///nPURRF0bPPPhsNGjQoiqIoGjhwYPTpp59GURRFo0aNiu68885o3759Ue/evaPNmzdHURRFjz/+eHTjjTdGURRFTZo0iR5++OEi/onlSNqxY0fUvn37KCsrK7r22mujGTNmRFEURQCid955J4qiKBoxYkQ0duzYKIqiqE+fPtG8efPy3zNp0qT8bfF6JOnjUPPg5/P8hx9+iLp37x6tXr06iqJDz4/27dvnjz377LPz15gtW7ZEBw4ciKIoih588MHozjvvjKIoii688MJo4sSJxfRTypH04IMPRr/73e+C1++7777oyiuvjKIoij755JMoJycniqIo2r59e/Tjjz9GURRFr7zySnTppZdGUfTTvcpDDz1UTHstxUXXluQ63GN/qN9Pfr7+RFEUXX/99dE///nPKIp+miuff/559Otf/zoaPXp0FEVRNHLkyGj8+PFRFEXRxo0bo9atW0cHDhwIfheS0ufna8CFF14YDRs2LDpw4EC0e/fuKDMzM1qxYkW0f//+6IwzzohefPHFKIrsevDQQw9Ft912WxRF/u+zw4cPj6ZPnx5FURQtXrw46tq1axRFkfmssiYx7TAs+v9PyX42ePBglC9fHrt27cKHH36IIUOG5L/v5z9N69q1K84//3wMGzYs/7+fcMIJuPzyy3HeeecpN6KU+/n/kABA7969cckll+Djjz8+rG1MmzYNr7/+OgDg7LPPxnXXXQcAGDZsGMaPH49u3brhxRdfxJNPPolFixZhzpw5OOmkkwAAP/74I9q1a5e/rWHDhh2Bn0qKS7Vq1TBr1ixMmTIF7777Lvr374/nn38eVatWxSmnnAIA6Ny5c/7/mTlYpUqVMHDgwGLeYykKh5oHixcvxmOPPYb9+/dj7dq1WLhwIRo3buzOj23btmHv3r3Iy8sD8FN73tixYwH81K75c17Enj170K1btxL7WaVoTZs2Lb91oXv37tizZw+2b9+Obdu2YcSIEVi2bBkOHDiQ/1cBUjbp2pJch3PsC/r9ZM6cObj11luxfft2bN++HZUqVcr/jIsuugiXXXYZ/vM//xMA8Pbbb2PSpEm48847AQDfffddfj7Rz78LSel31lln4aijjsKiRYvQunVrNG3aFMBP9xMffvghhg4desix3u+zkydPxpdffpn/nq1btwafVdYk9iHI7Nmz0aZNm/y6cuXKAH7qqaxbt677p6avvfYapk6digkTJuCBBx7AjBkz8Oijj+LTTz/FxIkTkZeXh3nz5pnFR0qPgzNBflauXDkcOHAgv967d+9hbfPnRWHIkCG46667cN1112HPnj1o1aoV5s6di44dO2LKlCnu2J/nnKSPcuXKoX///ujfvz8yMjLwyiuvoEKFCvn//eijj8b+/fuDcTrWZQvPgwceeAArV67EJ598gho1auCss87KX0sONT8OdUNx7bXX4pZbbsGAAQMwadIkPPnkk0X+80jRysrKys8CScUf//hHDBw4EJdffjnmz5+Piy66qOh2TkoFXVuSK9VjX9DvJ5dccglee+01ZGVl4eGHHzYPzHr27InJkyfjyiuvzL/nnThxIpo0aRJsR/MpfaRyrA6+zzj49xvv91ngpxb+cuXCRwNldV4kMhNkzZo1uPHGG02wzM+qV6+OunXrYuLEiQB+CoGZP38+Dhw4gDVr1uDkk0/GqFGjsHr1auzfvx/Lly9Hjx49cNddd6F8+fLYvHlzcf848gtkZmZiwYIF+PHHH7Fhw4bYvww58cQT8cwzzwAAXnjhhfyAuho1aqBVq1b4r//6r/y/CGrTpg3WrFmDzz//HMBPC9DChQuL8KeRorRo0aL8RP4oijB//nxkZmYWaluHuqGV0s+bB926dUPVqlVRvXp1rF27FpMnTy5wGzVr1kSFChXwxRdfAADGjRuX/9927NiBhg0bIoqi/L8OkfR2yimnYMeOHeaB1rRp05CXl5d/PZk+fToqV66MGjVq5M8BAGZMtWrVsHPnzuLcdSkGurYk1+Ec+0P9fgL89NccdevWxb59+8z1BACuueYadOrUCRdffDGiKMKAAQPwf//3f/n/XflC6a1169ZYvHgxVq1ahQMHDmDcuHHo3bs3gJ9+N1m1ahV++OGH/H9d7lC/z/br1w+PPvpo/nbnzJlTIj9PcUrMQ5CfWyHatm2LIUOG4IorrjAhZQd75pln8NBDDyE3NxfZ2dl49913sX//fpx33nnIyclBXl4e/vjHP+Loo4/ODy7Kzs7Gb37zGzRq1KiYfzL5JTIzM3HGGWegbdu2uPTSS9GxY8cC33/77bdj6tSpyMnJwSOPPIIHH3ww/78NGzYM48aNy29zKV++PJ577jlcd911yM3NRefOnROxqJRVu3btwvnnn4927dqhffv2OHDgAH77298WalsXXnghsrOzFV6Xhrx5cNNNNyErKwtt2rTBJZdcghNPPDF2O2PGjMEFF1yAjh07onbt2vmv33bbbRg0aBC6dOmCxo0bF+WPIsXkqKOOwoQJEzBhwgS0aNEC7dq1w0MPPYQRI0Zg27ZtyMnJwTXXXIMnnngCAHDTTTfh+uuvR6dOncyfpg8aNAjjxo1TMGoZo2tLch3usfd+PwF+ujfNy8tD7969kZOTE4y77bbbULNmTVx//fX5bTM5OTlo27atCfKW9FOpUiX87W9/w+DBg5GTk4Pjjz8+v8XlzjvvxEknnYS+ffuiefPmAHDI32cfeughTJ06Fbm5ucjKysp/QF+WHRVxOIaIiIiIiIiISBmUmL8EEREREREREZFk00MQEREREREREUkEPQQRERERERERkUTQQxARERERERERSQQ9BBERERERERGRRNBDEBERERERERFJhHKpvvGoo44qyv34RX71K/ss58CBA7FjKleubOrdu3cf9ue2adPG1Lfddlvwnssvv9zUO3fuPOzPKS6/5F9LPhLzw9sG7xO/pzD7XLt27eC1SpUqmXrt2rWHvd2KFSuaum/fvqaePn16MGbLli2H/TlH4jsojMJ+TmleO1Jx9dVXm7pBgwambtasman37dsXbKNp06amfvzxx0391FNPxe4Hr3OpHI+48yeVMako6bVDSreSnh+pXFsYn29AavcWcR555BFTb9u2zdR8LQKA+vXrm/rcc8/9xfvhfSdx91JFda0pC9eWcuXs7fSPP/4YO4bvEW655RZTf/XVV6auV69esI2aNWuaum7duqb2ri2jRo0qcL+8uc/HqLTfdwCla35I0SgLa0fjxo1NPWnSJFOvW7cuGMPXiV69epmaf9fctWtXsI1GjRqZeu/evabesWNHMGbp0qWmzsjIMHXPnj2DMZs2bQpeKw6pzA39JYiIiIiIiIiIJIIegoiIiIiIiIhIIughiIiIiIiIiIgkQsqZIKUJ93Kl0qc7fvx4U/fr18/UM2fONPV9990XbKN69eqmfumll0zt9T3xdjt06GDqPXv2+DucQEcq46Bbt26m5r633NzcYAwfW+5r47yPChUqBNs4/vjjTf3MM8/Efi7PmSVLlpj6gw8+CMYUVy9uujkSWSknnHBC8Noll1xi6qOPPtrUxxxzjKm5NxwAjj32WFNzftDXX38djHn33XdNnco6F5cbwvsOAPv374/drkg6K8xakMr5xlkdOTk5wXs4f6F9+/am3rp1a+zn1KpVy9Tdu3c39d///vdgzJQpU0w9bdo0U3vfSdxaUJhslaSIywDhawAQHqMVK1aYulq1aqb27hc5J6R8+fKmvummmwrcLyCco4XJvtHcECkc/n2C7ym93wOqVKlS4Hs4b7BTp07BNjZs2GBqXsN++OGHYMy3335r6hEjRpjayy0qqUyQVOgvQUREREREREQkEfQQREREREREREQSQQ9BRERERERERCQR9BBERERERERERBLhqCjF5CIv9KikxAUgnn766cGYSZMmmfq7774zdeXKlU3tBQju2rXL1N98802B2wSAzMxMU48ePdrUt9xySzCmpPySEKsjMT9SCdbioLCrr746dgzzQr8WL15sag6wbdCggam9ELQFCxYU+Dle6Nz27dtNzfOQw1YB4LHHHjP18uXLTV1UAWWF3UZRrR0cAppKmFvbtm1NfeaZZ5p67dq1wZjhw4ebmsMPt23bZuq9e/cG2+Bg3dWrV5uaAxaB8LjyWjF37txgDOOQ1rjwvsIq6bVDSrd0nB99+/YNXvvTn/5k6h49epia1wIA2LFjh6l5jecQPO8c5e3y/UmdOnWCMXw/wmvQP/7xj2DMgw8+aOqlS5cG7ykKRXltORKB2alo2LChqU8++WRT/+EPfwjGxAW916xZ09QrV64MtsHHuVKlSqbmYHggvC4sWrTI1I8++mgw5uWXXw5eKw7puHZI8Slt96WF0a5dO1M/8cQTpvbOvSZNmpia1wG+7jRv3jzYBl9rNm7caGrvH4DgcOYBAwaY+ve//30w5v333w9eKw6pzA39JYiIiIiIiIiIJIIegoiIiIiIiIhIIughiIiIiIiIiIgkQrn4t5Q+cX0+//7v/x68tnnz5gJr7o36/vvvg22UL1/e1NzLW6VKlWAM5wPk5eU5eyypuuCCC0zt5Wx8/fXXpuZ+O87/AMJ8Ce6FXrFiham9PIbOnTubmnu/vX7eevXqmZrnFOeMAEBubq6pOTuiqHqeSxvOAOG8j3PPPTcYw983f3ce7rfnY/TZZ5+Zuk2bNsE2eO3Yt2+fqblHGwB++OEHU48cOdLU3hrF71m/fr2pU+mDTcr8EaB79+6mbtWqVfCesWPHFtfuFKvq1aubmnMRatSoEYzhfmjOEPJyiXbv3m1qzoHKzs6O3dd58+YVuG/eOcu5IXytPOecc4Ixl156qamffvrpAv97Ur3wwgvBa5wv9dJLL5n68ccfD8Y0bdrU1JwrwnOUrzUA0LFjR1PzteTjjz8OxnB+wKxZs0zN1xEAGDdunKmvvPJKU3OOQSqKKr9MJJ1kZGSYmnN9vHOCMym9e8iDbdmyJfY1zlzk6x0Q3pfyNe+4444rcD9KG/0liIiIiIiIiIgkgh6CiIiIiIiIiEgi6CGIiIiIiIiIiCRCWmaCxOE+ZwDYtWuXqblflnv2vf4qfo2zAThXAgh7uerXr+/ssQCp9YJmZmaamnuyAeDYY481NecibNiwIRjD/+Y2zxc+bl4WCc8H/py6desGYypWrGjqqlWrmnrnzp3BGN4O9+Dxv/Wdjvhc8nrt+bsbNGiQqbdt2xaM4ePK2+3Zs2cwZv78+abetGmTqXkt6dKlS7CNG264ocDP4fwYINx/ngu1atUKxjzwwAOm/u1vf2tq3ncg/K7Vk314uK89rjeXe2q9bXjXEsbv8bbLzj77bFP379/f1F7fMPcnc59wuvb1v//++6bmewDOCAHC3m1e8znrBwhzHdgbb7xh6iVLlgTv4XwjznTwrke8LnE2iTdPOa+kX79+pvZ+lh07dgSvlSZHYi7ef//9pm7evHnwnhtvvNHUnD3WtWvXYExWVpapFy5caGqek40bNw62MWPGDFNzZhhvAwDmzJljap5PkydPDsbw/Qznw3jX25dffjl47WDpsE6IFLWaNWua2rsOx+H1nc977/6Ar+18PeN1wVOhQgVTc65Raae/BBERERERERGRRNBDEBERERERERFJBD0EEREREREREZFE0EMQEREREREREUmEMhGMymFvXjALB2hyIAwHNPF/91SpUqXAGgA2b95sagWjHp4GDRqYmkN4OEgUCMPbOODu1FNPDcZ88cUXpm7Xrp2pZ82aZWov5IyD0DiY6McffwzGrFixInjtYBzEBwBbt241Nc+7shCMmgoOQuWgyL1798Zug8OkJk6cGLyHg3br1Klj6ptvvtnUXlhvy5YtTb1q1SpTewFxHNr6/fffm9oLreKgvbvvvtvUl112WeznyOHhY5dKQGncNrygSxb3ntatWwevDR061NQcwjlmzJhgDAehsnQNN4wL0PZ+7q+//trUHJDpHRMOS23RooWp+brhheLWqFHD1LwWeGsdB59Wq1bN1F5IMq+HHLrNQboA8OKLLwavpbvTTjvN1Hw/wOs3EB43DuFet25dMIbDa/l+hueod7/DIYocVOwFlq5evdrUHG7rBbLzfQf/PCNHjgzG8H3TypUrg/eIJB0Ho/Ja4v0+yuc1h5ry7wXe7x9x7/GuK7wv/PsG/+MVpZ3+EkREREREREREEkEPQUREREREREQkEfQQREREREREREQSoUxkgnz22Wem5t5FIOyP5T5d7qOsWLFisA3u9eY+3cceeywY069fP1NzL6/X/6se/X/p1KmTqRcsWGDqtm3bBmPmzJljap4PM2bMCMbUrl3b1PPnzy9wGx999NEh9vhfevbsaeqPP/44eE+tWrUK3IbXC8i93LzvZaHvNpVzIDs729R8fvK55uHzz8v14WyXxo0bm/r222839YgRI4JtcC4Nfw5n3wDA9u3bTc39md4YVpj+TF4L0zXzobhwPhD313MulDe3+T3t27c3tZczwnOK18rf/e53wRheGx955BFT876XZXxP8O2335qa8z6AcA33MpsY5yvwNYz3Izc3N9gG36/wNnkb3nsWLlxoam9OHX/88QW+h7ONyoJzzjkneO3pp5829cMPP2xqPteAMDekXr16pva+u27dupmar2mTJk0ydeXKlYNt8L0Jn/de9gvPJ74O9urVKxjD9028jn3wwQfBmLlz55q6TZs2puaMHSC8/oiUdZznwdciL8+D84T4HOb8IM4BAsKMSn4P7xcQnvdr1qwxdSp5mqWJ/hJERERERERERBJBD0FEREREREREJBH0EEREREREREREEqFMZILwv9nO/9Y9EPbge5kfB/N6o7k3nvulrrrqqmAM90Qed9xxpvb6+teuXVvgviVJs2bNTM05CV5f85AhQ0z96aefmtrLiuB+ac4aqVatmqm9fz+b/71s3ib/u95A+O+Dc8+v18d37LHHmrpHjx6m/vzzz4MxpR33M3LfIefvAGGfJJ+P3pgNGzaYev/+/ab25gavHcuXLzc1f99ergivJzxvvdwNni+cE+F9DveJ8nrDfeoAsH79+uC1pIrLQ/H6XTkDho9B3bp1TZ1K7gbvx549e4L31KlTx9R8HfTyku64447D3peywPsuqlevbmo+3zgLCACaNGli6i1btpjaO06c18FziPMZvPsXPkeXLVtmam/94Dn03Xffmdr7Tng7vD7y9bi08XrYea3ldfT6668PxvAan5WVZWrOWwHCTBDOj/GOER97rlu1amVqzgoAgOHDh5ua7xm83Dme27xm8f0OEN53/OEPfzC1l3nG83bcuHGm7tOnTzBGGVSSNHG/X3iZT/wevh/ctWuXqTnvyRvD65p3vxN3r+7dl5Zm+ksQEREREREREUkEPQQRERERERERkUTQQxARERERERERSQQ9BBERERERERGRRCgTwaidO3c2NQfCAMDUqVNNXatWLVNzQCkHYQFhEOPWrVtj943Dov7jP/7D1F44mYJR/4UDbLlu3LhxMIZDvzici4MKgTDMZ/PmzabmECIvOIyDOHkMB7IB4Rzi8LrWrVsHYzi8c+bMmcF70k1c2FJubm4whr9vDs1t3rx5MIZD8jjM1gvW433jkKrs7GxTe+GqmZmZpuYQSy+ctEOHDgVug+cKEAbrcjhit27dgjGvvPJK8FpS8bHl+cLhhwCwYMECU48ePdrUHNTshdN27drV1LzOeWFjvM7xde+NN94IxixdujR47WA8X4D4oMLCjCluLVu2DF7zgqoP5oWcLlq0yNR8XLwAOz6WHDbK6wuHVAJhICZfW/i8B8LgVz4mFSpUCMbw2sX70rBhw2BMaeIFpbN/+7d/M3XVqlWD9/Cx55/bm/PffPONqfmY8P0jEF7/+brXqFEjU3Owt7evfC/rXVv4OPMcTSUgfOTIkaZ+6623gjF5eXmm5nBV3gYA3H333cFrImUZ/wMJvI555yOf97wNrseOHRts49RTTzV1RkaGqfmeGoi/H+Y1rLTTX4KIiIiIiIiISCLoIYiIiIiIiIiIJIIegoiIiIiIiIhIIpSJTJDBgweb+quvvgrew72XW7ZsMfX8+fNN7fU3/s///I+pOe/D889//tPUl1xyialr164du40k4+yNWbNmmbpXr17BmPr165uasyG8XnDuo12zZo2pmzRpYmrOdADCnl/uh509e3Ywhnu9ed+5Rw8IcywqVaoUvCfdeP3SB+P+RiDMROC+Se6bB8Le7lR6L/nYcz8477u3r5w1wr3R/LMAQKtWrUzNmQ/cxw2E/Zm7d+82tZcxI/8SNw/5nAaAJ5980tScvfDyyy/Hfu6jjz562J/75ptvmprn7o4dO4IxnEn1xBNPxO5bnNKW/+HhPmYgPNZengfjLDFePzjvAwiPC2cZcXaEtw2+DnKeh7fvnOfBx4kzQ4BwveDvjdetdMA/9/jx403N5xEADB8+3NSvvvqqqb3sqBkzZph6+fLlpvbyy7jnns9Znl8ffvhhsA3OGmvTpk2B2wSA119/3dQ8B1evXh2MadGihan5XNi3b18whrP4+Jy7+OKLgzEnnnhi8JoU3tVXX21qnst8r1tU0iE7qqTwecG5Gt59aVx+Ga/vX3zxRbANvh/Myckx9aZNm4IxfN/J1whv7SjN9JcgIiIiIiIiIpIIeggiIiIiIiIiIomghyAiIiIiIiIikghpmQnC/Zhdu3Y19dq1a4Mx3B/FPbVcL1y4MNjG6NGjTb1o0aLYfeV+O+71atu2bew2koz73Dhvweuf5twH7oXjvjcA+Oabb0zdqFEjU2/YsMHUXlYH99fxNo877rhgDGdBxO2H9x7u4y6LOLMFCM8l7jktTL+pN4bnGL+H56TXN89rUr169Qr8DCDMeuH8GO9zuC+bc0S8fut7773X1Enp002lR5nPWe847dmz57A+pzDfLx9HAPjoo49MzVkTI0eODMYsXbq0wM9JZd84P+Dcc88N3jNz5szY7ZQ071gezMvM4PU6LkMGCO9XOKuI54eXZcQ5Vrzv3lzmtYDvcTjTAQj7uTlXxLselXac7zFq1ChTv/fee8GYW2+91dQDBgwwdbNmzYIxfM/I3xWfn0B4XeO5wsfdy37hXLk6deqY2psbPLe55nMcALp162bq9evXm3revHnBGM5DGjt2rKnffvvtYMxll11m6qRcj1LB9659+vQx9X333ReM4Xl5zjnnmPqWW24JxnzwwQeF3cVDSuU45ubmmtpbozm/sSzg6zv/3sP3ugBQtWpVU/P6XqVKFVNz/hAArFu3ztQ9evQwtZelxfc7vFbwfWppp78EEREREREREZFE0EMQEREREREREUkEPQQRERERERERkUTQQxARERERERERSYS0DEblICgO7/JC5DhwikN6eBte6CAHzXhBl8wLsjoY/yxJxt8vEAbRccDk999/H4xZtmyZqTnIxxsTF9rE84NrIAwemj17tqk5sMzbVw66SmV+tGzZ0tQcqAnEhzeWdt7PxOcWBwF684kDbjlU1jtf+bvjQCo+7l6oMgd+xQWYep/LvO8kLjQ3HYMNS1KLFi1MPWvWrMPeRty1BogP6eR1AQC2bt1qap7vcSGoQBiIWL9+/eA91apVMzWHed94443BmOuuuy72s4uTd15z2BzXfM8AhNej7777ztSFCXHkz/GCLDl8jtcLbwxf5+LC4IEwkI/fw2tfOhg3bpyp+Vy7++67gzHPP/+8qefOnWtqL1SWQ0x5PnnrNR97rvka4AXx8udysK4XcsrHkWvveuS9drAvv/wyeO3BBx80defOnU29adOmAreZJL179w5e+9///V9T83o9Y8YMU48ZMybYBv8uw2u8N//vuOMOU7/11lvOHh+eVq1aBa/993//t6n79+9v6s8++ywYM3To0F+8L6UNX0f4HPauRd5rB0vlnn/jxo2m5nXAW7N4X3l+xa0TpY3+EkREREREREREEkEPQUREREREREQkEfQQREREREREREQSIS0zQbiPifvea9SoEYypWLGiqbmf+ttvvzW116PNeQI9e/aM3de4nmGvPzypvB43zl3h3u527doFYx5++GFTcx4D9+p6r/GcGjBggKlffvnlYBs7d+40ddOmTU196qmnBmOeeuopU3NPNmegAGGfNvftxeXQpCPuxfdwn7aXp/L111+b+le/ss+BuecWAGrWrGnquB5+njsA0KBBA1PzcfX2lecCz1Ev84R7SbmHfP369cGYpEolv4HPrWnTpv3iz43L//B4WS4rV640NWd3ePkNPGc4T8jr227cuHGBY+bNmxeM4blb0o4//vjgNc7M4POYv08gvLfgTAMvm4Ov8ZwHxN9VKpkgqcwh3m6zZs1MzWsFEK4XZfF+hfM+vvjii+A9zz33nKlfe+01U7du3ToY07dvX1Pz/PGuyz169DA19/HzPWdmZmawjSZNmpia74m8bAC+V+F1zruv4vtu3nfvnO/QoUPwWknyzi1+ja+zfI/gvScVeXl5pr7iiitM7a29nEWzYMECU9erV8/U3jrHGSB83nv3EW+++aapeR7yfgDh8W/evLmpvTWL72U3b94cu29lkbcWH8w7t3h94fvSuG0C4T0Cf4439/lz+BhxRlFpp78EEREREREREZFE0EMQEREREREREUkEPQQRERERERERkURIy/AA7i1LJROE+xkZ9/17fU1ZWVmm5l4o799j535D7oHjrJIk83qw+bhwT7J3rNesWWPqbt26mdr7t+lXrVplaj6WnAHi9W9yr/zrr79uau/fPK9Tp46pP/30U1N7PZ78M3/55ZcF/ncg7AEu7bjv0Du3GM8N7kkFwh7Hjz76yNTeHOTeyi1btpia+yo5xwYIv3/uffX6xVesWGFqzmbwbN++3dT883rfI69BnJWQDrw8Ic43YF4mCB9LzpDxenN5jU8la+RwrV69OniNrz9vv/22qTkLCQA6duxo6mOPPdbU/fv3D8bwesrXRi9zgLOwSpp3TLivP64GgF27dpmaf3Zezz18TeM1h/PJgDBDKJX+bz62fN/knR+8PvDPy9kRQPHM/6K0dOnS4LULLrjA1GPGjDE1r7NAuIZz9ouXp8LHhL9v5t2X8n0HX59SuS/lffOyADhzKzs729TLli0Ld5jwvC3u7CBvbsbN11TyPzIyMkzNeSlAmCtXt25dUy9evDgYw9/XwIEDC9yG97PwnOL54K0dM2bMMDX/3sI5I0A4//k+YseOHcEY/t2Nf15ew8oqvl7y9+Cdj7wW8/3AnDlzYj+Xx/Bx9uY+5/N5917pRH8JIiIiIiIiIiKJoIcgIiIiIiIiIpIIeggiIiIiIiIiIomQlpkgjPuWvJ5a7otr2LBhgdvg3igg7MfkMSeddFIwZtKkSab+85//bOoJEyYEY5LK63v76quvTM39r16GA2cc8HxYvnx5MIb7tLt3717gfnj9m5wFwfvm9R5zXsmSJUtMzXkfQJhN06VLF1O/9957wZh0k5mZaWqv75D78bk/lvueAf/ft4/DPbRxeQfef+fe1lTWG+7Z557bevXqBWO4x5xrL3uEsyO87JrS7kj1lvN3sXLlytgxcT3lqWQmxL3HWzvOOOMMU/P6OXLkyGDMX/7yF1Pfcccdpu7Xr18wJi7bwMvB8NbykuStBfwa7zNnKwDhGsNZY172Ep/7fGzjciAAoHbt2qbesGGDqb35z9efrVu3mtrLGIrLRfHWYT7+vG/piK+7PXv2NDVnGQDAs88+a2r+7rwMlrh8O56jXu4G5xjxve369euDMZ988ompeU56uWm83qSSAcLXm+LOAGHedZb3kb/P9u3bB2PatWtnar5f8fDPznlT1atXD8bw/nKWGGcFedd3Xjs4A4RzIbx95Xtqb33nucxrIWdJAOEaxTkXXt6Ut5anOz7OXHvrLn9XPGbhwoWxn8v3qjx/vAwonhs8F1LJ0ClNStedioiIiIiIiIhIEdFDEBERERERERFJBD0EEREREREREZFE0EMQEREREREREUmEtAxG5ZAeDivj8CAgDHPjcCAOhvLCd7Zv317gfqQSjHrzzTcH75FD4++Yg8NWrFgRjOGgJw668gLhOATvnXfeMXWDBg1i95VDnHg/vMA4Dhni/fBCq/g74RA3Dnn1tlva1apVy9ReEBd/361btzb16tWrgzEzZswwdf369U3N4V5AGGjH84nnJK81QLj//DleaB5/Dv88M2fODMZwoK8XrMuaNWtm6nQMRvVwmBifayeccEIwhsPb1q1bZ+pUQtn4WhIXnJrKe7yAOA6eGzJkiKm9aw2HNzJvbXz33XdNPWfOHFOfeeaZwRhvPpckL1iO95GPrRfkx6GTHLrthUVyYF2LFi1MzWvB/Pnzg23wceEx3vzgdYgDtb3vJG5+e/+dA8HTLRjVu7ZwuB/fY3JQOhCGF/N12PscPkYcQszhmRyECYThhTxvU7l28j2F9/OdeOKJBX4O7zsQvw5410rvGnyk8D0CAFxxxRWmbtKkiam9IH3+vrj2wiH5++Kf3btv4/ONgyv5v3vHgO9xOIDVC1NlHKDpBdx+//33puZzxgvZjPu9y/tO+H6lLODvM64GwsB+Pqe9c5jxesLXKu+6kkpoazrRX4KIiIiIiIiISCLoIYiIiIiIiIiIJIIegoiIiIiIiIhIIqRlJgj3DHJPPveiebiHj/uavF4o7p3j/ruNGzfGfm5cTx/g938lAfcueq9x3yFnuwBhzxr31Xr9mp07dzb1K6+8YuoOHToUuB8AsH79elNzz57Xr8nvqVGjhqkXLVoUjOH+cO5T93ov0w33mXvnBJ/3nKHx5ptvxn5OtWrVYj+He6q5t3fbtm2m5uMBALt27TI1z0Fej7z38Lr3xhtvBGMGDRpkal5vvPMlJyfH1HG5EemCj2XLli1N7fWeT5kypcBtppLvEZerkMo2+Hrk9UFfeumlpr744otNPXny5NjP4fWHc0aAMJ+Hv0ePd/0sSV62Es8P7tH3MkH4POYx9erVC8bwus/ZYnxOejkJqaxTjK97tWvXNrXXb8/zgffVm7u8VqebVM5H/u74ewIO/zgDYeYQ31fwNcBbW3i+7Nixw9ReLgfvG2c+cG4EEP58fJ/l3d/wtTPumnak8Xnj5e1cddVVBW6D78mA8L6U1+ff/OY3sduJmy9AuN5wNl0q2WlLliwxNe+7t+bzceH7CG9O8fxu1KiRqb08Gz73eF3zzplvvvkmeC3d8XnBNZ9rQPhd8Xt4bfGsWbOmwG1484nPqVQyZUoz/SWIiIiIiIiIiCSCHoKIiIiIiIiISCLoIYiIiIiIiIiIJEJaNvNwvyL3zXGPvieuz9/rb+SeNu6lnzt3buznck9nUvM/PF7fNvf/pZL/kpGRYWrur9u6dWswhucM50u8/vrrpj722GODbdSpU8fUnPPg9clv2bLF1Nxv5+0rz/9Vq1aZ2uv1TjeZmZmm9s5H7kXkrJS33norGNOnTx9Tx/X4A2GfNveDp5KrwPsat/54n8s/H/d+A+Fc52wJr+84Ozs7eK0s6N69u6n79etn6pUrVwZjzjzzTFPz+fnII48EY/g4pJIxwLhPm/vHr7jiimDMiBEjTP3++++b2ssP4H3j3m+v35p7yrkvvW3btsGYou71P1xe3zLvI5+T3nHk75TXD6+HndcUnlP8ud5x4/Oa7xu8TCH+XD73+Xp1qM+Ok+494amcJ5zz5F2PeL3hc9qbTwMHDjQ1X+/5ntO7t23durWpOd/MyxPgaxbPW77PAsJrVG5urqmnTZsWjClpfC/k5XXxdZZ/du96vnDhwgJrL6+L13Q+H717D34tLqvDm2Nx2/DWDn4Pf0fFtb57+XZeTkW64++Tf+5Urgn8vXDOjyfuPXz/6L3G+TDemNKs7M0mERERERERERGHHoKIiIiIiIiISCLoIYiIiIiIiIiIJIIegoiIiIiIiIhIIqRlohWHVrZs2dLUCxYsCMZwOBQHwmzevNnUXmgYB05xQOWKFSsOsceSCj4GQBg6yKGnfAyAMCyVw8batGkTjOHQLw4i6tWrl6k///zzYBvr16839SmnnGLqHj16BGNmz55t6tNOO83UEydODMZw+BXP7cKE25U2jRo1MrUX1MahZ2vWrDE1f7cAMGjQIFNz4Jf33XGgLYfKMS9sitcsDtbzgvbiAlm9kDwOTOSQKi/krVmzZsFrxYm/81SCRTmYmEP6gPCc5QDh008/PRjDQXPDhw839Z/+9KdgzCeffGJqDlflsFFvjvHPc88995j61VdfDcZwECpL5XvkUDMv8JmDT3m+LFq0KBjjrcslyQvT42A5Pi783QBhECpvw/vO+ZzkoEK+1/C2sXr16gL37bjjjgvG8HpRmHBD/vk8XvBmOknlepmVlWXqpk2bBu/h84LXEv7+gfDehI9JvXr1TN28efNgG/wenuteIDsfV74+cYgnEAa98nfiBaOW9L0IhxB7oY1878ffBf/cQHhsN27caOqePXsGY/h6tGnTJlOnEvjJ7+Hj6IWr8phU/jEGnoe83aeeeioYw0Gv/F3Xrl07GMPzmeedF/I/c+bMcIfTHK/V/N151wRed+PmRmF41wj+HH5Pul0P9JcgIiIiIiIiIpIIeggiIiIiIiIiIomghyAiIiIiIiIikghpmQnCPffcG+/1XnJfJPcBcg8WZ1F4261SpYqpvd5oSV3r1q2D17jX/5133jH1woULgzF8LDlvgXv0gfgshcWLF5va653nTJjly5fHjuF5OW/evAI/FwizaZYsWRK8J91w7zD3uC9btiwYwz2ma9euNXW1atWCMdzLy72WqfTl8nnPx5B7tIFwjeL55uWMcB8u76vX2+vl6hzM+/m4L537dHkeH2mpZFewkSNHmtpbB3gN5ywgL1Plyy+/NDX3bfOxB8J59vzzz5v6j3/8o6m97KiLL77Y1JmZmaa+//77gzFxvP5w7t/ljCXubQfC/IkZM2aY2suj8DJuSpKXTcDnQtz5BoTnOs8h7zzm1zhXhHuovXmZk5Njav55vGNdsWLFArfL5z0QzmX+HO97TLce8MJo0KCBqb28GM6f4LXCO658jPjeltd477uO68nne11vu5UqVTK1Nzd4jnn5JKUNf5+pXMv4HsE7t3ht4ByR5557Lhjz7LPPxn52uvC+E87z4N+XODMPAObOnWvqwtwLlAUNGzY0Nd/HeVk2fM562T+Ha+/evab2jjOvL7xvqdxDlybptbciIiIiIiIiIoWkhyAiIiIiIiIikgh6CCIiIiIiIiIiiZCWmSDc+8Q9Sl7fNvc2cR8l9zVzbxQQ9sNyj6f3byqzVN6TVF4Oy86dO03N/dWfffZZMKZp06am5p5Yr5+Xcc8szwev5z07O9vUqfToca83/3xe7gP3lnIvppeFUdo1btzY1HxucZ8uAFSvXt3Un3zyiam9DBZeK7we9zg8f7g30+vf5M/hOemtN7yv3D/OfaRAeA7xeuP9vLwv7du3N3VRZ4IUBmcF8TkPADNnzjR1q1atTH3gwIFgDH/n69atM7W3dkybNs3U3Ct/wgknmNpbF3r16mXqTp06Be9hfCy5n9o7Z3g+8PpSv379YMxrr71map4vPXv2DMZwPkJJ83qb+fvjY+/lL/B3zOekd+/B6wOv+Zw7w/MWCM9JPm68FgLhOsTHzbu28M/D7/F69r3MiXSSSg7Bhg0bTP35558H73nvvfdM7X2/jI8br9+8jenTpwfb4Awqzsby1vw33njD1HHZJEB4jV61alXwnrKA1/hU7hdL23pX1LzfY5L2HRxJnKPBv1/wNcR7jbOmCoN/3/I+l/c17nel0k5/CSIiIiIiIiIiiaCHICIiIiIiIiKSCHoIIiIiIiIiIiKJoIcgIiIiIiIiIpIIaRmMyqExHDTH4S5AGBLHoWEc7uKFIVWtWtXUW7dujd9ZkkoIV1LVrFkzeI0DMjnMkEPAAKBHjx6m5uBQLxCR8Xt4fuzevTsYw/OSQ069ADkew6F5b775ZjCmX79+pp43b56pOeQsHQwcONDUfMy8cLdatWqZmoNRvYBYPu855MkLUOTXeG7wNrxgqLjz3gsF5u+At+GFMPKaxIGhGzduDMZwAGS3bt1M/eqrr4Y7fASdf/75pva+vxUrVpj673//u6m9c4vPv48//tjUbdq0CcasXLnS1Bz2tmTJkmAMr1F16tQxNV+ffv/73wfbmDJliqk5eM6bl3HrmBfezHjN5X0HwuORkZFhai8Akt9T0ryAN15TeL3wzsnVq1ebulmzZqb2wgH5O+b5wqGbvE0A+PbbbwvcV2/Njwvm9OYUf0+pzCEvQLY04+Oeyv0AryVeEC0fE/7+vTBJvqfkfeGwQ29t5PWFj6G3NvJ5Hhd2CAANGjQwdYsWLYL3MP1DACLx+HfWrKwsU3tBxbzu8j/uURh8LfLWdr6H5jqV9bQ00V+CiIiIiIiIiEgi6CGIiIiIiIiIiCSCHoKIiIiIiIiISCKkZSYI9yt+//33pvayAPg93F+fSl8T95Jyn66H+27VI3loXibI+vXrTc29udxfDQATJkww9apVq0zt9cgy7pHlevv27cEYnjO8rzzHgDCjgefpokWLgjFffPFFgdtt2bJlMKa0e/HFF009Y8YMU3t98v379zf1N998Y2rOwwDC857PTz7OQHjec14Q/3evP5/zBXiueOsN7xvnVeTm5gZjOL+DM2b4ewbCObZp06bgPUWJ8yOys7OD9yxdutTUfG41adIkGMM/B8+P5cuXB2P4/MvLyzP1ggULgjH8Hl4bjj32WFM3bNgw2MawYcOC1w5WVNeN5s2bm5rXTgCoV6+eqfm79rKx+PiUNG/t5X5nPq+9/AWeq5xL5N178PWG86QaNWpk6mXLlgXb4LWMr3tePhCvS5yB8/777wdj+Nh612RW2vJfigJff7wefZ4LnOnkjeG1IC7Pg89FIFxfOK/Em/s8T/m6591Xcf4a/7wiUjic/cPrrnct4rXCyys7XHze169fP3YMrx3p9juu/hJERERERERERBJBD0FEREREREREJBH0EEREREREREREEiEtM0HYunXrTO315XK/b7ly5Qr8796/k85juEffk+79UsWJ8zEA4LzzzjM19+zfcsstwRiv178sqVu3rqk586S4Mx2OBM7X4XrmzJnBmPHjxxe4zSFDhgSveb2VB+McDiBcG7iuU6eOqbknGwjzKHgd8LJIOL+E16TevXsHY2699dbgtdKOj+OgQYOC93A/PX+fX331VTBmz549pubzxMse4eP/6aefmpozNIBwTRo1apSpp06dauq77ror2AbjPAeeC0cKrxVe5gD3/nMuStWqVYMxqVwbi5OXmbFz505Tc0aIN6ZBgwam5mPPxw0I1wv+nH379pnaW6P4+PMa460fnDPE6wffN3mfw8feu7Zw9kVpV5hzKzMz09RZWVnBe+LuQ/k4e9vZsGGDqfmek3MAgHAu9OjRw9Redl2fPn1MnUomCGdQzZ8/P3hPnOJa10TSSWHuS/mc9e6BCno/4GfRHcy7l+V95euZt6+lmf4SREREREREREQSQQ9BRERERERERCQR9BBERERERERERBJBD0FEREREREREJBHKRDDq9u3bTV2zZs3YMRzIVLFiRVN7AWf8Gn+ux9uO+DIyMoLXpk+fbmoOJqxfv34wpqwHo3JIJAcT8n9PBxzaxLUX4BQX6sSBTUAYZMi8YEj+Pvmc5vC6VMJVeb0pX758MIaD9CpXrmzq9evXB2Pi8L4C8d9j3H//pThQ8LTTTgve069fvwLrhg0bBmPmzJlj6iVLlpj6mGOOCcZ44cwHe+utt4LX+NhxOC2H+r7yyisFfgZQfIGB/L0df/zxwXvmzZtnar6+cmAiAHTs2PGX79wR1KVLl+A1Dvnk0FNvHW3UqJGpeS3w5g/PM54vvCbxfgDhusShv9584SBUDrnbvHlzMKZFixambtKkial5HQOAp59+OnitNCvMPVnt2rVNvWvXruA9vE7yNcz7XH4Phw7yNvm4A+GazvPJC2TlaxS/x7tOcJi39w8QxEnlu1dYqiQNn+d87+oFIvN7li5dWuBnpBKMyuuLd1/K1xUOqfc+pzRLr70VERERERERESkkPQQRERERERERkUTQQxARERERERERSYQykQmyZs0aU3NuhIf7mry+SZZKf6YUnpfvwT1pa9euNfXnn38eu13uQy2pnlOvH7Yw+/L111+bmr+jVLJqShs+t1LJoeBcDe6F9vrzs7KyChzj5Wzs3LnT1Mcdd5ypudfe66fetm2bqTlvgNcwIOzb5kwQ73PieD39pQ2vzQDw9ttvF1hzLg4AnHrqqQXWjRs3DsZwnzv32Xbt2jUYwzkRX3zxhamfeOKJYExpMXnyZFNfffXVwXv4e2rWrJmp69SpE4zx5nNJWrZsWfAa519w7gYfVyDMr+nTp4+pvZwEzpNo2bKlqfn789Yg3n/OvZo9e3YwhjNx4nrGAeBvf/ubqfnn8663CxcujN1uaXIkrrl87QHCc4mzYLyMGb4u8DWLrwHeceb7ilSyojjnh/NxvPthvs84EvdRyv8QCXO2+P7XywTh+78dO3b84v3g9SUvLy94T1xeX1FnyB1p+ksQEREREREREUkEPQQRERERERERkUTQQxARERERERERSYQykQmye/duU3fu3Dl4D/fUcr8m/xvoGzZsCLZRpUoVU6fSe6mex9RxXxwA9OjRw9SbN282tffvWHPOQ2nhzYXC5JVwLkGNGjVM3aBBg2DMW2+9lcouppW4HJ/XXnsteG3w4MGmbtKkiam9fkb+fjm7htcFr/e7VatWpua+Su4NB8LzgTNBJk6cGIxhcf2bZcWuXbuC11588cUS2JPSI5W1ZO/evaY+44wzimp3StTYsWOD1x5//HFT830E53943n///V+2Y6UM56DwfRFnVKSjwqyBfN3gGgCaNm1qav4uvRyR0047zdScecYZITxHve2ecMIJpvZ+3lq1apl6yZIlpvaurXy/+8YbbwTviVNWrz8iv0TDhg1NzdlSfB8HABUqVDB1XN6btw3G9wOcfweE2SO875wjBQAvvPBC7GeXFP0liIiIiIiIiIgkgh6CiIiIiIiIiEgi6CGIiIiIiIiIiCSCHoKIiIiIiIiISCKUiWDUf/zjH6b2AvHq1Kljag6PbN68eezncCjn7NmzY8fEhdXIv4wZMyZ4benSpabevn27qTmkJ90UJjj35ZdfNjWHF7355pu/aJ/SBYesccjst99+G4wZMmSIqU8//XRTe+sAh8hxWCCHLHvBqLz+cMipF8S8cOFCU3PQ6549e4IxTMHMIuF1AwjD0lO5nrNUwubiwq8LExbJAaWpnOepfM6gQYNMzdeWVMLgyyL+7vganAov+J0Dyxs3bmzq6tWrm5rnLBCGGS5evNjUHLYKhNcW7z1HQmGC30WShn9P5NBTL5Ca1yQvNLmg93u++uorU69evTp2zAcffGDqBx54IHZMaaK/BBERERERERGRRNBDEBERERERERFJBD0EEREREREREZFEOCpSk56IiIiIiIiIJID+EkREREREREREEkEPQUREREREREQkEfQQREREREREREQSQQ9BRERERERERCQR9BBERERERERERBJBD0FEREREREREJBH0EEREREREREREEkEPQUREREREREQkEfQQREREREREREQS4f8BuwmYZFZnNTgAAAAASUVORK5CYII=",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"label counts in subset: [1226 1199 1184 1208 1188 1183 1250 1133 1208 1221]\n"
]
}
],
"source": [
"# viz: always look at the data right before it enters the model (Karpathy's rule)\n",
"fig, ax = plt.subplots(1, 8, figsize=(11, 1.8))\n",
"for k in range(8):\n",
" ax[k].imshow(Xtr[k].reshape(28, 28), cmap=\"gray\"); ax[k].axis(\"off\")\n",
" ax[k].set_title(CLASSES[ytr[k]], fontsize=7)\n",
"plt.tight_layout(); plt.show()\n",
"print(\"label counts in subset:\", np.bincount(ytr, minlength=10))"
]
},
{
"cell_type": "markdown",
"id": "5cbe9c70",
"metadata": {},
"source": [
"> **Interpretation.** The images are recognizable garments and the labels match: the standardization did not destroy the signal. Looking at one batch before training is the cheapest bug-catcher in deep learning; a broken normalization or a label swap is obvious here and invisible in the loss curve.\n"
]
},
{
"cell_type": "markdown",
"id": "cc4ee2d9",
"metadata": {},
"source": [
"Before training, **verify the loss at init**. We want the first logits to be near zero so the softmax is near-uniform across 10 classes and the loss starts at $\\ln 10 \\approx 2.30$. The hidden layer uses Kaiming init ($\\sqrt{2/\\text{fan\\_in}}$) so ReLU activations keep unit-ish variance, but the *output* layer is initialized small (factor $0.01$) on purpose, so the logits start tiny and the init loss is correct. This is Karpathy's \"make the init loss what it should be\" recipe; without the small output init, the logits have std $\\approx 1$ and the init loss sits near $2.9$, not $2.3$.\n"
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "6a51998e",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:15:03.365428Z",
"iopub.status.busy": "2026-06-10T19:15:03.365327Z",
"iopub.status.idle": "2026-06-10T19:15:03.438468Z",
"shell.execute_reply": "2026-06-10T19:15:03.438177Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"loss at init 2.2947 (expected ~ ln 10 = 2.3026)\n",
"[ ok ] loss at init is ~ln(10), safe to train\n"
]
}
],
"source": [
"def init_params(d=784, h=128, q=10, seed=SEED):\n",
" g = np.random.default_rng(seed)\n",
" return {\"W1\": g.standard_normal((d, h)) * np.sqrt(2/d), \"b1\": np.zeros(h), # Kaiming for ReLU hidden\n",
" \"W2\": g.standard_normal((h, q)) * 0.01, \"b2\": np.zeros(q)} # small output -> logits ~0 -> loss ~ln(10)\n",
"\n",
"P0 = init_params()\n",
"logits0, _ = mlp_forward(Xtr[:256], P0[\"W1\"], P0[\"b1\"], P0[\"W2\"], P0[\"b2\"])\n",
"loss0, _ = softmax_xent(logits0, ytr[:256])\n",
"print(f\"loss at init {loss0:.4f} (expected ~ ln 10 = {np.log(10):.4f})\")\n",
"assert abs(loss0 - np.log(10)) < 0.25, \"init loss should be near ln(10); if not, the init or forward is off\"\n",
"print(\"[ ok ] loss at init is ~ln(10), safe to train\")"
]
},
{
"cell_type": "markdown",
"id": "d5d53340",
"metadata": {},
"source": [
"Now train. The loop is the four-comment skeleton repeated in every chapter: forward, backward, update, track stats. We use plain mini-batch SGD on the gradients you derived. Re-seed first so the run reproduces.\n",
"\n",
"> **Runtime:** this cell takes ~10-40 s on CPU depending on `FAST`.\n"
]
},
{
"cell_type": "code",
"execution_count": 28,
"id": "d1647985",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:15:03.445439Z",
"iopub.status.busy": "2026-06-10T19:15:03.445344Z",
"iopub.status.idle": "2026-06-10T19:17:32.102431Z",
"shell.execute_reply": "2026-06-10T19:17:32.102174Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 0: batch loss 0.6212 · test acc 0.7320\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 1: batch loss 0.5052 · test acc 0.8335\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 2: batch loss 0.4016 · test acc 0.8410\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 3: batch loss 0.3849 · test acc 0.8280\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 4: batch loss 0.2702 · test acc 0.8405\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 5: batch loss 0.4233 · test acc 0.8065\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 6: batch loss 0.2352 · test acc 0.8505\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 7: batch loss 0.2987 · test acc 0.8395\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 8: batch loss 0.4469 · test acc 0.8290\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 9: batch loss 0.2003 · test acc 0.8505\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 10: batch loss 0.2764 · test acc 0.8580\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 11: batch loss 0.3104 · test acc 0.8255\n",
"\n",
"final test accuracy: 0.8255\n",
"[ ok ] the pure-NumPy MLP trained on FashionMNIST\n"
]
}
],
"source": [
"def accuracy(P, X, y, bs=1000):\n",
" 'Test accuracy over X in batches; argmax of the logits vs the labels.'\n",
" correct = 0\n",
" for i in range(0, len(X), bs):\n",
" lg, _ = mlp_forward(X[i:i+bs], P[\"W1\"], P[\"b1\"], P[\"W2\"], P[\"b2\"])\n",
" correct += (lg.argmax(1) == y[i:i+bs]).sum()\n",
" return correct / len(X)\n",
"\n",
"def train_numpy_mlp(P, epochs, bs=128, lr=0.2, seed=SEED):\n",
" g = np.random.default_rng(seed)\n",
" n = len(Xtr); hist = []\n",
" for ep in range(epochs):\n",
" perm = g.permutation(n) # fresh shuffle each epoch\n",
" for i in range(0, n, bs):\n",
" idx = perm[i:i+bs]\n",
" logits, cache = mlp_forward(Xtr[idx], P[\"W1\"], P[\"b1\"], P[\"W2\"], P[\"b2\"]) # forward\n",
" loss, dlogits = softmax_xent(logits, ytr[idx])\n",
" grads = mlp_backward(dlogits, cache, P[\"W2\"]) # backward\n",
" for k in P:\n",
" P[k] -= lr * grads[k] # update\n",
" hist.append((ep, loss, accuracy(P, Xte, yte))) # track stats\n",
" print(f\"epoch {ep}: batch loss {loss:.4f} · test acc {hist[-1][2]:.4f}\")\n",
" return hist\n",
"\n",
"EPOCHS = 3 if FAST else 12\n",
"P = init_params()\n",
"hist = train_numpy_mlp(P, epochs=EPOCHS, lr=0.2)\n",
"final_acc = hist[-1][2]\n",
"print(f\"\\nfinal test accuracy: {final_acc:.4f}\")\n",
"# floor well below both modes (FAST 3-epoch ~0.75, full 12-epoch ~0.85): a 'did it learn at all' check,\n",
"# not a seed-fragile tight bound. The experiment-log table gives the per-mode expected values.\n",
"assert final_acc > 0.65, \"a working 2-layer MLP should clear ~0.65 even in the FAST smoke run\"\n",
"print(\"[ ok ] the pure-NumPy MLP trained on FashionMNIST\")"
]
},
{
"cell_type": "code",
"execution_count": 29,
"id": "7165d7c4",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:17:32.115631Z",
"iopub.status.busy": "2026-06-10T19:17:32.115531Z",
"iopub.status.idle": "2026-06-10T19:17:32.624851Z",
"shell.execute_reply": "2026-06-10T19:17:32.624180Z"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA3kAAAE1CAYAAABa9SnrAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAfLxJREFUeJzt3Xl8TNf/x/HXJLJZElskQkgsRYhd1FK0pbFUiy62Wkt/lNby7YaitJW2WtWi0oXqQmm1Squ1lKJFxVKtfY2lJGJNCBIy9/fHbYY0QcIkM0nez8djHpncOffO546YO58553yOxTAMAxEREREREckTXBwdgIiIiIiIiNiPkjwREREREZE8REmeiIiIiIhIHqIkT0REREREJA9RkiciIiIiIpKHKMkTERERERHJQ5TkiYiIiIiI5CFK8kRERERERPIQJXkiIiIiIiJ5iJI8ESc3a9YsLBYLhw4dcnQoIiIiIpILKMkTERERyUfWrVvHK6+8wrlz57L1eSZMmMD333+frc8hIhlTkiciIiKSj6xbt45x48YpyRPJw5TkiYiIiIhkQWJioqNDELkpJXkiudAHH3xA9erV8fDwICAggEGDBqX7Rnbfvn088sgj+Pv74+npSdmyZenSpQvx8fG2NsuXL6dp06YULVqUwoULU6VKFUaOHJnDZyMiIjnllVde4fnnnwcgODgYi8WSbt73l19+Sb169fDy8qJ48eJ06dKFo0ePpjnOra4xFouFxMREPvvsM9tz9O7d+4ZxJScnM2bMGOrVq4ePjw+FChXinnvu4ddff03X1mq18t577xEaGoqnpye+vr60bt2aTZs2pWn35ZdfEhYWRsGCBSlWrBjNmjVj2bJltsctFguvvPJKuuMHBQWliTV1bvzq1at5+umnKVWqFGXLlgXg8OHDPP3001SpUgUvLy9KlCjBY489luE8+nPnzjFs2DCCgoLw8PCgbNmy9OzZk1OnTnHhwgUKFSrEkCFD0u33zz//4OrqSkRExA1fP5H/KuDoAEQka1555RXGjRtHy5YtGThwIHv27GH69Ols3LiRtWvX4ubmRnJyMuHh4SQlJfHMM8/g7+/PsWPH+PHHHzl37hw+Pj7s2LGDBx98kJo1azJ+/Hg8PDzYv38/a9eudfQpiohINunUqRN79+7lq6++4t1336VkyZIA+Pr6AvD6668zevRoHn/8cfr168fJkyeZMmUKzZo1488//6Ro0aKZusZ88cUX9OvXj7CwMJ566ikAKlaseMO4EhIS+OSTT+jatSv9+/fn/PnzzJgxg/DwcKKioqhdu7at7ZNPPsmsWbNo06YN/fr14+rVq/z222/88ccf1K9fH4Bx48bxyiuv0LhxY8aPH4+7uzsbNmxg5cqVPPDAA7f12j399NP4+voyZswYW0/exo0bWbduHV26dKFs2bIcOnSI6dOn06JFC3bu3EnBggUBuHDhAvfccw+7du2ib9++1K1bl1OnTrFo0SL++ecfateuTceOHZk3bx6TJk3C1dXV9rxfffUVhmHQvXv324pb8ilDRJzap59+agBGdHS0ERcXZ7i7uxsPPPCAkZKSYmszdepUAzBmzpxpGIZh/PnnnwZgfPPNNzc87rvvvmsAxsmTJ7P9HERExHlMnDjRdl253qFDhwxXV1fj9ddfT7N927ZtRoECBWzbM3ONMQzDKFSokNGrV69MxXT16lUjKSkpzbazZ88afn5+Rt++fW3bVq5caQDGs88+m+4YVqvVMAzD2Ldvn+Hi4mJ07NgxzbXy+jaGYRiAMXbs2HTHKV++fJq4U6/DTZs2Na5evZqm7cWLF9Ptv379egMwPv/8c9u2MWPGGIDx3Xff3TDupUuXGoDx888/p3m8Zs2aRvPmzdPtJ3IzGq4pkov88ssvJCcnM3ToUFxcrv337d+/P97e3ixevBgAHx8fAJYuXcrFixczPFbRokUBWLhwIVarNXsDFxERp/fdd99htVp5/PHHOXXqlO3m7+9P5cqVbUMnM3ONySpXV1fc3d0BczjmmTNnuHr1KvXr12fLli22dt9++y0Wi4WxY8emO4bFYgHg+++/x2q1MmbMmDTXyuvb3I7+/fun6WED8PLyst2/cuUKp0+fplKlShQtWjRd3LVq1aJjx443jLtly5YEBAQwe/Zs22Pbt2/n77//5oknnrjtuCV/UpInkoscPnwYgCpVqqTZ7u7uToUKFWyPBwcHM3z4cD755BNKlixJeHg406ZNSzMfr3PnzjRp0oR+/frh5+dHly5d+Prrr5XwiYjkU/v27cMwDCpXroyvr2+a265du4iLiwMyd425HZ999hk1a9bE09OTEiVK4Ovry+LFi9Mc98CBAwQEBFC8ePEbHufAgQO4uLgQEhJyR/H8V3BwcLptly5dYsyYMQQGBuLh4UHJkiXx9fXl3Llz6eKuUaPGTY/v4uJC9+7d+f77723J8+zZs/H09OSxxx6z67lI3qckTySPeuedd/j7778ZOXIkly5d4tlnn6V69er8888/gPnt45o1a/jll1/o0aMHf//9N507d6ZVq1akpKQ4OHoREclpVqsVi8XCkiVLWL58ebrbhx9+aGt7q2tMVn355Zf07t2bihUrMmPGDFsM9913X45/+Xija+D1vXapnnnmGV5//XUef/xxvv76a5YtW8by5cspUaLEbcXds2dPLly4wPfff49hGMyZM4cHH3zQ1nsqklkqvCKSi5QvXx6APXv2UKFCBdv25ORkoqOjadmyZZr2oaGhhIaG8vLLL7Nu3TqaNGlCZGQkr732GmB+a3j//fdz//33M2nSJCZMmMCoUaP49ddf0x1LRETyhhsNWaxYsSKGYRAcHMxdd911y+Pc6hqTlaGR8+fPp0KFCnz33Xdp9vvvsMyKFSuydOlSzpw5c8PevIoVK2K1Wtm5c2eagi3/VaxYsXSVqZOTk4mJiclS3L169eKdd96xbbt8+XK641asWJHt27ff8ng1atSgTp06zJ49m7Jly3LkyBGmTJmS6XhEUqknTyQXadmyJe7u7rz//vsYhmHbPmPGDOLj42nXrh1gVim7evVqmn1DQ0NxcXEhKSkJgDNnzqQ7furFMLWNiIjkPYUKFQJIl4h06tQJV1dXxo0bl+YaA2AYBqdPnwYyd41JfZ7MLrieOtft+ufdsGED69evT9PukUcewTAMxo0bl+4Yqft26NABFxcXxo8fn6437frjV6xYkTVr1qR5/KOPPsrSaBZXV9d0r9WUKVPSHeORRx7hr7/+YsGCBTeMO1WPHj1YtmwZkydPpkSJErRp0ybT8YikUk+eSC7i6+vLiBEjGDduHK1bt+ahhx5iz549fPDBBzRo0MA2MXvlypUMHjyYxx57jLvuuourV6/yxRdf4OrqyiOPPALA+PHjWbNmDe3ataN8+fLExcXxwQcfULZsWZo2berI0xQRkWxUr149AEaNGkWXLl1wc3Ojffv2VKxYkddee40RI0Zw6NAhOnToQJEiRYiOjmbBggU89dRTPPfcc5m6xqQ+zy+//MKkSZMICAggODiYhg0bZhjTgw8+yHfffUfHjh1p164d0dHRREZGEhISwoULF2zt7r33Xnr06MH777/Pvn37aN26NVarld9++417772XwYMHU6lSJUaNGsWrr77KPffcQ6dOnfDw8GDjxo0EBATY1pvr168fAwYM4JFHHqFVq1b89ddfLF261LasRGY8+OCDfPHFF/j4+BASEsL69ev55ZdfKFGiRJp2zz//PPPnz+exxx6jb9++1KtXjzNnzrBo0SIiIyOpVauWrW23bt144YUXWLBgAQMHDsTNzS3T8YjYOKaop4hk1vVLKKSaOnWqUbVqVcPNzc3w8/MzBg4caJw9e9b2+MGDB42+ffsaFStWNDw9PY3ixYsb9957r/HLL7/Y2qxYscJ4+OGHjYCAAMPd3d0ICAgwunbtauzduzcHz05ERBzh1VdfNcqUKWO4uLiku8Z8++23RtOmTY1ChQoZhQoVMqpWrWoMGjTI2LNnj2EYmbvGGIZh7N6922jWrJnh5eVlADddTsFqtRoTJkwwypcvb3h4eBh16tQxfvzxR6NXr15G+fLl07S9evWqMXHiRKNq1aqGu7u74evra7Rp08bYvHlzmnYzZ8406tSpY3h4eBjFihUzmjdvbixfvtz2eEpKivHiiy8aJUuWNAoWLGiEh4cb+/fvv+ESChs3bkwX99mzZ40+ffoYJUuWNAoXLmyEh4cbu3fvTncMwzCM06dPG4MHDzbKlCljuLu7G2XLljV69eplnDp1Kt1x27ZtawDGunXrbviaidyMxTD+00csIiIiIiIO07FjR7Zt28b+/fsdHYrkUpqTJyIiIiLiJGJiYli8eDE9evRwdCiSi2lOnoiIiIiIg0VHR7N27Vo++eQT3Nzc+L//+z9HhyS5mHryREREREQcbPXq1fTo0YPo6Gg+++wz/P39HR2S5GKakyciIiIiIpKHqCdPREREREQkD8l3c/KsVivHjx+nSJEiWCwWR4cjIiK3yTAMzp8/T0BAAC4u+s7yZnTtExHJGzJ77ct3Sd7x48cJDAx0dBgiImInR48epWzZso4Ow6np2icikrfc6tqX75K8IkWKAOYL4+3t7eBoRETkdiUkJBAYGGh7X5cb07VPRCRvyOy1L98leanDVLy9vXWhExHJAzT88NZ07RMRyVtude3TJAYREREREZE8REmeiIiIHU2bNo2goCA8PT1p2LAhUVFRN20/efJkqlSpgpeXF4GBgQwbNozLly+naXPs2DGeeOIJSpQogZeXF6GhoWzatCk7T0NERHKxfDdcU0REJLvMmzeP4cOHExkZScOGDZk8eTLh4eHs2bOHUqVKpWs/Z84cXnrpJWbOnEnjxo3Zu3cvvXv3xmKxMGnSJADOnj1LkyZNuPfee/n555/x9fVl3759FCtWLKdPT0REcgkleSIiInYyadIk+vfvT58+fQCIjIxk8eLFzJw5k5deeild+3Xr1tGkSRO6desGQFBQEF27dmXDhg22Nm+++SaBgYF8+umntm3BwcE3jSMpKYmkpCTb7wkJCXd0XiIieUFKCkRtg7jTUKoEhIWCq6ujo8oeGq4pIiJiB8nJyWzevJmWLVvatrm4uNCyZUvWr1+f4T6NGzdm8+bNtiGdBw8e5KeffqJt27a2NosWLaJ+/fo89thjlCpVijp16vDxxx/fNJaIiAh8fHxsNy2fICL53c9roElX6DIMnn3N/Nmkq7k9L1KSJyIiYgenTp0iJSUFPz+/NNv9/PyIjY3NcJ9u3boxfvx4mjZtipubGxUrVqRFixaMHDnS1ubgwYNMnz6dypUrs3TpUgYOHMizzz7LZ599dsNYRowYQXx8vO129OhR+5ykiEgu9PMaGDgWYk6m3R570tyeFxO9fDtcMzk5GcMwbOVHU1JSSElJwcXFhQIFCqRpB+Dm5mbXtleuXMEwDAoUKGBbrd5qtXL16lUsFgtubm5O1fbq1atYrVZcXV1x/bdfOyttDcPgypUrALi7u2dL24xe96y0deS/fV7+O8mJf/s7/TvJqX97vUfY998z9fXMzVatWsWECRP44IMPaNiwIfv372fIkCG8+uqrjB49GjBfx/r16zNhwgQA6tSpw/bt24mMjKRXr14ZHtfDwwMPD48cOw8REWeVkgLjpoKRwWMGYMF8/IEmeWvoZr5N8qZNm8bzzz9PwYIFAdiwYQO//fYbNWvWpE2bNrZ2U6dO5cqVKwwYMAAfHx8AtmzZwsqVKwkJCaF9+/a2tpGRkVy6dIm+ffvi6+sLwLZt21i6dCmVK1emU6dOtraffPIJCQkJ9OzZk9KlSwOwa9cufvzxR8qXL0+XLl1sbT/77DNOnz5N165dKVeuHAD79+9nwYIFlClThieeeMLWds6cOcTGxvLoo49SsWJFAA4fPszXX39NqVKlbPNEAL755huOHj3Kww8/TNWqVQE4fvw4s2fPplixYjz11FO2tgsWLODgwYO0bduW0NBQAE6ePMmsWbMoXLgwgwYNsrX98ccf2bNnD61ataJu3bqAWTjg448/xsPDg6FDh9raLl26lO3bt9OiRQsaNmwIwIULF/jggw9wcXHh+eeft7VduXIlf/75J02aNKFp06aAOe/kvffeA+C5556zfQhcs2YNUVFRhIWFce+99wLmB6V3330XgCFDhuDp6QnA+vXrWbt2LXXq1OGBBx6wPd97772H1Wrl6aefti04uWnTJlatWkWNGjVo166dre0HH3xAUlIS/fv3p3jx4gD89ddfLF++nCpVqtChQwdb248++ogLFy7Qu3dv2zf+O3fu5KeffqJChQo89thjtraffvopZ8+epXv37pQtWxaAvXv3snDhQgIDA23zeAC+/PJL4uLiePzxx23zdaKjo5k/fz7+/v5pPgzOmzePY8eO0bFjR+666y4A/vnnH7766itKlChBv379bG2//fZbDh8+zIMPPkj16tUBOHHiBJ9//jne3t4MHDjQ1nbRokXs27eP8PBwateuDZi9GzNnzsTLy4tnn33W1vbnn39m586d3HfffTRo0AAw5w1FRkbi5ubG8OHDbW2XL1/O33//zT333EPjxo0BuHTpElOmTAHgxRdftLVdtWoVmzdvplGjRjRr1gwwk5XUf/thw4bZkoi1a9eyfv166tWrl2aIXWrbZ555Ru8RTvwe4WxJXsmSJXF1deXEiRNptp84cQJ/f/8M9xk9ejQ9evSw/Z8LDQ0lMTGRp556ilGjRuHi4kLp0qUJCQlJs1+1atX49ttvs+dERETykKht6XvwrmdgPh61DRrVzqmosp+Ga4qIiNiBu7s79erVY8WKFbZtVquVFStW0KhRowz3uXjxoq2XNNX1PZYATZo0Yc+ePWna7N27l/Lly9szfBGRPMVqhd82wRsfZa593OnsjSenWYzUq0g+kZCQgI+PDydPnqREiRIaiuXEQ7Gy2lbDNR3fVsM19R6Rk+8RCQkJ+Pr6Eh8fj7e3N85g3rx59OrViw8//JCwsDAmT57M119/ze7du/Hz86Nnz56UKVOGiIgIAF555RUmTZrERx99ZBuuOXDgQOrVq8e8efMA2LhxI40bN2bcuHE8/vjjREVF0b9/fz766CO6d++eqbhSr33O9FqJiGSHE6fg6yUw7yc4GpP5/ea+mzt68jL7fp5vkzxd6EREcjdnfT+fOnUqEydOJDY2ltq1a/P+++/bhpq2aNGCoKAgZs2aBZhJ7Ouvv84XX3zBsWPH8PX1pX379rz++usULVrUdswff/yRESNGsG/fPoKDgxk+fDj9+/fPdEzO+lqJiNhDSgqsioKvfoSVf0CK1dzuXQgeaglL1sDpsxnPywMo7Qtrv8odc/KU5N2ALnQiInmD3s8zT6+ViORFx06YPXZf/5x23l39GtC1HbRrAV6e16prQsaJ3pTR8NB9ORHxncvs+3m+LbwiIiIiIiK5y5Wr8Ms6+GoxrNkIqd1Vxbyh0wPQpR3cFZR2nzbNYPo4s4rm9cmgi4s5d+/w8RwLP8coyRMREREREad2+BjM/Qm++RlOnr22vVEd6PYghDcFD/cb79+mmblMQtQ2s8hKqRIQEwfDImDabHg03By2mVcoyRMREREREaeTlAxLfzfn2q3789p232LwaGuz1y6oTOaP5+qatriKYcDsH2DTdpgQaQ7bzCuU5ImIiIhIvpWSkrZ3Jyw0dxTgyM1u9ZrvP2Imdt8uhbMJ5jaLBZo1MOfatWwMbnbIYiwWGPcsPPh/sGgl9OwADULv/LjOQEmeiIiIiORLP69JP0+rtC+MHWwO7xP7u9FrPvL/zPl2Xy2GjduuPeZfEh5vA53bQll/+8dTo7KZOM75Eca8Dz9G5o0kX0meiIiIiOQ7qRUX/1ttMfakuX36OCV69naj1zzmJDzz2rXfXV3gvruh64PQPAwKZHPS9dyT8OOvsHO/Wa2zW/vsfb6c4OLoAEREREREclJKitmblFE5/dRt46aa7cQ+bvaap3J1geF9YP08+OR1uL9R9id4ACWKms8L8NYnEH8++58zuynJExEREZF8JWpb2uGC/2VgPh617cZtJGtu9ZqDuYh5WE3wK5kzMV3viYehcnlzDuC7s3L++e1NSZ6IiIiI5Ctxp+3bTm7N2V9ztwLwyjPm/c+/hz3RjonDXpTkiYiIiEi+UqpE5tp9uxTizmRvLPlFZl/zzLbLDk3rQfg9Zo/iuKnXFlrPjZTkiYiIiEi+EhZqVnS03KLd6o1wbw+InAvJV3IktDzryi1ePwvmv0mYg5cweHkgeLjB2i2w5DfHxnInlOSJiIiISL7i6mouk5BRR43l39sL/aBWVbhwESI+hAf6wor1ubt3x1F27oeBr1z7/b/JdervYwc7fvmCcqXhqS7m/demw+Ukx8Zzu5TkiYiIiEi+06YZ3BWUfru/r7l8wqDu8P00ePtF8C0G0f9A35HQ60XYdzjHw821jsdB7xFmsnx3LZgy2nyNr5f6mjvLkhVPdzV7Ff+JhQ/nOTqa22MxjPz1fURCQgI+Pj7Ex8fj7e3t6HBEROQ26f088/RaiaS36wC07gcuFpgyxizxX6qEOVzwv71J5xNh2myYMd8ctlnAFXp1hCG9wKewY+LPDeIvwKPPwN5DZuXKb6eATxHztY7aZhZZudFr7miLVsIzr4KnB/z6OQSUcnREpsy+n6snT0RERETynU+/M3+2awEPtoCH74dGtTNONooUgpeeguWfQqvGcDXFTPhaPAFzftB6ehlJSoanRpsJXqkS8NmbZoIH5mvcqPbNX3NHa3+vuZzD5SSYEOnoaLJOSZ6IiIiI5Ctn4uH7X8z7vTtlfr+gMuYi3Z+/CZXKm8cZMQkeHAAb/sqeWHMjqxWeexP+2AqFC8KsN6CMn6OjyhqLxVxSwcUFfvg19/37KskTERERkXzlqx/NnqbQu6Be9azv3zwMlnwCrwwG78JmYZHHh8KgceY8rvzuzY/N4Y4FXCFyHFSv5OiIbk/1StDtQfP+2PfNHtzcQkmeiIiIiOQbV67CFwvN+30fMXtsbodbAejzCKz6Ap54yOzx+XEV3NcL3p0Fly7bK+LcZdYCc8kJgLeeh3vqOzaeO/W/vmYiv+ug+eVAbqEkT0RERCQDKSmwfissXGH+1LyrvGHpbxBz0qyY2a7FnR+vRFF4fRj8+KFZPTIpGSZ/ZiZ7i1bmryUXlvwGr0wx7z/3JDwS7th47KG4j5noAbw9E84lODaezFKSJyIiIvIfP6+BJl2hyzB49jXzZ5Ou5va8LD8ktqkFV7o/BB7u9jtu9Uow91344BUo62cuHfDMq/DYENi+z37P46w27zD/rxiGOcRxcHdHR2Q/TzwEVYLNBO+dTx0dTeYoyRMRERG5zs9rYOBYs7fnerEnze15NdHLD4nttj2wabs51LJbe/sf32KBds1hxWcwvI9Zfn/jNnjw/+Clt+HU2bTt80pSffAoPDnS7MW87254dejtD4N1RgVczYXaAb5cBLsPOjaezHBokhcREUGDBg0oUqQIpUqVokOHDuzZs+eW+33zzTdUrVoVT09PQkND+emnn3IgWhERkcyZNm0aQUFBeHp60rBhQ6Kiom7afvLkyVSpUgUvLy8CAwMZNmwYly9nPKHnjTfewGKxMHTo0GyIXFJSYNxUyGiEXeq2cVNz74fxG8kvie31yyb4lci+5/H0gCE9zfXVHrrP7N36ajG06AEff22utZdXkupTZ6HXS3A2AWpWgWljzKQor2lS11ys3WqFsVOcfxiuQ5O81atXM2jQIP744w+WL1/OlStXeOCBB0hMTLzhPuvWraNr1648+eST/Pnnn3To0IEOHTqwffv2HIxcREQkY/PmzWP48OGMHTuWLVu2UKtWLcLDw4mLi8uw/Zw5c3jppZcYO3Ysu3btYsaMGcybN4+RI0ema7tx40Y+/PBDatasmd2nkW9FbUuf6FzPwHw8aluOhZTt8ktie/KMWQofzIIrOSGgFEwZDfPfhxqVzUXVX5sOTbvCgDyQVF+8BH1HwpHjUC4AZk6Agl6Ojir7jBpoDvH9Yyv85OT/Rg5N8pYsWULv3r2pXr06tWrVYtasWRw5coTNmzffcJ/33nuP1q1b8/zzz1OtWjVeffVV6taty9SpU3MwchERkYxNmjSJ/v3706dPH0JCQoiMjKRgwYLMnDkzw/br1q2jSZMmdOvWjaCgIB544AG6du2arvfvwoULdO/enY8//phixYrlxKnkOyfPmAtbZ0bc6eyNJSfll8R2zg9mD1qdEKhVNWefu0EoLJoObz5nFmo5cYO/n9yUVF9NgcGvwl+7oZg3fPYG+BZ3dFTZK9AfBnQx778+3bkrqDrVnLz4+HgAihe/8V/I+vXradmyZZpt4eHhrF+/PsP2SUlJJCQkpLmJiIhkh+TkZDZv3pzmOuXi4kLLli1veJ1q3LgxmzdvtiV1Bw8e5KeffqJt27Zp2g0aNIh27dqluwZmRNe+zLNaYXWU2aty9+NmNcTMKJWNQ/1yWmYT1tyc2CZfgS8Wmfdzqhfvv1xdoUs7mPjCzdvlhqTaMGD0ZFix3uzZmjEBKgQ6OqqcMbCr2UN77AR8OM/R0dxYAUcHkMpqtTJ06FCaNGlCjRo1btguNjYWPz+/NNv8/PyIjc145cmIiAjGjRtn11hFREQycurUKVJSUjK8Tu3evTvDfbp168apU6do2rQphmFw9epVBgwYkGa45ty5c9myZQsbN27MVBy69t1a7EmY9zN8/RP8c+La9tpVIfofSLiQ8fBFgAIFzHWz8gKrFbbuylzb3JzYLl5l9tT6lTTnVTnShYuZa+fMSfW02TDnR7O4ypTRt7egfG7l5QmjBsCg8fDBHHg0HMr6Ozqq9JymJ2/QoEFs376duXPn2vW4I0aMID4+3nY7evSoXY8vIiJyJ1atWsWECRP44IMP2LJlC9999x2LFy/m1VdfBeDo0aMMGTKE2bNn4+npmalj6tqXsasp8Ms6eHIUNOoCkz41EzzvwtC7IyydAQunw5vPm+1vVBzw6lXo8DREfuX8Q+pu5vAx6Po/mPntzdtZgNK+EBaaI2Fli1n/Flzp8bBZWdORMpssO+vQx2+XwsQZ5v1xz0B4U8fG4wjtWlxbE3FCpKOjyZhT9OQNHjyYH3/8kTVr1lC2bNmbtvX39+fEiRNptp04cQJ//4xTaA8PDzw8POwWq4iIyI2ULFkSV1fXLF2nRo8eTY8ePejXrx8AoaGhJCYm8tRTTzFq1Cg2b95MXFwcdevWte2TkpLCmjVrmDp1KklJSbi6pi1lp2tfWkdjYd5P8M3PEHvq2vawmtD1QWjbzKyGmKpNM5g+zpwXdf1ctdK+MLQ3LPvdHKYW8REsXweTXoLyZXLsdO6Y1WomPW/NMOcUeXlC+3vN1wfS92AamOXjXXNpxcQ/d8LW3eDhZq7f5mhhoebfUuzJG/cWA7z1CYx+2rl6yX7bBC9MNO8P6AK9Ojo2HkexWOCVZ6DtU7B4Naz7ExrXcXRUaTm0J88wDAYPHsyCBQtYuXIlwcHBt9ynUaNGrFixIs225cuX06hRo+wKU0REJFPc3d2pV69emuuU1WplxYoVN7xOXbx4EReXtJfj1KTNMAzuv/9+tm3bxtatW223+vXr0717d7Zu3ZouwRPTlavw02ro+QLc0w2mfGEmeMV9oP/j5jpm37wHnVqlTfBStWkGa78yF7d+/2Xz59qvoEtbmPE6vPU8FC5orrnWup+5dpazl1QHOHDEXJx73DQzwWtUB5bNMOeJTR8H/r7p9ylXOnf31qT2VD50v1n0xNFcr1tz7b+9xam/u7uZyWmnwTDwFbPX1dF27jfnrl5NMZeFeLG/oyNyrGoVofu/ay2Om2q+Ls7EoT15gwYNYs6cOSxcuJAiRYrY5tX5+Pjg5WXWX+3ZsydlypQhIiICgCFDhtC8eXPeeecd2rVrx9y5c9m0aRMfffSRw85DREQk1fDhw+nVqxf169cnLCyMyZMnk5iYSJ8+fYD017X27dszadIk6tSpQ8OGDdm/fz+jR4+mffv2uLq6UqRIkXRz1QsVKkSJEiVuOoc9v4r+B+YuhvlL0y483bQedG0HrZqYhSIyw9UVGtVOv91igc5toXFdeO4N+OMvGPUuLFsLbz2XcaLkaFdT4JNvzCGqSclQyAtGDjB7tlK/Y2jTDB5oYhb8iDttvk7DI+BIDHy/wkyIc5sTp8xkH6B3J8fGcr0b9Rb7+5oJYN0QeOdT+GaJGf/ytdCzAzzzBBTzyfl4j52A3iPM+YR314a3X7z2d5Of/a+PWaxp90GYvci5ejYdmuRNnz4dgBYtWqTZ/umnn9K7d28Ajhw5kuYbzsaNGzNnzhxefvllRo4cSeXKlfn+++91oRMREafQuXNnTp48yZgxY4iNjaV27dosWbLEVozlv9e1l19+GYvFwssvv8yxY8fw9fWlffv2vP766446BaeSknIt6ShVwhzq9t/Oy8vJsPQ3c7Hp9X9e2+5bHB5vY/a+lQuwf2yB/vDVJHOB7Tc/Nqt0tuoLrw6Bh+83k0FnsDcannvLLHUP0Kw+vPEclPFL3/a/ie2BI+awwYmfpB/Wmht8schMcMNqmuvUOZP/JtX//ft+63mzEuiED82/rRnzzSG1g3uYyYRnJr+suFPx56HXi2bCfFcQfPRq5r8oyeuK+cBzfWH0e2ZS/tB9jknCM2IxjNwwuMB+EhIS8PHxIT4+Hm9vb0eHIyIit0nv55mXW1+rn9dkPC9u7GDzA/LeQ2av3bfL4Ny/q0RYLNAizJxrd9/dOVdkY99hs9fr7z3m722bw+vDzOGhjnLlKkTOhfc/N5cQ8C4EowfBY60zn4BeToL7epk9Oc8/CYOfyN6Y7elyMjTuDKfPwfRXzH+T3Oq3Tea6bLsOmr+X9YMX+ptzKbOzRy0p2Rzy/MdfZmXS76eZywfINVdToN1TZm/eEw+Z/++zU2bfz5XkiYhIrqT388zLja/Vz2tg4Nj0hSksmNsqBsKB64qGBpQye+0eb5NxD1VOuHIVPpgN739hfvDzLWZW6rzfAWUDduyH59+CHfvM3+9vBBOG3d5Q0gXLYegEc4jn6i+dt+rjf32zBJ570/x7WDMbCuTy6aspKfDdcnh7xrUCQrWqmuX8G9ay//NZrTDkdXM4YuGCMP99cx6apLd+K3QZZibciz+EkErZ91yZfT/XaFoRERFxKikpZg9eRt9Cp247cBRcLGZBkE8j4Pc5MKy34xI8MHsNh/Qyezsql4eTZ6HvSLMa4fnEnIkh+Qq8MxMeGmAmeEW9YfJIs1jM7c4VfPh+qFkFEi/B5M/sG292MQxzGC1Az4dzf4IH5jDOx1rDqi/guSfNpPuv3fD4UOj/sjm01p7e+MhM8Aq4wofjleDdTKPa0K65mRi/MsU5ijApyRMRERGnErUt7RDNG5k21pwfdN/dzlXeP7QK/PiRWcXTYjGXb2j9pPltf3b6ew88+H/XehJb3wPLP4WOre5sfqCLC7w80Lw/50dzmKyz27TdTHI9PaBLO0dHY19enmYBltVfmsMDXV3Moj+t+sDLk9MWHLpdn34HH84z7098wSxcJDc3coD597bhb1i8ytHRKMkTERERJxN3OnPtrlzN3jjuhKe7mRjNexfK+puLrncZBuOnmfPc7Olystnr8vDTsCfaXCbgg7EQOQ5K2WloZcNaZq+p1eq8iz9fL3XZhI4tzd7MvMi3uDn/a+lMaNkYUqzwxUJo/gRMm337f2dLfjN70gFe6AedHrBfzHlZWX8Y2NW8/3qkuUSJIynJExEREadSqoR92zlSw1qwdIa5fAOYFRLb/d+1Ai13atN2aNsPpn9lJmAP3Wf23rVrYf/qni89ZQ7d+3UD/L7Zvse2p2MnzGqrAH0ecWwsOaFyeXM47tx3IfQuc5mDtz6BFj3g26Xm30VmbdoOz75mDjd84iF4ulv2xZ0XDehiFsU5Hmf+n3QkJXkiIiLiVMJCzSqaN8pRLJiPh4XmZFS3r3BBc8mCmRPM3pf9h6HD0/DurNvvjbx02ewVfPRZc36ib3H4+FWYMjr7FvyuEAg9Hjbvv/aBOXfSGX2x0OzValwHqgQ7Opqc06g2LJoO740y56bGnIThb5hfKmSUlKekmEOIF64wf+49BE+OMitqtmwM4551nmVAcgtPDxj579DmyLlwNNZxsSjJExEREafi6mouk5CR1M+cYwc71zy8zLi/ESyfaZa9T7GaRUw6Dsr6HLc/tkL4k2avoGHAo+Hwy6fwQNPsiDqtIT3Bu7BZyn/+0ux/vqy6dNlcLxHMNebyGxcX6NASVn4OI56CIoVg537o/hz0fslcMxHM6rVNuppDiJ99zfzZ+klzKZLaVWHKy3mjWI0jtG0GjeqYyfLr0x0Xh5I8ERERcTptmsF7L6ff7u8L08eZj+dGxXxg6hizx82nCGzbCw8+BR9/fW1Y3X97WFJ7zBIvmYsudx4Gh4+bvZmfRsA7L+XcvLNiPvBMD/P+2zPNmJzJ97+YiUpgabMgT37l6Q4DuppLR/TpdG2YbXg/6P4/GDA2fXGjlH///ro/BAW9cj7mvMJigVcGmwn3z2tg7RYHxaF18kREJDfS+3nm5dbXausus5iIT2F4dag5By8sNPf14N3IiVPw/ERYHWX+3rAmtL/PLJrx3wXgO7eF+UvMAi5gzvEbOcDsVctpScnQsg8cOQ5De5lLVzgDwzB7OPdEw+inod9jjo7IeUT/A29+bCYdt1LaF9Z+lXf+nznKmPfhswVwVxD8/In9eka1Tp6IiIjkajv3mz9rVzPXamtUO2998PQrCZ+9ARHDoaCnWXr95cnpe1hiTppDO/85YRZ1mP22OcfPEQkegIc7vNTfvP/hPDNZdQbrt5oJXkFPeKyNo6NxLsFlzWqrrzxz67YxJ81lTOTODO8NxbzN4dhfLMz551eSJyIiIk5px79JXkglx8aRnSwW6NYefvoY3N1u3ragl9kj4AxrlrVtDnWrm3Pg3p7p6GhMn/67bMIj4Wbvr6RX3Cdz7TK7jIncWFFvc9F6gEmfwpn4nH1+JXkiIiLilHYeMH/m5SQvVewpSL5y8zYXL11LfB3NYrm2QPo3S671ujrKkRhYvs6837uTY2NxZnlpeZLcoGs7CKkICRfMZS0ymmubXZTkiYiIiNNJSYHd+SjJy2zPiTP1sNSrDg+2MOfCvR5p/nSUzxaYz9+8AVQq57g4nF1eW57E2bm6Xhsi+9WPaauZNumauTmSt0tJnoiIiDidw8fh4mVz3angMo6OJvvl1h6WF58yh5n+vtms3ugIiZfg65/M+/lh8fM7cf3yJP9N9HLz8iTO7EbDNGNPwsCx2ZfoKckTERERp5M6/K9qcP74wJlbe1jKlb42PPL1SLjqgAXSv10KCYnmYu3NG+T88+c2bZqZy5D4+6bdntuXJ3FGKSkwbmrGj6V2fI+bmj1DNwvY/5AiIiIid2ZnPii6cr3UHpaBY82E7vqRj87ewzL4CfjmZ9h/GOYuhiceyrnntlph1gLzfq+O5tpkcmttmsEDTcwqmnGn897yJM4ialv6arnXM7hWzbRRbfs+t/4riIiIiNPJT0VXUuXWHhafwjCkl3l/0qdwPjHnnvv3zXDgCBQuCI+G59zz5gWurmZikReXJ3EWjpxrq548ERERcTr5McmD3NvD8sRD8Pn3cPAoTJ8DL/TPmeed+e+yCY+3MRM9EWfiyLm26skTERERp3LqrLnAtsUCVSs4Opqclxt7WNwKwIinzPufzIdjJ7L/OaP/MYu9WCzmUE0RZ+PIubZK8kRERMSp7Pq3Fy+4LBTycmwsknmtmsDdtSApGSbOyP7nm/Wd+fO+uyEoH1RgldzHkdVMleSJiIiIU0ld8DukomPjkKy5foH0Bcvh7z3Z91znE81F2AH6atkEcWKOmmurOXkiIiLiVFIra1ZTkpfrhFaBjq3MJO+1D2DeZDP5s7dvfjbXx6tcHprUtf/xRezJEXNt1ZMnIiJiZ9OmTSMoKAhPT08aNmxIVFTUTdtPnjyZKlWq4OXlRWBgIMOGDePy5cu2xyMiImjQoAFFihShVKlSdOjQgT17srGbxMHy2/IJec0L/cDDHTb8DcvW2v/41y+b0KdT9iSRIvaW03NtleSJiIjY0bx58xg+fDhjx45ly5Yt1KpVi/DwcOLi4jJsP2fOHF566SXGjh3Lrl27mDFjBvPmzWPkyJG2NqtXr2bQoEH88ccfLF++nCtXrvDAAw+QmJiDtepzyOUkOHDUvF9dSV6uFFAK+j1m3o/4EJKv2Pf4v26Aw8fBu7DZaygi6SnJExERsaNJkybRv39/+vTpQ0hICJGRkRQsWJCZM2dm2H7dunU0adKEbt26ERQUxAMPPEDXrl3T9P4tWbKE3r17U716dWrVqsWsWbM4cuQImzdvzvCYSUlJJCQkpLnlFnsPmT01JYpmT1lxyRlPd4OSxcwKmLMX2ffYn/5bcKVLOyiowjwiGVKSJyIiYifJycls3ryZli1b2ra5uLjQsmVL1q9fn+E+jRs3ZvPmzbak7uDBg/z000+0bdv2hs8THx8PQPHixTN8PCIiAh8fH9stMDDwdk8px+24bqimhuHlXoULwvDe5v3Jn0P8efscd+8h+G0TuLhAzw72OaZIXqQkT0RExE5OnTpFSkoKfn5+abb7+fkRGxub4T7dunVj/PjxNG3aFDc3NypWrEiLFi3SDNe8ntVqZejQoTRp0oQaNWpk2GbEiBHEx8fbbkePHr2zE8tBO1VZM8/o3M4sjHIuAaZ8aZ9jfvbvXLxWTSDQ3z7HFMmLlOSJiIg40KpVq5gwYQIffPABW7Zs4bvvvmPx4sW8+uqrGbYfNGgQ27dvZ+7cuTc8poeHB97e3mluuYWKruQdBVxh1L9LKny2AI4cv7PjxZ+Hb5eZ9/t2urNjieR1SvJERETspGTJkri6unLixIk020+cOIG/f8bdDqNHj6ZHjx7069eP0NBQOnbsyIQJE4iIiMBqtaZpO3jwYH788Ud+/fVXypYtm23n4ShW67WF0NWTlze0CIN76pvFV978+M6ONe9nuHQZqlWAhrXsE59IXqUkT0RExE7c3d2pV68eK1assG2zWq2sWLGCRo0aZbjPxYsXcXFJezl2/be2tmEYtp+DBw9mwYIFrFy5kuDg4Gw6A8c6EmOufebhBhXKOToasQeLBUYNMH/+uAo277i946SkXBuq2VvLJojckpI8EREROxo+fDgff/wxn332Gbt27WLgwIEkJibSp08fAHr27MmIESNs7du3b8/06dOZO3cu0dHRLF++nNGjR9O+fXtbsjdo0CC+/PJL5syZQ5EiRYiNjSU2NpZLly455ByzS2rRlSoVzKF+kjdUqwiPtzHvv/oB/PvdRZb8sh7+iYVi3tCh5a3bi+R3BRwdgIiISF7SuXNnTp48yZgxY4iNjaV27dosWbLEVozlyJEjaXruXn75ZSwWCy+//DLHjh3D19eX9u3b8/rrr9vaTJ8+HYAWLVqkea5PP/2U3r17Z/s55RTbUE3Nx8tz/tcHFq2EP3fC4lXw4L1Z23/mt+bPbg+Cp4fdwxPJcyyGcTvfp+ReCQkJ+Pj4EB8fn6smoouISFp6P8+83PJa9R0JK9bD+GehV0dHRyP2NvkzeHcWBJaGFbPAwz1z++06AK37gasLrJ0LpX2zM0oR55bZ9/MsD9fcsmUL27Zts/2+cOFCOnTowMiRI0lOTr69aEVERCTfU2XNvO2px8GvJByNgVkLMr9f6uLnbZopwRPJrCwnef/3f//H3r17AXPB1i5dulCwYEG++eYbXnjhBbsHKCIiInnfmXiIOWner6bKmnlSQS94rq95f+oX5r/5rZyJh+9/Me/3eST7YhPJa7Kc5O3du5fatWsD8M0339CsWTPmzJnDrFmz+Pbbb+0dn4iIiOQDqb14QWWgcEHHxiLZ55EHzOUxEhLhvc9u3X7uYkhKhtC7oF717I9PJK/IcpJnGIZt3Z5ffvmFtm3bAhAYGMipU6fsG52IiIjkCzv/LbqiXry8zfW6BdK/XAQHj9647ZWr8Pn35v0+WjZBJEuynOTVr1+f1157jS+++ILVq1fTrl07AKKjo22Vw0RERESyQvPx8o+m9eC+u+FqCrzx0Y3bLfvdHMJbsljWq3GK5HdZTvImT57Mli1bGDx4MKNGjaJSJfPdeP78+TRu3DhLx1qzZg3t27cnICAAi8XC999/f9P2q1atwmKxpLvFxsZm9TRERETEiaQmedWV5OULIweY1TKX/g5/bM24TeqyCd3bZ74Sp4iYsrxOXs2aNdNU10w1ceJE26KtmZWYmEitWrXo27cvnTp1yvR+e/bsSVMytFSpUll6XhEREXEel5PhwBHzvnry8ofK5aHrg+aQzdemw6LpcN3ykWzbA5u2g1sB6P6Q4+IUya2ynOQdPXoUi8VC2bJlAYiKimLOnDmEhITw1FNPZelYbdq0oU2bNlkNgVKlSlG0aNFMtU1KSiIpKcn2e0JCQpafT0RERLLPvkPm0L1i3uBf0tHRSE4Z1tusnLltL3y/Ajq1uvZY6rIJ7VqAXwlHRCeSu2V5uGa3bt349ddfAYiNjaVVq1ZERUUxatQoxo8fb/cAM1K7dm1Kly5Nq1atWLt27U3bRkRE4OPjY7sFBgbmSIwiIpI7PPLII7z55pvptr/11ls89thjDogo/7l+Pp6Ka+QfJYvB093M+xM/gcv/fid/8gz8YH7UpK+WTRC5LVlO8rZv305YWBgAX3/9NTVq1GDdunXMnj2bWbNm2Tu+NEqXLk1kZCTffvst3377LYGBgbRo0YItW7bccJ8RI0YQHx9vux09epMyTiIiku+sWbPGVin6em3atGHNmjUOiCj/UdGV/OvJR6GMHxyPg4/mwfqtMOZ9SL4CtatBraqOjlAkd8rycM0rV67g4eEBmEsoPPSQOVC6atWqxMTE2De6/6hSpQpVqlSx/d64cWMOHDjAu+++yxdffJHhPh4eHrZ4RURE/uvChQu4u6ev6uDm5qYh/jkkdfmEEC2fkO94esDzT8LQCTDpUzCue+zQP/DzGmjTzGHhieRaWe7Jq169OpGRkfz2228sX76c1q1bA3D8+HFKlMj5QdNhYWHs378/x59XRETyhtDQUObNm5du+9y5cwkJCXFARPmL1aqevPwutXKm8Z/t8edh4Fgz0RORrMlyT96bb75Jx44dmThxIr169aJWrVoALFq0yDaMMydt3bqV0qVL5/jziohI3jB69Gg6derEgQMHuO+++wBYsWIFX331Fd98842Do8v7/omFCxfB3Q0qlnN0NJLTUlJg/LSMHzMACzBuKjzQxFxIXUQyJ8tJXosWLTh16hQJCQkUK1bMtv2pp56iYMGCWTrWhQsX0vTCRUdHs3XrVooXL065cuUYMWIEx44d4/PPPwfMNfqCg4OpXr06ly9f5pNPPmHlypUsW7Ysq6chIiICQPv27fn++++ZMGEC8+fPx8vLi5o1a/LLL7/QvHlzR4eX56UO1bwryCyXL/lL1DZzwfMbMTAfj9oGjWrnVFQiud9tvZ26urpy9epVfv/9d8CcKxcUFJTl42zatIl7773X9vvw4cMB6NWrF7NmzSImJoYjR47YHk9OTuZ///sfx44do2DBgraL8PXHEBERyap27drRrl07R4eRL2moZv4Wd9q+7UTElOUkLzExkWeeeYbPP/8cq9UKmElfz549mTJlSpZ681q0aIFh/HcE9jX/rdb5wgsv8MILL2Q1ZBERkRvauHEjVquVhg0bptm+YcMGXF1dqV+/voMiyx+U5OVvpTJZziGz7UTElOXCK8OHD2f16tX88MMPnDt3jnPnzrFw4UJWr17N//73v+yIUUREJNsMGjQow+V1jh07xqBBgxwQUf6SOlyzupK8fCksFEr7mnPvMmLBfDwsNCejEsn9spzkffvtt8yYMYM2bdrg7e2Nt7c3bdu25eOPP2b+/PnZEaOIiEi22blzJ3Xr1k23vU6dOuzcudMBEeUf5xLg2AnzftUKjo1FHMPVFcYONu//N9FL/X3sYBVdEcmqLCd5Fy9exM/PL932UqVKcfHiRbsEJSIiklM8PDw4ceJEuu0xMTEUKKBKINkptRcvsDR4F3ZsLOI4bZrB9HHg75t2u7+vuV3r5IlkXZavXo0aNWLs2LF8/vnneHp6AnDp0iXGjRtHo0aN7B6giEhWWa1WkpOTHR2G3CE3Nzdcc+Dr+wceeIARI0awcOFCfHx8ADh37hwjR46kVatW2f78+Znm40mqNs3MZRKitplFVkqVMIdoqgdP5PZkOcl77733CA8Pp2zZsrY18v766y88PT1ZunSp3QMUEcmK5ORkoqOjbYWhJHcrWrQo/v7+WCw3mrFz595++22aNWtG+fLlqVOnDmCuwern58cXX3yRbc8r15K86hUdG4c4B1dXLZMgYi9ZTvJq1KjBvn37mD17Nrt37waga9eudO/eHS8vL7sHKCKSWYZhEBMTg6urK4GBgbi4ZHlEujgJwzC4ePEicXFxAJQuXTrbnqtMmTL8/fffzJ49m7/++gsvLy/69OlD165dcXNzu61jTps2jYkTJxIbG0utWrWYMmUKYWFhN2w/efJkpk+fzpEjRyhZsiSPPvooERERthEzt3PM3GDXv8M11ZMnImJftzXZoGDBgvTv39/esYiI3JGrV69y8eJFAgICsrScizin1C8O4+LiKFWqVLYO3SxUqBBPPfWUXY41b948hg8fTmRkJA0bNmTy5MmEh4ezZ88eSpUqla79nDlzeOmll5g5cyaNGzdm79699O7dG4vFwqRJk27rmLlB8hXYd9i8ryRPRMS+MpXkLVq0KNMHfOihh247GBGRO5GSkgKAu7u7gyMRe0lN1q9cuZLt8/N27tzJkSNH0s3nzOp1bdKkSfTv358+ffoAEBkZyeLFi5k5cyYvvfRSuvbr1q2jSZMmdOvWDYCgoCC6du3Khg0bbvuYSUlJJCUl2X5PSEjI0jnkhH2H4cpV8CkCAbkzTxURcVqZSvI6dOiQqYNZLBbbhywREUfJzvlbkrNy4t/y4MGDdOzYkW3btmGxWDAMI81zZ+W6lpyczObNmxkxYoRtm4uLCy1btmT9+vUZ7tO4cWO+/PJLoqKiCAsL4+DBg/z000/06NHjto8ZERHBuHHjMh23I1xfdEX/ZUVE7CtTE1asVmumbkrwREQktxkyZAjBwcHExcVRsGBBduzYwZo1a6hfvz6rVq3K0rFOnTpFSkpKuqWG/Pz8iI2NzXCfbt26MX78eJo2bYqbmxsVK1akRYsWjBw58raPOWLECOLj4223jBZ7dzRbkqeiKyIidqeqBCIikq+tX7+e8ePHU7JkSVxcXHBxcaFp06ZERETw7LPPZvvzr1q1igkTJvDBBx+wZcsWvvvuOxYvXsyrr75628f08PDA29s7zc3ZKMkTEck+SvJERBysRYsWDB06NMv7zZgxgwceeMCusSQnJxMUFMSmTZtu2m7VqlVYLBbOnTuX6WP37t0708P/c1JKSgpFihQBoGTJkhw/fhyA8uXLs2fPniwdq2TJkri6uqZbXP3EiRP4+/tnuM/o0aPp0aMH/fr1IzQ0lI4dOzJhwgQiIiKwWq23dUxnZxhaI09EJDspyRMRyYUuX77M6NGjGTt2rG3bjh07eOSRRwgKCsJisTB58uQM9502bRpBQUF4enrSsGFDoqKibI+5u7vz3HPP8eKLL970+Rs3bkxMTIxt8fDMeO+995g1a5bt99tNbu2tRo0a/PXXXwA0bNiQt956i7Vr1zJ+/HgqVKiQpWO5u7tTr149VqxYYdtmtVpZsWIFjRo1ynCfixcvplvuI7XIjGEYt3VMZ/fPCUhIBLcCUKm8o6MREcl7lOSJiORC8+fPx9vbmyZNmti2Xbx4kQoVKvDGG2/csIcntRT/2LFj2bJlC7Vq1SI8PNy2Hh1A9+7d+f3339mxY8cNn9/d3T3Li5T7+PhQtGjRTLfPKS+//DJWqxWA8ePHEx0dzT333MNPP/3E+++/n+XjDR8+nI8//pjPPvuMXbt2MXDgQBITE22VMXv27JmmiEr79u2ZPn06c+fOJTo6muXLlzN69Gjat29vS/ZudczcZte/vXiVg8D99pYiFBGRm1CSJyJ5XnJyMsnJybaqiWAO0UtOTubq1at2b3ungoKCmDBhAn379qVIkSKUK1eOjz76KE2buXPn0r59+zTbGjRowMSJE+nSpQseHh4ZHvv6UvwhISFERkZSsGBBZs6caWtTrFgxmjRpwty5c28Y43+Ha86aNYuiRYuydOlSqlWrRuHChWndujUxMTG2fa4frtm7d29Wr17Ne++9h8ViwWKxcOjQoSy8SvYTHh5Op06dAKhUqRK7d+/m1KlTxMXFcd9992X5eJ07d+btt99mzJgx1K5dm61bt7JkyRJb4ZQjR46keV1efvll/ve///Hyyy8TEhLCk08+SXh4OB9++GGmj5nb7Px3EfTqGqopIpItbmsxdKvVyv79+4mLi7N9+5mqWbNmdglMRMRe3n33XQCeeeYZ27prGzZs4LfffqNmzZq0adPG1nbq1KlcuXKFAQMG2IYibtmyhZUrVxISEpImsYqMjOTSpUv07dsXX19fALZt20bt2rXvOOZ33nmHV199lZEjRzJ//nwGDhxI8+bNqVKlCgC///67rcR+ZmWlFH9YWBi//fZblo5/8eJF3n77bb744gtcXFx44okneO6555g9e3a6tu+99x579+6lRo0ajB8/HsD2GjqD4sWL39H+gwcPZvDgwRk+9t+KnQUKFGDs2LFpht5m9Zi5jYquiIhkrywneX/88QfdunXj8OHDab69Bq2TJyJiL23btuXpp58G4MUXX+Tdd9/l119/pUqVKpw7d474+HgCAgKydMybleLfvXt3mm0BAQEcPnw4S8e/cuUKkZGRVKxofnIfPHiwLYH7Lx8fH9zd3SlYsGCuLR4it2+Hiq6IiGSrLCd5AwYMoH79+ixevJjSpUtr0WERcXrDhg0DwM3t2uSfhg0bUr9+/XQFL1J7Sq5vW7duXWrVqpWu7YABA9K1DQ0NtUvMNWvWtN23WCz4+/vb5s1dunQJAE9PT7s8V0a8vLy4ePFilvYpWLCgLcEDKF26dJq5fiIA8Rfgn3+X96umnjwRkWyR5SRv3759zJ8/n0qV9PWbiOQO7u7u6ba5urrailrYu609XJ84gpnopQ6PL1GiBBaLhbNnz2bpmFkpxX/mzJksD5/MKOb/jvgQSS26UtYPfIo4NhYRkbwqy4VXGjZsyP79+7MjFhERyQR3d3dCQkLYuXNnlvfLbCn+7du3U6dOHbvEe7N4nGGI/5o1a9IV1QG4evUqa9ascUBEeZvWxxMRyX6Z6sn7+++/bfefeeYZ/ve//xEbG0toaGi6b26vH2IkIiLZIzw8nN9//z3NOnPJycm2xC85OZljx46xdetWChcubBt9MXz4cHr16kX9+vUJCwtj8uTJGZbi/+2333j11Vez9RyCgoLYsGEDhw4donDhwhQvXjzdkNiccO+99xITE0OpUqXSbI+Pj+fee+91ikQ0L0mtrKkkT0Qk+2Qqyatdu3a6YTd9+/a13U99TIVXRERyxpNPPkn9+vWJj4+3VQE9fvx4mt63t99+m7fffpvmzZvbKjp27tyZkydPMmbMGGJjY6ldu3a6Uvzr168nPj6eRx99NFvP4bnnnqNXr16EhIRw6dIloqOjCQoKytbnzEjq9eu/Tp8+TaFChXI8nrxOSZ6ISPazGJmYMJGVCmvly5e/o4CyW0JCAj4+PsTHx+Pt7e3ocETEji5fvkx0dDTBwcHZWpTEWTz22GPUrVs3zZII9tC5c2dq1arFyJEj7Xrc23Gzf9M7fT9PXRtv4cKFtG7dOs3agikpKfz9999UqVKFJUuW3NlJOAFnufYlX4Hq7cyfv38FgSqsKiKSJZl9P89UT56zJ24iIvnRxIkT+eGHH+x6zOTkZEJDQ20VSfOy1B5QwzAoUqQIXl5etsfc3d25++676d+/v6PCy5MOHDETPO9CZuEVERHJHlmurhkREYGfn1+a4ZoAM2fO5OTJk7z44ot2C05ERG4sKCiIZ555xq7HdHd35+WXX7brMZ3Vp59+Cpiv43PPPaehmTng+qIrWoFJRCT7ZHmG+4cffkjVqlXTba9evTqRkZF2CUpERCSnvPDCC2nm5B0+fJjJkyezbNkyB0aVN6Uugq718UREsleWk7zY2FhKly6dbruvry8xMTF2CUpE5E5obba8Iyf+LR9++GE+//xzAM6dO0dYWBjvvPMODz/8MNOnT8/2589PVHRFRCRnZDnJCwwMZO3atem2r127loCAALsEJSJyO1IXIk9OTnZwJGIvFy9eBNIvtG5PW7Zs4Z577gFg/vz5+Pv7c/jwYT7//HPef//9bHve/MYwtEaeiEhOyfKcvP79+zN06FCuXLnCfffdB8CKFSt44YUX+N///mf3AEVEMqtAgQIULFiQkydP4ubm5pA118Q+DMPg4sWLxMXFUbRoUVsCnx0uXrxIkSJFAFi2bBmdOnXCxcWFu+++O0vVpeXmYk5C/Hko4AqVVc9NRCRbZTnJe/755zl9+jRPP/207dtyT09PXnzxRV566SW7BygiklkWi4XSpUsTHR2tD+d5RNGiRfH3z946+5UqVeL777+nY8eOLF261FZZNC4uTkvt2FFqL17l8uDh7thYRETyuiwneRaLhTfffJPRo0eza9cuvLy8qFy5cpr1hUREHMXd3Z3KlStryGYe4Obmlq09eKnGjBlDt27dGDZsGPfddx+NGjUCzF696xeXlzuzQ0M1RURyTJaTvL59+/Lee+9RpEgRGjRoYNuemJjIM888w8yZM+0aoIhIVrm4uOSLxdDFPh599FGaNm1KTEwMtWrVsm2///776dixowMjy1s0H09EJOdkecLKZ599xqVLl9Jtv3Tpkq06mYiISG7i7+9PkSJFWL58ue0a16BBgwyXDJLbk1pZU8sniIhkv0z35CUkJGAYBoZhcP78+TTfkqekpPDTTz9RqlSpbAlSREQku5w+fZrHH3+cX3/9FYvFwr59+6hQoQJPPvkkxYoV45133nF0iLlewgU4cty8H6IkT0Qk22W6J69o0aIUL14ci8XCXXfdRbFixWy3kiVL0rdvXwYNGpSdsYqIiNjdsGHDcHNz48iRIxQsWNC2vXPnzixZssSBkeUduw+aPwNKQTEfx8YiIpIfZLon79dff8UwDO677z6+/fZbihcvbnvM3d2d8uXLa508ERHJdZYtW8bSpUspW7Zsmu2VK1dWlVY7UdEVEZGclekkr3nz5gBER0cTGBio9adERCRPSExMTNODl+rMmTOqHG0nu/6dj6ehmiIiOSPLmVr58uVxcXHh4sWL7N69m7///jvNLSvWrFlD+/btCQgIwGKx8P33399yn1WrVlG3bl08PDyoVKkSs2bNyuopiIiI2Nxzzz1pCodZLBasVitvvfUW9957720dc9q0aQQFBeHp6UnDhg2Jioq6YdsWLVpgsVjS3dq1a2drc+HCBQYPHkzZsmXx8vIiJCSEyMjI24rNEVRZU0QkZ2V5CYWTJ0/Sp08ffv755wwfT0lJyfSxEhMTqVWrFn379qVTp063bB8dHU27du0YMGAAs2fPZsWKFfTr14/SpUsTHh6e6ecVERFJ9dZbb3H//fezadMmkpOTeeGFF9ixYwdnzpxh7dq1WT7evHnzGD58OJGRkTRs2JDJkycTHh7Onj17MixQ9t1336VZ1/H06dPUqlWLxx57zLZt+PDhrFy5ki+//JKgoCCWLVvG008/TUBAAA899NDtnXgOuXIV9kab96sryRMRyRFZ7skbOnQo586dY8OGDXh5ebFkyRI+++wzKleuzKJFi7J0rDZt2vDaa69leh2iyMhIgoODeeedd6hWrRqDBw/m0Ucf5d13383qaYiIiABQo0YN9u7dS9OmTXn44YdJTEykU6dO/Pnnn1SsmPXxhZMmTaJ///706dPH1uNWsGDBG64jW7x4cfz9/W235cuXU7BgwTRJ3rp16+jVqxctWrQgKCiIp556ilq1at20h9BZHDwKSVegSCEo6+/oaERE8ocs9+StXLmShQsXUr9+fVxcXChfvjytWrXC29ubiIiINMNL7G39+vW0bNkyzbbw8HCGDh16w32SkpJISkqy/Z6QkJBd4YmISC505MgRAgMDGTVqVIaPlStXLtPHSk5OZvPmzYwYMcK2zcXFhZYtW7J+/fpMHWPGjBl06dKFQoUK2bY1btyYRYsW0bdvXwICAli1ahV79+694ZecznTtSy26UrUCaDq/iEjOyPLbbWJiom24SbFixTh58iQAoaGhbNmyxb7R/UdsbCx+fn5ptvn5+ZGQkJDhAu0AERER+Pj42G6BgYHZGqOIiOQuwcHBtmvZ9U6fPk1wcHCWjnXq1ClSUlIyvFbFxsbecv+oqCi2b99Ov3790myfMmUKISEhlC1bFnd3d1q3bs20adNo1qxZhsdxpmuf5uOJiOS8LCd5VapUYc+ePQDUqlWLDz/8kGPHjhEZGUnp0qXtHuCdGjFiBPHx8bbb0aNHHR2SiIg4EcMwsFgs6bZfuHABT0/PHI1lxowZhIaGEhYWlmb7lClT+OOPP1i0aBGbN2/mnXfeYdCgQfzyyy8ZHseZrn22JE+VNUVEckyWh2sOGTKEmJgYAMaOHUvr1q2ZPXs27u7u2V7p0t/fnxMnTqTZduLECby9vfHy8spwHw8PD5XAFhGRdIYPHw6Y1TRHjx6dZhmFlJQUNmzYQO3atbN0zJIlS+Lq6prhtcrf/+YT0hITE5k7dy7jx49Ps/3SpUuMHDmSBQsW2KZE1KxZk61bt/L222+nm8YAznPtM4xryyeo6IqISM7JcpL3xBNP2O7Xq1ePw4cPs3v3bsqVK0fJkiXtGtx/NWrUiJ9++inNtuXLl9OoUaNsfV4REcl7/vzzT8Dsydu2bRvu7u62x9zd3alVqxbPPfdclo7p7u5OvXr1WLFiBR06dADAarWyYsUKBg8efNN9v/nmG5KSktJcZwGuXLnClStX0q1P6+rqitVqzVJ8Oe3EKTgTD64uUDlrI19FROQOZDnJu55hGHh5eVG3bt3b2v/ChQvs37/f9nt0dDRbt26lePHilCtXjhEjRnDs2DHb+kUDBgxg6tSpvPDCC/Tt25eVK1fy9ddfs3jx4js5DRERyYd+/fVXAPr06cN7772Ht7e3XY47fPhwevXqRf369QkLC2Py5MkkJibSp08fAHr27EmZMmWIiIhIs9+MGTPo0KEDJUqUSLPd29ub5s2b8/zzz+Pl5UX58uVZvXo1n3/+OZMmTbJLzNkltehKpfLg6X7ztiIiYj+3leTNmDGDd999l3379gFQuXJlhg4dmm6i+K1s2rQpzUKzqUNnevXqxaxZs4iJieHIkSO2x4ODg1m8eDHDhg3jvffeo2zZsnzyySdaI09ERG7bp59+atfjde7cmZMnTzJmzBhiY2OpXbs2S5YssRVjOXLkSLpeuT179vD777+zbNmyDI85d+5cRowYQffu3Tlz5gzly5fn9ddfZ8CAAXaN3d52/jtUU0VXRERylsUwDCMrO4wZM4ZJkybxzDPP2IZJrl+/nqlTpzJs2LB0cwmcTUJCAj4+PsTHx9vtW1sREcl5ej/PPEe9VgNfgZ9Ww6iB8NTjOfa0IiJ5Vmbfz7Pckzd9+nQ+/vhjunbtatv20EMPUbNmTZ555hmnT/JEREQkZ6iypoiIY2R5CYUrV65Qv379dNvr1avH1atX7RKUiIiI5G4XLsKhY+b9akryRERyVJaTvB49ejB9+vR02z/66CO6d+9ul6BEREQkd0tdOsG/JJQo6tBQRETynUwN10wtiALmekKffPIJy5Yt4+677wZgw4YNHDlyhJ49e2ZPlCIiIpKr7FLRFRERh8lUkpe6llCqevXqAXDggPkOXrJkSUqWLMmOHTvsHJ6IiIjkRrb5eEryRERyXKaSvNS1hEREREQyQ8sniIg4Tpbn5ImIiIjczNUU2H3QvF9dSZ6ISI5TkiciIiJ2dfAoJCVDIS8oV9rR0YiI5D9K8kRERMSuUufjVa0ALvqkISKS4/TWKyIiInaloisiIo6lJE9ERETsKnX5BM3HExFxDCV5IiIiYjeGATvUkyci4lBK8kRERMRu4s7A6XPmXLwqwY6ORkQkf1KSJyIiInaTOh+vYiB4ejg2FhGR/EpJnoiIiNiNiq6IiDiekjwRERGxG1uSV9GxcYiI5GdK8kRERMRudv5bWVM9eSIijqMkT0REROzi4iWI/se8X009eSIiDqMkT0REROxid7S5hEKpEuBb3NHRiIjkX0ryRERExC5UdEVExDkoyRMRERG7SE3yqmuopoiIQynJExEREbtQT56IiHNQkiciImJn06ZNIygoCE9PTxo2bEhUVNQN27Zo0QKLxZLu1q5duzTtdu3axUMPPYSPjw+FChWiQYMGHDlyJLtPJdNSUmDXQfO+iq6IiDiWkjwRERE7mjdvHsOHD2fs2LFs2bKFWrVqER4eTlxcXIbtv/vuO2JiYmy37du34+rqymOPPWZrc+DAAZo2bUrVqlVZtWoVf//9N6NHj8bT0zOnTuuWoo/B5STw8oSgMo6ORkQkfyvg6ABERETykkmTJtG/f3/69OkDQGRkJIsXL2bmzJm89NJL6doXL562DOXcuXMpWLBgmiRv1KhRtG3blrfeesu2rWLFG3eXJSUlkZSUZPs9ISHhts8ns1KHalatAK6u2f50IiJyE+rJy6IUq8H6A6dZuPUY6w+cJsVqODokERFxEsnJyWzevJmWLVvatrm4uNCyZUvWr1+fqWPMmDGDLl26UKhQIQCsViuLFy/mrrvuIjw8nFKlStGwYUO+//77Gx4jIiICHx8f2y0wMPCOziszbEVXNB9PRMThlORlwZLtMTR9cyVdP/6DIXO30vXjP2j65kqWbI9xdGgiIuIETp06RUpKCn5+fmm2+/n5ERsbe8v9o6Ki2L59O/369bNti4uL48KFC7zxxhu0bt2aZcuW0bFjRzp16sTq1aszPM6IESOIj4+33Y4ePXpnJ5YJOw+YP0M0H09ExOE0XDOTlmyPYeCXW/hvv11s/GUGfrmF6U/UpXWN0g6JTURE8oYZM2YQGhpKWFiYbZvVagXg4YcfZtiwYQDUrl2bdevWERkZSfPmzdMdx8PDAw8Pj5wJ+l+qrCki4jzUk5cJKVaDcT/sTJfgAbZt437YqaGbIiL5XMmSJXF1deXEiRNptp84cQJ/f/+b7puYmMjcuXN58skn0x2zQIEChISEpNlerVo1p6muGXcGTp4BFxdzTp6IiDiWkrxMiIo+Q0z85Rs+bgAx8ZeJij6Tc0GJiIjTcXd3p169eqxYscK2zWq1smLFCho1anTTfb/55huSkpJ44okn0h2zQYMG7NmzJ832vXv3Ur58efsFfwd2/duLF1zWrK4pIiKOpeGamRB3/sYJ3vW+/OMQgcW9KFusYDZHJCIizmr48OH06tWL+vXrExYWxuTJk0lMTLRV2+zZsydlypQhIiIizX4zZsygQ4cOlChRIt0xn3/+eTp37kyzZs249957WbJkCT/88AOrVq3KiVO6Jc3HExFxLkryMqFUkcx9Lbl4WyyLt8XSMLg4j9QtS5tQf4p4umVzdCIi4kw6d+7MyZMnGTNmDLGxsdSuXZslS5bYirEcOXIEF5e0A2n27NnD77//zrJlyzI8ZseOHYmMjCQiIoJnn32WKlWq8O2339K0adNsP5/M0Hw8ERHnYjEMI19NJEtISMDHx4f4+Hi8vb0ztU+K1aDpmyuJjb+c4bw8C+Dj5Ua10kX4I/oMqa+oRwEXHqjuT6e6ZbinUkkKuGp0rIiIvdzO+3l+ld2v1X294MAR+OxNaBF26/YiInJ7Mvt+rp68THB1sTC2fQgDv9yCBdIkepZ/f77xSCita5Tm2LlLfP/nMb7b8g8HTibyw1/H+eGv45Qs7EGH2gF0qluWkAB9GBERkbzh0mWI/se8r548ERHnoJ68LFiyPYZxP+xMU4SltI8nY9uHpFs+wTAMth2L57stx1j013HOJCbbHqvqX4ROdcvwcO0y+HlrhrqIyO1QT17mZedrtXUXPPw0+BaDTd/Z9dAiIvIf6snLBq1rlKZViD9R0WeIO3+ZUkU8CQsujquLJV1bi8VCzbJFqVm2KCPbVmP13pMs+PMfftkZx+7Y80z4aTdv/LybppV96VSnDA9U96Ogu/45REQkd9F8PBER56OsIotcXSw0qpi+8tnNuBdwoVWIH61C/Ii/eIUftx1nwZZjbDp8ljV7T7Jm70kKubvSJrQ0neqW4e7gErhkkDimWI1MJZgiIiI5ZYeSPBERp6MkL4f5FHSje8PydG9YnkOnElnw5zG++/Mfjp65xPzN/zB/8z8E+HjSoU4ZOtUtQ6VSRYCsDRUVERHJKbblE5TkiYg4Daco9zht2jSCgoLw9PSkYcOGREVF3bDtrFmzsFgsaW6enrlzXltQyUIMa3UXa56/l/kDGtE1rBxFPAtwPP4yH6w6QMtJa3h46u+8MP8vBny5Jd2C7LHxlxn45RaWbI9x0BmIiEh+lpICu/9N8qppjTwREafh8CRv3rx5DB8+nLFjx7JlyxZq1apFeHg4cXFxN9zH29ubmJgY2+3w4cM5GLH9WSwW6gcVJ6JTKBtHtWRat7q0rFaKAi4W/vonnq83/ZPhfqkVc8b9sJMUa76qnyMiIk7g8HG4eBk8PaBCWUdHIyIiqRye5E2aNIn+/fvTp08fQkJCiIyMpGDBgsycOfOG+1gsFvz9/W231AVmM5KUlERCQkKamzPzdHOlXc3SfNKrAX+MvJ9ejcrftL0BxMRfJir6TM4EKCIi8q/UoitVg8HV1bGxiIjINQ5N8pKTk9m8eTMtW7a0bXNxcaFly5asX7/+hvtduHCB8uXLExgYyMMPP8yOHTtu2DYiIgIfHx/bLTAw0K7nkJ1KFvagbvlimWq7dEcMiUlXszkiERGRazQfT0TEOTk0yTt16hQpKSnpeuL8/PyIjY3NcJ8qVaowc+ZMFi5cyJdffonVaqVx48b880/GQxpHjBhBfHy87Xb06FG7n0d2KlUkc/MNZ607TN1Xl/N/X2xi4dZjnL98JZsjE8k7UqwG6w+cZuHWY6w/cFrDn0UyScsniIg4p1xXXbNRo0Y0atTI9nvjxo2pVq0aH374Ia+++mq69h4eHnh4eORkiHYVFlyc0j6exMZf5kYfOwt5uFKikDtHzlxi6Y4TLN1xAndXF+6pXJI2oaVpVc0Pn4JuORq3SG6hyrUit089eSIizsmhSV7JkiVxdXXlxIkTabafOHECf3//TB3Dzc2NOnXqsH///uwI0eFcXSyMbR/CwC+3YIE0iV7qCnnvPFaL8Or+7IxJYMn2WBZvi+HgyURW7I5jxe44CrhYaFypJG1r+PNAdX+KF3J3wJlIVmhNxJyxZHsMA7/cku4LlNTKtdOfqKtET+QGTp2FE6fAYoGqFRwdjYiIXM+hSZ67uzv16tVjxYoVdOjQAQCr1cqKFSsYPHhwpo6RkpLCtm3baNu2bTZG6lita5Rm+hN10/U2+P+nt6F6gA/VA3wY3uou9sVd4KdtMSzZHsvu2PO2RddHfb+dhsHFaRNamvDqfpkeDio5Rz1LOSPFajDuh50Z9pAbmF+ijPthJ61C/JVgi2Rg17+9eEFloJCXY2MREZG0HD5cc/jw4fTq1Yv69esTFhbG5MmTSUxMpE+fPgD07NmTMmXKEBERAcD48eO5++67qVSpEufOnWPixIkcPnyYfv36OfI0sl3rGqVpFeKfqd4di8XCXX5FuMuvCENb3sWBkxdYsj2Wn7bFsON4AusOnGbdgdOMWbidBkHFaVPDn9Y1/Cntc+OrtHqWcoZ6lnJOVPSZdGtPXu/6yrWNKpbIucBEcokdqfPxtD6eiIjTcXiS17lzZ06ePMmYMWOIjY2ldu3aLFmyxFaM5ciRI7i4XKsPc/bsWfr3709sbCzFihWjXr16rFu3jpCQEEedQo5xdbHc1ofNir6FGXRvJQbdW4kjpy/y8/YYftoey19HzxEVfYao6DOM+2EndcsVpW1oaVrX8KdssYK2/dWzlDPUs5RzEi5fYc6GI5lqG3f+xomgSH6moisiIs7LYhhGviojl5CQgI+PD/Hx8Xh7ezs6HIc6du4SS7bH8vO2GDYdPpvmsZplfWhTozSF3F0Zu2hHusQjNcXI7p6l/NSDuP7Aabp+/Mct233V/271LN2muPOXmfn7IWb/cZjzmVxyRK+389L7eeZlx2vVsjfsOwyfRsB9d9vlkCIicguZfT93eE+eOE6Zol482TSYJ5sGcyLhMkt3mEM6o6LP8Pc/8fz9T/wN982JnqX81oO45cjZWzdCPUu349CpRD767SDzN/9D8lUrAJV8C3HyQjIJl67csHItwI7j8dxdoTgWS978ckHkdlxOgoP/rkhUXT15IiJOR0meAODn7UnPRkH0bBTEyfNJLNsZy9yoI2w7lnDDfVLnLLV7/zcCinpR2KMAhTwKUMSzAIXcC1DYswCFPVwp7OFGIQ9Xc7tHAQr/eyvkUQA314yXasxPc9M2Hz7L1JX7+HXPyUy1L+Kp5TAya/uxeKavPsDP22JIXfquTrmiPN2iEvdXLcWynbE3rFyb+vtri3ex9eg53nykJoU89JYpArD3EKRYoURRKKWObhERp6NPLJKObxEPujcsT2GPAgyZu/WW7XfHnmd37Pnbei6PAi5pkr9CHgUo7O7KuoOn8/TcNMMwF9+esnI/6w+eBszz8nBz4fIV6033fXH+X4xqF8LDtQPUu5QBwzBYf/A001cd4Ld9p2zbW1TxZWDzioQFX+uVu1nl2jEPhhCbcJnXF+/ix79j2BN7nsge9ajoWzjHz0nE2ey4bj6e3oZERJyPkjy5ocwur/Ds/ZUoU9SL85evkpiUwoWkK1xISuFC0lUSk65y4fJVLiRdtf1+Pumqbchc0lUrSReSOXUhOdNx5eaqh4Zh8OueOKau3M+WI+cAcHO18EjdsgxoXpHdsQkM/HKL2fa6/VJ7lnwLu3PyQjJD521lzoYjvPJQdUICNBcJwGo1WLYzlumrD/LX0XMAuFigfa0A/q9ZxRu+TreqXBtaxoenZ29hX9wFHp66lomP1qRNaN7oRRa5XTtVWVNExKkpyZMbCgsuTmkfT2LjL2fYq2bB7PEYcv9dWe5RS75qNRPA/yR/iUlX+X3fKeZuPHrLY7yzbDeD76tM00olKXCDYZ/Owmo1WLojlqm/7mfHcXMIrEcBF7o0COSp5hUpU9RcviKoZKGbronYokopZvwezZSV+4g6dIYHp/xGj7vLM7xVFXwK5s9hnMlXrXz/5zEi1xzg4MlEwHxtH68fSP97KlCuRMFbHOHmlWvrBxXnx2eb8sycP9kQfYaBs7fwVLMKvBBexen/7kSyS2qSV01JnoiIU1J1Tbmp1LlxkL5nCbKnumZmq0ym8i3iwUO1AuhYpwzVA7ydagjj1RQrP/4dw7Rf97Mv7gIABd1d6XF3eZ68J/iGvaW3qip67NwlJizexeJtMQAUL+TOi62r8Fi9QFxy6RDWrLqQdJW5UUf45LdoYhPMhLiIZwF6NipP78bB+BbxsOvzXU2x8tbSPXy05iAAd1cozpSude3+PJJ5zvx+Pm3aNCZOnEhsbCy1atViypQphIWFZdi2RYsWrF69Ot32tm3bsnjx4nTbBwwYwIcffsi7777L0KFDMxWPPV8rqxVqPAiJl2D5TLgr+I4OJyIiWaDqmmIXN5uzlF1VLjPTg1iisDuta/iz+O8YTp5PYsbv0cz4PZrKpQrTsW4ZOtQuQ0DRGy/unt2Sr1r5bss/fLDqAEfOXATMBKRP4yD6NAmmWCH3m+5/qzURyxT1Ylr3unTbf4qxi3awP+4CL367jTlRRxn/UHVqBRa15+k4ldMXkpi17hCfrTtEwmVzGYRSRTzod08wXcPKZVthmgKuLoxsW43agUV5/pu/+OOg2ZP6Qfe61CtfPFueM6/K60ujzJs3j+HDhxMZGUnDhg2ZPHky4eHh7Nmzh1KlSqVr/91335GcfG3I+unTp6lVqxaPPfZYurYLFizgjz/+ICAgIFvP4WaOxJgJnocbVCjnsDBEROQm1JMnmZLTH8oy24OYfNXKmr0nWfDnMZbvOmGb62exwN3BJehYpwxtQv1zrCLl5SspzNt4lMjVB2xJcfFC7jzZNJgejcrjnQ1xXEmx8tm6Q0z+ZR8Xkq5isUCXBoE8H16V4rdIJp3Jrf7Gjp65yCe/HWTepqO24jTBJQvxf80q0LFuGTwKuOZYrPvjLjDgy83sj7tAARcLL7erRq/GQU7Vi+ys7Lk0irO+nzds2JAGDRowdepUAKxWK4GBgTzzzDO89NJLt9x/8uTJjBkzhpiYGAoVKmTbfuzYMRo2bMjSpUtp164dQ4cOdUhP3k+rYeArULMK/BB5R4cSEZEsyuz7uZI8cVpZ/TAYf+kKP2+LYcGfx9gQfca23aOAC61C/OhUtwz3VPa94bINd+JC0lVm/3GYj3+L5tSFJMDsXXqqWQW6NSxHQffs7zSPS7jMGz/v5rs/jwHg4+XGcw/cRbeG5Z2+l+Rm/9ZBJQvx4eqDLPrrOCn/roMQWsaHp1tU5IHqjquwmph0lRe+/ZvFf5tDZh+uHUBEp9Ac+bfOrW60NMrtDv92xvfz5ORkChYsyPz58+nQoYNte69evTh37hwLFy685TFCQ0Np1KgRH330kW2b1WqlZcuWPPzwwwwZMoSgoKCbJnlJSUkkJSXZfk9ISCAwMNAur9XbM2HKF9ClHbz53B0dSkREskjDNSXXu1XVw//y8XKjS1g5uoSV45+zF1m49TjfbfmHAycT+fHvGH78O4YShdxp/+/8vZplfe645yX+4hVmrTvEp+uiOXfxCmAOpRzYoiKP1iuLp1vO9S6V8vZkUufadG1YjjELd7ArJoHRC3fwVdRRxj9cnfpBzjmk8EYf/GPiLzPg397cVE0rlWRgi4o0rljC4b1mhTwKMLVrHeqWK8aEn3axcOtxdsecZ/oTdamQC5ZZyOne+RSrwbgfdubppVEATp06RUpKCn5+fmm2+/n5sXv37lvuHxUVxfbt25kxY0aa7W+++SYFChTg2WefzVQcERERjBs3LvOBZ4Eqa4qIOD8leeLUbjU37UbKFivIoHsr8XSLimw7Fs+CP4/xw1/HOXUhmVnrDjFr3SEq+BaiU50yPFy7DIHF01dgvNmH4NMXzHmAn68/zIUkc15YhZKFePreSjxcOyBbegszq0FQcX4Y3ISvoo4wcekedsYk8GjkejrVKcNLbapSyjtzS2PkhJt98L9e2xr+DGxRidCyPjkSV2ZZLBaebBpMaBkfBs3Zwp4T581lFh6rResa/o4O74bsOWTyeoZhkHDpKjEJl4iNv2zeEsyfu2IT0jxfun3JvUuj2NOMGTMIDQ1NU6Rl8+bNvPfee2zZsiXTX26MGDGC4cOH235P7cmzh53XrZEnIiLOScM1Jd+4kmLl932n+O7PYyzbEUvS1WuLjocFFadj3TK0DS2Nj5fbDT8EP3t/ZfaduMCcqMO2eWFV/Ysw6N5KtA0t7XQ9EKcvJDFx6R7mbTqKYWAucH9/ZXo3CXJoIgrmshILtx5j2Nd/3bLtV/3vdvoP/nEJlxk850+iDplDhQc0r8hzD9zldMss3O6QyRSrwakLScTGXyYm/jInEq7/eYkTCUnExF+y/b+4Xe91qc3Dtctkqq0zvp/fyXDNxMREAgICGD9+PEOGDLFtnzx5MsOHD8fF5drfUkpKCi4uLgQGBnLo0KFbxmWv1+pMPNTpYN7fsRgK33qFEhERsSPNybsBZ/xQIDnv/OUrLNkey4I/j7H+4GlS/xe4u7pQPcCbP/9dTPtmapb14Zn7KnN/1VJOv2zB1qPnGLtwO3/9Ew9ApVKFGfdQdZpUKpkjz28YBodPX2TbsXi2H4vn73/i2X48nvP/Vse8lax88HekKylW3vh5NzN+jwagccUSvN+1DiULO8cyCylWg6Zvrrxpj1rRgm78X7MKnEhISpPIxZ1Pss2JvJViBd3w8/aktI8n/j6e+Ht7kZh8hY/WRN9y36wk9M76ft6wYUPCwsKYMmUKYM6nK1euHIMHD75p4ZVZs2YxYMAAjh07RokS116D06dPExMTk6ZteHg4PXr0oE+fPlSpUuWWMdnrtfp9M3R/DsoHwJrZt30YERG5TZqTJ3ITRTzdeKx+II/VDyQm/hILtx5nwZZj7Dlx/pYJnrurhY961Kd5FV+HzwvLrNqBRVnwdBO+2XyUN5fsYX/cBbp/soF2oaUZ2a6abTF2uPO5WoZhcOSMmdBt+yfeltglZJDQFXCxcDUTicON1hN0Nm6uLox+MIQ65Yrywvy/WXfgNA++/zsfPFGXuuWKOTo8oqLP3DTBAzh38QpvLtmT4WMuFvPfwkzc/v3pYyZzqUmdn7dnhnNRU6wGP/wVc9OlUfx9zL+33G748OH06tWL+vXrExYWxuTJk0lMTKRPnz4A9OzZkzJlyhAREZFmvxkzZtChQ4c0CR5AiRIl0m1zc3PD398/UwmePe08YP7UUE0REeemJE/yvdI+XgxoXpH/a1aBuRuPMOK77Tdtn5xi4OHmmmsSvFQuLhY6NyhH6+qlmbR8D1/8cZjF22JYuTuOwfdVot89wfy6Oy5Lc7UMw+DomUv8feycLZnb9k/GCZ27qwvVShehRhkfapb1oUYZHyr6Fubet1fluQ/+D9YMoIpfEQZ8uZkDJxPp/OF6xjwYwhN3l8/xv5vEpKtsOXKWjdFnWLwt5tY7APXLF6VeUHFK2xI5L/y9PSlZ2P22h5+6ulgY2z6EgV9uwULGS6OMbR/idEOeb0fnzp05efIkY8aMITY2ltq1a7NkyRJbMZYjR46kGXoJsGfPHn7//XeWLVvmiJAzTfPxRERyBw3XFLnOwq3HGDJ36y3b5Zbhgzez83gCryzaYZtD5lvYnZMXktO1S/3I/UH3ulQP8DF76I7Fs+3YObYfSyD+0pV0+7i7ulC1dBFCy/gQWsZM6O7yK4J7gfQJQmbXRMyNLiRd5YX5f/HTtlgAOtYpw4SOoXi5Z1/V1bOJyWw8dIaNh84QFX2G7ccTMj3MMlV2zoHMD+vkOSN7vVbhT8LugzBzAtzfyI4BiohIpmi4pshtyOywwNwyfPBmQgK8mfd/d7Por+O89uPODBM8uJZ4PT07fbEOuJbQ1SjjQ81bJHQZaV2jNNOfqJvug7+/Hao9OlphjwJM61aXT36L5o0lu1nw5zF2xSQQ+UQ9gkoWuvUBMiE2/jJRh84QFX2aqOgz7D1xIV2bMkW9CAsuTv2gYry7fC+nLyQ7rOc0q0ujiPO4nAz7D5v31ZMnIuLclOSJXCcsuDilfTzz3PDBG7FYLDxcuwxFPN3oO2vjTdsagKsLhJT2IbSsj62XLisJ3Y3k5Q/+FouF/s0qEFrWh8Fz/mR37HnaT/2dSY/XplWIX5bmQBqGwaHTF9kYfYYN0WZv3ZEzF9O1q1SqMA2CitMwuDgNgounmXNZopC7w4dM3u7SKOI4KSnw7VK4mmJW1PR1/BRTERG5CSV5ItfJT/OGrnf+cvohlxmZ+EgtOtUrmy0x5PUP/ndXKMHiZ5vy9OwtbD58lv6fb6J1DT+2HoknNiHjoYtWq8Hu2PO2oZdRh85w8nxSmuO6WKB6gA8NgooTFlycBkHFKHGTap55uedUssfPa2DcVIg5af5+4SI07QZjB0ObZo6NTUREMqY5eSIZyK7Fop3V+gOn6frxH7dslxvWq3N2yVetTPhpF7PWHbppu9Ay3hw+fTFdERt3VxdqBxalQXAxwoJLULdcUYp4umU5jjutouoM9H6eebf7Wv28BgaO5cbrKo5ToicikpM0J0/kDuTl4YMZyW/DVB3JvYC5zML3fx7jXAZFa1JtO5YAQCF3V+oFFScsyEzqapb1yXCJgqzK6z2ncudSUswevIzeEwzM94VxU+GBJuCafbWERETkNijJE7mB/PQhOL8OU3WUqOgzN03wUk3oUIPHGwTe9rIFInciatu1IZoZMTAfj9oGjWrnVFQiIpIZ+uQgIsC1uVr+Pmkrh/r7eObqZQycUdz5my9InqqQZwEleOIwcaft205ERHKOevJExCa/DVN1lPy0VIfkXqUyOZAhs+1ERCTnKMkTkTTy0zBVR9EcSMkNwkKhtC/Ensx4Xp4F8Pc124mIiHPROCARkRyWOgcSrs15TKU5kOIsXF3NZRLgJn+ng1V0RUTEGSnJExFxAM2BlNygTTNzmQR/37Tb/X21fIKIiDPTcE0REQfRHEjJDdo0M5dJiNpmFlkpVcIcoqkePBER56UkT0TEgTQHUnIDV1ctkyAikptouKaIiIiIiEgeoiRPREREREQkD8l3wzUNwywEnZCQ4OBIRETkTqS+j6e+r8uN6donIpI3ZPbal++SvPPnzwMQGBjo4EhERMQezp8/j4+Pj6PDcGq69omI5C23uvZZjHz2FajVauX48eMUKVIEi+X2KtglJCQQGBjI0aNH8fb2tnOEzis/nnd+PGfIn+etc85952wYBufPnycgIAAXF80+uBld+25ffjxvnXP+OGfIn+ed2885s9e+fNeT5+LiQtmyZe1yLG9v71z5x3Gn8uN558dzhvx53jrn3EU9eJmja9+dy4/nrXPOP/Ljeefmc87MtU9ffYqIiIiIiOQhSvJERERERETyECV5t8HDw4OxY8fi4eHh6FByVH487/x4zpA/z1vnLHJz+fXvJT+et845/8iP551fzjnfFV4RERERERHJy9STJyIiIiIikocoyRMREREREclDlOSJiIiIiIjkIUryRERERERE8hAlebdh2rRpBAUF4enpScOGDYmKinJ0SNkmIiKCBg0aUKRIEUqVKkWHDh3Ys2ePo8PKUW+88QYWi4WhQ4c6OpRsd+zYMZ544glKlCiBl5cXoaGhbNq0ydFhZauUlBRGjx5NcHAwXl5eVKxYkVdffZW8VJNqzZo1tG/fnoCAACwWC99//32axw3DYMyYMZQuXRovLy9atmzJvn37HBOsOC1d+3Tty6vy27UvP1z3QNc+JXlZNG/ePIYPH87YsWPZsmULtWrVIjw8nLi4OEeHli1Wr17NoEGD+OOPP1i+fDlXrlzhgQceIDEx0dGh5YiNGzfy4YcfUrNmTUeHku3Onj1LkyZNcHNz4+eff2bnzp288847FCtWzNGhZas333yT6dOnM3XqVHbt2sWbb77JW2+9xZQpUxwdmt0kJiZSq1Ytpk2bluHjb731Fu+//z6RkZFs2LCBQoUKER4ezuXLl3M4UnFWuvbp2pdX5cdrX3647oGufRiSJWFhYcagQYNsv6ekpBgBAQFGRESEA6PKOXFxcQZgrF692tGhZLvz588blStXNpYvX240b97cGDJkiKNDylYvvvii0bRpU0eHkePatWtn9O3bN822Tp06Gd27d3dQRNkLMBYsWGD73Wq1Gv7+/sbEiRNt286dO2d4eHgYX331lQMiFGeka5+ufXlVfrz25bfrnmHkz2ufevKyIDk5mc2bN9OyZUvbNhcXF1q2bMn69esdGFnOiY+PB6B48eIOjiT7DRo0iHbt2qX5987LFi1aRP369XnssccoVaoUderU4eOPP3Z0WNmucePGrFixgr179wLw119/8fvvv9OmTRsHR5YzoqOjiY2NTfN37uPjQ8OGDfPN+5rcnK59uvblZfnx2pffr3uQP659BRwdQG5y6tQpUlJS8PPzS7Pdz8+P3bt3OyiqnGO1Whk6dChNmjShRo0ajg4nW82dO5ctW7awceNGR4eSYw4ePMj06dMZPnw4I0eOZOPGjTz77LO4u7vTq1cvR4eXbV566SUSEhKoWrUqrq6upKSk8Prrr9O9e3dHh5YjYmNjATJ8X0t9TPI3Xft07cvL8uO1L79f9yB/XPuU5EmmDRo0iO3bt/P77787OpRsdfToUYYMGcLy5cvx9PR0dDg5xmq1Ur9+fSZMmABAnTp12L59O5GRkXn2Qgfw9ddfM3v2bObMmUP16tXZunUrQ4cOJSAgIE+ft4hkjq59eVt+vPbpupc/aLhmFpQsWRJXV1dOnDiRZvuJEyfw9/d3UFQ5Y/Dgwfz444/8+uuvlC1b1tHhZKvNmzcTFxdH3bp1KVCgAAUKFGD16tW8//77FChQgJSUFEeHmC1Kly5NSEhImm3VqlXjyJEjDoooZzz//PO89NJLdOnShdDQUHr06MGwYcOIiIhwdGg5IvW9Kz++r0nm6Nqna5+ufXlLfr/uQf649inJywJ3d3fq1avHihUrbNusVisrVqygUaNGDows+xiGweDBg1mwYAErV64kODjY0SFlu/vvv59t27axdetW261+/fp0796drVu34urq6ugQs0WTJk3SlQjfu3cv5cuXd1BEOePixYu4uKR9K3R1dcVqtTooopwVHByMv79/mve1hIQENmzYkGff1yRrdO3TtU/Xvrwlv1/3IH9c+zRcM4uGDx9Or169qF+/PmFhYUyePJnExET69Onj6NCyxaBBg5gzZw4LFy6kSJEitnHKPj4+eHl5OTi67FGkSJF08y4KFSpEiRIl8vR8jGHDhtG4cWMmTJjA448/TlRUFB999BEfffSRo0PLVu3bt+f111+nXLlyVK9enT///JNJkybRt29fR4dmNxcuXGD//v2236Ojo9m6dSvFixenXLlyDB06lNdee43KlSsTHBzM6NGjCQgIoEOHDo4LWpyKrn269uVV+fHalx+ue6Brn5ZQuA1TpkwxypUrZ7i7uxthYWHGH3/84eiQsg2Q4e3TTz91dGg5Kj+UkTYMw/jhhx+MGjVqGB4eHkbVqlWNjz76yNEhZbuEhARjyJAhRrly5QxPT0+jQoUKxqhRo4ykpCRHh2Y3v/76a4b/j3v16mUYhllKevTo0Yafn5/h4eFh3H///caePXscG7Q4HV37dO3Lq/LbtS8/XPcMQ9c+i2HkseXtRURERERE8jHNyRMREREREclDlOSJiIiIiIjkIUryRERERERE8hAleSIiIiIiInmIkjwREREREZE8REmeiIiIiIhIHqIkT0REREREJA9RkiciIiIiIpKHKMkTEZtVq1ZhsVg4d+6co0MRERHJEbr2SV6kJE9ERERERCQPUZInIiIiIiKShyjJE3EiVquViIgIgoOD8fLyolatWsyfPx+4Npxk8eLF1KxZE09PT+6++262b9+e5hjffvst1atXx8PDg6CgIN555500jyclJfHiiy8SGBiIh4cHlSpVYsaMGWnabN68mfr161OwYEEaN27Mnj17svfERUQk39K1T8T+lOSJOJGIiAg+//xzIiMj2bFjB8OGDeOJJ55g9erVtjbPP/8877zzDhs3bsTX15f27dtz5coVwLxAPf7443Tp0oVt27bxyiuvMHr0aGbNmmXbv2fPnnz11Ve8//777Nq1iw8//JDChQuniWPUqFG88847bNq0iQIFCtC3b98cOX8REcl/dO0TyQaGiDiFy5cvGwULFjTWrVuXZvuTTz5pdO3a1fj1118NwJg7d67tsdOnTxteXl7GvHnzDMMwjG7duhmtWrVKs//zzz9vhISEGIZhGHv27DEAY/ny5RnGkPocv/zyi23b4sWLDcC4dOmSXc5TREQkla59ItlDPXkiTmL//v1cvHiRVq1aUbhwYdvt888/58CBA7Z2jRo1st0vXrw4VapUYdeuXQDs2rWLJk2apDlukyZN2LdvHykpKWzduhVXV1eaN29+01hq1qxpu1+6dGkA4uLi7vgcRURErqdrn0j2KODoAETEdOHCBQAWL15MmTJl0jzm4eGR5mJ3u7y8vDLVzs3NzXbfYrEA5pwJERERe9K1TyR7qCdPxEmEhITg4eHBkSNHqFSpUppbYGCgrd0ff/xhu3/27Fn27t1LtWrVAKhWrRpr165Nc9y1a9dy11134erqSmhoKFarNc08BxEREUfRtU8ke6gnT8RJFClShOeee45hw4ZhtVpp2rQp8fHxrF27Fm9vb8qXLw/A+PHjKVGiBH5+fowaNYqSJUvSoUMHAP73v//RoEEDXn31VTp37sz69euZOnUqH3zwAQBBQUH06tWLvn378v7771OrVi0OHz5MXFwcjz/+uKNOXURE8ild+0SyiaMnBYrINVar1Zg8ebJRpUoVw83NzfD19TXCw8ON1atX2yaG//DDD0b16tUNd3d3IywszPjrr7/SHGP+/PlGSEiI4ebmZpQrV86YOHFimscvXbpkDBs2zChdurTh7u5uVKpUyZg5c6ZhGNcmn589e9bW/s8//zQAIzo6OrtPX0RE8iFd+0Tsz2IYhuHIJFNEMmfVqlXce++9nD17lqJFizo6HBERkWyna5/I7dGcPBERERERkTxESZ6IiIiIiEgeouGaIiIiIiIieYh68kRERERERPIQJXkiIiIiIiJ5iJI8ERERERGRPERJnoiIiIiISB6iJE9ERERERCQPUZInIiIiIiKShyjJExERERERyUOU5ImIiIiIiOQh/w8K9NtZTvWeYwAAAABJRU5ErkJggg==",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# viz: loss/accuracy over epochs and a small confusion-ish view of predictions\n",
"eps_, losses_, accs_ = zip(*hist)\n",
"fig, ax = plt.subplots(1, 2, figsize=(9, 3.2))\n",
"ax[0].plot(eps_, losses_, marker=\"o\"); ax[0].set_xlabel(\"epoch\"); ax[0].set_ylabel(\"batch loss\")\n",
"ax[0].axhline(np.log(10), ls=\":\", c=\"#888\", label=\"ln(10) init\"); ax[0].legend(); ax[0].set_title(\"loss\")\n",
"ax[1].plot(eps_, accs_, marker=\"o\", color=\"#1E40FF\"); ax[1].set_xlabel(\"epoch\"); ax[1].set_ylabel(\"test acc\")\n",
"ax[1].set_title(\"test accuracy\")\n",
"plt.tight_layout(); plt.show()"
]
},
{
"cell_type": "markdown",
"id": "a98a74bc",
"metadata": {},
"source": [
"> **Interpretation.** Loss falls from $\\ln 10$ and test accuracy climbs into the low-to-mid 80s on this subset (the full 60k set with more epochs reaches the high 80s for an MLP; the gap is data and epochs, not the algorithm). Every gradient driving this came from the six-line backward pass you verified against finite differences.\n"
]
},
{
"cell_type": "markdown",
"id": "fa51960b",
"metadata": {},
"source": [
"**Experiment log.** Expected numbers under the committed seed (yours land within a point or two; CPU/BLAS shifts the last digits).\n",
"\n",
"| Run | subset | epochs | lr | FAST | final test acc |\n",
"|---|---|---|---|---|---|\n",
"| NumPy MLP | 12000 | 12 | 0.2 | off | ~0.83 |\n",
"| NumPy MLP (smoke) | 4000 | 3 | 0.2 | on | ~0.75 |\n",
"| lr=50 (Part 5 failure demo) | 12000 | 40 steps | 50 | either | diverges / NaN |\n",
"\n",
"The log doubles as your expected-value reference: if the full run prints accuracy far from ~0.83, something is wrong before you tune anything. The smoke run trains on a smaller subset for fewer epochs, so its accuracy is lower by design.\n"
]
},
{
"cell_type": "markdown",
"id": "3ebf2631",
"metadata": {},
"source": [
"### A second failure to recognize: the learning rate too high\n",
"\n",
"The deliberate `=`-vs-`+=` failure was in Part 3. Here is the other failure every beginner meets: a learning rate so large the update overshoots, so the loss climbs instead of falling and stays high (left running long enough, it overflows to NaN). We run it briefly, watch it diverge, then show the fix is a smaller step.\n"
]
},
{
"cell_type": "code",
"execution_count": 30,
"id": "0215ac0f",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:17:32.625854Z",
"iopub.status.busy": "2026-06-10T19:17:32.625770Z",
"iopub.status.idle": "2026-06-10T19:17:38.084577Z",
"shell.execute_reply": "2026-06-10T19:17:38.084187Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"loss over the first few steps: ['2.31', '19.41', '24.82', '22.45', '23.75', '26.34']\n",
"loss went from 2.31 to 24.73 (climbed instead of fell; non-finite: False)\n",
"[ ok ] reproduced the lr-too-high divergence\n"
]
}
],
"source": [
"# scale-up flag pattern: this short broken run is ON by default because it is the lesson, and it is cheap\n",
"P_bad = init_params()\n",
"bad_losses = []\n",
"g = np.random.default_rng(SEED)\n",
"for i in range(40): # only 40 steps; that is enough to see it blow up\n",
" idx = g.integers(0, len(Xtr), size=128)\n",
" logits, cache = mlp_forward(Xtr[idx], P_bad[\"W1\"], P_bad[\"b1\"], P_bad[\"W2\"], P_bad[\"b2\"])\n",
" loss, dlogits = softmax_xent(logits, ytr[idx])\n",
" grads = mlp_backward(dlogits, cache, P_bad[\"W2\"])\n",
" for k in P_bad:\n",
" P_bad[k] -= 50.0 * grads[k] # lr=50 is absurd on purpose\n",
" bad_losses.append(loss)\n",
"print(f\"loss over the first few steps: {[f'{l:.2f}' for l in bad_losses[:6]]}\")\n",
"print(f\"loss went from {bad_losses[0]:.2f} to {bad_losses[-1]:.2f} (climbed instead of fell; non-finite: {not np.isfinite(bad_losses[-1])})\")\n",
"assert not np.isfinite(bad_losses[-1]) or bad_losses[-1] > 2 * bad_losses[0], \\\n",
" \"lr=50 should make the loss climb sharply or go non-finite; if it trained, the data is too easy\"\n",
"print(\"[ ok ] reproduced the lr-too-high divergence\")"
]
},
{
"cell_type": "code",
"execution_count": 31,
"id": "04aa165b",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:17:38.085808Z",
"iopub.status.busy": "2026-06-10T19:17:38.085721Z",
"iopub.status.idle": "2026-06-10T19:17:44.364944Z",
"shell.execute_reply": "2026-06-10T19:17:44.364183Z"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAEiCAYAAAAPh11JAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAW25JREFUeJzt3Xd4FFXbx/HvpveEkJAQShKqgHSEB5EeCb2J1EcDKrwiKN3yKE0RFBtSBBXBhopKswCCQAClNxFBEAwQSkJPSC973j8OWViSkE3dlPtzXXslmZ2dvWdnk/3lnDNnDEophRBCCCGEyJGNtQsQQgghhCgpJDgJIYQQQlhIgpMQQgghhIUkOAkhhBBCWEiCkxBCCCGEhSQ4CSGEEEJYSIKTEEIIIYSFJDgJIYQQQlhIgpMQQgghhIUkOAmRjU8//RSDwcDp06etXco9hYeHYzAYCA8Pt3Yp4g7Tpk3DYDBw5cqVHNcNCgpi6NCheXqeoKAgunfvnqfHCiFyT4KTEMKqduzYwbRp07hx40aJ2K4QomyT4CRECdemTRsSExNp06aNtUvJkx07djB9+vRCCU6Fsd3CcPz4cT7++GNrlyGEsIAEJyGKmfj4+Fytb2Njg5OTEzY2xePXObf1C3B0dMTe3t7aZQghLFA8/tIKUYKsW7eO1q1b4+rqiru7O926deOvv/4yW+fw4cMMHTqUatWq4eTkhL+/P0888QRXr141Wy9jHMzRo0cZPHgw5cqV46GHHgJuj1357bffaN68OU5OTlSrVo3PP//cbBtZjXFq164d999/P0ePHqV9+/a4uLhQqVIlZs+enWl/zpw5Q8+ePXF1daVChQqMGzeOX375xaJxU/eq35LXYNq0aUyaNAmA4OBgDAZDpnFlX375JU2bNsXZ2Rlvb28GDhxIZGRkjnXda7tpaWm89tprVK9eHUdHR4KCgvjf//5HcnJypm198MEH1KtXD0dHRwICAhg1alSuWrFu3LjB0KFD8fLywtPTk2HDhpGQkGC2TlZjnA4fPkzbtm1xdnamcuXKzJgxg6VLl2Y77i6n98ndUlNT8fb2ZtiwYZnui42NxcnJiYkTJ5qWzZs3j3r16uHi4kK5cuVo1qwZX331VY77n9Pjzpw5wzPPPEPt2rVxdnamfPnyPProo5n2MWPM4e+//8748ePx9fXF1dWVPn36cPny5UzPa8nvqRB5YWftAoQoSb744gvCwsIIDQ3lzTffJCEhgYULF/LQQw9x8OBBgoKCANi4cSP//vsvw4YNw9/fn7/++ouPPvqIv/76i127dmEwGMy2++ijj1KzZk1mzpyJUsq0/OTJk/Tr148nn3ySsLAwlixZwtChQ2natCn16tW7Z63Xr1+nc+fO9O3bl/79+/P999/zwgsvUL9+fbp06QLo1qEOHTpw8eJFxowZg7+/P1999RVbtmzJ1euSVf2WvAZ9+/blxIkTfP3117z33nv4+PgA4OvrC8Drr7/O5MmT6d+/P0899RSXL19m3rx5tGnThoMHD+Ll5ZVlPTlt96mnnuKzzz6jX79+TJgwgd27dzNr1iyOHTvGqlWrTNuZNm0a06dPJyQkhJEjR3L8+HEWLlzI3r17+f333y1qJerfvz/BwcHMmjWLAwcOsHjxYipUqMCbb76Z7WPOnz9P+/btMRgMvPTSS7i6urJ48WIcHR2zXD8v7xN7e3v69OnDypUr+fDDD3FwcDDdt3r1apKTkxk4cCAAH3/8Mc899xz9+vVjzJgxJCUlcfjwYXbv3s3gwYOz3Q9LHrd371527NjBwIEDqVy5MqdPn2bhwoW0a9eOo0eP4uLiYrbNZ599lnLlyjF16lROnz7NnDlzGD16NMuXLzetY+nvqRB5ooQQWVq6dKkCVEREhFJKqZs3byovLy81fPhws/WioqKUp6en2fKEhIRM2/v6668VoLZt22ZaNnXqVAWoQYMGZVo/MDAw0/qXLl1Sjo6OasKECaZlW7ZsUYDasmWLaVnbtm0VoD7//HPTsuTkZOXv768eeeQR07J33nlHAWr16tWmZYmJieq+++7LtM2s3Kt+S1+Dt956y+x1znD69Glla2urXn/9dbPlf/75p7Kzs8u0/G7ZbffQoUMKUE899ZTZ8okTJypAbd68WSmlX2sHBwfVqVMnlZ6eblpv/vz5ClBLliy55/NnvDZPPPGE2fI+ffqo8uXLmy0LDAxUYWFhpp+fffZZZTAY1MGDB03Lrl69qry9vTPtk6Xvk6z88ssvClA//vij2fKuXbuqatWqmX7u1auXqlev3j23lRVLHpfV+2Tnzp2Z3r8Zv48hISHKaDSalo8bN07Z2tqqGzduKKVy93sqRF5IV50QFtq4cSM3btxg0KBBXLlyxXSztbWlRYsWZq00zs7Opu+TkpK4cuUK//nPfwA4cOBApm0//fTTWT5n3bp1ad26telnX19fateuzb///ptjvW5ubvz3v/81/ezg4EDz5s3NHrt+/XoqVapEz549TcucnJwYPnx4jtvPqf7cvgZ3W7lyJUajkf79+5u93v7+/tSsWTPXrWIZ1q5dC8D48ePNlk+YMAGAn3/+GYBff/2VlJQUxo4dazZ+bPjw4Xh4eJjWy8ndr03r1q25evUqsbGx2T5m/fr1tGzZkkaNGpmWeXt7M2TIkCzXz+v7pEOHDvj4+Ji11ly/fp2NGzcyYMAA0zIvLy/OnTvH3r1777m9u1nyuDvfJ6mpqVy9epUaNWrg5eWV5ftkxIgRZi22rVu3Jj09nTNnzgC5+z0VIi+kq04IC/3zzz+A/rDJioeHh+n7a9euMX36dL755hsuXbpktl5MTEymxwYHB2e5zapVq2ZaVq5cOa5fv55jvZUrV87UJViuXDkOHz5s+vnMmTNUr14903o1atTIcft3yqr+3L4Gd/vnn39QSlGzZs0s78/rYOozZ85gY2OTaR/9/f3x8vIyfQBnfK1du7bZeg4ODlSrVs10f07uPoblypUDdEC58z1zd40tW7bMtDy745LX94mdnR2PPPIIX331FcnJyTg6OrJy5UpSU1PNgtMLL7zAr7/+SvPmzalRowadOnVi8ODBtGrV6p7bt+RxiYmJzJo1i6VLl3L+/Hmzruqs3if3ej0hd7+nQuSFBCchLGQ0GgE9fsLf3z/T/XZ2t3+d+vfvz44dO5g0aRKNGjXCzc0No9FI586dTdu5053/dd/J1tY2y+V3frhkJz+Pza2s6s/ta3A3o9GIwWBg3bp1We6Lm5tbvmq+OywWlqI4Dvl5joEDB/Lhhx+ybt06evfuzbfffst9991Hw4YNTevUqVOH48eP89NPP7F+/XpWrFjBBx98wJQpU5g+fXq227bkcc8++yxLly5l7NixtGzZEk9PTwwGAwMHDszyfZLTvubm91SIvJB3kBAWql69OgAVKlQgJCQk2/WuX7/Opk2bmD59OlOmTDEtz/hPuDgJDAzk6NGjKKXMgsTJkyfztd3cvAbZBZjq1aujlCI4OJhatWrluobsthsYGIjRaOSff/6hTp06puXR0dHcuHGDwMBA03qg51iqVq2aab2UlBQiIiLu+R7Ir8DAwCyPQX6PS1batGlDxYoVWb58OQ899BCbN2/m5ZdfzrSeq6srAwYMYMCAAaSkpNC3b19ef/11XnrpJZycnLLdfk6P+/777wkLC+Odd94xPSYpKSnP829Z+nsqRF7JGCchLBQaGoqHhwczZ84kNTU10/0Zp0Rn/Ed893/7c+bMKfQacys0NJTz58/zww8/mJYlJSXlezLG3LwGrq6uAJk+KPv27YutrS3Tp0/PtB2lVKapHSzdbteuXbOs5d133wWgW7duAISEhODg4MDcuXPNnv+TTz4hJibGtF5hCA0NZefOnRw6dMi07Nq1ayxbtqzAn8vGxoZ+/frx448/8sUXX5CWlmbWTQdkeq0dHByoW7cuSqksfxdy8zhbW9tMx3fevHmkp6fnaX8s/T0VIq+kxUkIC3l4eLBw4UIee+wxmjRpwsCBA/H19eXs2bP8/PPPtGrVivnz5+Ph4UGbNm2YPXs2qampVKpUiQ0bNhAREWHtXcjk//7v/5g/fz6DBg1izJgxVKxYkWXLlplaEPLanZWb16Bp06YAvPzyywwcOBB7e3t69OhB9erVmTFjBi+99BKnT5+md+/euLu7ExERwapVqxgxYoTZPEOWbrdhw4aEhYXx0UcfcePGDdq2bcuePXv47LPP6N27N+3btwf0AOuXXnqJ6dOn07lzZ3r27Mnx48f54IMPeOCBB8wG3he0559/ni+//JKHH36YZ5991jQdQdWqVbl27VqBdzMOGDCAefPmMXXqVOrXr2/WEgfQqVMn/P39adWqFX5+fhw7doz58+fTrVs33N3ds92uJY/r3r07X3zxBZ6entStW5edO3fy66+/Ur58+Tzti6W/p0LkWZGfxydECXH3dAQZtmzZokJDQ5Wnp6dycnJS1atXV0OHDlX79u0zrXPu3DnVp08f5eXlpTw9PdWjjz6qLly4oAA1depU03oZp6xfvnw50/MHBgaqbt26ZVretm1b1bZtW7N6yGI6gqxOAw8LC1OBgYFmy/7991/VrVs35ezsrHx9fdWECRPUihUrFKB27dp1z9foXvVb+hoopdRrr72mKlWqpGxsbDK95itWrFAPPfSQcnV1Va6uruq+++5To0aNUsePH79nbffabmpqqpo+fboKDg5W9vb2qkqVKuqll15SSUlJmbYxf/58dd999yl7e3vl5+enRo4cqa5fv57jc2f32mT1vrp7OgKllDp48KBq3bq1cnR0VJUrV1azZs1Sc+fOVYCKiooye6wl75N7MRqNqkqVKgpQM2bMyHT/hx9+qNq0aaPKly+vHB0dVfXq1dWkSZNUTEzMPbdryeOuX7+uhg0bpnx8fJSbm5sKDQ1Vf//9d6bXJON127t3r9lzZPX+z1ie0++pEHlhUKoQRooKIUq0OXPmMG7cOM6dO0elSpWsXY64ZezYsXz44YfExcVlO0haCFG4JDgJUcYlJiZmmnOpcePGpKenc+LECStWVrbdfVyuXr1KrVq1aNKkCRs3brRiZUKUbTLGSYgyrm/fvlStWpVGjRoRExPDl19+yd9//10oA5GF5Vq2bEm7du2oU6cO0dHRfPLJJ8TGxjJ58mRrlyZEmSbBSYgyLjQ0lMWLF7Ns2TLS09OpW7cu33zzTaYzq0TR6tq1K99//z0fffQRBoOBJk2a8Mknn9CmTRtrlyZEmSZddUIIIYQQFpJ5nIQQQgghLCTBSQghhBDCQmV+jJPRaOTChQu4u7sX2bWrhBBCCFF8KKW4efMmAQEB2Njcu02pzAenCxcuUKVKFWuXIYQQQggri4yMpHLlyvdcp8wHp4xp/yMjI/Hw8LByNUIIIYQoarGxsVSpUuWelxDKUOaDU0b3nIeHhwQnIYQQogyzZMiODA4XQgghhLCQBCchhBBCCAtJcBJCCCGEsJAEJyGEEEIIC0lwEkIIIYSwUJk/q04IIfLLmJxM8p9/knruHHb+/thXroydvz8GO/kTK0RpI7/VQgiRSxlBKXH/fpL27yf5yBFUSor5Sra22FWsiH3lyjpI3fpqX7kydpUqYePsXKg1qvR00q9fx2Bri8HeHoODA9jbyxUShMgnCU6iQKjUVBJ27iQ1MrJQn8fGxQX7KlWwr1IFW19fDDlMjZ8TpRTGmBhSIyNJPXsWg709Lg89hI2LSwFVrCX/8w83V6wg+cQJ/SHm6Gj6MMv0NeNmb4/ByQkbNzdsXF3Nvhoyvjo4yAdhETAmJZF85IgpKCX9+SekppqtY+vjg0O1aqRdukTahQuolBTSzp0j7dw5ErPYpm358vq9XK0aDtWrm2625crluj6Vnk7q6dMkHztG8t9/k3zsGCknTqASs3jme7zfbNzdcfnPf3Dt0AH7qlVzXUdhMiYnY7x+nfTr10m/dk1/vfN27RoYjXg8+iiubdpYu1xRihmUUsraRVhTbGwsnp6exMTEyASYuaSMRpIOHSJu3TriN23CGBtbpM9vcHTU/8VXrar/k69aNctQdXc4Sj13jtSzZ0mLjCQ1MhLjzZvm23Vxwa1TJ9x79cLx/vvzHExUSgrxmzcT8913JP/xR773N0t2dmaBytbLC/sqVbCrUgWHwEDsqlTBvlKlUtNlZExOJuWff0j+6y+Sjx4l7cIFyM2fMIMhU2DA3h4bR8fbgeKOIGGMiyPpwIFsg5JT06Y4N22KU9Om2FetanqvKKOR9MuX9XvtVnhKPXeOtPPnST137p6/K7be3tjfEaQcqlfHoVo1bNzc9LbT0m6HpFtBKeXECVRSUpb7m6vX5w721avj2qEDru3b41CzZqEFdKUUxuvXSYuKIi06Wn/NuF2+bApJKj7e4m26tG5N+UmTsA8IKJSaizOVloYxLg5jfLz+esdNJSdjX7kyDjVqYOvtbe1Si5XcZAEJThKcci3l5ElurltH/C+/kBYVZVpu6+ODU5MmGGxtC+2502NjdeA5fx7S07Ndz+DoiH2VKhgcHLIMR3ez9fPDvkoV/Yf7jlYz+2rVcO/ZE/euXS3+Q5N68SI3V67k5po1+r9gAFtbXNu3x7V9e0C30KmUFPNbxrI7vyYm6j+AGX8Eb33NzYcItrbYBQToUFm16u1blSrYli+f6Y/rvW6kp2Nbrhy25ctj6+1tfitfHhsPj3y3AmZQaWmknDqlw8GtoJRy8uQ9j3thuldQyq302Fgdps6eJeXUKdMt7fz57J/fzw9bb29S//0XlZyc6X6DszOOtWvjUKcOjvfdh2OdOtgHBoLBYPa+4s733l3vubQLF4jfupXEPXvMXme7SpVMIcrx/vtzdYyNSUn69+rixUzhKD06mrTo6Cz3J+sXwVa/18qV0zdvb2y8vEzLUs+cIearryA9HYOjI15PPonXf/+rA3I+pV29StLevai0tHxvK1tK3T4mycn6+3t9TUnJFJCyDNBZsPX2xqFGDX2rWROHGjWwDw7Gxsmp8PavGJPglAslNTilnj9P9PPPk37tGq4dO+LWqROO9esX2n+FaVFRxK1fT9z69frD6xaDqyuuHTrg3qULTk2bFmpoupNKSyMtKkq3IJ09q1uTbt3SLlzI8sPVtkKF2+HhjhYqu8qVTX8slFIkHTzIzTVriP/119t/0G1tcWnTBo9evXBu2TLTfiqjkcRdu4j97jsSfv8djEb9MF9fPPr2xb13b+x8fQtu/41GVEKC+X+W8fGkX7lyu2Xt1ldL/5AWCFtbbL28TMHKxtMTG0fH2604Gd9ndFfeuczBAWNcnKklJeX48Sw/UG29vXGsWxeHOnVwqFYtV+85ZTRmHVpvBQpjcrLZV+zscGrQIN9ByVLGhARSIiJIPXWKlH//NQWq9EuXzNYzuLjocHTffaagZF+1aoH9/qXHxpKwfTvxW7aQuHOn2XGw9fXFtV07XDt0wKlxY4wJCToUZQSjqCizn03/PNyLwYBt+fLY+fvrm58fdv7+2FaogF358thkhCQ3txyPQcq//3LlzTdJ2r8fAPuqVSn/wgu4tGiR69dBKUXS/v3ErlhB/JYtUJihqYAZHB11a/QdN4O9PSlnzpB27lzWLZE2NthXqWIKVIUdpJRSqKQkjPHx+p/EhATTV2NCgv4bd+fy+HjcOnXCe9SoAq9FglMulMTglHToEFETJ2K8ccNsuV3Firh26oRbp0441KqV7z/yaVeukLB1K3Hr15N08ODtO+ztcWnVCrfOnfV4oGL2H4pKSyPtwgUdGlJTM4UjSxnj4oj75RdurllD8tGjpuW2FSrg3r077j17YuPmxs0ffyR2xQr9x+gW5+bN8ejXD5c2bazaTaaU0l1GdwbMjO/PndPhwGC4PYYqhxs2NrfHmFy7RvrVq6bvC6Or1sbNDce6dU1ByalePWz9/MrcuK702FhS//2XtKtXcaheXYekAmrZy4kxMZGEHTuI37yZhN9+M2/ttLW1qAXQ4OyMXcWKmYKR6VahAgZ7+wKrWSlF/C+/cPW990i/ehUA106dKD9unEX/wKTHxhL388/ErlhB6unTpuUOtWphW758gdWZSRZdyTndsvvdvdffHWNiog7lJ0/evv3zD8aYmMLbtwLi1r07FaZNK/DtSnDKhZIWnG6uXcvl116D1FQc7rsPr7AwErZtIz483GwgqH1goA5RoaE4BAXdc5tKKdLOnyf5+HFS/v5bfz1+3PQHBwCDAacmTXDr0gXXDh2wLQGvVUFKOXmS2DVriFu71vyPi729aeyLjZsbbj164PHIIzm+5sWBSk9HJSZicHEpkA9hlZp6O1RlBKqYmMwtPHd1Ndx5M9jZ4VC7Nk716uFQp47ubi2igCByplJSSNyzh/gtW4gPDzf9LtiUK2cKQfYVK94OSbe+2nh6WiXsGuPiuLZwIbHffQdGIwZXV7z/7//w6N8/U7BQSpH811+6dWnDBlMrm8HZGbfOnfF45BEc77uvyPehqCilSL96lZR//jGFqdQzZ1CF2DVuMBj0CTAuLhicnfVXFxdsXFywcXa+/X3GcmdnbH19C2XsmgSnXCgpwUkZjVxftIgbS5YA4NK+PRVefdV0SrMxKYmE334jfsMG/V/hHadGO9SqhVunTrh26oRdhQp6YOkdASnlxAk9fuVuBoN+bGgobqGh2Pn5Fcm+FmcqJYX4bdu4uWYNibt2gVI41K6Nx6OP4hYaWuinmAtRXKi0NNKio/XYtmLW6ny35L//5sobb5B85AgADjVr4vPiizg1bIgxIYG49euJXbGClOPHTY9xqFEDj379cOvc2TQwX5ReEpxyoSQEJ2NSEpenTiV+0yYAvIYOpdwzz2T7n7gxLo74rVuJ27BBf7jf8R+DwcEh83wzAPb2OFSvrgeX1q6tx07UrClB4B7SLl3CGB+PfVBQmes6EqKkUUYjN3/4gWvz5playpxbtCDpyBFT96PBwQHXkBDdutSggfxelyESnHKhuAentCtXiB4/Xo+xsbPD9+WXce/Rw+LHp9+4QfyWLcRt2EDSvn2glB5YWqvW7YBUuzYOwcEFOsZACCGKo/QbN7g2fz43V682LbOvWhX3vn1x794dWy8vq9UmrEeCUy4U5+CUfPw4UePHkx4djY2nJ35vvYVzkyZ53l76tWsYExKwCwiQcSNCiDIt6fBhEnbuxLlxY5yaNZO/iWVcbrJA6ZgVrxSKDw/n0uTJqMRE7IOC8J8zB/vKlfO1zYz5doQQoqxzatAApwYNrF2GKIEkOBUzSilivvySa3PnglI4t2hBhTfewNbd3dqlCSGEEGWeBKdiRKWmcuWNN7i5Zg0A7o88gs+kSaXmchlCCCFESSefyMWEUoroF18kYetWsLGh/PjxeAwYIGd1CCGEEMWIBKdiwnj9ug5NBgP+776Ly0MPWbskIYQQQtxFTiMoJlJvXa7DtkIFCU1CCCFEMSXBqZjICE75PXNOCCGEEIVHglMxkSbBSQghhCj2JDgVExktTnYSnIQQQohiS4JTMSFddUIIIUTxJ8GpmEg7fx6Q4CSEEEIUZxKcigFjQgLpV68C0lUnhBBCFGcSnIqB1FutTTaennJpFSGEEKIYk+BUDMgZdUIIIUTJIMGpGEiNjASkm04IIYQo7iQ4FQNyRp0QQghRMkhwKgakq04IIYQoGSQ4FQMZg8Olq04IIYQo3iQ4WZlKSyMtKgqQFichhBCiuJPgZGVpFy9CejoGR0dsfXysXY4QQggh7kGCk5WZrlFXqRIGg8HK1QghhBDiXiQ4WZmcUSeEEEKUHBKcrEzOqBNCCCFKDglOVmbqqqtSxcqVCCGEECInEpysTLrqhBBCiJKjVASnPn36UK5cOfr162ftUnJFKSVddUIIIUQJUiqC05gxY/j888+tXUaupV+5gkpOBltb7CpWtHY5QgghhMhBqQhO7dq1w93d3dpl5JppfJO/PwY7OytXI4QQQoicWD04bdu2jR49ehAQEIDBYGD16tWZ1lmwYAFBQUE4OTnRokUL9uzZU/SFFgJTN12lSlauRAghhBCWsHpwio+Pp2HDhixYsCDL+5cvX8748eOZOnUqBw4coGHDhoSGhnLp0qUirrTgmVqcZHyTEEIIUSJYvX+oS5cudOnSJdv73333XYYPH86wYcMAWLRoET///DNLlizhxRdfzPXzJScnk5ycbPo5NjY290UXEDmjTgghhChZrN7idC8pKSns37+fkJAQ0zIbGxtCQkLYuXNnnrY5a9YsPD09TbcqVpw/Sc6oE0IIIUqWYh2crly5Qnp6On5+fmbL/fz8iIqKMv0cEhLCo48+ytq1a6lcufI9Q9VLL71ETEyM6RYZGVlo9edEuuqEEEKIksXqXXUF4ddff7V4XUdHRxwdHQuxGssY4+IwxsQAMjhcCCGEKCmKdYuTj48Ptra2REdHmy2Pjo7G39/fSlUVjIzWJtvy5bFxdbVyNUIIIYSwRLEOTg4ODjRt2pRNmzaZlhmNRjZt2kTLli2tWFn+mbrppLVJCCGEKDGs3lUXFxfHyZMnTT9HRERw6NAhvL29qVq1KuPHjycsLIxmzZrRvHlz5syZQ3x8vOksu5JKBoYLIYQQJY/Vg9O+ffto37696efx48cDEBYWxqeffsqAAQO4fPkyU6ZMISoqikaNGrF+/fpMA8ZLmtRbg9IlOAkhhBAlh9WDU7t27VBK3XOd0aNHM3r06CKqqGjIGXVCCCFEyVOsxziVZjL5pRBCCFHySHCyApWSQvqtS8ZIcBJCCCFKDglOVpB64QIohcHFBZty5axdjhBCCCEsJMHJCu48o85gMFi5GiGEEEJYSoKTFcgcTkIIIUTJJMHJCmRguBBCCFEySXCyApn8UgghhCiZJDhZganFqUoVK1cihBBCiNyQ4FTElNFI2oULgEx+KYQQQpQ0EpyKWPqlS6iUFLCzw66EXzZGCCGEKGskOBUxUzddQAAGW1srVyOEEEKI3JDgVMTkGnVCCCFEySXBqYiZzqiTOZyEEEKIEkeCUxGTFichhBCi5JLgVMRk8kshhBCi5JLgVISUUjL5pRBCCFGCSXAqQsaYGIxxcYBcp04IIYQoiSQ4FaGMbjpbX19snJysXI0QQgghckuCUxGSbjohhBCiZJPgVITkjDohhBCiZLOzdgFlSer584C0OAkhyp709HRSU1OtXYYoo2xtbbGzs8NgMOR7WxKcilBaZCQgwUkIUbbExcVx7tw5lFLWLkWUYS4uLlSsWBEHB4d8bUeCUxGSrjohRFmTnp7OuXPncHFxwdfXt0D+4xciN5RSpKSkcPnyZSIiIqhZsyY2NnkfqSTBqYgYk5JIv3IFkBYnIUTZkZqailIKX19fnJ2drV2OKKOcnZ2xt7fnzJkzpKSk4JSPM9tlcHgRyTijzsbdHVtPTytXI4QQRUtamoS15aeVyWw7BbIVkSPpphNCCCFKPglORUSuUSeEECVHu3btGDt2rLXLsFhKSgo1atRgx44d2a5z+vRpDAYDhw4dAiA8PByDwcCNGzeKpshC9OKLL/Lss88WyXNJcCoiaTIVgRBClGkZQeXuW1RUlNl6CxYsICgoCCcnJ1q0aMGePXty3PaiRYsIDg7mwQcftLieBx98kIsXL+JZCoaPTJw4kc8++4x///230J9LglMRMXXVyTXqhBCixEtJScnzY48fP87FixdNtwoVKpjuW758OePHj2fq1KkcOHCAhg0bEhoayqVLl7LdnlKK+fPn8+STT+aqDgcHB/z9/Qt1/Fl+Xqfc8PHxITQ0lIULFxb6c0lwKiLSVSeEECVXUFAQr732Go8//jgeHh6MGDEiz9uqUKEC/v7+ptudg5bfffddhg8fzrBhw6hbty6LFi3CxcWFJUuWZLu9/fv3c+rUKbp162a2fM+ePTRu3BgnJyeaNWvGwYMHze6/s6suNjYWZ2dn1q1bZ7bOqlWrcHd3JyEhAYDIyEj69++Pl5cX3t7e9OrVi9OnT5vWHzp0KL179+b1118nICCA2rVrA7Bjxw4aNWpkqmX16tVm3YYAR44coUuXLri5ueHn58djjz3GlVtnowN8//331K9fH2dnZ8qXL09ISAjx8fGm+3v06ME333yTw6uffxKcioBKSyPtwgUA7KtUsXI1QghhPUopjImJVrnldwLOt99+m4YNG3Lw4EEmT54MQL169XBzc8v21qVLl0zbadSoERUrVuThhx/m999/Ny1PSUlh//79hISEmJbZ2NgQEhLCzp07s61r+/bt1KpVC3d3d9OyuLg4unfvTt26ddm/fz/Tpk1j4sSJ2W7Dw8OD7t2789VXX5ktX7ZsGb1798bFxYXU1FRCQ0Nxd3dn+/bt/P7777i5udG5c2ezlqVNmzZx/PhxNm7cyE8//URsbCw9evSgfv36HDhwgNdee40XXnjB7Hlu3LhBhw4daNy4Mfv27WP9+vVER0fTv39/AC5evMigQYN44oknOHbsGOHh4fTt29fsmDZv3pxz586ZBbnCIPM4FYG06GhIT8fg4ICtr6+1yxFCCKtRSUmcbt3aKs8dtH07hnzMJdWhQwcmTJhgtmzt2rX3vJTMnXNXVaxYkUWLFtGsWTOSk5NZvHgx7dq1Y/fu3TRp0oQrV66Qnp6On5+f2Tb8/Pz4+++/s32OM2fOEBAQYLbsq6++wmg08sknn+Dk5ES9evU4d+4cI0eOzHY7Q4YM4bHHHiMhIQEXFxdiY2P5+eefWbVqFaC7EY1GI4sXLzZ17y1duhQvLy/Cw8Pp1KkTAK6urixevNg0Q/eiRYswGAx8/PHHODk5UbduXc6fP8/w4cNNzz1//nwaN27MzJkzTcuWLFlClSpVOHHiBHFxcaSlpdG3b18CAwMBqF+/vln9Ga/BmTNnCAoKynY/80uCUxG4c3yToYDmkRBCCFG0mjVrlmlZxoe4JWrXrm3qugI9OPvUqVO89957fPHFF3muKzExMdOEjseOHaNBgwZmy1u2bHnP7XTt2hV7e3t++OEHBg4cyIoVK/Dw8DC1gP3xxx+cPHnSrGULICkpiVOnTpl+rl+/vtllTY4fP56plubNm5tt448//mDLli24ubllquvUqVN06tSJjh07Ur9+fUJDQ+nUqRP9+vWjXLlypvUyQmpGt2JhkeBUBOQadUIIoRmcnAjavt1qz50frq6umZbVq1ePM2fOZPuY1q1bZxo3dKfmzZvz22+/AXqAs62tLdHR0WbrREdH4+/vn+02fHx8+PPPP3MqP0cODg7069ePr776ioEDB/LVV18xYMAA7Ox0VIiLi6Np06YsW7Ys02N97+hNyep1yklcXBw9evTgzTffzHRfxYoVsbW1ZePGjezYsYMNGzYwb948Xn75ZXbv3k1wcDAA165dy1RLYchTcPrss8/w8fExDUR7/vnn+eijj6hbty5ff/11rhJ4WZB6ayoCmfxSCFHWGQyGfHWXFTe56arLyqFDh6hYsSKgg0vTpk3ZtGkTvXv3BsBoNLJp0yZGjx6d7TYaN27MwoULUUqZutDq1KnDF198QVJSkqmlZ9euXTnuz5AhQ3j44Yf566+/2Lx5MzNmzDDd16RJE5YvX06FChXw8PDIcVsZateuzZdffklycjKOjo4A7N2712ydJk2asGLFCoKCgkxB7W4Gg4FWrVrRqlUrpkyZQmBgIKtWrWL8+PGAHlxub29PvXr1LK4tL/LUbzRz5kzTm2Hnzp0sWLCA2bNn4+Pjw7hx4wq0wNLAdEadTEUghBClSmBgIDVq1Mj2VumOv/tz5sxhzZo1nDx5kiNHjjB27Fg2b97MqFGjTOuMHz+ejz/+mM8++4xjx44xcuRI4uPjGTZsWLY1tG/fnri4OP766y/TssGDB2MwGBg+fDhHjx5l7dq1vP322znuT5s2bfD392fIkCEEBwfTokUL031DhgzBx8eHXr16sX37diIiIggPD+e5557j3K3PuawMHjwYo9HIiBEjOHbsGL/88ouploygN2rUKK5du8agQYPYu3cvp06d4pdffmHYsGGkp6eze/duZs6cyb59+zh79iwrV67k8uXL1KlTx/Q827dvp3Xr1oV+TcQ8BafIyEhq1KgBwOrVq3nkkUcYMWIEs2bNYruVmmCLszSZikAIIcq8lJQUJkyYQP369Wnbti1//PEHv/76Kx07djStM2DAAN5++22mTJlCo0aNOHToEOvXr880YPxO5cuXp0+fPmZdaG5ubvz444/8+eefNG7cmJdffjnLbrC7GQwGBg0axB9//MGQIUPM7nNxcWHbtm1UrVqVvn37UqdOHZ588kmSkpLu2QLl4eHBjz/+yKFDh2jUqBEvv/wyU6ZMATC1hgUEBPD777+Tnp5Op06dqF+/PmPHjsXLywsbGxs8PDzYtm0bXbt2pVatWrzyyiu88847ZmctfvPNN2YDzguLQeXh/MwKFSrwyy+/0LhxYxo3bsz48eN57LHHOHXqFA0bNiQuLq4wai0UsbGxeHp6EhMTk6umR0sppTjdpg0qMZHK33+PQyGO9BdCiOImKSmJiIgIgoOD83VFenFvhw8f5uGHH+bUqVNZDrAubpYtW8awYcOIiYkpkBaidevWMWHCBA4fPpxtV9+93ou5yQJ5GuP08MMP89RTT9G4cWNOnDhB165dAfjrr78K9RTAkij92jVUYiIYDNjfdbqoEEIIURAaNGjAm2++SURERKbT9IuDzz//nGrVqlGpUiX++OMPXnjhBfr3719g3Wrx8fEsXbo029BUkPL0DAsWLOCVV14hMjKSFStWUL58eUDPXjpo0KACLbCky+ims/Pzw3DH6ZlCCCFEQRo6dKi1S8hWVFQUU6ZMISoqiooVK/Loo4/y+uuvF9j2+/XrV2DbykmegpOXlxfz58/PtHz69On5Lqi0Mc3hJOObhBBClFHPP/88zz//vLXLKBB5Ghy+fv1607wToFugGjVqxODBg7l+/XqBFVcayDXqhBBCiNIjT8Fp0qRJxMbGAvDnn38yYcIEunbtSkREhGk+BaHJGXVCCCFE6ZGnrrqIiAjq1q0LwIoVK+jevTszZ87kwIEDpoHiQpPJL4UQQojSI08tTg4ODqZrwfz666+mC/t5e3ubWqKEliqXWxFCCCFKjTy1OD300EOMHz+eVq1asWfPHpYvXw7AiRMnqCwBwcQYH4/x1pgvCU5CCCFEyZenFqf58+djZ2fH999/z8KFC01Tyq9bt47OnTsXaIElWcbAcBsvL2xKwIRkQgghhLi3PLU4Va1alZ9++inT8vfeey/fBZUmckadEEIIUbrkqcUJID09nRUrVjBjxgxmzJjBqlWrSE9PL8jaSjw5o04IIUqmdu3aMXbsWGuXkS+TJ09mxIgRhbb9oKAg5syZUyDbunLlChUqVLjnxYKLizwFp5MnT1KnTh0ef/xxVq5cycqVK/nvf/9LvXr1OHXqVEHXWGLJ5JdCCCHuFB4eTpMmTXB0dKRGjRp8+umnOa7fq1cvKlasiKurK40aNTK7mG92oqKieP/993n55ZdNywo6DO7du7fAgpmPjw+PP/44U6dOLZDtFaY8BafnnnuO6tWrExkZyYEDBzhw4ABnz54lODiY5557rqBrLLHSbk1FIC1OQghRuqSkpOT6MREREXTr1o327dtz6NAhxo4dy1NPPcUvv/yS7WN27NhBgwYNWLFiBYcPH2bYsGE8/vjjWQ6XudPixYt58MEHCQwMzFWNSinS0tIsWtfX1xcXF5dcbf9ehg0bxrJly7h27VqBbbNQqDxwcXFRhw8fzrT80KFDytXVNS+btJqYmBgFqJiYmALf9vIuU9XvzbqrxIMHC3zbQghREiQmJqqjR4+qxMREpZRSRqNS8QnWuRmNltfdtm1bNWbMGNPPgYGB6tVXX1WPPfaYcnd3V2FhYbl+LZ5//nlVr149s2UDBgxQoaGhudpO165d1bBhw+65Tr169dT8+fNNP4eFhSnA7BYREaG2bNmiALV27VrVpEkTZW9vr7Zs2aJOnjypevbsqSpUqKBcXV1Vs2bN1MaNG82eIzAwUL333numnwH18ccfq969eytnZ2dVo0YNtWbNGtP9165dU4MHD1Y+Pj7KyclJ1ahRQy1ZssRsm8HBwWrx4sW5ej0sdfd78U65yQJ5Ghzu6OjIzZs3My2Pi4vDQS5kC8DlS6lMTxhPiqsDz+xMZ2RdcJKXRghRxiUmQR0rzZN8bC24OOf98W+//TZTpkwx606qV68eZ86cyfYxrVu3Zt26dQDs3LmTkJAQs/tDQ0Nz3X0WExNDnTp1sr3/2rVrHD16lGbNmpmWvf/++5w4cYL777+fV199FdAtRqdPnwbgxRdf5O2336ZatWqUK1eOyMhIunbtyuuvv46joyOff/45PXr04Pjx41StWjXb554+fTqzZ8/mrbfeYt68eQwZMoQzZ87g7e3N5MmTOXr0KOvWrcPHx4eTJ0+SmJho9vjmzZuzfft2nnzyyVy9JkUpT8Gpe/fujBgxgk8++YTmzZsDsHv3bp5++ml69uxZoAWWVPFXblLL8SIHUuox51tY+RtMHQUhD1q7MiGEEHnRoUMHJkyYYLZs7dq1pKamZvsYZ+fbSS0qKgo/Pz+z+/38/IiNjSUxMdFs3ex8++237N27lw8//DDbdc6ePYtSioCAANMyT09PHBwccHFxwd/fP9NjXn31VR5++GHTz97e3jRs2ND082uvvcaqVav44YcfGD16dLbPPXToUAYNGgTAzJkzmTt3Lnv27KFz586cPXuWxo0bmwJdUFBQpscHBARw8ODB7F+AYiBPwWnu3LmEhYXRsmVL7O3tAUhNTaVXr14FNsK+pAuq683K9d78tAVmLISzF+DJl6HDf2DqaAiqZO0KhRCi6Dk76ZYfaz13ftzZgpMht2OI8mPLli0MGzaMjz/+mHr16mW7XkYrjpOT5Tt8977FxcUxbdo0fv75Zy5evEhaWhqJiYmcPXv2nttp0KCB6XtXV1c8PDy4dOkSACNHjuSRRx7hwIEDdOrUid69e/Pgg+atCc7OzqYrkxRXeQpOXl5erFmzhpMnT3Ls2DEA6tSpQ40aNQq0uJLOYIAeHaBDS5j3BSz+Djbvgt/2w/8NhFGD8/+LLIQQJYnBkL/uMmtydXXNtCw3XXX+/v5ER0eb3R8dHY2Hh0eOrU1bt26lR48evPfeezz++OP3XNfHxweA69ev4+vre891M9y9bxMnTmTjxo28/fbb1KhRA2dnZ/r165fjoPiMxpQMBoMBo9EIQJcuXThz5gxr165l48aNdOzYkVGjRvH222+b1r927ZrFNVuLxcFp/Pjx97x/y5Ytpu/ffffdvFdUCrk6w4sj4NHOMG0ebNung9TKDTD5GejcWv8xEUIIUbLkpquuZcuWrF1r3ty2ceNGWrZsec/nCA8Pp3v37rz55psWnf5fvXp1PDw8OHr0KLVq1TItd3BwsHi+xd9//52hQ4fSp08fQLdAZYyHyg9fX1/CwsIICwujdevWTJo0ySw4HTlyhHbt2uX7eQqTxcHJ0j5HgySAbFWvCp/PhvXb4bUP4Hw0PD0VWjeD6c/q+4UQQpQcuemqe/rpp5k/fz7PP/88TzzxBJs3b+bbb7/l559/Nq0zf/58Vq1axaZNmwDdKNG9e3fGjBnDI488QlRUFKBDkLe3d5bPY2NjQ0hICL/99hu9e/c2LQ8KCmL37t2cPn0aNze3bB8PULNmTVauXEmPHj0wGAxMnjzZ1HKUV1OmTKFp06bUq1eP5ORkfvrpJ7NB7gkJCezfv5+ZM2fm63kKm8XB6c4WJZF3BgN0aQPtmsOCr+DDb2D7Pgh9Ep56FJ59TLdQCSGEKF2Cg4P5+eefGTduHO+//z6VK1dm8eLFhIaGmta5cuWK2UTSn332GQkJCcyaNYtZs2aZlrdt25bw8PBsn+upp55i+PDhzJ49GxsbPWXjxIkTCQsLo27duiQmJhIREZHt4999912eeOIJHnzwQXx8fHjhhReIjY3Nx97rsPfSSy9x+vRpnJ2dad26Nd98843p/jVr1lC1alVat26dr+cpbAallLJ2EdYUGxuLp6cnMTExeHh4FPnznz4P0+frsU8AFX3hjYk6WAkhREmXlJREREQEwcHBuRqsLPJHKUWLFi0YN26c6Sy34u4///kPzz33HIMHDy6U7d/rvZibLJDna9WJghFUCZbOgiUzoUpFuHgZwl6ASbMhJs7a1QkhhCiJDAYDH330kcWzgFvblStX6Nu3b4kIedLiZOUWpzslJMJbn8DSlaAU+PvArAl6CgMhhCiJpMVJFBfS4lQKuTjrOZ6+e1+3REVdgWEvwcQ3ISbzRO1CCCGEKGISnIqhB+rD+sXwZD89mPy79fDwE7Bpp7UrE0IIIco2CU7FlLMTTBkF38+FalUg+go88T8YP0tan4QQJU8ZHxUiioGCeg9KcCrmmt0P6z6GEQN069OKDRAyDH7dYe3KhBAiZ7a2tgA5zjgtRGHLuJTL3bOb55YMDi9Gg8Nzsv8vmPQmnIrUP/d5GKaNBq/iXbYQogxTSnH27FlSU1MJCAgwzSkkRFFRSpGQkMClS5fw8vKiYsWKmdbJTRaQ4FSCghNAUjK89yl89C0YjRBQAVbNB//ifWkfIUQZlpKSQkRERL5nnhYiP7y8vPD398/yCicSnHKhpAWnDAePwpjX4cwFaFAbvp0jFwwWQhRfRqNRuuuE1djb25u6jbMiwSkXSmpwAjh7AXqOhOux0KM9zJssFwsWQgghckvmcSojqgbAh6+CvR38uAXmfm7tioQQQojSTYJTCdeiIcwYq79/91P4OdyKxQghhBClnASnUmBgNz1ZJsD4N+DP49atRwghhCitJDiVEv97Gto212fdPfUKRF+1dkVCCCFE6SPBqZSws4X5k6FGoL7G3fBXdIgSQgghRMGR4FSKeLjBktf1hJh//A2TZkPZPmdSCCGEKFgSnEqZwEqwaLpugfphM8xfZu2KhBBCiNJDglMp1LIRvDZGf//2J7Bum1XLEUIIIUoNCU6l1OAeMKyv/n7cLDjyj3XrEUIIIUoDCU6l2CvPQNsHIDEJnnoZLl2zdkVCCCFEySbBqRSzs4V5U6B6Fbh4GUa8AklyqSghhBAizyQ4lXKebvDJTPB0h4PHYPR0PUGmnG0nhBBC5J4EpzIguDIsnKZboDbugO5PQ7vHYPZiOHpSQpQQQghhKYNSZftjMzdXRC7pdh2Cz1fDpl3mk2NWrwLd20P3dlAr2ErFCSGEEFaSmywgwakMBacM8YmwaSf8FA7huyA59fZ9tYJuh6jqVa1UoBBCCFGEJDjlQlkMTne6Ga+7734Oh617IDXt9n11q8OQnjCkBxgMVitRCCGEKFQSnHKhrAenO8XEwcbf4Mdw+G0fpKXr5dNGw7BHrFmZEEIIUXhykwVkcLgw8XSDfp3hszdg3woYNVgvf/UD3RolhBBClHUSnESWynnCpKfg0c5gNMLoV+HkWWtXJYQQQliXBCeRLYMBXh8HD9SH2Hh48n9wI9baVQkhhBDWI8FJ3JOjA3z4KlT2h9PnYeQ08wHkQgghRFkiwUnkqLwXfPI6uDrDjoMwbZ61KxJCCCGsQ4KTsMh91WDuK7r77ssf4LNV1q5ICCGEKHoSnITFQh6EF4fr76fPh+37rFuPEEIIUdQkOIlc+b+B8EgnSDfCM9Ph30hrVySEEEIUHQlOIlcMBpg1AZrUg9g4ePJliLlp7aoKhlKQlGLtKoQQQhRnEpxErjk6wEevQiU/3eL0zPSSf6ZdXAL0GQWtBsKxU9auRgghRHElwUnkia83LJ4BLk7w23547QNrV5R36ekw5nU4eAyuXIfhk+F6jLWrEkIIURxJcBJ5VrcGzHlZf//ZKn223b2kpsHZi7DzEKz4BeZ+oR+TlFzopd7TW5/ArzvA0R4CKkDkRd2KlnGtPqGt2wYvvwcXLlm7EiGEsB65yK9c5DffFiyD2YvB1kYHKVdnOB+tP2DPR9++RV/V44juVtkPXhgBPdrrMVRFaeUGGDdLf//+y1CnOvR+BhKS4Ml+MGVU0dZTXJ0+D52GQXIqeLjBGxOhW1trVyWEEAUjN1lAgpMEp3xTCsbOhNW/5ryuoz0E+OmWnYAKupvv4mV9X+O6Oqg0qVu49WY4cBQGjIWUVBg1BJ5/Si9fvx3+b4r+/p0X9IWPyzKlIOxFfaFnB3v9egH07wLTntVBWQghSjIJTrkgwalgJKXAiFfgj+M6EFW+FY4q+UOlCnogeYAf+HiBzR0dxIlJ8NG3sPBr/T1Azw7wwnB9mZfCcj4aeo2Ey9ehUyt9WZk763p3Kbz/uQ56374PjeoUXi3F3bpt8PRUHZp+/ghWb4QPvtaBKriynhi1QW1rVymEEHknwSkXJDgVD9FX9Fij73/RH8iO9vDUozByMLi7FuxzJSTCI8/B0ZNQpxqsmJ+51cRo1K1OG34HPx/4cRH4lS/YOkqC+EToGKZbBZ97DCY8oZfvPATjZurldrYw8Un4vwHm4VMIIUqK3GQB+TMnigU/H3j7BfjpQ/hPIz2WZsFX0O4x+PonfeZbQTAaYfwbOjSV94LFr2fd1WRjA+++BDUDdah7eiokl8E5nuZ+rsNRlYq6OzNDy0awfjF0basH0b/xEQyeeLvbVQghSisJTqJYub8mfPMufPya7ga6ch1efAe6jtDjofJrzme668neTnfP3as70N1VBysPNzjwF0x+P+vB7aXViQhY/J3+fvqz4ORofr+XB3wwFWZPAmcn2HkQQp/Ur68QQpRWpSI4/fTTT9SuXZuaNWuyePFia5cj8slggE4PwYYlMHUUeLrD3//CkInw+PM6QOUlwPy4WY9bAj37+QP1c35MUCWYP1m3QC1fC5+vzv3zlkRK6aCYlq7HgHVsmfV6BgMM6AprP4L6tfQs8k9PhRff1l2iQghR2pT4MU5paWnUrVuXLVu24OnpSdOmTdmxYwfly1s2IEXGOBV/N2J14Pl89e25lWoEwtA+0LeTZWd1HT4O/Z7T3W3D+8MrI3NXw4fLYeYiPZ5n2du6O7E0W7lRj2FycoRNn1o2UD8lVQ+qX/SNDl7Vq8D7r+hAJYQQxVmZGuO0Z88e6tWrR6VKlXBzc6NLly5s2LDB2mWJAuTlAVNHw6bPIKyPDkonz8Arc+A/j8KrC+DM+ewfH30FnnpFh6b2LeClEbmvYUR/6B2ig9vIaXAuKq97U/zFxMHMhfr75x6z/OxGB3t4cQR89Q74+8CpSOg7Gvb+WXi1CiFEUbN6cNq2bRs9evQgICAAg8HA6tWrM62zYMECgoKCcHJyokWLFuzZs8d034ULF6hUqZLp50qVKnH+/D0+RUWJFVQJXn0Odn8H00brn2Pj4ZPvoe1j8MT/YPs+8268pGR9CZXoK3qg97zJYGub++c2GODNiXoM1rUYvc2M6RNKm3eW6GkaqlfRrXO59WBjPXC8XQvdCjX6Vbh6o8DLFEIIq7B6cIqPj6dhw4YsWLAgy/uXL1/O+PHjmTp1KgcOHKBhw4aEhoZy6ZJc96GscneFYY/Als/h0zegbXMdljbthP9Ogo5DdbdeXAJMmg1//K1brT55PX9TGzg5wkev6bPxjp7U2y7ZHd2Z/XkCvlijv391jG5FyotynnrgePWqEHVFd/sZjQVXpxBCWIvVg1OXLl2YMWMGffr0yfL+d999l+HDhzNs2DDq1q3LokWLcHFxYcmSJQAEBASYtTCdP3+egICAbJ8vOTmZ2NhYs5somWxsdNfb52/C5s/0mCdXZzh1Vg9sbtIbftisxyUtmg6BlXLcZI4q+elt2dnCj1tg/rLSE56MRpg8R3/t2QEeapq/7bk6w8JpOnBu3asnORVCiJLO6sHpXlJSUti/fz8hISGmZTY2NoSEhLBz504AmjdvzpEjRzh//jxxcXGsW7eO0NDQbLc5a9YsPD09TbcqVaoU+n6Iwle9KkzP6MZ7Vk9lkHzr0iCvjdHzDhWU5g30cwG8/Qm0GgQzFsLBoyU7RC1fCwePgZtL7gfPZ6d2sH79Ad5eArv/KJjtCiGEtRTr4HTlyhXS09Px8/MzW+7n50dUlB6da2dnxzvvvEP79u1p1KgREyZMuOcZdS+99BIxMTGmW2RkZKHugyha7q4wrK9ugfryLVgyEwb3KPjn+W9PGDf09gWNP/4Weo+ChwbB64t092BJClHXYuCNj/X344fpCUkLyqOd4ZFOuiXr2Rl6bi4hhCip7KxdQEHo2bMnPXv2tGhdR0dHHB0dc15RlGg2NtC6WeE+x9gweHqg7ob6aQv8ugPORcNHy/Wtsj90bwfd2ulT8g2Gwq0nP974SE/7UKeaPnOxIBkMMGOsvo7hyTN6vNNnb8rlWYQQJVOxDk4+Pj7Y2toSHR1ttjw6Ohp//0K8AqwQFnJyhNCH9C0pGbbshp/D4dedesqCRd/oW5WKOkQ92Bjs7G63Rt3dKnX38nKe+ky+wgxd+47objqAGeP0+K2C5uIMC6dCj5GwbZ++nM6z/y345xF5k5AIu/6AMxegVWOoFWztioqf1DSIOKfPzi3O/wSJwlesg5ODgwNNmzZl06ZN9O7dGwCj0cimTZsYPXq0dYsT4i5OjtCljb4lJsHm3fDzFti0CyIv6sHReRkgXdkfenWAXiF6zFBBSkvXA8JBd6k1u79gt3+nWsG65Wnim3qizGb3F+zYM2E5oxGOntLTd2zdC/uP6KkjMtQK0kG/azsdFMq6rXtg+gJ94snAbvDGBAlPZZnVZw6Pi4vj5MmTADRu3Jh3332X9u3b4+3tTdWqVVm+fDlhYWF8+OGHNG/enDlz5vDtt9/y999/Zxr7lBcyc7gobAmJuiXqp3A4cRoM3P6je+fXjL/Dpj/IBj2xZ/wdly6pHQy9OkKPDlC1Yv5rW/K9/kDwdNfTO5T3yv82czLxTfhuPfh6w7qP9VdR+C5dg99uBaXf9mcea1bZT595uuewbl3JUDtYdzd3b6dPwihLIs7Bax/oqU7u9GQ/mPyMhKfSJDdZwOrBKTw8nPbt22daHhYWxqeffgrA/Pnzeeutt4iKiqJRo0bMnTuXFi1aFMjzS3ASxVliEmzeBWs26fB1Z6tAk3p62oDu7fIWPqKvQofH9XxXM8fBEMuGCeZbQiL0HAn/nNFTHnz+Zt4mJRX3lpYOuw7Btr36duxf8/tdnODBJnosYJtm+kxUg0HPHL/xN/h5q26RujNE1akG3drr91xw5aLcm6J1Mx7mfQFLVuj9t7OFoX31PytT5up1xg3V4xxF6VCigpO1SXASJUVMHKzfpkPUzkO3J5S0sYFWTXRLVMtGOphcj4UbN/WA7+ux+uK7Gd/fuAk3YvTElNdjodF9sHJ+0YaXE6d1eEpM0mfxjXm86J67LDgXBaOmw6G/by8zGPR4uTYP6KDUpF7OE5zG3IQNv+vW0t/23b5WJEDdGtCjPfTvAj7lCmU3ipzRCN//ArM/1rPng54Bf8ozt1vblq6EafP096+MzNvs+qL4keCUCxKcREkUfVWPn/phs557Ka8c7WHFfOtciHfFLzD+DR38lr2tB86L/Nu0E8bN0qHHwxVCW+uw1KpJ/rpib8TCL7/pkx9+2w/pt4K7oz08EgpPPVqyu/L2/6UD0eHj+ufgyro7rmPLzOvO+1LP4QbwxkQY1K3o6hSFQ4JTLkhwEiXdmfPwwxZY8yv8G6nHK3l53Lq561s5z1vLb91X7tb9lf3B29N6tU+aDd+uA99ysHYxVJDxTnmWlq4/zDNOQGh4HyyYClUK4QTkazE6RH39k56zLMPDD8KIAfBA/ZIz/ifqMsz6CFb/qn92c4ExYfpKBNm1yCmlp/BY9I3ez3mv6HGHouSS4JQLEpxEaaJUyfnAAt1V13Ok7rp7sLGetFTGO+Ve9BUY/Zoe2A16Etj/PZ33aw1aSinY+6eeAHbjjtvTaDS6T3dhdW5TONNbFISkZPj4O1iwTL8PDQZ9ZunzT1k2ZlApeGUOfPmD3sePXsu6dUpkLyW18N+jlpLglAsSnISwrn/OQM+nISFJBtzmxW/74bkZcPWGbi2Z/Tx0a1v0dZw6C598D9+vv325o8r+8FQ/6N9Vz7JfHCSl6JayD76CS1f1sqb19KWaGtTO3baMRt0tuvpX3WX56ZvS5WwJoxHeWQoffqPHyE1/DuytPDmSBKdckOAkhPWt3KhnFAdo2xz+2wM6tCy+rRXFQXo6zP8S3vtMt37UrQ4fTLP+2W5XrsMXa+CzVfrkAwAPN3isJ4T1Bb/sr4hVqJJTdLfw/C/1iRGgL9r9/FP6xIq8ttSmpsEz0/Qgehcn+OodaFy3wMoudRKTdNhct+32srYP6G5ld1fr1SXBKRckOAlRPLy+SF+qJkNFXz3odmC3gr12nrWlpsEv2/XZWX+e0Ge6/achtGgIzepb1jJz5TqMnamnCwD9Gk1/Vk/CWlwkJsGKDbD4Oz0fEuhWhS5t9LUemzcomm7l1DTdCjbvS31dSdDvrdH/1a0dBdFVlJQCT/5Pt/55usPy96BO9fxvt7SJvgpPvawH4Nvb6SkevvxBv1fqVINP3wB/X+vUJsEpFyQ4CVF8nDkPy37ULQMZrRW2NvBwK3isl+4Gyc017pKS4cg/cOiYHsQcn6i306550Y8Fux4DX/2kW2MuXs56HVsbfYZji1tB6oH6urXmTnv/hFGv6nFNzk7w+lh9VltxlZ6uL0H00XJ9eZ8MtYL03GF9H868jwUhLR1WboC5X+iZ+wEqlIdRQ3TQdHIo2OdLSIT/TtJn5/mWg2/fh2pVCvY5SrK/TupwefGyPjnlo9d0eD58HJ54SU//UNEXls6yTuiU4JQLEpyEKH6SUmDdVh2i9v55e3lQJf1h+2ioPlPwTkYjnDyrA9KhY/r297/mcw9laN5Ad9E8UL9w9wN0DUtXwqqNursI9LQAQ3roqQKOntQTVe7+Q18k+k42NlCvhg5R/2moxxHNXqynAqgRqK//V5KuK/fncfjyRz0XWWKSXubipLvK/ttLt77lV1o6/LAJ3v8cTp/Xy3zKwTOD9WtemK1yMXEwcJw+pgEV4Pu5ujuwrNv4ux6Hl5Ckp6xYOlPPUp8hMgrCXtDvbzcXWDhNT6FRlCQ45YIEJyGKt+MRujl/5QY9yznogbhd2+mxESdO65B0+Pjt++/kWw4a1YGGdfRcRF+svj14ucN/YNKTejLHgpSerudTWrISdh68vbxeTXiiL3TvkHWLx7koHaB2/aG/nrmQ9fZ7h8DM8cVnwHVuxcTp4/nlD3DyzO3ljevoYNyjfe4CTmoaXL2uX7e5n8OpSL3c2xOeHqTHV7kU0Wt15Tr0H6NrCKoEzw/X+1XRt2Sd8VoQlIKPvoVZH+rvH2qqx+F5ZtHCGHMTRkzWx9DOVr+/B3QtulolOOWCBCchSob4RN2S8MUP8Nc/Wa/j7AQNaumQ1KiOPi0+oIL5B9bFy/D+Z7o7MGMSx54dYMIT+oMuP2Li4Nu18Nnq291Dtja6ZemJR/SFjXPz4Rl1+XaI2v0HxMbD+KEwqHvp+BBWSu/Xsh/1YOGMy7t4ukO/UOjbCZRRX2fvynXz253LbsSab9fLA54eCI/3tk64vHgZ+j1r3oLo56Pfk01uvTcb1C66MGcNKakw+X345mf985AeOZ89l5yi53Zbs0n//Nxj+soCRfFel+CUCxKchChZlNLdcV/+AP+chvuq64DUqA7UDLL8TLx/I+HdpfDjFv2zna3+D3fM45YPRo++osdQ/XlC33Yc0N0RoD+8B3XTY6qkuyZnl6/pMPvVj5m7LC1ha6NbdQZ215NXWvMMLdAD0Rd9o8c8/X3qdkjPYGMD9wXr923juvprjaq5G8NXXN2Ihaen6dZWGxuYPBKGPWJZAFIK3l6iz34EPQbuzUmFP9+TBKdckOAkRNl25B946xMI361/dnTQH7zPDNbhB/Qf86greozOnyduh6XL1zJvr1aQbl3qHaJbwETupKdD+B4djHce0gPHfcrpSSl9y+nvM372KXdrmbeeFb+4ho7EJP1+OXgMDh7VXctZnSDg7qrHrtWoeusWqMcEVamYt6k5lNLze52P1reYON1dVhizyWeIOAdP/E//Y+LqDPMm521i0K9/hpff1YGzZWP48NWsu/gKigSnXJDgJIQAPev2mx/fPvPL3VUPWo6MgiMn9AfQ3WxsoGagHtRcv9at7sE6paMbTRSuqMs6SB26FaYOn7g9YP5uDva6GzkjVFW/Faqq+OvL35yPhguXbgekC5du3aJvj+fLYDDogdeDu+tAU5ATT+48BP83RY9XquQHn7yevzPkwvfoObLiE/Xv2adv6ElVC4MEp1yQ4CSEyKAUbN4Fby2GY/+a32dro1uT7q+lQ9L9tfSkk9KqJApCWro+q+zkGX12aMbXfyP1tBp5ZTDoaRgCKuj38J1TQlQor7unB3bNeyA5cx627tVzim3epfejcR34aEbBXHvyr5Mw7CXdLe7rrc/Iq5/LGd4tIcEpFyQ4CSHuZjTCT+Fw4C/93339WnosVUHP/SNEToxGOH/pjlB1Rp+xd/KMbm1ydtKtO5Uq6HAU4AeV/W5/7+9jPj7ozHndDfbdej2wHnS4atschnTPecb+uAQ9dmnrXti2N/OZnz07wFvPF+y0Dxcu6fD09786lK1aUPCtuhKcckGCkxBCiJIoMUkHlLyEiJRUPb/Ssh/h9wO3l/v76FaoAV11IDMa9Vms2/bpoLTviPncaHa2+mzRNg/o2/01C6erOjYOpszV04cUxskWEpxyQYKTEEKIsizinJ424Lv1t8fy2djoix//G5l5fF9ggA5JbR/QA7fdXIq64oInwSkXJDgJIYQQeh6lDb/Bsp/MJ251ddaXO8oIS4H5nO+sOMpNFijA8fRCCCGEKKkcHaBHB337N1J34dUMhCb1Cn8epZJEgpMQQgghzFSrIhcpzk4xnS5MCCGEEKL4keAkhBBCCGEhCU5CCCGEEBaS4CSEEEIIYSEJTkIIIYQQFpLgJIQQQghhoTI/HUHG/J+xsbFWrkQIIYQQ1pCRASyZE7zMB6ebN28CUKWKTFghhBBClGU3b97E09PznuuU+UuuGI1GLly4gLu7O4YCvjJhbGwsVapUITIyskxczkX2t3ST/S3dZH9Lv7K2z7nZX6UUN2/eJCAgABube49iKvMtTjY2NlSuXLlQn8PDw6NMvEkzyP6WbrK/pZvsb+lX1vbZ0v3NqaUpgwwOF0IIIYSwkAQnIYQQQggLSXAqRI6OjkydOhVHR0drl1IkZH9LN9nf0k32t/Qra/tcWPtb5geHCyGEEEJYSlqchBBCCCEsJMFJCCGEEMJCEpyEEEIIISwkwakQLViwgKCgIJycnGjRogV79uyxdkmFYtq0aRgMBrPbfffdZ+2yCsy2bdvo0aMHAQEBGAwGVq9ebXa/UoopU6ZQsWJFnJ2dCQkJ4Z9//rFOsQUgp/0dOnRopuPduXNn6xRbAGbNmsUDDzyAu7s7FSpUoHfv3hw/ftxsnaSkJEaNGkX58uVxc3PjkUceITo62koV548l+9uuXbtMx/jpp5+2UsX5s3DhQho0aGCay6dly5asW7fOdH9pOraQ8/6WpmOblTfeeAODwcDYsWNNywr6GEtwKiTLly9n/PjxTJ06lQMHDtCwYUNCQ0O5dOmStUsrFPXq1ePixYum22+//WbtkgpMfHw8DRs2ZMGCBVneP3v2bObOncuiRYvYvXs3rq6uhIaGkpSUVMSVFoyc9hegc+fOZsf766+/LsIKC9bWrVsZNWoUu3btYuPGjaSmptKpUyfi4+NN64wbN44ff/yR7777jq1bt3LhwgX69u1rxarzzpL9BRg+fLjZMZ49e7aVKs6fypUr88Ybb7B//3727dtHhw4d6NWrF3/99RdQuo4t5Ly/UHqO7d327t3Lhx9+SIMGDcyWF/gxVqJQNG/eXI0aNcr0c3p6ugoICFCzZs2yYlWFY+rUqaphw4bWLqNIAGrVqlWmn41Go/L391dvvfWWadmNGzeUo6Oj+vrrr61QYcG6e3+VUiosLEz16tXLKvUUhUuXLilAbd26VSmlj6e9vb367rvvTOscO3ZMAWrnzp3WKrPA3L2/SinVtm1bNWbMGOsVVcjKlSunFi9eXOqPbYaM/VWq9B7bmzdvqpo1a6qNGzea7WNhHGNpcSoEKSkp7N+/n5CQENMyGxsbQkJC2LlzpxUrKzz//PMPAQEBVKtWjSFDhnD27Flrl1QkIiIiiIqKMjvWnp6etGjRotQea4Dw8HAqVKhA7dq1GTlyJFevXrV2SQUmJiYGAG9vbwD2799Pamqq2TG+7777qFq1aqk4xnfvb4Zly5bh4+PD/fffz0svvURCQoI1yitQ6enpfPPNN8THx9OyZctSf2zv3t8MpfHYjho1im7dupkdSyic398yf626wnDlyhXS09Px8/MzW+7n58fff/9tpaoKT4sWLfj000+pXbs2Fy9eZPr06bRu3ZojR47g7u5u7fIKVVRUFECWxzrjvtKmc+fO9O3bl+DgYE6dOsX//vc/unTpws6dO7G1tbV2efliNBoZO3YsrVq14v777wf0MXZwcMDLy8ts3dJwjLPaX4DBgwcTGBhIQEAAhw8f5oUXXuD48eOsXLnSitXm3Z9//knLli1JSkrCzc2NVatWUbduXQ4dOlQqj212+wul79gCfPPNNxw4cIC9e/dmuq8wfn8lOIl869Kli+n7Bg0a0KJFCwIDA/n222958sknrViZKAwDBw40fV+/fn0aNGhA9erVCQ8Pp2PHjlasLP9GjRrFkSNHStUYvXvJbn9HjBhh+r5+/fpUrFiRjh07curUKapXr17UZeZb7dq1OXToEDExMXz//feEhYWxdetWa5dVaLLb37p165a6YxsZGcmYMWPYuHEjTk5ORfKc0lVXCHx8fLC1tc00aj86Ohp/f38rVVV0vLy8qFWrFidPnrR2KYUu43iW1WMNUK1aNXx8fEr88R49ejQ//fQTW7ZsoXLlyqbl/v7+pKSkcOPGDbP1S/oxzm5/s9KiRQuAEnuMHRwcqFGjBk2bNmXWrFk0bNiQ999/v9Qe2+z2Nysl/dju37+fS5cu0aRJE+zs7LCzs2Pr1q3MnTsXOzs7/Pz8CvwYS3AqBA4ODjRt2pRNmzaZlhmNRjZt2mTWz1xaxcXFcerUKSpWrGjtUgpdcHAw/v7+Zsc6NjaW3bt3l4ljDXDu3DmuXr1aYo+3UorRo0ezatUqNm/eTHBwsNn9TZs2xd7e3uwYHz9+nLNnz5bIY5zT/mbl0KFDACX2GN/NaDSSnJxc6o5tdjL2Nysl/dh27NiRP//8k0OHDpluzZo1Y8iQIabvC/wY538su8jKN998oxwdHdWnn36qjh49qkaMGKG8vLxUVFSUtUsrcBMmTFDh4eEqIiJC/f777yokJET5+PioS5cuWbu0AnHz5k118OBBdfDgQQWod999Vx08eFCdOXNGKaXUG2+8oby8vNSaNWvU4cOHVa9evVRwcLBKTEy0cuV5c6/9vXnzppo4caLauXOnioiIUL/++qtq0qSJqlmzpkpKSrJ26XkycuRI5enpqcLDw9XFixdNt4SEBNM6Tz/9tKpataravHmz2rdvn2rZsqVq2bKlFavOu5z29+TJk+rVV19V+/btUxEREWrNmjWqWrVqqk2bNlauPG9efPFFtXXrVhUREaEOHz6sXnzxRWUwGNSGDRuUUqXr2Cp17/0tbcc2O3efOVjQx1iCUyGaN2+eqlq1qnJwcFDNmzdXu3btsnZJhWLAgAGqYsWKysHBQVWqVEkNGDBAnTx50tplFZgtW7YoINMtLCxMKaWnJJg8ebLy8/NTjo6OqmPHjur48ePWLTof7rW/CQkJqlOnTsrX11fZ29urwMBANXz48BL9D0FW+wqopUuXmtZJTExUzzzzjCpXrpxycXFRffr0URcvXrRe0fmQ0/6ePXtWtWnTRnl7eytHR0dVo0YNNWnSJBUTE2PdwvPoiSeeUIGBgcrBwUH5+vqqjh07mkKTUqXr2Cp17/0tbcc2O3cHp4I+xgallMpbW5UQQgghRNkiY5yEEEIIISwkwUkIIYQQwkISnIQQQgghLCTBSQghhBDCQhKchBBCCCEsJMFJCCGEEMJCEpyEEEIIISwkwUkIIYQQwkISnIQQQgghLCTBSQhRJgwdOpTevXtbuwwhRAknwUkIIYQQwkISnIQQpcr3339P/fr1cXZ2pnz58oSEhDBp0iQ+++wz1qxZg8FgwGAwEB4eDkBkZCT9+/fHy8sLb29vevXqxenTp03by2ipmj59Or6+vnh4ePD000+TkpJinR0UQliVnbULEEKIgnLx4kUGDRrE7Nmz6dOnDzdv3mT79u08/vjjnD17ltjYWJYuXQqAt7c3qamphIaG0rJlS7Zv346dnR0zZsygc+fOHD58GAcHBwA2bdqEk5MT4eHhnD59mmHDhlG+fHlef/11a+6uEMIKJDgJIUqNixcvkpaWRt++fQkMDASgfv36ADg7O5OcnIy/v79p/S+//BKj0cjixYsxGAwALF26FC8vL8LDw+nUqRMADg4OLFmyBBcXF+rVq8err77KpEmTeO2117CxkYZ7IcoS+Y0XQpQaDRs2pGPHjtSvX59HH32Ujz/+mOvXr2e7/h9//MHJkydxd3fHzc0NNzc3vL29SUpK4tSpU2bbdXFxMf3csmVL4uLiiIyMLNT9EUIUP9LiJIQoNWxtbdm4cSM7duxgw4YNzJs3j5dffpndu3dnuX5cXBxNmzZl2bJlme7z9fUt7HKFECWQBCchRKliMBho1aoVrVq1YsqUKQQGBrJq1SocHBxIT083W7dJkyYsX76cChUq4OHhke02//jjDxITE3F2dgZg165duLm5UaVKlULdFyFE8SNddUKIUmP37t3MnDmTffv2cfbsWVauXMnly5epU6cOQUFBHD58mOPHj3PlyhVSU1MZMmQIPj4+9OrVi+3btxMREUF4eDjPPfcc586dM203JSWFJ598kqNHj7J27VqmTp3K6NGjZXyTEGWQtDgJIUoNDw8Ptm3bxpw5c4iNjSUwMJB33nmHLl260KxZM8LDw2nWrBlxcXFs2bKFdu3asW3bNl544QX69u3LzZs3qVSpEh07djRrgerYsSM1a9akTZs2JCcnM2jQIKZNm2a9HRVCWI1BKaWsXYQQQhRXQ4cO5caNG6xevdrapQghigFpZxZCCCGEsJAEJyGEEEIIC0lXnRBCCCGEhaTFSQghhBDCQhKchBBCCCEsJMFJCCGEEMJCEpyEEEIIISwkwUkIIYQQwkISnIQQQgghLCTBSQghhBDCQhKchBBCCCEsJMFJCCGEEMJC/w8oG+i96gEREgAAAABJRU5ErkJggg==",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"lr=0.2 first->last loss: 2.307 -> 0.581\n"
]
}
],
"source": [
"# the fix: the same loop with a sane lr trains (we already saw it above); contrast the two curves\n",
"P_good = init_params()\n",
"good_losses = []\n",
"g = np.random.default_rng(SEED)\n",
"for i in range(40):\n",
" idx = g.integers(0, len(Xtr), size=128)\n",
" logits, cache = mlp_forward(Xtr[idx], P_good[\"W1\"], P_good[\"b1\"], P_good[\"W2\"], P_good[\"b2\"])\n",
" loss, dlogits = softmax_xent(logits, ytr[idx])\n",
" grads = mlp_backward(dlogits, cache, P_good[\"W2\"])\n",
" for k in P_good:\n",
" P_good[k] -= 0.2 * grads[k] # the lr that worked\n",
" good_losses.append(loss)\n",
"fig, ax = plt.subplots(figsize=(6, 3))\n",
"ax.plot(bad_losses, label=\"lr=50 (diverges)\", color=\"#d33\")\n",
"ax.plot(good_losses, label=\"lr=0.2 (trains)\", color=\"#1E40FF\")\n",
"ax.set_xlabel(\"step\"); ax.set_ylabel(\"loss\"); ax.set_yscale(\"symlog\"); ax.legend()\n",
"ax.set_title(\"learning rate too high vs sane\"); plt.tight_layout(); plt.show()\n",
"print(f\"lr=0.2 first->last loss: {good_losses[0]:.3f} -> {good_losses[-1]:.3f}\")"
]
},
{
"cell_type": "markdown",
"id": "b44e6dee",
"metadata": {},
"source": [
"> **Interpretation.** The red curve jumps to ~20 on the first step and never recovers; the blue one descends from $\\ln 10$. Same model, same gradients, same data: only the step size differs. The signature (loss climbing and staying high, eventually overflowing to NaN if you let it run) is one of the three failures from the draft, and the fix is mechanical, not mysterious.\n",
"\n",
"> **Key takeaways**\n",
"> - The tensor backward pass is the matmul rule plus a broadcast-bias sum plus the ReLU mask; six lines, finite-difference-verified.\n",
"> - Loss at init is $\\ln K$ for a $K$-class problem; check it before every training run.\n",
"> - A learning rate too high makes the loss climb and overflow to NaN; the cure is a smaller step.\n"
]
},
{
"cell_type": "markdown",
"id": "eb88cfc2",
"metadata": {},
"source": [
"## Safety lens\n",
"\n",
"Backpropagation is differentiable end to end, which is wonderful for training and dangerous at inference: the same gradient that updates weights can be taken with respect to the *input*, producing a tiny perturbation that flips the prediction (Goodfellow et al. 2015, FGSM). We make this concrete on the model we just trained. We compute $\\partial L / \\partial x$ for one test image, step a small amount in the sign of that gradient, and watch the predicted class change while the image looks unchanged.\n"
]
},
{
"cell_type": "code",
"execution_count": 32,
"id": "5cc97b18",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:17:44.370256Z",
"iopub.status.busy": "2026-06-10T19:17:44.370157Z",
"iopub.status.idle": "2026-06-10T19:17:44.389546Z",
"shell.execute_reply": "2026-06-10T19:17:44.388896Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"true Pullover · clean pred Pullover · adversarial pred Coat\n"
]
}
],
"source": [
"# FGSM-style single-image attack using OUR backward pass (no torch needed)\n",
"def input_gradient(P, x_row, y_true):\n",
" 'dL/dx for a single standardized image row, via the chain rule back to the input.'\n",
" X = x_row[None, :] # (1,784)\n",
" logits, cache = mlp_forward(X, P[\"W1\"], P[\"b1\"], P[\"W2\"], P[\"b2\"])\n",
" _, dlogits = softmax_xent(logits, np.array([y_true]))\n",
" # one more step of the chain rule: dL/dX = dZ1/dX path -> dX = dZ1 @ W1.T\n",
" _, Z1, _ = cache\n",
" dH = dlogits @ P[\"W2\"].T\n",
" dZ1 = dH * (Z1 > 0)\n",
" dX = dZ1 @ P[\"W1\"].T # (1,784)\n",
" return dX[0], int(logits.argmax(1)[0])\n",
"\n",
"i0 = 0\n",
"g_x, pred_clean = input_gradient(P, Xte[i0], yte[i0])\n",
"eps_atk = 0.5 # small step in standardized units\n",
"x_adv = Xte[i0] + eps_atk * np.sign(g_x) # FGSM: move along the sign of the input gradient\n",
"lg_adv, _ = mlp_forward(x_adv[None, :], P[\"W1\"], P[\"b1\"], P[\"W2\"], P[\"b2\"])\n",
"pred_adv = int(lg_adv.argmax(1)[0])\n",
"print(f\"true {CLASSES[yte[i0]]} · clean pred {CLASSES[pred_clean]} · adversarial pred {CLASSES[pred_adv]}\")"
]
},
{
"cell_type": "code",
"execution_count": 33,
"id": "16acf8d7",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:17:44.390553Z",
"iopub.status.busy": "2026-06-10T19:17:44.390467Z",
"iopub.status.idle": "2026-06-10T19:17:44.779491Z",
"shell.execute_reply": "2026-06-10T19:17:44.779178Z"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAuIAAAEICAYAAAAa1JFGAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAKwJJREFUeJzt3Wl0lGWah/F/CJstGLYOq+ybkIUABmSPAiE4MgwgtgsKKGIjc6a7BYRRQXEUu+WICnOEgXbrphVbwcalycgimzACikCDCKgshrAmQAhLQt754Okcaen7KaxKniRcv3P8IFel6q1KFXVThPuNCoIgEAAAAIBiVc73AQAAAABXIgZxAAAAwAMGcQAAAMADBnEAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAAwZxAAAAwAMGcQAAAMADBvFi0rhxY23evNnLbQ8fPlz169dXu3bt1Lp1a40ePVp5eXnm13z77beqVq1a4f9HRUUpOzu7aA8UKEM2btyo2267LSLXNW7cOL355puXbLNmzdLw4cMlSR9//LHatWv3T69n+vTp+u1vf/ujX3///ffVq1cv53HMmjVLTz/9dCiHDJQKOTk5ioqK8n0YlzR58mTNnz/febnhw4fr+eefD+k69+zZoyFDhqhJkybq0KGDkpOTNW/evLCO8/HHH9fZs2fDuo4rGYP4FWL8+PHavHlz4X+zZ8/2fUiXlJ+f7/sQgIjo2LGjFixYEPb1fPfdd/rwww8jMtS/++67Gjhw4E/++vvvv1+///3vdeLEibCPBSiLCgoKVFBQEPb15Ofna+rUqbrzzjsjcFTfy8zMVLdu3ZSamqpvvvlGmzZtUnp6etjvu0888QSDeBgYxCNs3bp16tatmxITE5WQkKC//OUvP7pMZmamhg4dquTkZMXHx+vRRx8tbOPGjdP111+vdu3aqUePHtq5c2dhi4qK0tNPP63k5GQ1adJEr7zyymUfX+XKldWzZ0/t3LnzR5+ebdu2TY0bN3Zex8aNG9WlSxclJCQoOTlZa9eulSSNGjVK06dPL7zcN998ozp16igvL095eXmaOHGikpOT1a5dOw0dOlRZWVmSvv/T/MiRI9WjRw/FxcVd9n0CfDtz5oxuu+02tWnTRomJierbt++PXl9z5sxRy5Yt1b59ez355JMXfQpnvbZffvllDR48uPDyp06d0m233aZWrVqpW7du2rp1a0jHeOjQIWVnZ6tVq1bKy8vTmDFj1KJFCyUnJ2vFihWFl5s/f746duyoc+fOKQgC3XLLLXrqqackSRUrVlTfvn31pz/9KZyHCyhyd955pzp27KiEhATdfPPNyszMLGxz5sxRixYtlJSUpBkzZhT++lNPPaWxY8cW/n9OTo5q1KihI0eOSPr+b5SSk5PVvn179evXT3v37pX0/SfCgwcPVmpqquLi4nTw4EGNHTtW1113nRITE9WhQwedPXtW+fn5Sk1NVceOHdW2bVvdcccdOn36tKTv/zarbdu2uvfee9WuXTstWrTook+6ly1bphtuuEFJSUlq27atfv/731/2Y/Lf//3f6t69u0aNGlX4a9WrV9cDDzwgSTp8+LAGDRqk+Ph4xcXFac6cOYWX+2ezyd+/tnv37mrXrp0OHz582cd1xQsQMceOHQtiY2ODVatWBUEQBBcuXAiOHTsWBEEQNGrUKPj888+DIAiCvn37Bh9//HEQBEGQl5cXpKamBm+99VYQBEFw+PDhwut74403gtTU1ML/lxRMnz49CIIg2LFjR1ClSpUgLy8vCIIgeOyxx4KXXnrpksd1zz33BDNmzAiCIAiOHz8exMfHBy+//HKwYsWKIDExsfByW7duDRo1ahQEQRB88803QUxMzEW3nZWVFZw7dy649tprgyVLlgRBEASrV68OateuHZw6dSpYu3Zt0LZt28KvmTx5cvCb3/wmCIIgeOqpp4KpU6cWtqlTpwZjxowpPL6EhITg5MmT1sMLlFgLFy4M+vbtW/j/x44du+j1tXXr1qBOnTrBwYMHgyD4/rXxw99+rdf2jTfeGLz33nuFlx03blwwbNiwoKCgIMjOzg5at24d3HPPPUEQBD96Tf/QnDlzgkmTJgVBEASzZs0KbrzxxuDcuXPBuXPngl69egU9e/YsvOz9998fPPjgg8Hvfve7oF+/fkFBQUFhe+2114LBgwf/tAcKKCY/fC+dNm1aMHr06CAIvn8t1q5dO8jIyAiCIAgmTZpU+Frct29f8POf/zw4e/ZsEARB8PLLLweDBg0KgiAI5s+fH9x3331Bfn5+EARB8Prrrwf9+/cPgiAIpkyZEtStWzfIzMwMgiAIPvvss6B169bBhQsXgiAIguzs7ODChQtBQUFBcPTo0SAIgqCgoCB44IEHgmnTpgVB8P1rNyoqqnA2CIIfv3f//baPHTsWNGzYMNi/f/+PLrdhw4YgLS3tko9JWlpa8Nxzz/3Tx2zo0KHBxIkTgyAIgkOHDgUNGjQI1q1b96PH81KzSVZW1j+9XtjK+/xDQFmzbt06tWrVSt27d5cklStXTjVq1LjoMqdPn9ayZct06NChwl/Lyckp/NPlRx99pJkzZ+rUqVMqKCjQ8ePHL/r6v/81VevWrVW+fHllZmaqQYMGmjp1qnlszz77rF599VWVK1dOQ4YM0fDhw7Vy5crLvo87d+5UuXLllJqaKknq1q2bateurc2bN6tbt27Kz8/Xhg0b1LFjR73++ut67733JH3/V+InTpzQO++8I0k6f/78RZ++33rrrapateplHw9QEiQmJmrHjh0aM2aMevbsqf79+1/Uly9frn79+qlOnTqSvv/bo398zf6z1/aBAwdUu3btwsstW7ZMM2bMUFRUlGJiYnTHHXdoz549zmN899139cQTTxRex913362KFStKkkaOHHnRJ2wvvPCCOnXqpMWLF+uzzz676NP7OnXq6MCBA5fz8ADF7k9/+pP+8Ic/6OzZszp79qxq1aol6fvXYlpamurWrStJ+uUvf6lp06ZJkq699lolJSVp8eLFuvXWW/Xqq69q/Pjxkr5//WzYsEEdOnSQJF24cOGi2+vfv3/h67Rp06bKz8/XyJEjlZKSoptvvlnlypVTQUGBZsyYoQ8++ED5+fk6ceKEunTpUngdTZs2Vc+ePS95f44dO6Z7771XX331lcqXL69jx45p27ZtatCgwUWX69ixoz788MOf9JgtXbpUmzZtkiTFxsZq0KBBWrp0qTp37uycTfDTMYgXsyAIJEnr169X5cqVL2r79u3T2LFjtWHDBjVr1kxbtmxRjx49LrrMD78mOjo65J/tGj9+vH71q19d9Gvly5e/6DeTn/ozXj98kx4xYoReeeUV5eTkqFatWoU/ahIEgWbOnKm+ffte8jqqVKnyk24bKAmaNm2q7du3a/ny5Vq6dKkmTJhg/uOpS/3jsH/22v7Zz35mvjZD+Ydmp06d0s6dO9WxY8eQruPw4cPKyspSQUGBsrOzC4cY6fvfJ6666irnbQK+rFmzRi+++KLWrVun2NhYLV68WJMnT77kZf/xuT9y5Ei98sor6tChg3bv3q1+/fpJ+v49bNKkSbr//vsveT0/fA+LiYnRtm3btHLlSq1YsUKTJk3SqlWrtH79ei1fvlwrV67UNddcoxdffFHLly+/5HX8owceeED9+/fXO++8o6ioKLVv3/6y37M7dOigdevW6de//nVIl//7YxPKbIKfjp8Rj6AuXbpo165dWr16tSRd8k+NVapUUUpKip555pnCX8vIyNCBAwd04sQJVahQQXXr1lUQBJo1a1aRHm/Tpk21d+/ewp9/+8Mf/uD8mlatWqmgoEAfffSRJOmTTz5RZmZm4c/CDhs2TH/+8581e/ZsjRw5svDrBg4cqBkzZig3N1eSlJubq7/97W8RvkeAHwcOHFBUVJQGDBig6dOnKwgC7d+/v7CnpKQoPT298OcnL+fnOxMSEi76tyK9e/fWK6+8oiAIdPLkSb3xxhvO6/jwww+VlpZW+Mbau3dv/fGPf1ReXp7Onz9/0c+k5+fn6xe/+IWefPJJPffccxo6dKjOnTtX2Hfs2KHExMSQjx8obllZWapatapq1qyp8+fPX/SzzjfeeKOWLFlS+DPj/7i4YODAgdqwYYOmTZumu+66S+XLly/89dmzZxe+p+fl5enzzz+/5O0fOXJEp0+fVt++ffX000+rcePG2r59u7KyslSrVi1dc801OnXqlF599dXLuk+NGjVSVFSUVq1apS+++OJyHhJJ0pgxY7Ry5cqLXu/Z2dmFj0/v3r01d+7cwvuwcOFC9enTxzmbVK1alX/AHQYG8QiqXr26Fi1apIkTJyohIUHt27cv/IeMPzR//nzt3r1bcXFxio+P16BBg3Ts2DHFx8frF7/4hdq2bavrr79eDRs2DPm2J0+efNmbUOrVq6cJEyYoOTlZnTt3/tGP0VxKxYoVtXDhQk2ZMkUJCQn61a9+pbfffrvwT/L16tVTcnKyFi9erNtvv73w6x5++GFdf/316tSpkxISEtS5c2dv6xyBSNu6dau6du2qxMREJSUladiwYUpISCjsf/9H2V27di38JCsmJiak6x4yZIjS09ML//+xxx7TmTNn1Lp1a/Xv31/dunW76PLbt29XgwYNCv+79dZbtWjRoou2pYwaNUotWrRQmzZt1K1bt4v+UenEiRPVqlUr3XPPPRo6dKhuuOGGi/42bcmSJRoyZMjlPUBAMerXr59atWpV+KOiP3x+x8XF6fHHH1f37t2VlJSkSpUqXfS1lSpV0tChQzVv3jyNGDGi8NfvvPNODR8+XCkpKUpMTFS7du0u+jT7h/bv368+ffooISFBcXFxiouLU1pamu6++27l5uaqVatWSktLK/wx1lA888wzmjhxotq1a6eXX35ZnTp1uuTlNm7c+KMfjfu7unXras2aNXr//ffVpEkTJSQk6KabblKFChUkSS+++KJ27Nih+Ph4paSk6JFHHlGnTp2cs8lDDz2kPn368I81f6Ko4O8/KwEAKDKnTp0q/HcQL7zwgpYsWaK//vWvzq8rKChQcnKy3n333R/9PGgozp8/r5YtW2r37t2Fn+79VNu3b9fo0aML/9YPABAeBnEAKAYPPvig1q5dq7y8PNWrV09z5sxR06ZNQ/raTZs26cyZMz/69Lu4paen69prr1WbNm28HgcAlBUM4gAAAIAH/Iw4AAAA4AGDOAAAAOABgzgAAADgAYM4AAAA4AGDOAAAAOBByEtlQzmNMsLXuXNns2dlZZn9hhtuMHsoJxH56quvzH7o0CGzlytn//lu48aNzmOAm++FR8Xxe0JR38dQ7oPrGMJ9HMK9/kg8RqXh93ee726pqalhff3//u//hvX1oejbt2+RXn8o98F1DK7riIuLM7vrfTgjI8PsgwYNMruki87ceymuM+66ns9ffvml8xjCVdTPBZfieL6H8prmE3EAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAAwZxAAAAwAMGcQAAAMCDkNcXIjSxsbFm79evn9krV65s9kqVKpl92bJlZj937pzZJalBgwZm79Onj9lvuukms7vWF/7nf/6n2VEyhLKWqahX77HaLzIrGEuC0nCMLqX9PoSyTs618i3clXRFff2StGHDBrOnpaWZ3fU+3KlTJ7P/13/9l9k3b95sdkkaNmyY2ZOTk81+/fXXm33Hjh1mf+mll8wu+V9PGAnFcR/4RBwAAADwgEEcAAAA8IBBHAAAAPCAQRwAAADwgEEcAAAA8IBBHAAAAPCAQRwAAADwgD3iEeba3TllyhSzP/vss2afP3++2ceOHWv2mjVrml2SNm3aZPbc3FyzT5w40ezt27d3HgOKXnHs4A5XuDu8I7ED3PU4hPs4lobvQ7j3IRS+97W77mNJ+D64FMcO7nC5jrGov16Sfv3rX5u9bdu2Zj9y5IjZf/Ob35h97ty5Zo+JiTG7JH355Zdm37Nnj9l37dpl9u+++855DC6R+F6Fw/V8LwmvB4lPxAEAAAAvGMQBAAAADxjEAQAAAA8YxAEAAAAPGMQBAAAADxjEAQAAAA8YxAEAAAAPGMQBAAAADzihT4R16tTJ7IsXLzZ7rVq1zL59+3azDx8+3OwVK1Y0uyRt2bLF7C1btjT78ePHzV6nTh2zV61a1eySdOrUKbOHe3KQ0nDyjnCFex9DeYyL40Qw4dx+cSjqE/ZE4vsQidsoytsvKcfg4vvkTOGeoCSUE7C4bqOoT+ISiZOwxMXFmX3VqlVmP3HihNld74FPPvmk2ZctW2Z2SWrevLnZ9+7da/Y1a9aYvV+/fmYfOHCg2SX3yf/Cfa4Uxwl5iuMkWXwiDgAAAHjAIA4AAAB4wCAOAAAAeMAgDgAAAHjAIA4AAAB4wCAOAAAAeMAgDgAAAHgQFYS42LSo9/2WFbNnzzb7J598Yvbs7Oywbt+1XzsrK8t5HfHx8WbPz883e6tWrcweHR1t9pdeesnskpSRkeG8TFlXEnZku4S7V9n3XuZQjsGlNHyfwhWJ75Pv95iS8FwK9xhSU1PD+vpQhLtXuTj2MrskJSWZ3XUujS5duoR1+6792q73cUlq1qyZ2S9cuGD2Ro0amX3p0qVmz8nJMbskzZ8/33mZolQS9oyH8prmE3EAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAg/K+D6A0ue+++5yX+eKLL8x+8uRJs1999dVmr1OnjtldO75XrFhhdkkaPHiw2c+dO2f2PXv2mP3s2bNmb9iwodklqaCgwOyZmZnO60B4fO99LinHEO7u55KwKx1lYxd6uFw7kUvDMQwYMMB5md27d5u9W7duZq9UqZLZN27caPbevXubfdOmTWaXpJSUFLPn5eWZ/cCBA2Y/f/682V2ziCTVqFHD7MePH3deRzgisbO+OF4TfCIOAAAAeMAgDgAAAHjAIA4AAAB4wCAOAAAAeMAgDgAAAHjAIA4AAAB4wCAOAAAAeBAVhLiktrTvR5Xc92HChAlmr1atmvM21q1bZ3bX7s2YmBizf/rpp2avXr262V37TyX3/lDXLvScnByz79+/3+z/9m//ZnZJKlfO/jPkzJkznddR2pWF/dW+f1+JxG7o0rDnu6gf55LwGIR7H133IZTr9/04uI4xlL3J4SrqvcvDhg0zu+s8GJI0ffp0s99xxx1mr1Klitm3b99u9qpVq5q9YsWKZpek999/3+ynT582+5kzZ8zerl07s/fs2dPskvt9+q233jJ7uM8l1/M9lOsP9zWTnp7uvAyfiAMAAAAeMIgDAAAAHjCIAwAAAB4wiAMAAAAeMIgDAAAAHjCIAwAAAB4wiAMAAAAelPd9AMUpPj7e7BUqVDC7a3+2JB0+fNjs0dHRZv/ggw/MPnToULNv3rzZ7Hl5eWaXpKSkJLO7dqV3797d7Hv27DG7a8+45H6cXfvaMzMznbdR1vneeXylKOr91pEQ7g7tSOxa970H3PdO+0gojj3hRa158+ZmL1/eHltyc3Odt5GdnR1WnzJlitlvuukmszdp0sTsFy5cMLskxcbGmr1+/fpmd70H7t271+yHDh0yuyQtWLDA7BkZGWYPdw94Ue+0jxQ+EQcAAAA8YBAHAAAAPGAQBwAAADxgEAcAAAA8YBAHAAAAPGAQBwAAADxgEAcAAAA8uKL2iLv2W8fExJj9xIkTztv46quvzD548GCzz5071+y1atUye1pamtkHDBhgdkmaPHmy2a+++mqz5+fnm33VqlVm79Gjh9klqVevXmbfuXOn2RctWuS8jbIuErufrwTh7qcO93EsDd+ncB+jSFxHadjXXtRce5NLw57xxMREs1epUsXs5cq5P19s3bq12Q8ePGh213ksUlNTzb5v3z6zP/roo2aXpFmzZpn9X//1X83u2lVeu3Zts6enp5tdkh5++GGzz5s3z+zh7gEvDc93iU/EAQAAAC8YxAEAAAAPGMQBAAAADxjEAQAAAA8YxAEAAAAPGMQBAAAADxjEAQAAAA8YxAEAAAAPytQJfVxL+uvXr2/2r7/+2uyuBfiS9M4775j9008/NfuaNWvM3rVrV7NXqlQprOuXpISEBLOPGTPG7OfOnTP7unXrzN6nTx+zS9Jnn31m9o0bN5q9NJwkpagV9YlmSoNInGgm3NtwXX9xPBfD/V5G4rnAazJ84Z7AJNwTqISiUaNGZo+NjTV7RkaG2d98803nMUyfPt3sM2bMMPuzzz5rdtdJiapVq2b2L774wuyS1Lx5c7O75qG8vDyzv/3222b/l3/5F7NL0oIFC8zesGFDs7tOjFRWfk/gE3EAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAgzK1R/zw4cNmr1Wrltn/53/+x+yu3aGStG/fPrOvX78+rK9fu3at2QcOHGj2W265xeySNHz4cLPfe++9Zh8xYoTZH3vsMbOfP3/e7JJUpUoVs7v2j7r2m546dcp5DFe60rDfOhLHWNT70q+E/dnF8X0o6scpEjvnfd8Hl3D3kIfC9R4YExNj9jNnzph97NixzmN4/fXXzb5t2zazZ2Zmmt11PpGnnnrK7N26dTO7JE2aNMnsAwYMMLtrD/icOXPMnpKSYnbJvct88eLFZr/qqqvMnpub6zwGSyh7812viUjs3ucTcQAAAMADBnEAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAAwZxAAAAwAMGcQAAAMCDMrVHPC8vz+yu3dA1atQwe2xsrPMYZs+ebfYtW7aY/cKFC2Z//PHHzf7Xv/7V7HPnzjW7JD344INmd+0XHTlypNlde8rr1atndkl6/vnnzd6rVy+zR0dHO2+jrCsJO419H0NR7wiXwt8tXRzCfZwjcR/CvY6i3jnve8d3KFw7jYtjT7jrGFznuujevbvZlyxZYvYNGzaYXZLWrVtn9t27d5u9d+/eZq9evbrZjx49anbXfm1JGjx4sNlPnz5t9smTJ5s9OTnZ7K7zskjuc4p06NDB7KtXrzZ7Tk6O2V3P9+J4PYSCT8QBAAAADxjEAQAAAA8YxAEAAAAPGMQBAAAADxjEAQAAAA8YxAEAAAAPGMQBAAAAD8rUHvFrr73W7Nu2bTO7a494RkaG8xgyMzPNPmDAALPXqVPH7A8//LDZe/bsafbbbrvN7JI0ceJEs7v2qbt2nbt2g6anp5tdkiZMmGD25cuXmz0mJsbs2dnZzmMo6yKx49v3nnCXK2FXemlR1LvMS8Lj7PsYIrFnPNzrqF27ttm//vprs7t+727fvr3ZJWnfvn1mv+OOO8y+cePGsK7fdYyuPeWSe5a4/fbbzT5q1Ciz//a3vzX7//3f/5ldkv74xz+a3fU4fv7552bv0qWL8xiKWiR2kfOJOAAAAOABgzgAAADgAYM4AAAA4AGDOAAAAOABgzgAAADgAYM4AAAA4AGDOAAAAOBBVBDiYlPXjtaSoGvXrmZ37d188cUXw/p6SYqPjzf72rVrzR4dHW32cuXsPzsVFBSE9fWSdO7cubCuo1atWmYvX95eX79s2TKzS1Ljxo3NXq1aNbO79pe6ds6XBL53Eofye0JRH2NJ2B0d7jGUhPvgUhKOMdz3oLLwfXBJTU0N6+tdO8Kl8PcmJyYmmj03N9fsrnM89OjRw3kMzZo1M/uzzz5r9rS0NLN/9NFHZu/Tp4/ZQ3mfPn/+fFjX4doD3r9/f7Nv2LDB7JJUt25ds3/yySdm37Fjh9mbN29u9kjszQ9XKOdF4RNxAAAAwAMGcQAAAMADBnEAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAAwZxAAAAwAN7oXMpk5WVZfbNmzebvUuXLmZ37d2UpJiYGLPv2bPH7HFxcWZfv3692V17xK+77jqzS9KhQ4fM7trzWq9ePbNXrVrV7IcPHza7JD3yyCNmHzdunNkrV67svI2yrjScG8Al3N3OkdiFXhr2Sxf1Du4rQXG8XsJ9nEPZA+7byZMnzd6iRQuzX7hwwezPPfec8xgeffRRs7vehytUqGB21/7rBg0amN11ngxJevPNN81+9uxZs995551mP3LkiNld85YkDR8+3OwzZ840e8WKFZ23EY6S8nrhE3EAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAAwZxAAAAwIMydUKfli1bmt11wp5nnnnG7OXKuf/c4jrpQ0JCgtm7d+9u9vfff9/sPXv2NHv//v3NLknjx483e9u2bc3uug9//vOfzZ6Tk2N2Sfroo4/MPmTIELNv3LjR7Dt37nQew5UulJOPuF4PrusI9+uLQ1GfLKc4HoNwjyHc64+EkvA4lnZ9+/Z1XsZ1EhTXdbhOdjN48GCzv/baa2YP5UQz0dHRZnedeK9KlSpm79SpU1jXf/fdd5tdkn73u9+Z/YEHHjC768R9y5YtM3soJwf89NNPzZ6SkmL2Ro0amd01s4XyfC4J+EQcAAAA8IBBHAAAAPCAQRwAAADwgEEcAAAA8IBBHAAAAPCAQRwAAADwgEEcAAAA8CAqCHF5arh7ZItDx44dzd6sWTOz16hRw+zvvPOO8xg2bdpk9lGjRpk9Ozvb7K4d3bt27TL7tm3bzC5Jt9xyi9kzMzPN7tod+vHHH5s9KSnJ7JI0ZcoUs//7v/+72Tt37mz29evXO4/Bt3D3HpeG3dAuxfH7ku8d25HYf+37e10S3j+K4/nqe1d5uI9zcexdbtOmjdld71ExMTFmX758ufMYDh48aPZevXqZ3XWuiwkTJph93rx5Zm/RooXZJen06dNmP3bsmNkbNGhg9tmzZ5s9NjbW7JJ73pk+fbrZ4+LizF6vXj3nMYQr3L356enpztvgE3EAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAg/K+DyCSWrVqZfauXbuafdq0aWYvV87955b33nvP7GfOnDH7I488YnbXjm/XrvSpU6eaXZLuu+8+s9evX9/sAwYMMPtjjz1mdtdjJEmnTp0y+y9/+Uuzf/PNN87bQPh871UuCcK9j8XxGBX1bUTi+kvCLnKX4tgJ71u4e5UbNmxo9goVKpj9tddeM3soz5M1a9aY/ezZs2Z/4403zP7QQw+Z3fU+7dq/LUk9e/Y0+1133WX2bt26mX3OnDlmd+1al9wzl+t9+LvvvnPeRlFzPZ9dr4dQ8Ik4AAAA4AGDOAAAAOABgzgAAADgAYM4AAAA4AGDOAAAAOABgzgAAADgAYM4AAAA4EFUEOLi0tKwwzUtLS2sr3ftx16wYIHzOlx7urds2WL2I0eOmD0+Pt7sBw4cMHtOTo7ZJalmzZpmL1/eXj+fkZFh9mrVqpl98eLFZpeke++91+zPP/+82W+++eawj8G3ot45XBp2HhfH70sl4X6WdJF4roR7HeE+F8rC9zk1NdXs4e4Aj4QuXbqY3XWMtWrVMvvSpUudx/D222+bfc+ePWbPysoye/Pmzc1+6NAhs4dyLo2YmBizR0dHm901a3zyySdm/9nPfmZ2SapYsaLZs7Ozze46X4hrH3xxPJ9d0tPTnZfhE3EAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAA3shdClzzTXXmL1///5mf/jhh83u2jMuSSdOnDD7/v37zd65c2ezz5492+wNGzY0e2JiotklafXq1Wa/+uqrzX7dddeZ/euvvzZ75cqVzS5JsbGxZnftlK9UqZLzNq50kdjb7Hs3cyRuv6j3U5eEfe0l4Rh8P1dcysLOetdeZdcO71Cuw8X1/tG7d2+z79q1y+x33XWX8xgWLlxo9s2bN5t93LhxZh89erTZ7777brO3bNnS7JL7GFetWmV21zG6zgfSpk0bs0vS7bffbnbXfVixYoXzNsIRyvO9OPCJOAAAAOABgzgAAADgAYM4AAAA4AGDOAAAAOABgzgAAADgAYM4AAAA4AGDOAAAAOABgzgAAADgQZk6oY/LokWLzN69e3ez79ixw3kbrhPqxMfHmz0mJsbsjRs3NnuLFi3M3rNnT7NL0tatW83+85//3OyukxKdOnXK7MnJyWaXpCVLlpg9NzfX7DVq1HDeBvzjRDMl4zEojmMI9zaK+nEI5fpLwveqpEtPTzd7SkqK2du1a2f2b7/91nkMrpP+dO3a1eyukxLVrVvX7F9++aXZhw4danZJ2rNnj9lr1qxpdtcJe4YMGeI8BpcnnnjC7D169DC7a16pUKHCZR/TD4VycirXSX/CPcGVxCfiAAAAgBcM4gAAAIAHDOIAAACABwziAAAAgAcM4gAAAIAHDOIAAACABwziAAAAgAdX1B5x177Hxx9/3Ozz58933sbMmTPNfvr0abO7dmy79pB//fXXZp87d67ZJSkuLs7srmOcM2eO2V270FNTU80uSX/729/MvnPnTrM3atTIeRsIX0nfDR0JRb07ujh2dBf19YdyH1yXCfc+FPX1h3IbZUG4e5Vdv78nJSWZfd68eWZ37a6WpLfeesvsf/nLX8w+YsQIszdv3tzsrvfYxYsXm12Sdu3aZfZKlSqZfeHChWZ37UKfMmWK2SX3OUdc84jrvCwurudqKCKxJ9yFT8QBAAAADxjEAQAAAA8YxAEAAAAPGMQBAAAADxjEAQAAAA8YxAEAAAAPGMQBAAAAD8rUHvFatWqZfcmSJWbv2bOn2Q8fPnzZx/SPmjRpYvaUlBSzP/TQQ2bv2LGj2e+55x6zS9Lo0aPNftVVV5n9ueeeM/uMGTPM/sUXX5hdcu8yj46ONrtr3zpKx17lot7hHYnb8P31oSgJx1DUinoPeSi34ftxKgl7lWNiYszuOp/HLbfcYvasrCznMaSnp5vdtSf86NGjZq9atarZV69ebfYXXnjB7JJ04403mt31OLm+Pjs72+yuPeaSlJuba3bX6+W7774z+6effuo8Bksoz+Vw9+aHgk/EAQAAAA8YxAEAAAAPGMQBAAAADxjEAQAAAA8YxAEAAAAPGMQBAAAADxjEAQAAAA+ighAXm0Zip3BRGzhwoNlvuOEGs7v2Rbr2lEtSbGys2V27NytXrmz29u3bm/3QoUNmz8jIMLskJSUlmf3MmTNmX79+vdld9+HAgQNml9y7YuvVq2f2xo0bm921y7Yk8L2TOBRFvUPbpTgeI9+/N5aE/dbFsXPe93MhlNv3vY89NTU1rK8PRbh7lc+ePWv2+Ph4s1erVi2sLknVq1c3+7x588xesWJFs7du3drsx44dM3uHDh3MLknffvut2Xv16mX2rVu3hnX9R44cMbvknrk+++wzs9etW9fs+/btcx6DJZS9+q7ns+s6QnlN84k4AAAA4AGDOAAAAOABgzgAAADgAYM4AAAA4AGDOAAAAOABgzgAAADgAYM4AAAA4EGZ2iM+fvx4s7t2h7722mtmHz58uPMYtmzZYnbX3swmTZqY/ejRo2bPz883e7ly7j97FRQUmD0nJ8fs586dc96GJZRjTEtLM3tmZqbZXbvKN27c6DwG34p6R3ZR7zyOhNLw+5Lv3dKRUBzHWNTfy5LwffC9RzzcHeCRcNddd5l93bp1Zv/ggw/MfvPNNzuPYffu3WbfuXOn2UeMGGH2BQsWmL13795mj8TOetf5Pt577z2zu55LobxPnzx50uyufeqHDx82e8OGDc1eHM939ogDAAAApRSDOAAAAOABgzgAAADgAYM4AAAA4AGDOAAAAOABgzgAAADgAYM4AAAA4EGZ2iPeuXNns1999dVmX7Zsmdn37t3rPAbXdfzHf/yH2ZOTk81+9uxZ5zFYLly4ENbXS1J0dLTZXftLq1atavZQ7uPp06fNXrNmTbO79sS69pCXBOHuHC4L+6tdwt0dXRy3URyPc1EfQ0l4HEv77YdyDC6uYyyOPeEuGRkZZq9cubLZa9SoYfYxY8Y4j8F1noht27aZ3XUeipSUFOcxWFzn8pCk9PR0s7vOteF6n129erXZ8/LyzC5JPXr0MLtrZ/y+ffvMfvz4cbO7nu+uHeCRwB5xAAAAoIRiEAcAAAA8YBAHAAAAPGAQBwAAADxgEAcAAAA8YBAHAAAAPGAQBwAAADxgEAcAAAA8KO/7ACKpWrVqZned0Oeqq64y+/z5853HcOTIEbO7FvUfPHjQ7OXK2X92cp0MIT8/3+yhXMZ1H86fP2/23Nxcs588edLskpSQkGD28uXtp3alSpWct1HWlYYT0YR7jCXhRGQl4cRIRS0SJ8MpCd8rSyjfR9+vmXBPYFIcJ/wZPHiw2VetWmX2lStXmr1Zs2bOY8jKyjK76/vUoUMHs+fk5Ji9YsWKZg/lxHvhvo+63uddJxdcuHCh2SXp888/N7trJnM9TuEK5fleHK8ZPhEHAAAAPGAQBwAAADxgEAcAAAA8YBAHAAAAPGAQBwAAADxgEAcAAAA8YBAHAAAAPIgKQlxsWtJ3vEpS48aNze7aHX3s2DGzHz169HIPCUXEtSvWtUfcte/9+PHjl31MxS3cncTF8ZqOxH7pku5K2BPuUha+jy6R2CMeidsoytsPRbi7yuvWrWv2ChUqmP3EiRNh9VAUxz71ks71fQxF/fr1zR4dHW327Oxss4dyzhFLJPaIu4TymuYTcQAAAMADBnEAAADAAwZxAAAAwAMGcQAAAMADBnEAAADAAwZxAAAAwAMGcQAAAMCDkPeIAwAAAIgcPhEHAAAAPGAQBwAAADxgEAcAAAA8YBAHAAAAPGAQBwAAADxgEAcAAAA8YBAHAAAAPGAQBwAAADxgEAcAAAA8+H8wIK0FqKx01gAAAABJRU5ErkJggg==",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# viz: the clean image, the perturbation, and the adversarial image side by side\n",
"fig, ax = plt.subplots(1, 3, figsize=(7.5, 2.6))\n",
"ax[0].imshow(Xte[i0].reshape(28, 28), cmap=\"gray\"); ax[0].set_title(f\"clean: {CLASSES[pred_clean]}\", fontsize=8)\n",
"ax[1].imshow(np.sign(g_x).reshape(28, 28), cmap=\"gray\"); ax[1].set_title(\"sign(dL/dx)\", fontsize=8)\n",
"ax[2].imshow(x_adv.reshape(28, 28), cmap=\"gray\"); ax[2].set_title(f\"adversarial: {CLASSES[pred_adv]}\", fontsize=8)\n",
"for a in ax: a.axis(\"off\")\n",
"plt.tight_layout(); plt.show()"
]
},
{
"cell_type": "markdown",
"id": "2a92e4a1",
"metadata": {},
"source": [
"> **Caveat:** whether one specific image flips depends on the seed and the model, so we do not assert that it does. The robust, assertable claim is that a small step along $\\mathrm{sign}(\\partial L/\\partial x)$ raises the loss on the true class; we check that next.\n"
]
},
{
"cell_type": "code",
"execution_count": 34,
"id": "e7dbf2b5",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:17:44.784413Z",
"iopub.status.busy": "2026-06-10T19:17:44.784315Z",
"iopub.status.idle": "2026-06-10T19:17:44.818446Z",
"shell.execute_reply": "2026-06-10T19:17:44.818175Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"mean loss clean 0.334 -> adversarial 15.403\n",
"[ ok ] differentiability is a property attackers can use too\n"
]
}
],
"source": [
"# the assertable version: averaged over a handful of images, the FGSM step raises the loss\n",
"rows = range(0, 40)\n",
"clean_loss, adv_loss = 0.0, 0.0\n",
"for r in rows:\n",
" gx, _ = input_gradient(P, Xte[r], yte[r])\n",
" lg_c, _ = mlp_forward(Xte[r][None, :], P[\"W1\"], P[\"b1\"], P[\"W2\"], P[\"b2\"])\n",
" lg_a, _ = mlp_forward((Xte[r] + 0.5 * np.sign(gx))[None, :], P[\"W1\"], P[\"b1\"], P[\"W2\"], P[\"b2\"])\n",
" clean_loss += softmax_xent(lg_c, np.array([yte[r]]))[0]\n",
" adv_loss += softmax_xent(lg_a, np.array([yte[r]]))[0]\n",
"clean_loss /= len(rows); adv_loss /= len(rows)\n",
"print(f\"mean loss clean {clean_loss:.3f} -> adversarial {adv_loss:.3f}\")\n",
"assert adv_loss > clean_loss, \"an FGSM step in the input-gradient direction must raise the loss\"\n",
"print(\"[ ok ] differentiability is a property attackers can use too\")"
]
},
{
"cell_type": "markdown",
"id": "ac9be75b",
"metadata": {},
"source": [
"Three habits the draft argues for, all cheap, all catching real bugs:\n",
"- **Verify the loss at init is $\\ln K$.** A 30-second check; you ran it before training.\n",
"- **Overfit a tiny batch first.** If the model cannot drive loss toward zero on a handful of examples, it will not on the full set.\n",
"- **Look at the data right before it enters the model.** A broken normalization or a swapped label is obvious in one plotted batch and invisible in the loss curve.\n",
"\n",
"The deeper point: a falling loss is evidence the optimizer worked, not that the model learned what you wanted. Differentiability, the thing that makes training possible, is also what makes the model attackable.\n"
]
},
{
"cell_type": "markdown",
"id": "3f37b2ac",
"metadata": {},
"source": [
"## Test yourself\n",
"\n",
"Three parts: concept self-checks, auto-checked problems, and a capstone. Solutions are folded; try before you peek. Every answer is in this notebook; if unsure, re-run that section.\n"
]
},
{
"cell_type": "markdown",
"id": "c663df47",
"metadata": {},
"source": [
"### Part A: Concepts\n",
"\n",
"1. Why does an MLP with no activation collapse to a linear model? AnswerComposing two affine maps gives $X(W_1 W_2) + (b_1 W_2 + b_2) = X W' + b'$, a single affine map (you proved it in Exercise 9.1). Only hyperplane boundaries are reachable; the non-linearity is what adds expressive power.\n",
"2. In the `Value` engine, why does every `_backward` use `+=` rather than `=`? AnswerA node can feed multiple downstream ops; the chain rule sums the contributions from each path. `=` keeps only the last, halving (or worse) the gradient of any reused node. Exercise 9.3 reproduced this: `a*a` gave `3` instead of `6`.\n",
"3. You see the loss curve from Part 4 fall to near zero, but the lr=50 curve in Part 5 climbing and going NaN. Name the failure and its fix. AnswerLearning rate too high: the update overshoots, the loss climbs, logits overflow to NaN. The fix is a smaller step (lr=0.2 trained fine on the same model and gradients).\n",
"4. What is the cross-entropy loss at initialization for a 10-class balanced classifier, and why is checking it valuable? Answer$\\ln 10 \\approx 2.30$, because untrained logits are near-uniform. Checking it (you did, before FashionMNIST training) catches normalization bugs, all-zero inputs, and broken inits in 30 seconds.\n",
"5. Why is the softmax-cross-entropy gradient $p - y$ \"elegant for implementation\"? AnswerIt is \"predicted minus actual\": no division by tiny probabilities, no $\\log 0$. The off-diagonal softmax-Jacobian terms cancel against the cross-entropy derivative, leaving this clean form. You finite-difference-verified it in Exercise 9.5.\n",
"6. Why is the FashionMNIST model standardized with TRAIN statistics, even for the test set? AnswerUsing test statistics leaks information about the test distribution into preprocessing. The model must see test data through exactly the transform it learned on, so `mu`/`sd` come from train only.\n",
"7. One-liner: create a length-10 vector of the per-class counts in `ytr`. Answer`np.bincount(ytr, minlength=10)`. We printed exactly this under the data-viz cell to confirm class balance.\n"
]
},
{
"cell_type": "markdown",
"id": "b8e6473e",
"metadata": {},
"source": [
"### Part B: Auto-checked problems\n",
"\n",
"Two problems that make you compute something new with the chapter's pieces. Stubs first, folded solutions after the checks.\n"
]
},
{
"cell_type": "markdown",
"id": "b07579f0",
"metadata": {},
"source": [
"**Problem B.1: `tanh` for the `Value` engine** · `Difficulty 2/5 · ~8 min`\n",
"\n",
"Add a `value_tanh(v)` free function that returns a new `Value` equal to $\\tanh(v)$ with a correct `_backward` (derivative $1-\\tanh^2$). It must agree with `torch.autograd`. This is the activation Karpathy uses in the original micrograd video.\n"
]
},
{
"cell_type": "code",
"execution_count": 35,
"id": "1ac37406",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:17:44.819828Z",
"iopub.status.busy": "2026-06-10T19:17:44.819731Z",
"iopub.status.idle": "2026-06-10T19:17:44.834864Z",
"shell.execute_reply": "2026-06-10T19:17:44.834616Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] B.1 value_tanh: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 35,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import math\n",
"def value_tanh(v):\n",
" \"\"\"Return tanh(v) as a Value with a correct backward closure. v is a Value.\"\"\"\n",
" # TODO 1: t = math.tanh(v.data); out = Value(t, (v,), \"tanh\")\n",
" # TODO 2: define _backward: v.grad += (1 - t*t) * out.grad ; attach and return out\n",
" out = None\n",
" attempted(out)\n",
" return out\n",
"\n",
"def _check_value_tanh(fn):\n",
" g = np.random.default_rng(5)\n",
" xv = float(g.standard_normal())\n",
" x = Value(xv); y = fn(x); y.backward()\n",
" tx = torch.tensor(xv, requires_grad=True); ty = torch.tanh(tx); ty.backward()\n",
" check_close(y.data, ty.item(), msg=\"tanh forward must match torch\")\n",
" check_close(x.grad, tx.grad.item(), msg=\"tanh backward must match torch (1 - tanh^2)\")\n",
"\n",
"check(\"B.1 value_tanh\", lambda: _check_value_tanh(value_tanh))"
]
},
{
"cell_type": "markdown",
"id": "5f91fa78",
"metadata": {},
"source": [
"Hint 1Compute `t = math.tanh(v.data)` once, reuse it in both the forward value and the derivative `1 - t*t`.\n",
"\n",
"Solution\n",
"\n",
"```python\n",
"def value_tanh(v):\n",
" t = math.tanh(v.data)\n",
" out = Value(t, (v,), \"tanh\")\n",
" def _backward():\n",
" v.grad += (1 - t * t) * out.grad\n",
" out._backward = _backward\n",
" return out\n",
"```\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 36,
"id": "e7898c4c",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:17:44.836130Z",
"iopub.status.busy": "2026-06-10T19:17:44.836042Z",
"iopub.status.idle": "2026-06-10T19:17:44.853484Z",
"shell.execute_reply": "2026-06-10T19:17:44.853182Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] B.1 value_tanh\n"
]
},
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def value_tanh(v):\n",
" t = math.tanh(v.data)\n",
" out = Value(t, (v,), \"tanh\")\n",
" def _backward():\n",
" v.grad += (1 - t * t) * out.grad\n",
" out._backward = _backward\n",
" return out\n",
"\n",
"check(\"B.1 value_tanh\", lambda: _check_value_tanh(value_tanh), required=True)"
]
},
{
"cell_type": "markdown",
"id": "975f3e74",
"metadata": {},
"source": [
"**Problem B.2: dead-ReLU fraction** · `Difficulty 2/5 · ~10 min`\n",
"\n",
"Write `dead_relu_fraction(P, X)` returning the fraction of the 128 hidden units that are zero (dead) for *every* example in a batch `X`. A unit is dead if its post-ReLU activation `H[:, j]` is zero across the whole batch. This is the diagnostic the draft recommends logging during training.\n"
]
},
{
"cell_type": "code",
"execution_count": 37,
"id": "eb778824",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:17:44.857595Z",
"iopub.status.busy": "2026-06-10T19:17:44.857496Z",
"iopub.status.idle": "2026-06-10T19:17:44.978435Z",
"shell.execute_reply": "2026-06-10T19:17:44.978175Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] B.2 dead_relu_fraction: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 37,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def dead_relu_fraction(P, X):\n",
" \"\"\"Fraction of hidden units whose ReLU output is 0 for every row of X.\"\"\"\n",
" _, cache = mlp_forward(X, P[\"W1\"], P[\"b1\"], P[\"W2\"], P[\"b2\"])\n",
" _, _, H = cache # H is (n, h), the post-ReLU hidden activations\n",
" # TODO 1: a unit j is dead if H[:, j] is 0 for ALL rows\n",
" # TODO 2: return the fraction of dead units (a float in [0,1])\n",
" frac = None\n",
" attempted(frac)\n",
" return frac\n",
"\n",
"def _check_dead_relu(fn):\n",
" # a healthy trained model has few dead units; an all-negative-bias model has all dead\n",
" healthy = fn(P, Xte[:256])\n",
" assert 0.0 <= healthy < 0.5, f\"a trained model should have <50% dead units, got {healthy:.2f}\"\n",
" P_dead = init_params(); P_dead[\"b1\"] = np.full(128, -1e6) # force every pre-activation negative\n",
" assert abs(fn(P_dead, Xte[:256]) - 1.0) < 1e-9, \"with hugely negative bias, all units must be dead\"\n",
"\n",
"check(\"B.2 dead_relu_fraction\", lambda: _check_dead_relu(dead_relu_fraction))"
]
},
{
"cell_type": "markdown",
"id": "081ec706",
"metadata": {},
"source": [
"Hint 1`(H == 0).all(axis=0)` is a length-`h` boolean: True where a unit is zero across the whole batch. The dead fraction is its `.mean()`.\n",
"\n",
"Solution\n",
"\n",
"```python\n",
"def dead_relu_fraction(P, X):\n",
" _, cache = mlp_forward(X, P[\"W1\"], P[\"b1\"], P[\"W2\"], P[\"b2\"])\n",
" _, _, H = cache\n",
" return float((H == 0).all(axis=0).mean())\n",
"```\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 38,
"id": "f41c9a05",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:17:44.983589Z",
"iopub.status.busy": "2026-06-10T19:17:44.983494Z",
"iopub.status.idle": "2026-06-10T19:17:45.267959Z",
"shell.execute_reply": "2026-06-10T19:17:45.267566Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] B.2 dead_relu_fraction\n",
"dead units in the trained model: 0.0%\n"
]
}
],
"source": [
"def dead_relu_fraction(P, X):\n",
" _, cache = mlp_forward(X, P[\"W1\"], P[\"b1\"], P[\"W2\"], P[\"b2\"])\n",
" _, _, H = cache\n",
" return float((H == 0).all(axis=0).mean())\n",
"\n",
"check(\"B.2 dead_relu_fraction\", lambda: _check_dead_relu(dead_relu_fraction), required=True)\n",
"print(f\"dead units in the trained model: {dead_relu_fraction(P, Xte[:256]):.1%}\")"
]
},
{
"cell_type": "markdown",
"id": "893dd75e",
"metadata": {},
"source": [
"### Part C: Capstone: cross-check the whole NumPy MLP against PyTorch\n",
"\n",
"Redo the chapter's MLP, but this time prove your hand-derived gradients match PyTorch's autograd on the *same weights and the same batch*, then train both and compare. This is the strongest possible self-check: your code is right iff it reproduces a real implementation.\n",
"\n",
"**Deliverables**\n",
"1. Build a `torch.nn` MLP with the same architecture (784 -> 128 -> 10, ReLU) and copy your NumPy `P` into it (remember `nn.Linear` stores `weight` as `(out, in)`, so transpose).\n",
"2. Run one forward + backward in torch on a batch; extract `weight.grad` / `bias.grad`.\n",
"3. Assert your `mlp_backward` gradients match torch's to a tight tolerance.\n",
"4. Train the torch model a few epochs and confirm it lands near your NumPy accuracy.\n",
"\n",
"**Self-assessment (pass / partial / fail)**\n",
"- (a) the torch model's logits match your `mlp_forward` logits on the same batch (`atol=1e-4`);\n",
"- (b) all four gradient tensors match (`atol=1e-4`), with the `nn.Linear` transpose handled correctly;\n",
"- (c) you used `loss.backward()` once and read `.grad`, with `p.grad = None` (never `.data`) if you loop;\n",
"- (d) the torch model trains to within a couple points of your NumPy accuracy;\n",
"- (e) the notebook runs top-to-bottom.\n",
"\n",
"My solution (reference, runs in seconds on CPU)\n",
"\n",
"```python\n",
"import torch.nn as nn\n",
"import torch.nn.functional as F\n",
"\n",
"# 1. mirror architecture and copy weights (transpose for nn.Linear's (out,in) layout)\n",
"net = nn.Sequential(nn.Linear(784, 128), nn.ReLU(), nn.Linear(128, 10)).double()\n",
"with torch.no_grad():\n",
" net[0].weight.copy_(torch.tensor(P[\"W1\"].T)); net[0].bias.copy_(torch.tensor(P[\"b1\"]))\n",
" net[2].weight.copy_(torch.tensor(P[\"W2\"].T)); net[2].bias.copy_(torch.tensor(P[\"b2\"]))\n",
"\n",
"# 2-3. forward + backward on one batch, compare to your hand grads\n",
"xb = torch.tensor(Xtr[:128]); yb = torch.tensor(ytr[:128])\n",
"logits_t = net(xb)\n",
"my_logits, cache = mlp_forward(Xtr[:128], P[\"W1\"], P[\"b1\"], P[\"W2\"], P[\"b2\"])\n",
"assert np.allclose(logits_t.detach().numpy(), my_logits, atol=1e-4) # (a)\n",
"loss_t = F.cross_entropy(logits_t, yb)\n",
"net.zero_grad(); loss_t.backward()\n",
"_, dlogits = softmax_xent(my_logits, ytr[:128])\n",
"my = mlp_backward(dlogits, cache, P[\"W2\"])\n",
"assert np.allclose(net[0].weight.grad.numpy(), my[\"W1\"].T, atol=1e-4) # (b), note the transpose\n",
"assert np.allclose(net[2].weight.grad.numpy(), my[\"W2\"].T, atol=1e-4)\n",
"print(\"[ ok ] hand-derived gradients match torch.autograd\")\n",
"\n",
"# 4. train torch a few epochs\n",
"opt = torch.optim.SGD(net.parameters(), lr=0.2)\n",
"for ep in range(EPOCHS):\n",
" for i in range(0, len(Xtr), 128):\n",
" xb = torch.tensor(Xtr[i:i+128]); yb = torch.tensor(ytr[i:i+128])\n",
" loss = F.cross_entropy(net(xb), yb)\n",
" opt.zero_grad(); loss.backward(); opt.step()\n",
"acc_t = (net(torch.tensor(Xte)).argmax(1).numpy() == yte).mean()\n",
"print(f\"torch test acc {acc_t:.4f} vs numpy {final_acc:.4f}\")\n",
"```\n",
"The lesson lands when (b) passes: the `nn.Linear` transpose is the one-hour gotcha the draft warned about, and your six-line backward pass is doing exactly what PyTorch does.\n"
]
},
{
"cell_type": "markdown",
"id": "a6729053",
"metadata": {},
"source": [
"## Reflection\n",
"\n",
"Write ~150 words, in the cell below, on the dumbest bug you hit in this notebook and how you found it. Maybe it was a `=` where you needed `+=` and the `a*a` check caught it; maybe a transpose in the matmul backward; maybe the `nn.Linear` weight layout in the capstone. Nobody grades this. Writing it is the point: the habit of naming a bug and the signal that exposed it is what turns \"my model is not training\" from a panic into a checklist. The three diagnostics from the safety lens (loss at init, overfit a tiny batch, look at the data) exist precisely because someone wrote down the bug that motivated each one.\n"
]
},
{
"cell_type": "markdown",
"id": "776794b3",
"metadata": {},
"source": [
"*(your reflection here)*\n"
]
},
{
"cell_type": "markdown",
"id": "67e9f70b",
"metadata": {},
"source": [
"## Going further\n",
"\n",
"- Karpathy, *The spelled-out intro to neural networks and backpropagation: building micrograd* (YouTube), the two-hour walkthrough of the engine you just built; the best autograd-intuition resource there is.\n",
"- ARENA 3.0, *chapter0 part4 backprop*, autograd at the tensor level, where micrograd leaves off.\n",
"- Howard & Gugger, *fastbook* ch. 4 *MNIST Basics*, a second pass at building a digit classifier from foundations.\n",
"- Prince, *Understanding Deep Learning* (free PDF), ch. 3-7, the most modern textbook treatment of MLPs and backprop.\n",
"- D2L, *Multilayer Perceptrons* and *Implementation from Scratch*, a parallel view of this chapter's NumPy MLP.\n",
"- Goodfellow, Shlens, Szegedy (2015), *Explaining and Harnessing Adversarial Examples*, the FGSM paper behind the safety lens.\n",
"\n",
"## What this enables\n",
"\n",
"- **Ch 10, PyTorch Foundations**: you now know what `.backward()` does, so `nn.Module` is an abstraction you trust rather than a black box. The next chapter is the PyTorchify ladder on top of this worldview.\n",
"- **Ch 11, Training Deep Networks**: you have met the dead-ReLU and lr-too-high failures here; Ch 11 fixes init, normalization, and optimizers properly (we reached ~0.85 on the subset; better init + Adam closes much of the gap to a CNN).\n",
"- **Ch 12, CNNs**: an MLP on images is the baseline; CNNs swap `nn.Linear` for `nn.Conv2d` and add spatial structure, but the training loop and backward pass are identical to what you built here.\n"
]
},
{
"cell_type": "markdown",
"id": "f56a8ae0",
"metadata": {},
"source": [
"---\n",
"*Built top-to-bottom. If every check above printed `[ ok ]`, you've reproduced the chapter. Total running time and verification stamp written by CI.*\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "obvix-nb",
"language": "python",
"name": "obvix-nb"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
},
"obvix": {
"title": "Ch 09, Intro to Neural Networks"
}
},
"nbformat": 4,
"nbformat_minor": 5
}