{
"cells": [
{
"cell_type": "markdown",
"id": "a56df005",
"metadata": {},
"source": [
"# Ch 12 — CNNs & Computer Vision (notebook)\n",
"\n",
"`[<- 11 training-deep-networks]` · **this notebook** · `[13 sequences-and-time-series ->]`\n",
"\n",
"Runs top-to-bottom in ~5 min on free Colab CPU. Last verified 2026-06-11.\n",
"\n",
"**What you'll build**\n",
"- A 2D convolution from scratch in NumPy that reproduces the hand-computed `[[19, 25], [37, 43]]` and then agrees with `nn.Conv2d` to machine precision.\n",
"- The output-shape formula `floor((W - k + 2P)/S) + 1`, tested against PyTorch on a dozen layer configs.\n",
"- A learned kernel: start from random weights and recover a hand-designed vertical-edge detector by gradient descent (ground-truth recovery).\n",
"- A small CNN trained on FashionMNIST, with its parameters and FLOPs counted by hand and checked against the library.\n",
"- A deliberate failure: a model that scores 10% because you forgot one line, then the same model fixed.\n",
"- A pretrained checkpoint you reload and verify outputs against, bit-identically.\n",
"\n",
"**How this notebook works.** Code cells with a `# TODO` are yours to fill in. Run the cell to grade yourself: `[ ok ]` passed, `[FAIL]` shows what went wrong, `[ -- ]` means not attempted yet. Every exercise has a hint ladder (open only as many as you need) and a folded solution below it. The notebook runs top-to-bottom even if you fill in nothing: the solution cells redefine the functions so later cells work. See Ch 00 for the full protocol.\n"
]
},
{
"cell_type": "markdown",
"id": "d744fc19",
"metadata": {},
"source": [
"## Before you start\n",
"\n",
"1. A fully-connected layer between two 224x224 RGB images has about 22 billion weights. A 3x3 convolution kernel has nine. Where did the other 22 billion go? AnswerThey were never needed. A conv shares one small filter across every spatial location instead of learning a separate weight for every (input pixel, output pixel) pair. That weight sharing, not a clever optimizer, is what makes vision tractable at scale.\n",
"2. A 3x3 conv with no padding turns a 5x5 input into what size output? Answer3x3. You lose `k - 1 = 2` pixels total along each axis. Stack ten such layers and you lose twenty pixels each way. Padding exists to stop that bleed.\n",
"3. Predict before you run: a `nn.Conv2d(3, 16, kernel_size=5)` with bias. How many learnable parameters? Answer`3 * 16 * 5 * 5 + 16 = 1216`. Each of the 16 output channels is a `3x5x5` filter plus one bias. Image size does not enter the count, which is the whole point.\n"
]
},
{
"cell_type": "markdown",
"id": "c31385de",
"metadata": {},
"source": [
"## Setup\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "7433e087",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:41.084417Z",
"iopub.status.busy": "2026-06-10T19:24:41.084332Z",
"iopub.status.idle": "2026-06-10T19:24:50.614426Z",
"shell.execute_reply": "2026-06-10T19:24:50.613990Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"numpy 2.2.6 · torch 2.12.0+cpu · torchvision 0.27.0+cpu · device cpu\n"
]
}
],
"source": [
"import numpy as np\n",
"import torch\n",
"import torch.nn as nn\n",
"import torch.nn.functional as F\n",
"import matplotlib.pyplot as plt\n",
"import torchvision\n",
"\n",
"device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
"print(f\"numpy {np.__version__} · torch {torch.__version__} · torchvision {torchvision.__version__} · device {device}\")\n",
"if np.__version__ < \"2.0\":\n",
" print(\"WARN: written for NumPy 2.x; older versions may differ slightly\")"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "20bd88c1",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.626721Z",
"iopub.status.busy": "2026-06-10T19:24:50.626485Z",
"iopub.status.idle": "2026-06-10T19:24:50.669556Z",
"shell.execute_reply": "2026-06-10T19:24:50.668833Z"
}
},
"outputs": [],
"source": [
"import os, random\n",
"SEED = 0\n",
"FAST = bool(os.environ.get('NB_FAST')) # CI smoke mode: ~10x fewer steps, same code paths\n",
"\n",
"# Training-step budgets. FAST keeps the SAME code paths, fewer iterations.\n",
"KERNEL_STEPS = 80 if FAST else 600 # steps to recover the edge kernel (Part 3)\n",
"TRAIN_BATCHES = 80 if FAST else 300 # mini-batches for the small CNN (Part 4)\n",
"N_TRAIN = 4000 if FAST else 12000 # FashionMNIST training subset size\n",
"N_TEST = 1000 if FAST else 2000 # FashionMNIST test subset size\n",
"BATCH = 128 # mini-batch size (fits CPU RAM comfortably)\n",
"\n",
"# One CPU thread keeps timing stable and deterministic on shared/loaded machines;\n",
"# this tiny CNN does not benefit from more, and oversubscription can slow it 10x.\n",
"torch.set_num_threads(1)\n",
"\n",
"rng = np.random.default_rng(SEED)\n",
"torch.manual_seed(SEED); random.seed(SEED)\n",
"torch.use_deterministic_algorithms(False) # CPU conv is deterministic enough; see the reproducibility note\n",
"\n",
"# ── house self-check harness (identical across all chapter notebooks) ──\n",
"import numpy as _np\n",
"\n",
"def check(label, test_fn, required=False):\n",
" \"\"\"Run one self-check. test_fn raises AssertionError (with a teaching\n",
" message) on failure, NotImplementedError if the stub is unfilled.\n",
" required=True is used only in solution cells; it is what CI grades.\"\"\"\n",
" try:\n",
" test_fn()\n",
" except NotImplementedError:\n",
" if required:\n",
" raise AssertionError(f\"{label}: reference solution incomplete\")\n",
" print(f\"[ -- ] {label}: not attempted yet — fill in the TODO above, then re-run.\")\n",
" return False\n",
" except AssertionError as e:\n",
" if required:\n",
" raise\n",
" print(f\"[FAIL] {label}: {e}\")\n",
" return False\n",
" print(f\"[ ok ] {label}\")\n",
" return True\n",
"\n",
"def attempted(*vals):\n",
" \"\"\"Treat None placeholders as 'not attempted'.\"\"\"\n",
" if any(v is None for v in vals):\n",
" raise NotImplementedError\n",
"\n",
"def check_shape(x, want):\n",
" assert tuple(x.shape) == tuple(want), \\\n",
" f\"shape {tuple(x.shape)}, expected {tuple(want)} — check your reshape/transpose order\"\n",
"\n",
"def check_close(got, want, atol=1e-5, rtol=1e-4, msg=\"\"):\n",
" g, w = _np.asarray(got, dtype=float), _np.asarray(want, dtype=float)\n",
" assert g.shape == w.shape, f\"shape {g.shape} vs expected {w.shape}. {msg}\"\n",
" bad = ~_np.isclose(g, w, atol=atol, rtol=rtol)\n",
" assert not bad.any(), \\\n",
" f\"{bad.mean():.2%} of values wrong (max diff {abs(g - w).max():.3g}). {msg}\""
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "1d4d62fa",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.670595Z",
"iopub.status.busy": "2026-06-10T19:24:50.670503Z",
"iopub.status.idle": "2026-06-10T19:24:50.683184Z",
"shell.execute_reply": "2026-06-10T19:24:50.682575Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"helpers ready: out_size, n_params, layer_summary, show_grid\n"
]
}
],
"source": [
"# house helpers: tiny, defined here, never imported (spec: self-contained)\n",
"def out_size(w, k, stride=1, pad=0):\n",
" \"\"\"Spatial output size of a conv/pool along one axis: floor((w - k + 2p)/s) + 1.\"\"\"\n",
" return (w - k + 2 * pad) // stride + 1\n",
"\n",
"def n_params(module):\n",
" \"\"\"Total learnable parameters in a module.\"\"\"\n",
" return sum(p.numel() for p in module.parameters())\n",
"\n",
"@torch.no_grad()\n",
"def layer_summary(model, input_shape):\n",
" \"\"\"Dummy-forward printout: name, output shape, and param count per submodule\n",
" (d2l's lenet layer_summary, inlined). input_shape excludes the batch dim.\"\"\"\n",
" x = torch.zeros(1, *input_shape)\n",
" for name, layer in model.named_children():\n",
" x = layer(x)\n",
" print(f\"{name:14} {str(tuple(x.shape)):>22} params={n_params(layer):,}\")\n",
" return x.shape\n",
"\n",
"def show_grid(imgs, titles=None, ncols=6, cmap=\"gray\"):\n",
" # viz: small image grid for FashionMNIST samples / feature maps\n",
" n = len(imgs)\n",
" nrows = (n + ncols - 1) // ncols\n",
" fig, axes = plt.subplots(nrows, ncols, figsize=(1.5 * ncols, 1.6 * nrows))\n",
" for i, ax in enumerate(np.array(axes).ravel()):\n",
" if i < n:\n",
" ax.imshow(np.asarray(imgs[i]), cmap=cmap)\n",
" if titles is not None:\n",
" ax.set_title(titles[i], fontsize=8)\n",
" ax.axis(\"off\")\n",
" plt.tight_layout(); plt.show()\n",
"\n",
"print(\"helpers ready: out_size, n_params, layer_summary, show_grid\")"
]
},
{
"cell_type": "markdown",
"id": "1abcfd73",
"metadata": {},
"source": [
"> **Note:** seeds make this notebook's printed numbers reproduce on CPU. Library versions and BLAS threading can shift the last digit or two, so the checks below use tolerances and behavioral properties, never bitwise float equality. If your training loss is 0.41 and the page says 0.40, you did nothing wrong. The only exact-equality check in the notebook is a checkpoint round-trip (Part 5), where exactness is the point.\n"
]
},
{
"cell_type": "markdown",
"id": "c342d8d8",
"metadata": {},
"source": [
"## The map\n",
"\n",
"> **Part 1 — Convolution as feature detection.** Hand-compute a 2x2 cross-correlation to get `[[19, 25], [37, 43]]`, write it in NumPy, and prove `nn.Conv2d` computes the same thing.\n",
"> **Part 2 — Padding, strides, pooling, receptive field.** Derive the output-shape formula and test it against PyTorch; build max-pool from scratch; compute receptive fields with the recurrence.\n",
"> **Part 3 — Learn the kernel.** Hand-design a vertical-edge detector, then recover it from random initialization by gradient descent. The fit is the check.\n",
"> **Part 4 — A small CNN, accounted for.** Train a LeNet-shaped CNN on FashionMNIST, count its parameters and FLOPs by hand, then break it (10% accuracy) and fix it.\n",
"> **Part 5 — Load a pretrained net and verify.** Save the trained CNN's weights, reload them into a fresh module, and verify the outputs are bit-identical.\n"
]
},
{
"cell_type": "markdown",
"id": "19ac329c",
"metadata": {},
"source": [
"## Part 1 — Convolution as feature detection\n",
"\n",
"> **Objectives**\n",
"> - Hand-compute a 2D cross-correlation and reproduce the canonical `[[19, 25], [37, 43]]`.\n",
"> - Write `corr2d` in NumPy and check it against the hand value and against `nn.Conv2d`.\n",
"> - See multi-channel convolution: the weight tensor is `(C_out, C_in, kH, kW)`.\n",
"\n",
"A convolution is a sliding dot product. Take a small filter, slide it over the input, and at each position multiply the filter by the underlying patch and sum. The result is a feature map. The filter is shared across every position, which buys translation equivariance (shift the input, the output shifts the same way) for free.\n",
"\n",
"> **Note:** what deep learning calls \"convolution\" is technically *cross-correlation*: the kernel is not flipped. PyTorch's `nn.Conv2d` does cross-correlation. We use the deep-learning convention throughout, and the math below matches it exactly.\n"
]
},
{
"cell_type": "markdown",
"id": "8ca75747",
"metadata": {},
"source": [
"### 1.1 The hand computation\n",
"\n",
"Take the 3x3 input on the left and the 2x2 kernel on the right. Slide the kernel over the four valid positions. The top-left output is `0*0 + 1*1 + 3*2 + 4*3 = 0 + 1 + 6 + 12 = 19`.\n",
"\n",
"$$X = \\begin{bmatrix} 0 & 1 & 2 \\\\ 3 & 4 & 5 \\\\ 6 & 7 & 8 \\end{bmatrix}, \\quad K = \\begin{bmatrix} 0 & 1 \\\\ 2 & 3 \\end{bmatrix} \\;\\Rightarrow\\; \\begin{bmatrix} 19 & 25 \\\\ 37 & 43 \\end{bmatrix}$$\n",
"\n",
"> **Predict:** the bottom-right output uses the patch `[[4,5],[7,8]]`. Compute it on paper before running. Answer`4*0 + 5*1 + 7*2 + 8*3 = 0 + 5 + 14 + 24 = 43`. That is the `43` in the corner.\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "06ff5275",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.684405Z",
"iopub.status.busy": "2026-06-10T19:24:50.684032Z",
"iopub.status.idle": "2026-06-10T19:24:50.693952Z",
"shell.execute_reply": "2026-06-10T19:24:50.693227Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"top-left output = 19 (expected 19)\n"
]
}
],
"source": [
"# the four output positions, computed by hand, one term at a time\n",
"X = np.array([[0, 1, 2], [3, 4, 5], [6, 7, 8]], dtype=float)\n",
"K = np.array([[0, 1], [2, 3]], dtype=float)\n",
"top_left = X[0, 0]*K[0, 0] + X[0, 1]*K[0, 1] + X[1, 0]*K[1, 0] + X[1, 1]*K[1, 1]\n",
"print(f\"top-left output = {top_left:.0f} (expected 19)\")"
]
},
{
"cell_type": "markdown",
"id": "76b32fa0",
"metadata": {},
"source": [
"> **Interpretation.** The single number 19 is one dot product between the kernel and a 2x2 window of the input. The whole feature map is that dot product repeated at every valid window. Everything else in this chapter is bookkeeping around this one operation.\n"
]
},
{
"cell_type": "markdown",
"id": "0231dba2",
"metadata": {},
"source": [
"### Exercise 12.1 — `corr2d` from scratch\n",
"`Difficulty 2/5 · ~12 min`\n",
"\n",
"Implement single-channel 2D cross-correlation. Given input `X` of shape `(H, W)` and kernel `K` of shape `(kH, kW)`, return the `(H-kH+1, W-kW+1)` feature map. No padding, stride 1. The check verifies you reproduce `[[19, 25], [37, 43]]` and then matches an independent reference on random inputs.\n",
"\n",
"Harder: after it works, add a `stride` argument and confirm `corr2d(X, K, stride=2)` returns a single value.\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "a4fa60ea",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.694973Z",
"iopub.status.busy": "2026-06-10T19:24:50.694880Z",
"iopub.status.idle": "2026-06-10T19:24:50.708464Z",
"shell.execute_reply": "2026-06-10T19:24:50.707892Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] 12.1 hand value: not attempted yet — fill in the TODO above, then re-run.\n",
"[ -- ] 12.1 vs reference: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def corr2d(X, K):\n",
" \"\"\"2D cross-correlation, single channel, no padding, stride 1.\n",
" X: (H, W), K: (kH, kW) -> (H-kH+1, W-kW+1).\"\"\"\n",
" X, K = np.asarray(X, dtype=float), np.asarray(K, dtype=float)\n",
" kH, kW = K.shape\n",
" H_out, W_out = X.shape[0] - kH + 1, X.shape[1] - kW + 1\n",
" out = np.zeros((H_out, W_out))\n",
" # TODO 1: for each output position (i, j), take the kH x kW window of X\n",
" # starting at (i, j) and set out[i, j] to the sum of window * K.\n",
" # TODO 2: leave the loop body raising until you fill it in.\n",
" raise NotImplementedError\n",
" return out\n",
"\n",
"def _toy_corr():\n",
" got = corr2d(X, K)\n",
" check_close(got, [[19, 25], [37, 43]],\n",
" msg=\"top-left should be 0*0+1*1+3*2+4*3 = 19; full map [[19,25],[37,43]]\")\n",
"\n",
"def _ref_corr():\n",
" # independent reference via numpy stride tricks (different code path)\n",
" A = rng.standard_normal((7, 9)); W = rng.standard_normal((3, 4))\n",
" kH, kW = W.shape\n",
" ref = np.zeros((A.shape[0]-kH+1, A.shape[1]-kW+1))\n",
" for i in range(ref.shape[0]):\n",
" for j in range(ref.shape[1]):\n",
" ref[i, j] = (A[i:i+kH, j:j+kW] * W).sum()\n",
" check_close(corr2d(A, W), ref, msg=\"disagrees with the reference cross-correlation\")\n",
"\n",
"check(\"12.1 hand value\", _toy_corr)\n",
"check(\"12.1 vs reference\", _ref_corr)"
]
},
{
"cell_type": "markdown",
"id": "06b18770",
"metadata": {},
"source": [
"Hint 1 (conceptual)Two nested loops over the output positions `(i, j)`. The window of `X` for output `(i, j)` is `X[i:i+kH, j:j+kW]`. The output value is the elementwise product with `K`, summed.\n",
"\n",
"Hint 2 (pseudocode)\n",
"\n",
"```python\n",
"for i in range(H_out):\n",
" for j in range(W_out):\n",
" window = X[i:i+kH, j:j+kW] # (kH, kW)\n",
" out[i, j] = (window * K).sum()\n",
"return out\n",
"```\n",
"\n",
"\n",
"Help — my output is shifted or the wrong shapeThe window must start at `(i, j)`, not `(i*kH, j*kW)`. Stride-1 windows overlap. If the shape is off by one, recheck `H_out = H - kH + 1` (you keep only fully-covered positions).\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "efbee8cb",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.709402Z",
"iopub.status.busy": "2026-06-10T19:24:50.709323Z",
"iopub.status.idle": "2026-06-10T19:24:50.714596Z",
"shell.execute_reply": "2026-06-10T19:24:50.714224Z"
},
"collapsed": true,
"jupyter": {
"source_hidden": true
},
"tags": [
"hide-input"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] 12.1 hand value\n",
"[ ok ] 12.1 vs reference\n",
"corr2d(X, K) =\n",
" [[19. 25.]\n",
" [37. 43.]]\n"
]
}
],
"source": [
"#@title Solution { display-mode: \"form\" }\n",
"# solution: redefines corr2d; the checks below re-verify the reference.\n",
"def corr2d(X, K):\n",
" X, K = np.asarray(X, dtype=float), np.asarray(K, dtype=float)\n",
" kH, kW = K.shape\n",
" H_out, W_out = X.shape[0] - kH + 1, X.shape[1] - kW + 1\n",
" out = np.zeros((H_out, W_out))\n",
" for i in range(H_out):\n",
" for j in range(W_out):\n",
" out[i, j] = (X[i:i+kH, j:j+kW] * K).sum()\n",
" return out\n",
"\n",
"check(\"12.1 hand value\", _toy_corr, required=True)\n",
"check(\"12.1 vs reference\", _ref_corr, required=True)\n",
"print(\"corr2d(X, K) =\\n\", corr2d(X, K))"
]
},
{
"cell_type": "markdown",
"id": "f0ac06d1",
"metadata": {},
"source": [
"> **Interpretation.** The from-scratch loop matches the hand value exactly. This is the operation `nn.Conv2d` runs, with the loops fused inside a single kernel and the patch extraction done via `im2col` so the inner step is one matmul. Read the loop once, then never write it again.\n"
]
},
{
"cell_type": "markdown",
"id": "96c112bd",
"metadata": {},
"source": [
"### 1.2 PyTorch computes the same thing\n",
"\n",
"`nn.Conv2d` adds batch and channel dimensions: input `(N, C_in, H, W)`, weight `(C_out, C_in, kH, kW)`, output `(N, C_out, H', W')`. If we load our `K` into a `Conv2d(1, 1, 2)` with zero bias, the output must equal `corr2d(X, K)`. This is the implementation ladder's agreement assert: from-scratch vs library.\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "aba1bd6f",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.728781Z",
"iopub.status.busy": "2026-06-10T19:24:50.728641Z",
"iopub.status.idle": "2026-06-10T19:24:50.739789Z",
"shell.execute_reply": "2026-06-10T19:24:50.739178Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch : [19. 25. 37. 43.]\n",
"scratch: [19. 25. 37. 43.]\n",
"[ ok ] nn.Conv2d agrees with corr2d to 1e-5\n"
]
}
],
"source": [
"conv = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=2, bias=False)\n",
"with torch.no_grad():\n",
" conv.weight.copy_(torch.tensor(K, dtype=torch.float32).view(1, 1, 2, 2))\n",
"Xb = torch.tensor(X, dtype=torch.float32).view(1, 1, 3, 3) # (N=1, C=1, H=3, W=3)\n",
"torch_out = conv(Xb).detach().numpy().reshape(2, 2)\n",
"scratch_out = corr2d(X, K)\n",
"print(\"torch :\", torch_out.ravel())\n",
"print(\"scratch:\", scratch_out.ravel())\n",
"assert np.allclose(torch_out, scratch_out, atol=1e-5), \\\n",
" \"nn.Conv2d should equal corr2d; if not, you flipped the kernel (that would be true convolution)\"\n",
"print(\"[ ok ] nn.Conv2d agrees with corr2d to 1e-5\")"
]
},
{
"cell_type": "markdown",
"id": "d666c463",
"metadata": {},
"source": [
"> **Common confusion:** if `nn.Conv2d` disagreed by a flip, you would be computing true convolution (kernel reversed). Deep learning does not flip; the network learns whatever orientation it needs, so the flip is wasted work. The assert above is the proof.\n"
]
},
{
"cell_type": "markdown",
"id": "f85c4525",
"metadata": {},
"source": [
"### 1.3 Multi-channel: the filter sees all input channels at once\n",
"\n",
"An RGB input has `C_in = 3`. One output channel is a filter of shape `(C_in, kH, kW)` that dots against all input channels and sums them into a single feature map. With `C_out` output channels, the weight tensor is `(C_out, C_in, kH, kW)`. The parameter count is `C_out * C_in * kH * kW (+ C_out bias)`, and crucially it does not depend on the image size.\n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "a1b9aa97",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.740762Z",
"iopub.status.busy": "2026-06-10T19:24:50.740680Z",
"iopub.status.idle": "2026-06-10T19:24:50.750408Z",
"shell.execute_reply": "2026-06-10T19:24:50.749633Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"input (8, 3, 32, 32) -> output (8, 16, 32, 32)\n",
"params 1216 (expected 1216)\n",
"[ ok ] parameter count matches the formula\n"
]
}
],
"source": [
"conv_rgb = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=5, padding=2)\n",
"x_rgb = torch.randn(8, 3, 32, 32) # (batch, C_in, H, W)\n",
"y_rgb = conv_rgb(x_rgb)\n",
"print(\"input \", tuple(x_rgb.shape), \"-> output\", tuple(y_rgb.shape))\n",
"expected = 3 * 16 * 5 * 5 + 16 # C_out*C_in*kH*kW + bias\n",
"print(f\"params {n_params(conv_rgb)} (expected {expected})\")\n",
"assert n_params(conv_rgb) == expected, \"param count is C_out*C_in*kH*kW + C_out, independent of H,W\"\n",
"print(\"[ ok ] parameter count matches the formula\")"
]
},
{
"cell_type": "markdown",
"id": "8fecd23a",
"metadata": {},
"source": [
"> **Key takeaways**\n",
"> - Convolution is a sliding dot product; the from-scratch `corr2d` reproduces the hand value `[[19,25],[37,43]]` and agrees with `nn.Conv2d`.\n",
"> - Deep-learning \"convolution\" is cross-correlation; no kernel flip.\n",
"> - A conv's parameter count is `C_out*C_in*kH*kW (+bias)`, independent of image size. That independence is the inductive bias.\n"
]
},
{
"cell_type": "markdown",
"id": "468f4731",
"metadata": {},
"source": [
"## Part 2 — Padding, strides, pooling, receptive field\n",
"\n",
"> **Objectives**\n",
"> - Derive and test the output-shape formula `floor((W - k + 2P)/S) + 1` against PyTorch.\n",
"> - Implement max pooling from scratch and check it against `nn.MaxPool2d`.\n",
"> - Compute receptive fields with the recurrence and verify the VGG \"two 3x3 = one 5x5\" claim.\n",
"\n",
"Two parameters sit on every conv layer, and both answer one question: how should the output spatial size compare to the input?\n",
"\n",
"- **Padding** `P` zero-pads the borders before convolving. With `P = (k-1)/2` for odd `k`, the output keeps the input size (\"same\" padding). `P = 0` is \"valid\": the output shrinks.\n",
"- **Stride** `S` is how far the filter jumps each step. `S = 2` halves the spatial dimensions, which is how modern ResNets downsample without a pooling layer.\n",
"\n",
"The formula you will use constantly:\n",
"\n",
"$$H_{\\text{out}} = \\left\\lfloor \\frac{H_{\\text{in}} - k + 2P}{S} \\right\\rfloor + 1$$\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "f184d98b",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.751704Z",
"iopub.status.busy": "2026-06-10T19:24:50.751335Z",
"iopub.status.idle": "2026-06-10T19:24:50.756729Z",
"shell.execute_reply": "2026-06-10T19:24:50.755550Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"H=32 k=3 s=1 p=1: formula 32 torch 32 [ok]\n",
"H=32 k=3 s=1 p=0: formula 30 torch 30 [ok]\n",
"H=32 k=3 s=2 p=1: formula 16 torch 16 [ok]\n",
"H=28 k=5 s=1 p=2: formula 28 torch 28 [ok]\n",
"H=31 k=3 s=2 p=0: formula 15 torch 15 [ok]\n",
"[ ok ] out_size matches nn.Conv2d on every config (note the floor on H=31,s=2)\n"
]
}
],
"source": [
"# our helper out_size implements the formula; check it against torch on several configs\n",
"configs = [ # (H_in, k, stride, pad)\n",
" (32, 3, 1, 1), # same padding -> 32\n",
" (32, 3, 1, 0), # valid -> 30\n",
" (32, 3, 2, 1), # stride-2 -> 16\n",
" (28, 5, 1, 2), # same padding, k=5 -> 28\n",
" (31, 3, 2, 0), # odd input with floor -> 15\n",
"]\n",
"for H, k, s, p in configs:\n",
" c = nn.Conv2d(1, 1, kernel_size=k, stride=s, padding=p)\n",
" torch_h = c(torch.zeros(1, 1, H, H)).shape[-1]\n",
" formula_h = out_size(H, k, s, p)\n",
" flag = \"ok\" if torch_h == formula_h else \"MISMATCH\"\n",
" print(f\"H={H} k={k} s={s} p={p}: formula {formula_h:>3} torch {torch_h:>3} [{flag}]\")\n",
" assert torch_h == formula_h, \"formula disagrees with PyTorch; check the floor and the +1\"\n",
"print(\"[ ok ] out_size matches nn.Conv2d on every config (note the floor on H=31,s=2)\")"
]
},
{
"cell_type": "markdown",
"id": "18c20a36",
"metadata": {},
"source": [
"> **Common confusion:** PyTorch's default padding is `0`, not \"same\". Forget it and your spatial dims silently shrink through every layer; you find out at the flatten before the classifier head, where the dimension is wrong. This bug is in everyone's first CNN.\n"
]
},
{
"cell_type": "markdown",
"id": "60278f4c",
"metadata": {},
"source": [
"### Exercise 12.2 — `conv_out_shape` for a whole layer\n",
"`Difficulty 2/5 · ~10 min`\n",
"\n",
"Write `conv_out_shape((C_in, H, W), C_out, k, stride, pad)` returning the full output shape `(C_out, H_out, W_out)` for a conv layer (ignore batch). Use the formula on `H` and `W` independently. The check compares against `nn.Conv2d` on several rectangular inputs, including a non-square one.\n"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "81b1b381",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.757609Z",
"iopub.status.busy": "2026-06-10T19:24:50.757532Z",
"iopub.status.idle": "2026-06-10T19:24:50.765340Z",
"shell.execute_reply": "2026-06-10T19:24:50.763610Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] 12.2 conv_out_shape: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def conv_out_shape(in_shape, c_out, k, stride=1, pad=0):\n",
" \"\"\"in_shape: (C_in, H, W). Returns (c_out, H_out, W_out).\"\"\"\n",
" c_in, H, W = in_shape\n",
" # TODO 1: H_out via the output-shape formula (use out_size)\n",
" H_out = None\n",
" # TODO 2: W_out via the same formula\n",
" W_out = None\n",
" attempted(H_out, W_out)\n",
" return (c_out, H_out, W_out)\n",
"\n",
"def _check_shape_formula():\n",
" cases = [((3, 32, 32), 16, 3, 1, 1), ((1, 28, 28), 6, 5, 1, 0),\n",
" ((8, 20, 30), 4, 3, 2, 1)] # last is non-square on purpose\n",
" for (c_in, H, W), c_out, k, s, p in cases:\n",
" layer = nn.Conv2d(c_in, c_out, k, stride=s, padding=p)\n",
" ref = tuple(layer(torch.zeros(1, c_in, H, W)).shape[1:])\n",
" got = conv_out_shape((c_in, H, W), c_out, k, s, p)\n",
" assert got == ref, f\"got {got}, torch says {ref} for C_in={c_in} {H}x{W} k={k} s={s} p={p}\"\n",
"\n",
"check(\"12.2 conv_out_shape\", _check_shape_formula)"
]
},
{
"cell_type": "markdown",
"id": "4fd81c14",
"metadata": {},
"source": [
"Hint 1 (conceptual)`H` and `W` are independent. Apply the same formula to each with the same `k`, `stride`, `pad`. The channel dimension of the output is just `c_out`.\n",
"\n",
"Hint 2 (pseudocode)\n",
"\n",
"```python\n",
"H_out = out_size(H, k, stride, pad)\n",
"W_out = out_size(W, k, stride, pad)\n",
"return (c_out, H_out, W_out)\n",
"```\n",
"\n",
"\n",
"Help — off by one on the non-square caseMake sure you pass `W` (not `H`) into the second call. A copy-paste bug that uses `H` twice passes the square cases and fails only the `20x30` one.\n"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "c9d9d435",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.766594Z",
"iopub.status.busy": "2026-06-10T19:24:50.766230Z",
"iopub.status.idle": "2026-06-10T19:24:50.773338Z",
"shell.execute_reply": "2026-06-10T19:24:50.772801Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] 12.2 conv_out_shape\n",
"Conv2d(3->16, k3, s1, p1) on 32x32 -> (16, 32, 32)\n"
]
}
],
"source": [
"# solution\n",
"def conv_out_shape(in_shape, c_out, k, stride=1, pad=0):\n",
" c_in, H, W = in_shape\n",
" H_out = out_size(H, k, stride, pad)\n",
" W_out = out_size(W, k, stride, pad)\n",
" return (c_out, H_out, W_out)\n",
"\n",
"check(\"12.2 conv_out_shape\", _check_shape_formula, required=True)\n",
"print(\"Conv2d(3->16, k3, s1, p1) on 32x32 ->\", conv_out_shape((3, 32, 32), 16, 3, 1, 1))"
]
},
{
"cell_type": "markdown",
"id": "509df855",
"metadata": {},
"source": [
"> **Interpretation.** With the formula you can compute every spatial dimension in a network in your head, which is the only way to catch a shape bug before the kernel does it for you with a cryptic matmul error.\n"
]
},
{
"cell_type": "markdown",
"id": "23b591f8",
"metadata": {},
"source": [
"### 2.1 Pooling: a fixed, parameter-free downsample\n",
"\n",
"Pooling reduces a feature map by taking the max (or mean) over a small window. The standard choice is 2x2 max pooling with stride 2: halve both spatial dims, keep the strongest activation per window. Max specifically, because the strongest activation is the best evidence a feature is present *somewhere* in the window. The exact location is discarded, which is the move from translation *equivariance* (conv) to a degree of translation *invariance* (pool).\n"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "1d6e029d",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.774251Z",
"iopub.status.busy": "2026-06-10T19:24:50.774170Z",
"iopub.status.idle": "2026-06-10T19:24:50.781553Z",
"shell.execute_reply": "2026-06-10T19:24:50.780994Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"pool: (8, 32, 32, 32) -> (8, 32, 16, 16) (spatial halved, channels unchanged)\n",
"global avg pool: (8, 32, 32, 32) -> (8, 32, 1, 1)\n"
]
}
],
"source": [
"pool = nn.MaxPool2d(kernel_size=2, stride=2)\n",
"x_fm = torch.randn(8, 32, 32, 32)\n",
"print(\"pool:\", tuple(x_fm.shape), \"->\", tuple(pool(x_fm).shape), \"(spatial halved, channels unchanged)\")\n",
"gap = nn.AdaptiveAvgPool2d((1, 1)) # global average pool: the standard pre-classifier layer in modern CNNs\n",
"print(\"global avg pool:\", tuple(x_fm.shape), \"->\", tuple(gap(x_fm).shape))"
]
},
{
"cell_type": "markdown",
"id": "ae1d0c53",
"metadata": {},
"source": [
"### Exercise 12.3 — `max_pool2d` from scratch\n",
"`Difficulty 2/5 · ~12 min`\n",
"\n",
"Implement max pooling on a `(N, C, H, W)` tensor (NumPy array). Window `k`, stride `s`. For each output position, take the max over the `k x k` window across `H, W` only (not across channels or batch). The check matches `nn.MaxPool2d` on a random batch.\n"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "6b09355f",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.782580Z",
"iopub.status.busy": "2026-06-10T19:24:50.782501Z",
"iopub.status.idle": "2026-06-10T19:24:50.792425Z",
"shell.execute_reply": "2026-06-10T19:24:50.791952Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] 12.3 max_pool2d: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def max_pool2d(x, k=2, stride=2):\n",
" \"\"\"x: (N, C, H, W) numpy array. Returns (N, C, H_out, W_out) of window maxima.\"\"\"\n",
" x = np.asarray(x, dtype=float)\n",
" N, Cc, H, W = x.shape\n",
" H_out, W_out = out_size(H, k, stride, 0), out_size(W, k, stride, 0)\n",
" out = np.zeros((N, Cc, H_out, W_out))\n",
" # TODO 1: loop over output positions (i, j); slice the k x k window over H, W.\n",
" # TODO 2: out[:, :, i, j] = the max over the last two axes of that window.\n",
" raise NotImplementedError\n",
" return out\n",
"\n",
"def _check_pool():\n",
" arr = rng.standard_normal((4, 3, 8, 8)).astype(np.float32)\n",
" ref = nn.MaxPool2d(2, 2)(torch.tensor(arr)).numpy()\n",
" check_close(max_pool2d(arr, 2, 2), ref, msg=\"disagrees with nn.MaxPool2d; pool over H,W only\")\n",
"\n",
"check(\"12.3 max_pool2d\", _check_pool)"
]
},
{
"cell_type": "markdown",
"id": "712b4128",
"metadata": {},
"source": [
"Hint 1 (conceptual)The window for output `(i, j)` is `x[:, :, i*stride:i*stride+k, j*stride:j*stride+k]`, shape `(N, C, k, k)`. Take the max over the last two axes with `.max(axis=(-1, -2))` (or reshape and max over the flattened window).\n",
"\n",
"Hint 2 (pseudocode)\n",
"\n",
"```python\n",
"for i in range(H_out):\n",
" for j in range(W_out):\n",
" win = x[:, :, i*stride:i*stride+k, j*stride:j*stride+k]\n",
" out[:, :, i, j] = win.max(axis=(-1, -2))\n",
"```\n",
"\n",
"\n",
"Help — my values are too large / look like sumsYou took a sum, not a max. `np.max` over `axis=(-1, -2)`, not `np.sum`. If the shape is wrong, you may be maxing over channels too; keep the first two axes intact.\n"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "d0c877fd",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.795253Z",
"iopub.status.busy": "2026-06-10T19:24:50.795155Z",
"iopub.status.idle": "2026-06-10T19:24:50.801337Z",
"shell.execute_reply": "2026-06-10T19:24:50.799711Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] 12.3 max_pool2d\n",
"max_pool2d matches nn.MaxPool2d\n"
]
}
],
"source": [
"# solution\n",
"def max_pool2d(x, k=2, stride=2):\n",
" x = np.asarray(x, dtype=float)\n",
" N, Cc, H, W = x.shape\n",
" H_out, W_out = out_size(H, k, stride, 0), out_size(W, k, stride, 0)\n",
" out = np.zeros((N, Cc, H_out, W_out))\n",
" for i in range(H_out):\n",
" for j in range(W_out):\n",
" win = x[:, :, i*stride:i*stride+k, j*stride:j*stride+k]\n",
" out[:, :, i, j] = win.max(axis=(-1, -2))\n",
" return out\n",
"\n",
"check(\"12.3 max_pool2d\", _check_pool, required=True)\n",
"print(\"max_pool2d matches nn.MaxPool2d\")"
]
},
{
"cell_type": "markdown",
"id": "7f73a0c0",
"metadata": {},
"source": [
"> **Caveat:** pooling discards information, including information you may later want. Tasks where exact location matters (segmentation, detection) avoid heavy pooling or recover the lost resolution with skip connections. U-Net is the canonical answer to \"I pooled it all away and now need pixel-level output back\".\n"
]
},
{
"cell_type": "markdown",
"id": "53f40ffa",
"metadata": {},
"source": [
"### 2.2 The receptive field\n",
"\n",
"The receptive field of a unit is the region of the input it depends on. One 3x3 conv: 3x3. Two stacked 3x3 convs: 5x5. Three: 7x7. A 2x2 stride-2 pool doubles the per-step jump. The recurrence:\n",
"\n",
"$$\\text{RF} \\mathrel{+}= (k - 1)\\cdot\\text{jump}, \\qquad \\text{jump} \\mathrel{*}= \\text{stride}$$\n",
"\n",
"This arithmetic is load-bearing: two stacked 3x3 convs cover the same 5x5 receptive field as one 5x5 conv, but with `2*9 = 18` parameters versus `25`, and one extra nonlinearity. That is the entire argument behind VGG using only 3x3 filters.\n"
]
},
{
"cell_type": "markdown",
"id": "88e5b7e8",
"metadata": {},
"source": [
"### Exercise 12.4 — `receptive_field` from the recurrence\n",
"`Difficulty 2/5 · ~8 min`\n",
"\n",
"Given `layers` as a list of `(kernel_size, stride)`, return the receptive field in pixels using the recurrence above, starting from `rf = 1, jump = 1`. The check verifies two stacked 3x3 convs give 5 (the VGG claim) and a VGG block with a pool gives 6.\n"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "e460bca9",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.802216Z",
"iopub.status.busy": "2026-06-10T19:24:50.802134Z",
"iopub.status.idle": "2026-06-10T19:24:50.807748Z",
"shell.execute_reply": "2026-06-10T19:24:50.807226Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] 12.4 receptive_field: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def receptive_field(layers):\n",
" \"\"\"layers: list of (kernel_size, stride). Returns RF in pixels.\"\"\"\n",
" rf, jump = 1, 1\n",
" # TODO 1: for each (k, s): rf += (k - 1) * jump ; then jump *= s\n",
" # (order matters: grow rf with the CURRENT jump, then update jump)\n",
" raise NotImplementedError\n",
" return rf\n",
"\n",
"def _check_rf():\n",
" assert receptive_field([(3, 1)]) == 3, \"one 3x3 conv has RF 3\"\n",
" assert receptive_field([(3, 1), (3, 1)]) == 5, \\\n",
" \"two stacked 3x3 convs cover RF 5 (the VGG 'two 3x3 = one 5x5' claim)\"\n",
" assert receptive_field([(3, 1), (3, 1), (2, 2)]) == 6, \"VGG block + 2x2 pool -> RF 6\"\n",
" assert receptive_field([(3, 1), (3, 1), (2, 2)] * 2) == 16, \"two such blocks -> RF 16\"\n",
"\n",
"check(\"12.4 receptive_field\", _check_rf)"
]
},
{
"cell_type": "markdown",
"id": "a79e91de",
"metadata": {},
"source": [
"Hint 1 (conceptual)Grow the receptive field using the *current* jump, then multiply the jump by this layer's stride for the next iteration. Doing it in the wrong order gives the wrong answer on any strided layer.\n",
"\n",
"Hint 2 (pseudocode)\n",
"\n",
"```python\n",
"for k, s in layers:\n",
" rf += (k - 1) * jump\n",
" jump *= s\n",
"return rf\n",
"```\n",
"\n",
"\n",
"Help — I get 4 instead of 5 for two 3x3 convsYou probably updated `jump` before growing `rf`, or used `k` instead of `k - 1`. With stride 1 the jump stays 1, so two convs add `(3-1)*1 = 2` each: `1 + 2 + 2 = 5`.\n"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "5a77d4e5",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.808591Z",
"iopub.status.busy": "2026-06-10T19:24:50.808515Z",
"iopub.status.idle": "2026-06-10T19:24:50.813793Z",
"shell.execute_reply": "2026-06-10T19:24:50.813501Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] 12.4 receptive_field\n",
"two 3x3 convs -> RF 5 | VGG-style 2 blocks -> RF 16\n"
]
}
],
"source": [
"# solution\n",
"def receptive_field(layers):\n",
" rf, jump = 1, 1\n",
" for k, s in layers:\n",
" rf += (k - 1) * jump\n",
" jump *= s\n",
" return rf\n",
"\n",
"check(\"12.4 receptive_field\", _check_rf, required=True)\n",
"print(\"two 3x3 convs -> RF\", receptive_field([(3, 1), (3, 1)]),\n",
" \"| VGG-style 2 blocks -> RF\", receptive_field([(3, 1), (3, 1), (2, 2)] * 2))"
]
},
{
"cell_type": "markdown",
"id": "37585277",
"metadata": {},
"source": [
"> **Key takeaways**\n",
"> - Output size is `floor((W - k + 2P)/S) + 1`, applied to `H` and `W` independently; tested against PyTorch.\n",
"> - Max pooling is a fixed, parameter-free downsample that trades equivariance for a little invariance, and throws away location.\n",
"> - Receptive field grows by `(k-1)*jump`; two 3x3 convs match one 5x5 with fewer parameters. That is why VGG is all 3x3.\n"
]
},
{
"cell_type": "markdown",
"id": "cdbbd33f",
"metadata": {},
"source": [
"## Part 3 — Learn the kernel\n",
"\n",
"> **Objectives**\n",
"> - Hand-design a vertical-edge detector and apply it (a falsification demo: it sees vertical edges, not horizontal).\n",
"> - Recover that kernel from random initialization by gradient descent, using only the input and the target output.\n",
"> - Use ground-truth recovery as the correctness check: the learned kernel must match the hand-designed one.\n",
"\n",
"We have been *setting* kernels by hand. The whole point of a CNN is that gradient descent *learns* them. To make the lesson assertable, we set up a problem where we already know the answer: a hand-designed `[1, -1]` kernel that detects vertical edges. We generate the target output with it, then throw the kernel away and learn it back from scratch. If the learner's training loop is correct, the recovered kernel must converge to the one we hid.\n"
]
},
{
"cell_type": "markdown",
"id": "d6da53eb",
"metadata": {},
"source": [
"### 3.1 A hand-designed vertical-edge detector\n",
"\n",
"The kernel `[[1, -1]]` subtracts each pixel from its left neighbor. On a flat region it outputs 0; at a vertical edge (a column where the value jumps) it outputs the jump. Apply it to an image that is black on the left half and white on the right: the output is zero everywhere except the seam.\n"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "63c1efbf",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.815429Z",
"iopub.status.busy": "2026-06-10T19:24:50.815067Z",
"iopub.status.idle": "2026-06-10T19:24:50.821271Z",
"shell.execute_reply": "2026-06-10T19:24:50.819903Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"response to vertical edge (one nonzero column at the seam):\n",
" [[ 0 0 0 -1 0 0 0]\n",
" [ 0 0 0 -1 0 0 0]\n",
" [ 0 0 0 -1 0 0 0]\n",
" [ 0 0 0 -1 0 0 0]\n",
" [ 0 0 0 -1 0 0 0]\n",
" [ 0 0 0 -1 0 0 0]\n",
" [ 0 0 0 -1 0 0 0]\n",
" [ 0 0 0 -1 0 0 0]]\n",
"\n",
"response to horizontal edge (all zeros: this kernel is blind to it):\n",
" [[0 0 0 0 0 0 0]\n",
" [0 0 0 0 0 0 0]\n",
" [0 0 0 0 0 0 0]\n",
" [0 0 0 0 0 0 0]\n",
" [0 0 0 0 0 0 0]\n",
" [0 0 0 0 0 0 0]\n",
" [0 0 0 0 0 0 0]\n",
" [0 0 0 0 0 0 0]]\n",
"[ ok ] falsification check: vertical detector is blind to horizontal edges\n"
]
}
],
"source": [
"# viz: vertical-edge image and the hand-designed detector's response\n",
"edge_img = np.zeros((8, 8)); edge_img[:, 4:] = 1.0 # left half 0, right half 1\n",
"Kv = np.array([[1.0, -1.0]]) # detects vertical edges\n",
"resp_v = corr2d(edge_img, Kv)\n",
"print(\"response to vertical edge (one nonzero column at the seam):\\n\", resp_v.astype(int))\n",
"# falsification: the SAME kernel sees nothing on a horizontal edge (transpose the image)\n",
"resp_h = corr2d(edge_img.T, Kv)\n",
"print(\"\\nresponse to horizontal edge (all zeros: this kernel is blind to it):\\n\", resp_h.astype(int))\n",
"assert np.abs(resp_h).sum() == 0, \"a vertical-edge kernel must give zero on a horizontal edge\"\n",
"print(\"[ ok ] falsification check: vertical detector is blind to horizontal edges\")"
]
},
{
"cell_type": "markdown",
"id": "d374df26",
"metadata": {},
"source": [
"> **Interpretation.** The negative control is a real check: a vertical-edge detector that fired on a horizontal edge would not be a vertical-edge detector. The seam shows up as a column of `-1`s (dark-to-bright transition); flip the image and the kernel goes silent.\n"
]
},
{
"cell_type": "markdown",
"id": "e95c5f55",
"metadata": {},
"source": [
"### Exercise 12.5 — Recover the kernel by gradient descent\n",
"`Difficulty 3/5 · ~20 min`\n",
"\n",
"Fill in the four-line training loop (the same four comments you will see in every training loop in this curriculum) to learn a `(1, 1, 1, 2)` conv weight that reproduces the target edge map. You are given the input `X`, the target `Y` (produced by the hidden kernel), and a randomly initialized `conv`. Minimize the mean-squared error between `conv(X)` and `Y`. The check asserts the recovered weight is close to `[1, -1]` and that the loss dropped by a large factor.\n",
"\n",
"This is the smallest possible \"train a CNN\" and it has a known answer, which is why we can grade it.\n"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "537a9168",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.822474Z",
"iopub.status.busy": "2026-06-10T19:24:50.822097Z",
"iopub.status.idle": "2026-06-10T19:24:50.841426Z",
"shell.execute_reply": "2026-06-10T19:24:50.841182Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] 12.5 learn the kernel: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def learn_edge_kernel(steps, lr=1.0):\n",
" \"\"\"Recover the [1,-1] kernel from (X, Y). Returns (weight_after, loss_history).\"\"\"\n",
" torch.manual_seed(SEED) # re-seed so re-running this cell alone reproduces the page\n",
" big = np.zeros((6, 8)); big[:, 4:] = 1.0\n",
" X = torch.tensor(big, dtype=torch.float32).view(1, 1, 6, 8)\n",
" Y = torch.tensor(corr2d(big, np.array([[1.0, -1.0]])), dtype=torch.float32).view(1, 1, 6, 7)\n",
" conv = nn.Conv2d(1, 1, kernel_size=(1, 2), bias=False) # random init; must learn [1,-1]\n",
" losses = []\n",
" for _ in range(steps):\n",
" # TODO 1 (forward): pred = conv(X)\n",
" pred = None\n",
" # TODO 2 (loss): loss = mean squared error between pred and Y\n",
" loss = None\n",
" attempted(pred, loss)\n",
" # TODO 3 (backward): zero existing grads, then loss.backward()\n",
" # set conv.weight.grad = None (NOT .data); then call loss.backward()\n",
" # TODO 4 (update): with torch.no_grad(): conv.weight -= lr * conv.weight.grad\n",
" losses.append(float(loss.item()))\n",
" return conv.weight.detach().view(-1).numpy(), losses\n",
"\n",
"def _check_kernel():\n",
" w, losses = learn_edge_kernel(KERNEL_STEPS)\n",
" check_close(w, [1.0, -1.0], atol=0.05,\n",
" msg=f\"recovered kernel {w}, expected ~[1,-1]; check the update sign and lr\")\n",
" assert losses[-1] < 0.01 * losses[0], \\\n",
" f\"loss only fell from {losses[0]:.3g} to {losses[-1]:.3g}; the loop is not learning\"\n",
"\n",
"check(\"12.5 learn the kernel\", _check_kernel)"
]
},
{
"cell_type": "markdown",
"id": "1f629e46",
"metadata": {},
"source": [
"Hint 1 (conceptual)The four steps are: forward (`pred = conv(X)`), loss (`((pred - Y)**2).mean()`), backward (clear old grads with `conv.weight.grad = None`, then `loss.backward()`), update (under `torch.no_grad()`, subtract `lr * grad` from the weight). This is the canonical loop.\n",
"\n",
"Hint 2 (pseudocode)\n",
"\n",
"```python\n",
"pred = conv(X)\n",
"loss = ((pred - Y) ** 2).mean()\n",
"conv.weight.grad = None # never use .data; this is the modern idiom\n",
"loss.backward()\n",
"with torch.no_grad():\n",
" conv.weight -= lr * conv.weight.grad\n",
"```\n",
"\n",
"\n",
"Help — my loss explodes / the kernel divergesThe learning rate is too high for this toy problem, or you added the gradient instead of subtracting it. Gradient descent subtracts `lr * grad`. If it still diverges, lower `lr`. If the loss is flat, you forgot `loss.backward()` or you never cleared the grad and it accumulated wrong.\n",
"\n",
"Help — loss decreases but the kernel is wrongPrint `conv.weight` each step. If it drifts to something like `[0.6, -0.6]`, you have not trained long enough; more steps converge it to `[1, -1]`. If it goes to `[-1, 1]`, you flipped the sign of the update.\n"
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "bc0ecb38",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.847089Z",
"iopub.status.busy": "2026-06-10T19:24:50.846989Z",
"iopub.status.idle": "2026-06-10T19:24:50.993801Z",
"shell.execute_reply": "2026-06-10T19:24:50.993178Z"
},
"collapsed": true,
"jupyter": {
"source_hidden": true
},
"tags": [
"hide-input"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] 12.5 learn the kernel\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"recovered kernel: [ 1. -1.] (target [1, -1])\n",
"loss: 0.3317 -> 1.37e-14 over 600 steps\n"
]
}
],
"source": [
"#@title Solution { display-mode: \"form\" }\n",
"# solution: the canonical four-comment training loop\n",
"def learn_edge_kernel(steps, lr=1.0):\n",
" torch.manual_seed(SEED)\n",
" big = np.zeros((6, 8)); big[:, 4:] = 1.0\n",
" X = torch.tensor(big, dtype=torch.float32).view(1, 1, 6, 8)\n",
" Y = torch.tensor(corr2d(big, np.array([[1.0, -1.0]])), dtype=torch.float32).view(1, 1, 6, 7)\n",
" conv = nn.Conv2d(1, 1, kernel_size=(1, 2), bias=False)\n",
" losses = []\n",
" for _ in range(steps):\n",
" pred = conv(X) # forward\n",
" loss = ((pred - Y) ** 2).mean() # loss (MSE)\n",
" conv.weight.grad = None # backward: clear grads (modern idiom, not .data)\n",
" loss.backward()\n",
" with torch.no_grad(): # update\n",
" conv.weight -= lr * conv.weight.grad\n",
" losses.append(float(loss.item()))\n",
" return conv.weight.detach().view(-1).numpy(), losses\n",
"\n",
"check(\"12.5 learn the kernel\", _check_kernel, required=True)\n",
"w_learned, loss_hist = learn_edge_kernel(KERNEL_STEPS)\n",
"print(f\"recovered kernel: {w_learned.round(3)} (target [1, -1])\")\n",
"print(f\"loss: {loss_hist[0]:.4f} -> {loss_hist[-1]:.2e} over {KERNEL_STEPS} steps\")"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "a0316d51",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:50.995043Z",
"iopub.status.busy": "2026-06-10T19:24:50.994686Z",
"iopub.status.idle": "2026-06-10T19:24:51.625123Z",
"shell.execute_reply": "2026-06-10T19:24:51.624688Z"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAekAAAEiCAYAAADd4SrgAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAQQJJREFUeJzt3XlcVFX/B/DPsKNsIgKiCKiUoggGSrgrKJGJaKlZTwFWamlalKZPT2ql2fLUj1LK0keptNwS0koUEYQMNVHcNRdUUtlc2GQR5vz+GLk5gjjgwB2Gz/v1mhd3Odz7PXdgvnPvPedchRBCgIiIiHSOgdwBEBERUe2YpImIiHQUkzQREZGOYpImIiLSUUzSREREOopJmoiISEcxSRMREekoJmkiIiIdxSRNRESko5ikiR6AQqHAggUL5A5DzZAhQ9CzZ89G34+rqysUCgUUCgWmT5/e6Purj9DQUCm2pjgWQ4YMwZAhQxp9P3dLTk6GQqHAxo0bm3zfDyomJgYKhQLnz5+XOxSdxiRN9VL9j1X9MjIyQocOHRAeHo5Lly7JHV6LcfnyZSxYsAAZGRmyxjFw4EB8//33CAsLU1v+1VdfYdy4cejUqRMUCgXCw8O1sr9FixYhJCQEDg4OdX5Bev311/H999+jW7duWtkvkVyM5A6Amqf33nsPbm5uKCsrw549exATE4Pff/8dR48ehZmZmdzhNZnS0lIYGTX9v9Hly5fx7rvvwtXVFd7e3k2+/2qdO3fGv/71rxrLP/roIxQVFaFv3764cuWK1vb3n//8B46Ojujduze2bdt2z3KDBw8GAKxYsQL5+fla2z9RU2OSpgYJDg6Gr68vAODFF1+EnZ0dPvroI2zevBnjx4+XObrGpVQqUVFRATMzsxb1haQ+du3aJZ1FW1hYaG27mZmZcHV1RX5+Ptq1a6e17VLthBAoKyuDubm53KG0WLzcTVoxcOBAAMDZs2fVlp88eRJPPfUUbG1tYWZmBl9fX2zevLnG79+4cQOvv/46XF1dYWpqio4dO+L5559XOwvKzc3FCy+8AAcHB5iZmcHLywvffvuttP7WrVuwtbVFREREje0XFhbCzMwMb775prSsvLwc8+fPR9euXWFqagpnZ2fMnj0b5eXlar9bfc91zZo16NGjB0xNTREfHy+tu/OS64IFC6BQKHDmzBmEh4fDxsYG1tbWiIiIwM2bN9W2W1paihkzZsDOzg6WlpYICQnBpUuX7nufOzk5GX369AEARERESLceYmJi1ModP34cQ4cORatWrdChQwd8/PHHNbal6TGoLxcXFygUigfaRm1cXV21vs1qq1evho+PD8zNzWFra4unn34aWVlZNcp988036NKlC8zNzdG3b1+kpqbWur0LFy4gJCQErVu3hr29PV5//XVs27YNCoUCycnJamX37t2Lxx57DNbW1mjVqhUGDx6M3bt3N6ge5eXleOKJJ2BtbY0//vgDgOqLZVRUFHr06AEzMzM4ODhgypQpuH79utrvurq64oknnsC2bdvg6+sLc3NzfP3119K97/Xr12PRokXo2LEjzMzMEBAQgDNnztSIQZv1ael4Jk1aUd34o02bNtKyY8eOoX///ujQoQPmzJmD1q1bY/369QgNDcVPP/2EMWPGAACKi4sxcOBAnDhxApMmTcIjjzyC/Px8bN68GX///Tfs7OxQWlqKIUOG4MyZM5g+fTrc3NywYcMGhIeH48aNG5g5cyaMjY0xZswYbNq0CV9//TVMTEykWOLi4lBeXo6nn34agOpDKyQkBL///jsmT56M7t2748iRI/i///s//PXXX4iLi1Or386dO7F+/XpMnz4ddnZ2900W48ePh5ubGxYvXowDBw5gxYoVsLe3x0cffSSVCQ8Px/r16/Hcc8/h0Ucfxa5duzBy5Mj7Huvu3bvjvffew7x58zB58mTpC1K/fv2kMtevX8djjz2GsWPHYvz48di4cSPeeusteHp6Ijg4uEHHQJ8tWrQI77zzDsaPH48XX3wReXl5WLJkCQYNGoSDBw/CxsYGAPC///0PU6ZMQb9+/fDaa6/h3LlzCAkJga2tLZydnaXtlZSUYNiwYbhy5QpmzpwJR0dH/PDDD0hKSqqx7507dyI4OBg+Pj6YP38+DAwMsGrVKgwbNgypqano27evxvUoLS3F6NGjsX//fuzYsUP6MjdlyhTExMQgIiICM2bMQGZmJpYuXYqDBw9i9+7dMDY2lrZx6tQpTJw4EVOmTMFLL72Ehx9+WFr34YcfwsDAAG+++SYKCgrw8ccf49lnn8XevXsbpT4EQBDVw6pVqwQAsWPHDpGXlyeysrLExo0bRbt27YSpqanIysqSygYEBAhPT09RVlYmLVMqlaJfv37C3d1dWjZv3jwBQGzatKnG/pRKpRBCiKioKAFArF69WlpXUVEh/P39hYWFhSgsLBRCCLFt2zYBQGzZskVtO48//rjo3LmzNP/9998LAwMDkZqaqlZu2bJlAoDYvXu3tAyAMDAwEMeOHasRHwAxf/58aX7+/PkCgJg0aZJauTFjxoi2bdtK8+np6QKAeO2119TKhYeH19hmbf78808BQKxatarGusGDBwsA4rvvvpOWlZeXC0dHR/Hkk0826BjUxsXFRYSFhdVZRgghWrdurVG5+sjLy9PoOA0ePFj06NGjzjLnz58XhoaGYtGiRWrLjxw5IoyMjKTlFRUVwt7eXnh7e4vy8nKp3DfffCMAiMGDB0vLPv30UwFAxMXFSctKS0tFt27dBACRlJQkhFD9fbu7u4ugoCDpb10IIW7evCnc3NzE8OHD64w9KSlJABAbNmwQRUVFYvDgwcLOzk4cPHhQKpOamioAiDVr1qj9bnx8fI3lLi4uAoCIj4+vdT/du3dXq/vnn38uAIgjR47Uuz7VnyWZmZl11rGl4+VuapDAwEC0a9cOzs7OeOqpp9C6dWts3rwZHTt2BABcu3YNO3fuxPjx41FUVIT8/Hzk5+fj6tWrCAoKwunTp6XW4D/99BO8vLykM+s7VV8y/e233+Do6IiJEydK64yNjTFjxgwUFxdj165dAIBhw4bBzs4O69atk8pdv34dCQkJmDBhgrRsw4YN6N69O7p16ybFlp+fj2HDhgFAjTOewYMHw8PDQ+PjM3XqVLX5gQMH4urVqygsLAQA6XL5K6+8olbu1Vdf1XgfdbGwsFBr0GViYoK+ffvi3Llz0rL6HgN9tWnTJiiVSowfP17tODg6OsLd3V06Dvv370dubi6mTp2qdpUmPDwc1tbWatuMj49Hhw4dEBISIi0zMzPDSy+9pFYuIyMDp0+fxjPPPIOrV69K+y4pKUFAQABSUlKgVCrvW4eCggKMGDECJ0+eRHJyslpjwg0bNsDa2hrDhw9Xq5+Pjw8sLCxqvM9ubm4ICgqqdT8RERFqda++ilP9d6Wt+tA/eLmbGiQ6OhoPPfQQCgoKsHLlSqSkpMDU1FRaf+bMGQgh8M477+Cdd96pdRu5ubno0KEDzp49iyeffLLO/V24cAHu7u4wMFD/Xtm9e3dpPQAYGRnhySefxA8//IDy8nKYmppi06ZNuHXrllqSPn36NE6cOHHPxke5ublq825ubnXGd7dOnTqpzVffBrh+/TqsrKxw4cIFGBgY1Nhu165d67Wfe+nYsWONe8Jt2rTB4cOHpfn6HoOmkp2drTZvbW3dqA2XTp8+DSEE3N3da11ffSm4+m/s7nLGxsbo3Lmz2rILFy6gS5cuNd6Du9/f06dPA0CNLmx3KigoULuNVJvXXnsNZWVlOHjwIHr06FFjHwUFBbC3t6/1d+vzt17X33X1voAHrw/9g0maGqRv375S6+7Q0FAMGDAAzzzzDE6dOgULCwvp2/Kbb755z2/l2kpId3v66afx9ddfY+vWrQgNDcX69evRrVs3eHl5SWWUSiU8PT3x2Wef1bqNO+8vAqh3kjA0NKx1uRCiXttpKE32X99j0FTat2+vNr9q1Sqt9bOujVKphEKhwNatW2s9btpsnV7bvgHgk08+uWdXOk32P3r0aKxduxYffvghvvvuO7Uvs0qlEvb29lizZk2tv3v3l7S6/tbv93elrfrQP5ik6YEZGhpi8eLFGDp0KJYuXYo5c+ZIZxbGxsYIDAys8/e7dOmCo0eP1lnGxcUFhw8fhlKpVPsAOnnypLS+2qBBg9C+fXusW7cOAwYMwM6dO/H222/X2OehQ4cQEBDQKK2Q78fFxQVKpRKZmZlqZ2a1tZStjTZilvsY3EtCQoLa/N1nhtrWpUsXCCHg5uaGhx566J7lqv/GTp8+Ld0SAFS9CjIzM9W+BLq4uOD48eMQQqgd27vf3y5dugAArKys7vt/UpfQ0FCMGDEC4eHhsLS0xFdffaW2jx07dqB///6N3pVKW/Whf/CeNGnFkCFD0LdvX0RFRaGsrAz29vYYMmQIvv7661oHs8jLy5Omn3zySRw6dAixsbE1ylV/Q3/88ceRnZ2tdq+5srISS5YsgYWFhTR4BQAYGBjgqaeewpYtW/D999+jsrJS7VI3oGp9fenSJSxfvrzGPktLS1FSUlL/g1AP1VcXvvzyS7XlS5Ys0ej3W7duDUDVda2h5D4G9xIYGKj2uvvMWtvGjh0LQ0NDvPvuuzWudAghcPXqVQCAr68v2rVrh2XLlqGiokIqExMTU+N9CAoKwqVLl9S6G5aVldU41j4+PujSpQv++9//ori4uEZsd/6f3M/zzz+PL774AsuWLcNbb70lLR8/fjyqqqrw/vvv1/idysrKB/obups260MqPJMmrZk1axbGjRuHmJgYTJ06FdHR0RgwYAA8PT3x0ksvoXPnzsjJyUFaWhr+/vtvHDp0SPq9jRs3Yty4cZg0aRJ8fHxw7do1bN68GcuWLYOXlxcmT56Mr7/+GuHh4UhPT4erqys2btyI3bt3IyoqCpaWlmqxTJgwAUuWLMH8+fPh6ekp3buu9txzz2H9+vWYOnUqkpKS0L9/f1RVVeHkyZNYv3691E+0sfj4+ODJJ59EVFQUrl69KnXB+uuvvwDc/0y5S5cusLGxwbJly2BpaYnWrVvDz8+vXvfOG/MYbNmyRXp/b926hcOHD2PhwoUAgJCQEPTq1QuAquuem5sbwsLCavTzrs3333+PCxcuSH3OU1JSpO0+99xzaldUNNWlSxcsXLgQc+fOxfnz5xEaGgpLS0tkZmYiNjYWkydPxptvvgljY2MsXLgQU6ZMwbBhwzBhwgRkZmZi1apVNe5JT5kyBUuXLsXEiRMxc+ZMtG/fHmvWrJEGv6l+fw0MDLBixQoEBwejR48eiIiIQIcOHXDp0iUkJSXBysoKW7Zs0bgu06dPR2FhId5++21YW1vj3//+NwYPHowpU6Zg8eLFyMjIwIgRI2BsbIzTp09jw4YN+Pzzz/HUU0/V+7jVRtv1IbALFtVPdbeJP//8s8a6qqoq0aVLF9GlSxdRWVkphBDi7Nmz4vnnnxeOjo7C2NhYdOjQQTzxxBNi48aNar979epVMX36dNGhQwdhYmIiOnbsKMLCwkR+fr5UJicnR0RERAg7OzthYmIiPD09a+2CJISqK4izs7MAIBYuXFhrmYqKCvHRRx+JHj16CFNTU9GmTRvh4+Mj3n33XVFQUCCVAyCmTZtW6zZwjy5YeXl5tR63O7ublJSUiGnTpglbW1thYWEhQkNDxalTpwQA8eGHH9a6vzv9/PPPwsPDQxgZGal1x7pXt6OwsDDh4uLSoGNQm7q6YIWFhQkAtb7ufM+OHDkiAIg5c+bct77VdbvXdqu7Nd1d/n5dsKr99NNPYsCAAaJ169aidevWolu3bmLatGni1KlTauW+/PJL4ebmJkxNTYWvr69ISUkRgwcPVuuCJYQQ586dEyNHjhTm5uaiXbt24o033hA//fSTACD27NmjVvbgwYNi7Nixom3btsLU1FS4uLiI8ePHi8TExDpjvrML1p1mz54tAIilS5dKy7755hvh4+MjzM3NhaWlpfD09BSzZ88Wly9flsq4uLiIkSNHaryfzMzMWrsCalIfdsHSjEKIJmrJQkT3lZGRgd69e2P16tV49tln5Q6nTq6urvD398eSJUtgbm4uXYKvjy+//BKzZ8/G2bNn4eDgoLXYioqKUF5ejtGjR6OgoOC+bR6aSlRUFF5//XX8/fff6NChg9zhUDPAe9JEMiktLa2xLCoqCgYGBhg0aJAMEdXf2rVr0a5dO7V7oPWRlJSEGTNmaDVBA6pL3+3atZOGxZTD3e9vWVkZvv76a7i7uzNBk8Z4Jk0kk3fffRfp6ekYOnQojIyMsHXrVmzdulW6/67rdu/eLSUiZ2dnteEj5Xb48GGp/6+FhQUeffTRJo8hODgYnTp1gre3NwoKCrB69WocO3YMa9aswTPPPNPk8VDzxCRNJJOEhAS8++67OH78OIqLi9GpUyc899xzePvtt2V5/CVpV1RUFFasWIHz58+jqqoKHh4emD17do2eBkR1YZImIiLSUS36nvQvv/yChx9+GO7u7lixYoXc4RAREalpsWfSlZWV8PDwQFJSEqytreHj44M//vgDbdu2lTs0IiIiAC14MJN9+/ahR48eUivL4OBgbN++Xe0pS3VRKpW4fPkyLC0tdWpIRSIi0n1CCBQVFcHJyanGg4Pu1GyTdEpKCj755BOkp6fjypUriI2NRWhoqFqZ6OhofPLJJ8jOzoaXlxeWLFkiPXD88uXLat0gqkfF0dTly5dlewABERHph6ysLOkRv7Vptkm6pKQEXl5emDRpEsaOHVtj/bp16xAZGYlly5bBz88PUVFRCAoKwqlTp+75yLb6qB6GMisrC1ZWVg+8PSIiajkKCwvh7OxcY0jjuzXbJB0cHIzg4OB7rv/ss8/w0ksvISIiAgCwbNky/Prrr1i5ciXmzJkDJycntTPnS5cuSWfZtSkvL0d5ebk0X1RUBED1tBcmaSIiaoj73S7Vy9bdFRUVSE9PV3tUmoGBAQIDA5GWlgZA9Tzko0eP4tKlSyguLsbWrVvv+dxjAFi8eDGsra2lFy91ExFRY9PLJJ2fn4+qqqoaQw06ODggOzsbAGBkZIRPP/0UQ4cOhbe3N9544406W3bPnTsXBQUF0isrK6tR60BERNRsL3drQ0hICEJCQjQqa2pqClNT00aOiIiI6B96eSZtZ2cHQ0ND5OTkqC3PycmBo6OjTFERERHVj14maRMTE/j4+CAxMVFaplQqkZiYCH9/fxkjIyIi0lyzvdxdXFyMM2fOSPOZmZnIyMiAra0tOnXqhMjISISFhcHX1xd9+/ZFVFQUSkpKpNbeuqCkFNiWClTcAp4eKXc0RESka5ptkt6/fz+GDh0qzUdGRgIAwsLCEBMTgwkTJiAvLw/z5s1DdnY2vL29ER8fr/Xn1j6I3enA64sB+7bAuMcAQ0O5IyIiIl3SYsfuflCFhYWwtrZGQUFBg/tJV9wC+jwF3CgEvv8YGNRHy0ESEZFO0jSH6OU96ebCxBh4YohqOjZB1lCIiEgHMUnLbOwI1c/4VOBmqbyxEBGRbmGSltkjHoCLE3CzDIj/Xe5oiIhIlzBJy0yhAMYMV03H8ZI3ERHdgUlaB1Qn6dR0IOeqvLEQEZHuYJLWAa4dgN4egFIJbE68f3kiImoZmKR1xNjbZ9ObeMmbiIhuY5LWEaOGAkaGwPEzwKlMuaMhIiJdwCStI9pYA0P9VNPsM01ERACTtE4Zc7vPdNwO1f1pIiJq2ZikdUiAP2DVGriSB+w5JHc0REQkNyZpHWJmAjw+RDW9abusoRARkQ5gktYx1a28t6YApWXyxkJERPJiktYxfTyBjg5A8U1gxx9yR0NERHJiktYxBgbA6EDVNPtMExG1bEzSOqj6yVi79gH51+WNhYiI5MMkrYO6dgJ6PQxUKYEtSXJHQ0REcmGS1lFjbl/y5sAmREQtF5O0jgoJAAwNgEMngbMX5Y6GiIjkwCSto+zaAIP6qKZjd8gbCxERyYNJWodVNyCLTeAwoURELRGTtA4b3g+waAX8nQ3sPyp3NERE1NSYpHWYuRkQPEg1zQZkREQtD5O0jhtze5jQX5KBsgpZQyEioibGJK3jHvUCHO2AwmJgZ5rc0RARUVNiktZxhoZAKPtMExG1SEzSzUB1K++kvcD1AnljISKiptNik/SNGzfg6+sLb29v9OzZE8uXL5c7pHt62A3w6ALcqlTdmyYiopahxSZpS0tLpKSkICMjA3v37sUHH3yAq1evyh3WPY25fTbNJ2MREbUcLTZJGxoaolWrVgCA8vJyCCEghJA5qnsbPUz1GMsDx4ALl+SOhoiImoLOJumUlBSMGjUKTk5OUCgUiIuLq1EmOjoarq6uMDMzg5+fH/bt21evfdy4cQNeXl7o2LEjZs2aBTs7Oy1Fr30OdkD/R1TTHCaUiKhl0NkkXVJSAi8vL0RHR9e6ft26dYiMjMT8+fNx4MABeHl5ISgoCLm5uVKZ6vvNd78uX74MALCxscGhQ4eQmZmJH374ATk5OU1St4Yae7vP9KbtgA6f9BMRkZYohC5f471NoVAgNjYWoaGh0jI/Pz/06dMHS5cuBQAolUo4Ozvj1VdfxZw5c+q9j1deeQXDhg3DU089Vev68vJylJeXS/OFhYVwdnZGQUEBrKys6r2/higpBXzGAqVlQGw08IhHk+yWiIi0rLCwENbW1vfNITp7Jl2XiooKpKenIzAwUFpmYGCAwMBApKVpNuJHTk4OioqKAAAFBQVISUnBww8/fM/yixcvhrW1tfRydnZ+sEo0QGtzIGiAanrT9ibfPRERNbFmmaTz8/NRVVUFBwcHteUODg7Izs7WaBsXLlzAwIED4eXlhYEDB+LVV1+Fp6fnPcvPnTsXBQUF0isrK+uB6tBQ1Ze8tyQBFbdkCYGIiJqIkdwByKVv377IyMjQuLypqSlMTU0bLyAN9fcB2tkCedeAXfuA4f3ljoiIiBpLszyTtrOzg6GhYY2GXjk5OXB0dJQpqqZhZAiEDFNNs880EZF+a5ZJ2sTEBD4+PkhMTJSWKZVKJCYmwt/fX8bImkb1MKGJfwAFxfLGQkREjUdnk3RxcTEyMjKkS9KZmZnIyMjAxYsXAQCRkZFYvnw5vv32W5w4cQIvv/wySkpKEBERIWPUTaNHV+AhV6D8FvDbLrmjISKixqKz96T379+PoUOHSvORkZEAgLCwMMTExGDChAnIy8vDvHnzkJ2dDW9vb8THx9doTKaPFArV2fSH3wCx24GJI+WOiIiIGkOz6CetizTt49ZYLucC/Z5WDWry+4+As37fiici0it63U+aACd74FFv1fTPHCaUiEgvMUk3Y9IwoQkcJpSISB8xSTdjwYMAUxPg7EXgyF9yR0NERNrGJN2MWbYGRtwezCSWfaaJiPQOk3QzN+b2Je/NO4HKKnljISIi7WKSbuYG9QHa2gD514GUP+WOhoiItIlJupkzNgJG3e5OzkveRET6hUlaD1QPE7rtd6CoRN5YiIhIe5ik9UCvh4HOzkB5BRCfKnc0RESkLUzSekCh+KcB2abt8sZCRETawyStJ0IDVT/TMoArebKGQkREWsIkrSc6tQf6eKpGHuMwoURE+oFJWo9wmFAiIv3CJK1HRg4BTIyBU5nAibNyR0NERA+KSVqPWFsCwx5VTW9in2kiomaPSVrPVPeZ/jkRqOIwoUREzRqTtJ4Z6gfYWAG5V4E/DsodDRERPQgmaT1jYgw8MUQ1zT7TRETNG5O0Hqoe2CQ+FbhZKm8sRETUcEzSesinB+DiBNwsU43nTUREzZNRfQrfuHEDsbGxSE1NxYULF3Dz5k20a9cOvXv3RlBQEPr169dYcVI9VA8TGvWt6slY1WfWRETUvGh0Jn358mW8+OKLaN++PRYuXIjS0lJ4e3sjICAAHTt2RFJSEoYPHw4PDw+sW7eusWMmDVQPE5qaDuRclTcWIiJqGI3OpHv37o2wsDCkp6fDw8Oj1jKlpaWIi4tDVFQUsrKy8Oabb2o1UKoft45Abw/g4HFgy07gxXFyR0RERPWlEOL+A0hevXoVbdu21Xij9S3fHBUWFsLa2hoFBQWwsrKSO5xafRcHvPM50MMd+O0buaMhIqJqmuYQjS531zfh6nuCbi6eGAoYGQLHTgN/ZcodDRER1Ve9Go4BwObNm2tdrlAoYGZmhq5du8LNze2BA6MHZ2utGtwk4Q/VMKFzJssdERER1Ue9k3RoaCgUCgXuvkpevUyhUGDAgAGIi4tDmzZttBYoNcyYEaokHbcDmP0iYMBOd0REzUa9P7ITEhLQp08fJCQkoKCgAAUFBUhISICfnx9++eUXpKSk4OrVqzrfcCwzMxNDhw6Fh4cHPD09UVJSIndIjSLAH7BqDVzJA/YckjsaIiKqj3qfSc+cORPffPONWp/ogIAAmJmZYfLkyTh27BiioqIwadIkrQaqbeHh4Vi4cCEGDhyIa9euwdTUVO6QGoWZCfD4EGDtr6phQvv1ljsiIiLSVL3PpM+ePVtrSzQrKyucO3cOAODu7o78/PwHj66RHDt2DMbGxhg4cCAAwNbWFkZG9f6+0myMvT2YydYUoKxc3liIiEhz9U7SPj4+mDVrFvLy8qRleXl5mD17Nvr06QMAOH36NJydnRscVEpKCkaNGgUnJycoFArExcXVKBMdHQ1XV1eYmZnBz88P+/bt03j7p0+fhoWFBUaNGoVHHnkEH3zwQYNjbQ76eAIdHIDim0DCbrmjISIiTdU7Sf/vf/9DZmYmOnbsiK5du6Jr167o2LEjzp8/jxUrVgAAiouL8Z///KfBQZWUlMDLywvR0dG1rl+3bh0iIyMxf/58HDhwAF5eXggKCkJubq5UxtvbGz179qzxunz5MiorK5Gamoovv/wSaWlpSEhIQEJCQoPj1XUGBv+MQLZJf6tJRKR3NBrM5G5KpRLbt2/HX3/9BQB4+OGHMXz4cBg0QtNhhUKB2NhYhIaGSsv8/PzQp08fLF26VIrH2dkZr776KubMmXPfbaalpWHBggXYtm0bAOCTTz4BAMyaNUvjuJrDYCZ3On0BCAwHDA2AP38C2trIHRERUculaQ5p0I1YAwMDPPbYY3jssccaHGBDVVRUID09HXPnzlWLJzAwEGlpaRpto0+fPsjNzcX169dhbW2NlJQUTJkypc7fKS8vR3n5Pzd0CwsLG1YBmbi7AJ4PAUf+AjbvBCLGyh0RERHdT4NOfXft2oVRo0ZJl7tDQkKQmpqq7dhqlZ+fj6qqKjg4OKgtd3BwQHZ2tkbbMDIywgcffIBBgwahV69ecHd3xxNPPFHn7yxevBjW1tbS60HuuculugFZLC95ExE1C/VO0qtXr0ZgYCBatWqFGTNmYMaMGTAzM0NAQAB++OGHxoixUQQHB+PIkSM4evQoPvvss/uWnzt3rtQvvKCgAFlZWU0QpXaFBKgudx86CZy9KHc0RER0P/W+3L1o0SJ8/PHHeP3116VlM2bMwGeffYb3338fzzzzjFYDvJudnR0MDQ2Rk5OjtjwnJweOjo6Ntl9TU9Nm35farg0wqA+QtBeI3QG8qdtd2YmIWrx6n0mfO3cOo0aNqrE8JCQEmZmN/xQHExMT+Pj4IDExUVqmVCqRmJgIf3//Rt9/czfmjkveSqW8sRARUd3qfSbt7OyMxMREdO3aVW35jh07tHaftri4GGfOnJHmMzMzkZGRAVtbW3Tq1AmRkZEICwuDr68v+vbti6ioKJSUlCAiIkIr+9dnI/oDFq2Av7OB9GOqPtRERKSb6p2k33jjDcyYMQMZGRnS0KC7d+9GTEwMPv/8c60EtX//fgwdOlSaj4yMBACEhYUhJiYGEyZMQF5eHubNm4fs7Gx4e3sjPj6+RmMyqsncDAgeBGyIVw0TyiRNRKS7GtRPOjY2Fp9++ilOnDgBAOjevTtmzZqF0aNHaz1AXdXc+knf6fd04Nk3ASsLVZ9pMxO5IyIialk0zSENStLUvJN0VRXQ72kgOx9Y9q7qzJqIiJqOpjmETxdugQwN/xkmlH2miYh0l0b3pNu0aQOFQqHRBq9du/ZAAVHTGDMcWLYW2LkHuF4AtLGWOyIiIrqbRkk6KiqqkcOgptatM+DRBTh+FvglGXiu5TQnICJqNjRK0mFhYY0dB8lgzAjg+FeqS95M0kREukeje9IlJSX12mh9y5M8Rg9TPcYy/Rhw4ZLc0RAR0d00StJdu3bFhx9+iCtXrtyzjBACCQkJCA4OxhdffKG1AKnxONgB/R9RTcfukDcWIiKqSaPL3cnJyfj3v/+NBQsWwMvLC76+vnBycoKZmRmuX7+O48ePIy0tDUZGRpg7d+59H/tIumPscCB1v+qS98znAQ3bBxIRUROoVz/pixcvYsOGDUhNTcWFCxdQWloKOzs79O7dG0FBQQgODoahoWFjxqszmnM/6TuVlAI+Y4HSMiA2GnjEQ+6IiIj0HwczaWT6kqQBYOYiIG6HqvHYwtfkjoaISP9xMBPS2NjbT8b6JQmouCVvLERE9A8maUJ/H6BdG+B6IbBrn9zREBFRNSZpgpEhEBKgmt7EYUKJiHQGkzQBAMaOUP1M/AMoKJY3FiIiUmGSJgBAj67AQ65A+S3gt11yR0NEREA9kvTHH3+M0tJSaX737t0oLy+X5ouKivDKK69oNzpqMgqF6qEbABC7Xd5YiIhIReMuWIaGhrhy5Qrs7e0BAFZWVsjIyEDnzp0BADk5OXByckJVVVXjRatD9KkLVrXLuarnTAsB7P4R6Ogod0RERPpJ612w7s7l7F6tf5zsgUe9VdNxHCaUiEh2vCdNasYEqn5uSlCdURMRkXyYpEnN44MBUxPg7EXg6F9yR0NE1LJp9ICNaitWrICFhQUAoLKyEjExMbCzswOgajhGzZ9la2BEf2BLkups2vNhuSMiImq5NG445urqCoUGj0jKzMx84KCaA31sOFYtMQ2Y9G/Arg2wd4NqsBMiItIeTXOIxmfS58+f10Zc1AwM6gO0tQHyr6seYznUT+6IiIhaJt6TphqMjYBRQ1XTm9hnmohINhon6bS0NPzyyy9qy7777ju4ubnB3t4ekydPVhvchJq36oFNtu8Gim/KGwsRUUulcZJ+7733cOzYMWn+yJEjeOGFFxAYGIg5c+Zgy5YtWLx4caMESU3PqxvQ2RkoKwe2psgdDRFRy6Rxks7IyEBAQIA0v3btWvj5+WH58uWIjIzEF198gfXr1zdKkNT01IYJ5ZOxiIhkoXGSvn79OhwcHKT5Xbt2ITg4WJrv06cPsrKytBsdySr09sAmfxwEruTJGwsRUUukcZJ2cHCQuldVVFTgwIEDePTRR6X1RUVFMDY21n6EjeS///0vevTogZ49e2L16tVyh6OTOrUH+niqRh77mcOEEhE1OY2T9OOPP445c+YgNTUVc+fORatWrTBw4EBp/eHDh9GlS5dGCVLbjhw5gh9++AHp6en4888/sXTpUty4cUPusHTS2OpL3kzSRERNTuMk/f7778PIyAiDBw/G8uXLsXz5cpiYmEjrV65ciREjRjRKkNp24sQJ+Pv7w8zMDObm5vDy8kJ8fLzcYemkkUMAE2Pg5Dng+Bm5oyEialk0TtJ2dnZISUnB9evXcf36dYwZM0Zt/YYNGzB//nytBJWSkoJRo0bByckJCoUCcXFxNcpER0fD1dUVZmZm8PPzw759+zTefs+ePZGcnIwbN27g+vXrSE5OxqVLl7QSu76xtgSG3b6rsYkNyIiImlS9BzOxtraGoWHNcSJtbW3VzqwfRElJCby8vBAdHV3r+nXr1iEyMhLz58/HgQMH4OXlhaCgIOTm5kplvL290bNnzxqvy5cvw8PDAzNmzMCwYcMwduxYPProo7XWiVTG3r5A8nMi0EIeF05EpBM0Hrt70qRJGm1w5cqVDxTQ3RQKBWJjYxEaGiot8/PzQ58+fbB06VIAgFKphLOzM1599VXMmTOn3vt48cUXMWbMGIwcOfKeZcrLy9UGayksLISzs7Nejt19t4pbQJ+ngBuFwOpPgIG+ckdERNS8aTp2t8Zn0jExMUhKSpIuEd/r1dgqKiqQnp6OwMBAaZmBgQECAwORlpam8Xaqz7pPnTqFffv2ISgoqM7yixcvhrW1tfRydnZuWAWaIRNj4IkhqmkOE0pE1HQ0fsDGyy+/jB9//BGZmZmIiIjAv/71L9ja2jZmbLXKz89HVVWVWp9tQNVF7OTJkxpvZ/To0SgoKEDr1q2xatUqGBnVfSjmzp2LyMhIab76TLqlGDMcWL0ZiE8FFpUCrczljoiISP9pfCYdHR2NK1euYPbs2diyZQucnZ0xfvx4bNu2DRpeMdcpaWlpOH78OP7880/4+Pjct7ypqSmsrKzUXi2JTw+gkxNwswzY9rvc0RARtQz1ajhmamqKiRMnIiEhAcePH0ePHj3wyiuvwNXVFcXFxY0Voxo7OzsYGhoiJydHbXlOTg4cHR2bJIaWSKEAxty+w8BhQomImkaDH1VpYGAAhUIBIQSqmrDJr4mJCXx8fJCYmCgtUyqVSExMhL+/f5PF0RJVj+Wdmg7kXpM3FiKilqBeSbq8vBw//vgjhg8fjoceeghHjhzB0qVLcfHiRVhYWGgtqOLiYmRkZCAjIwMAkJmZiYyMDFy8eBEAEBkZieXLl+Pbb7/FiRMn8PLLL6OkpAQRERFai4FqcusI9PYAlEpgc+L9yxMR0YPRuOHYK6+8grVr18LZ2RmTJk3Cjz/+CDs7u0YJav/+/Rg6dKg0X91gKywsDDExMZgwYQLy8vIwb948ZGdnw9vbG/Hx8TUak5H2jQkEDh5XDWzy4ji5oyEi0m8a95M2MDBAp06d0Lt3bygUinuW27Rpk9aC02Wa9nHTN9cKgD5PApVVQMJK4CE3uSMiImp+NM0hGp9JP//883UmZ2oZbK2BoX5Awh+qh2689ZLcERER6S+Nz6RJXUs9kwaAX5OBV94FnOyB3T8CBg1ufkhE1DJpfcQxomoB/QCr1sDlXGDPIbmjISLSX0zSVG9mJsDjQ1TT7DNNRNR4mKSpQaoHNtmaApSV112WiIgahkmaGqRvL6CDA1BUAiTsljsaIiL9xCRNDWJgAIRWDxO6Q95YiIj0FZM0NVj1MKG79gFXb8gaChGRXmKSpgZzdwE8H1INbLJ5p9zREBHpHyZpeiBjb59Nx7GVNxGR1jFJ0wMZNQwwNAAyTgJnL8odDRGRfmGSpgfSzhYY2Ec1zQZkRETaxSRND6z6kndsAsBBZomItIdJmh7YiP5Aa3Pg72xg/1G5oyEi0h9M0vTAzM2A4EGq6U3b5Y2FiEifMEmTVlT3mf4lGSivkDUUIiK9wSRNWuHvDTjaAYXFwM49ckdDRKQfmKRJKwwNgdEBqmk+GYuISDuYpElrxo5Q/dy5B7hRKG8sRET6gEmatKZbZ8CjC3CrUnVvmoiIHgyTNGlV6O0GZGzlTUT04JikSatCA1SPsUw/Bly8LHc0RETNG5M0aZWDHdD/EdU0G5ARET0YJmnSujGBqp+bOEwoEdEDYZImrXtskGoUsvOXgIMn5I6GiKj5YpImrWttDgQNUE3zkjcRUcMxSVOjqB4mdMtOoOKWvLEQETVXLSJJjxkzBm3atMFTTz2ltjwrKwtDhgyBh4cHevXqhQ0bNsgUof4Z4AO0awNcLwR27ZM7GiKi5qlFJOmZM2fiu+++q7HcyMgIUVFROH78OLZv347XXnsNJSUlMkSof4wMgRAOE0pE9EBaRJIeMmQILC0tayxv3749vL29AQCOjo6ws7PDtWvXmjg6/VU9TOiOP4CCYnljISJqjmRP0ikpKRg1ahScnJygUCgQFxdXo0x0dDRcXV1hZmYGPz8/7Nun/eun6enpqKqqgrOzs9a33VL16Aq4uwDlt4DfdskdDRFR8yN7ki4pKYGXlxeio6NrXb9u3TpERkZi/vz5OHDgALy8vBAUFITc3FypjLe3N3r27FnjdfmyZkNeXbt2Dc8//zy++eYbrdSJVBSKf86m43jJm4io3ozkDiA4OBjBwcH3XP/ZZ5/hpZdeQkREBABg2bJl+PXXX7Fy5UrMmTMHAJCRkdHg/ZeXlyM0NBRz5sxBv3796ixXXl4uzRcW8jFPmggNBD5aDuw5BPydDXR0lDsiIqLmQ/Yz6bpUVFQgPT0dgYGB0jIDAwMEBgYiLS3tgbcvhEB4eDiGDRuG5557rs6yixcvhrW1tfTiZXHNONkDj3qrpuN2yBoKEVGzo9NJOj8/H1VVVXBwcFBb7uDggOzsbI23ExgYiHHjxuG3335Dx44dpQS/e/durFu3DnFxcfD29oa3tzeOHDlS6zbmzp2LgoIC6ZWVldXwirUwY6ufjMVhQomI6kX2y91NYceO2k/hBgwYAKVSqdE2TE1NYWpqqs2wWozgQcA7nwNnLwJH/wI8H5Y7IiKi5kGnz6Tt7OxgaGiInJwcteU5OTlwdOTNzebCygIYfvt2/yY2ICMi0phOJ2kTExP4+PggMTFRWqZUKpGYmAh/f38ZI6P6qm7lvXknUFklbyxERM2F7Je7i4uLcebMGWk+MzMTGRkZsLW1RadOnRAZGYmwsDD4+vqib9++iIqKQklJidTam5qHQX0AW2sg/zqQuh8Y6id3REREuk/2JL1//34MHTpUmo+MjAQAhIWFISYmBhMmTEBeXh7mzZuH7OxseHt7Iz4+vkZjMtJtxkbAqGHAt7HApu1M0kREmlAIwfa2DVFYWAhra2sUFBTAyspK7nCahYwTwOhXADNTIH0TYNFK7oiIiOShaQ7R6XvSpF+8ugGdnYGyciA+Re5oiIh0H5M0NRmFQjUCGQDEJdZdloiImKSpiT0xRPVz7yGgpFTWUIiIdB6TNDWpzs5AJyeg4hbwxwG5oyEi0m1M0tSkFApgSF/VdLL2nzhKRKRXmKSpyVUn6Z17AA1HZSUiapGYpKnJ9esNWLYGLucCiQ/+MDMiIr3FJE1NztwMeOYJ1fT6rfLGQkSky5ikSRYhAaqfqels5U1EdC9M0iSLHl2BDg5AaRmw6Cu5oyEi0k1M0iQLhQJ4f6Zqev1W4HqBvPEQEekiJmmSTYA/4NEVuFUJ/PCL3NEQEekeJmmS1UvjVD+j1wBlFfLGQkSka5ikSVahgYBdG1XjsRNn7l+eiKglYZImWRkYAJ4PqaYzTsobCxGRrjGSOwAir25A0l5gwRLg8Engs7nAoZPA7E+AazdUZ9lVVYCJiaqPNQDcugXcLAWgUD2X2shINXpZaZlqnZER0Mpc1UBNqQSKSgAIwNwcMDFWbaOsHCgvBwwMAQtzQGEACCVQXAooqwBTE8DsPvu7WQpUVqrmW99+PnZVFVB8U7W/VuaquIUAysqA8oqa+yu6qfppaqp61jagGtu89Pb+LFurvswIAZTcVG3f2Fi1bUC1/5JS1f5a344NUB2LigrA0FC1vPpYFN/en5nZP/srK1fFpzBQ1e/u/d157CsrVcuhAFqb/7O/m6Wq43T3/uo69goDwLLVvY+9QqGqw53H3tBQtY2Sm/8c+1bmqpgrK9WPvXH1/u5x7KX9aXDsq99rY2PVsahrf9XHvnp/Bob//F3c/V6XV2h27BUK1fEtuan6vXu+17frd+d7fa9jX308lVX/HIs793f3sTcy+udY3Pl/dvexUNvfXcfe4K793evY1/Z/1tBjr6y6//9Z9bGo7f/szmP/bAgwb5pqWVNQCCFE0+xKv2j6wG66v8u5wNjpwJU81fy0Z4Cv1wGVVfLGRURUmy3LgF4PP9g2NM0hvNxNsnOyB5JXA8GDVPPRPzBBE5HuitvRdPtikiadYGYCRM8DJjwudyRERHUrKGq6ffGeNOkMQ0Pg41nAn0eAc1nA/80Fxo6QOyoiIpVblcCNQqCdbdPtk0madM63H6kakI0cInckRET/MDZq2gQNMEmTDurUXvUiImrpeE+aiIhIRzFJExER6SgmaSIiIh3FJE1ERKSjmKSJiIh0FJM0ERGRjmIXrAaqHvK8sLBQ5kiIiKi5qc4d93t8BpN0AxUVqcaFc3Z2ljkSIiJqroqKimBtbX3P9XwKVgMplUpcvnwZlpaWUDzAM8sKCwvh7OyMrKysFvM0LdaZddZHLa2+AOv8IHUWQqCoqAhOTk4wMLj3nWeeSTeQgYEBOnbsqLXtWVlZtZg/8mqsc8vQ0urc0uoLsM4NVdcZdDU2HCMiItJRTNJEREQ6iklaZqamppg/fz5MTU3lDqXJsM4tQ0urc0urL8A6NwU2HCMiItJRPJMmIiLSUUzSREREOopJmoiISEcxScsoOjoarq6uMDMzg5+fH/bt2yd3SA2WkpKCUaNGwcnJCQqFAnFxcWrrhRCYN28e2rdvD3NzcwQGBuL06dNqZa5du4Znn30WVlZWsLGxwQsvvIDi4uImrEX9LF68GH369IGlpSXs7e0RGhqKU6dOqZUpKyvDtGnT0LZtW1hYWODJJ59ETk6OWpmLFy9i5MiRaNWqFezt7TFr1ixUVlY2ZVU08tVXX6FXr15S/1B/f39s3bpVWq9Pdb2XDz/8EAqFAq+99pq0TN/qvWDBAigUCrVXt27dpPX6Vt9qly5dwr/+9S+0bdsW5ubm8PT0xP79+6X1sn2GCZLF2rVrhYmJiVi5cqU4duyYeOmll4SNjY3IycmRO7QG+e2338Tbb78tNm3aJACI2NhYtfUffvihsLa2FnFxceLQoUMiJCREuLm5idLSUqnMY489Jry8vMSePXtEamqq6Nq1q5g4cWIT10RzQUFBYtWqVeLo0aMiIyNDPP7446JTp06iuLhYKjN16lTh7OwsEhMTxf79+8Wjjz4q+vXrJ62vrKwUPXv2FIGBgeLgwYPit99+E3Z2dmLu3LlyVKlOmzdvFr/++qv466+/xKlTp8S///1vYWxsLI4ePSqE0K+61mbfvn3C1dVV9OrVS8ycOVNarm/1nj9/vujRo4e4cuWK9MrLy5PW61t9hRDi2rVrwsXFRYSHh4u9e/eKc+fOiW3btokzZ85IZeT6DGOSlknfvn3FtGnTpPmqqirh5OQkFi9eLGNU2nF3klYqlcLR0VF88skn0rIbN24IU1NT8eOPPwohhDh+/LgAIP7880+pzNatW4VCoRCXLl1qstgfRG5urgAgdu3aJYRQ1dHY2Fhs2LBBKnPixAkBQKSlpQkhVF9uDAwMRHZ2tlTmq6++ElZWVqK8vLxpK9AAbdq0EStWrND7uhYVFQl3d3eRkJAgBg8eLCVpfaz3/PnzhZeXV63r9LG+Qgjx1ltviQEDBtxzvZyfYbzcLYOKigqkp6cjMDBQWmZgYIDAwECkpaXJGFnjyMzMRHZ2tlp9ra2t4efnJ9U3LS0NNjY28PX1lcoEBgbCwMAAe/fubfKYG6KgoAAAYGtrCwBIT0/HrVu31OrdrVs3dOrUSa3enp6ecHBwkMoEBQWhsLAQx44da8Lo66eqqgpr165FSUkJ/P399bquADBt2jSMHDlSrX6A/r7Hp0+fhpOTEzp37oxnn30WFy9eBKC/9d28eTN8fX0xbtw42Nvbo3fv3li+fLm0Xs7PMCZpGeTn56OqqkrtjxgAHBwckJ2dLVNUjae6TnXVNzs7G/b29mrrjYyMYGtr2yyOiVKpxGuvvYb+/fujZ8+eAFR1MjExgY2NjVrZu+td23GpXqdrjhw5AgsLC5iammLq1KmIjY2Fh4eHXta12tq1a3HgwAEsXry4xjp9rLefnx9iYmIQHx+Pr776CpmZmRg4cCCKior0sr4AcO7cOXz11Vdwd3fHtm3b8PLLL2PGjBn49ttvAcj7GcYHbBBpwbRp03D06FH8/vvvcofSqB5++GFkZGSgoKAAGzduRFhYGHbt2iV3WI0mKysLM2fOREJCAszMzOQOp0kEBwdL07169YKfnx9cXFywfv16mJubyxhZ41EqlfD19cUHH3wAAOjduzeOHj2KZcuWISwsTNbYeCYtAzs7OxgaGtZoEZmTkwNHR0eZomo81XWqq76Ojo7Izc1VW19ZWYlr167p/DGZPn06fvnlFyQlJak9Gc3R0REVFRW4ceOGWvm7613bcalep2tMTEzQtWtX+Pj4YPHixfDy8sLnn3+ul3UFVJd3c3Nz8cgjj8DIyAhGRkbYtWsXvvjiCxgZGcHBwUEv630nGxsbPPTQQzhz5ozevs/t27eHh4eH2rLu3btLl/nl/AxjkpaBiYkJfHx8kJiYKC1TKpVITEyEv7+/jJE1Djc3Nzg6OqrVt7CwEHv37pXq6+/vjxs3biA9PV0qs3PnTiiVSvj5+TV5zJoQQmD69OmIjY3Fzp074ebmprbex8cHxsbGavU+deoULl68qFbvI0eOqP1zJyQkwMrKqsaHhi5SKpUoLy/X27oGBATgyJEjyMjIkF6+vr549tlnpWl9rPediouLcfbsWbRv315v3+f+/fvX6D75119/wcXFBYDMn2ENbnJGD2Tt2rXC1NRUxMTEiOPHj4vJkycLGxsbtRaRzUlRUZE4ePCgOHjwoAAgPvvsM3Hw4EFx4cIFIYSq+4KNjY34+eefxeHDh8Xo0aNr7b7Qu3dvsXfvXvH7778Ld3d3ne6C9fLLLwtra2uRnJys1l3l5s2bUpmpU6eKTp06iZ07d4r9+/cLf39/4e/vL62v7q4yYsQIkZGRIeLj40W7du10srvKnDlzxK5du0RmZqY4fPiwmDNnjlAoFGL79u1CCP2qa13ubN0thP7V+4033hDJyckiMzNT7N69WwQGBgo7OzuRm5srhNC/+gqh6l5nZGQkFi1aJE6fPi3WrFkjWrVqJVavXi2VkeszjElaRkuWLBGdOnUSJiYmom/fvmLPnj1yh9RgSUlJAkCNV1hYmBBC1YXhnXfeEQ4ODsLU1FQEBASIU6dOqW3j6tWrYuLEicLCwkJYWVmJiIgIUVRUJENtNFNbfQGIVatWSWVKS0vFK6+8Itq0aSNatWolxowZI65cuaK2nfPnz4vg4GBhbm4u7OzsxBtvvCFu3brVxLW5v0mTJgkXFxdhYmIi2rVrJwICAqQELYR+1bUudydpfav3hAkTRPv27YWJiYno0KGDmDBhglp/YX2rb7UtW7aInj17ClNTU9GtWzfxzTffqK2X6zOMT8EiIiLSUbwnTUREpKOYpImIiHQUkzQREZGOYpImIiLSUUzSREREOopJmoiISEcxSRMREekoJmkiIiIdxSRNRESko5ikiajBwsPDERoaKncYRHqLSZqIiEhHMUkT0X1t3LgRnp6eMDc3R9u2bREYGIhZs2bh22+/xc8//wyFQgGFQoHk5GQAQFZWFsaPHw8bGxvY2tpi9OjROH/+vLS96jPwd999F+3atYOVlRWmTp2KiooKeSpIpKOM5A6AiHTblStXMHHiRHz88ccYM2YMioqKkJqaiueffx4XL15EYWEhVq1aBQCwtbXFrVu3EBQUBH9/f6SmpsLIyAgLFy7EY489hsOHD8PExAQAkJiYCDMzMyQnJ+P8+fOIiIhA27ZtsWjRIjmrS6RTmKSJqE5XrlxBZWUlxo4dCxcXFwCAp6cnAMDc3Bzl5eVwdHSUyq9evRpKpRIrVqyAQqEAAKxatQo2NjZITk7GiBEjAAAmJiZYuXIlWrVqhR49euC9997DrFmz8P7778PAgBf5iABe7iai+/Dy8kJAQAA8PT0xbtw4LF++HNevX79n+UOHDuHMmTOwtLSEhYUFLCwsYGtri7KyMpw9e1Ztu61atZLm/f39UVxcjKysrEatD1FzwjNpIqqToaEhEhIS8Mcff2D79u1YsmQJ3n77bezdu7fW8sXFxfDx8cGaNWtqrGvXrl1jh0ukV5ikiei+FAoF+vfvj/79+2PevHlwcXFBbGwsTExMUFVVpVb2kUcewbp162Bvbw8rK6t7bvPQoUMoLS2Fubk5AGDPnj2wsLCAs7Nzo9aFqDnh5W4iqtPevXvxwQcfYP/+/bh48SI2bdqEvLw8dO/eHa6urjh8+DBOnTqF/Px83Lp1C88++yzs7OwwevRopKamIjMzE8nJyZgxYwb+/vtvabsVFRV44YUXcPz4cfz222+YP38+pk+fzvvRRHfgmTQR1cnKygopKSmIiopCYWEhXFxc8OmnnyI4OBi+vr5ITk6Gr68viouLkZSUhCFDhiAlJQVvvfUWxo4di6KiInTo0AEBAQFqZ9YBAQFwd3fHoEGDUF5ejokTJ2LBggXyVZRIBymEEELuIIioZQkPD8eNGzcQFxcndyhEOo3XlYiIiHQUkzQREZGO4uVuIiIiHcUzaSIiIh3FJE1ERKSjmKSJiIh0FJM0ERGRjmKSJiIi0lFM0kRERDqKSZqIiEhHMUkTERHpKCZpIiIiHfX/HOmuD1lHYTkAAAAASUVORK5CYII=",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# viz: the loss curve of the kernel recovery\n",
"plt.figure(figsize=(5, 3))\n",
"plt.plot(loss_hist, color=\"#1E40FF\")\n",
"plt.yscale(\"log\"); plt.xlabel(\"step\"); plt.ylabel(\"MSE (log)\")\n",
"plt.title(\"Recovering the [1,-1] edge kernel\"); plt.tight_layout(); plt.show()"
]
},
{
"cell_type": "markdown",
"id": "00b99135",
"metadata": {},
"source": [
"> **Key takeaways**\n",
"> - A conv kernel is just weights; gradient descent learns it from data instead of you setting it.\n",
"> - Ground-truth recovery is the strongest cheap check: we hid `[1,-1]`, learned it back, and asserted the match.\n",
"> - The four-comment loop (forward / loss / backward / update) is identical everywhere; the clear-grads step uses `grad = None`, never `.data`.\n"
]
},
{
"cell_type": "markdown",
"id": "a9499016",
"metadata": {},
"source": [
"## Part 4 — A small CNN, accounted for\n",
"\n",
"> **Objectives**\n",
"> - Load FashionMNIST (idempotent, cached), build a LeNet-shaped CNN, and read its layer summary.\n",
"> - Count its parameters and FLOPs by hand and check against the library.\n",
"> - Train it on a subset, then stage a deliberate failure (10% accuracy) and fix it.\n",
"\n",
"FashionMNIST is 28x28 grayscale clothing images in 10 classes. It is a drop-in, harder replacement for MNIST and downloads once to `data/` (the second run is a no-op).\n"
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "73c183e5",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:51.626292Z",
"iopub.status.busy": "2026-06-10T19:24:51.626207Z",
"iopub.status.idle": "2026-06-10T19:24:51.858447Z",
"shell.execute_reply": "2026-06-10T19:24:51.858175Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"train 60000 · test 10000 · classes 10\n"
]
}
],
"source": [
"from torchvision import transforms\n",
"# Tier-2 dataset: torchvision built-in, root=\"data\", idempotent (cached after first run)\n",
"tfm = transforms.Compose([transforms.ToTensor()]) # -> float tensor in [0,1], shape (1,28,28)\n",
"train_full = torchvision.datasets.FashionMNIST(root=\"data\", train=True, download=True, transform=tfm)\n",
"test_full = torchvision.datasets.FashionMNIST(root=\"data\", train=False, download=True, transform=tfm)\n",
"CLASSES = [\"T-shirt\", \"Trouser\", \"Pullover\", \"Dress\", \"Coat\",\n",
" \"Sandal\", \"Shirt\", \"Sneaker\", \"Bag\", \"Ankle boot\"]\n",
"print(f\"train {len(train_full)} · test {len(test_full)} · classes {len(CLASSES)}\")"
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "7ddc80f5",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:51.869633Z",
"iopub.status.busy": "2026-06-10T19:24:51.869525Z",
"iopub.status.idle": "2026-06-10T19:24:53.960925Z",
"shell.execute_reply": "2026-06-10T19:24:53.960654Z"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA3cAAAE8CAYAAABjM0iQAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAYTJJREFUeJzt3XlgVdW1+PGFoMwhjJHBJgxCgCSEUUCZEXBgdMCCghM4tSjVlr5OzlrBseqvWkFRVMQKAipqwULrE6mi8AAVGcMU5jlMEji/P/rM61l7wd1cbiA5fD9/uRf7nHvuufvuc465a+0SQRAEAgAAAAAo1s463QcAAAAAADh5PNwBAAAAQATwcAcAAAAAEcDDHQAAAABEAA93AAAAABABPNwBAAAAQATwcAcAAAAAEcDDHQAAAABEAA93AAAAABABxfrhbu/evVKhQgW56aabvLeZM2eOZGdnm//WuXNnmTp16gkdw/H2F6/x48fL0qVLE7pPJE52drZkZ2dLkyZNpGTJkgXtgQMHem1/3333yV133WX+2/Tp02XkyJHH3Hbq1Kkyb968eA4bRRTjCYUhLS1NGjVqJM2aNZMGDRpI3759Ze7cuaf7sFCMMDchXj/OPz+On+eff/64/cePHy/9+vUTkcK5rz7TlDrdB3AyJk2aJC1btpQpU6bIM888IxUqVDjdh5QQ48ePl+TkZElPTz/dhwLDwoULRUQkJydHsrOzC9qJ0KdPH+nTp4/5b/n5+TJ16lTJzs6Wtm3bJuw1cXoxnlBYJk2aVHCTNGXKFLn00kvl448/lgsuuKCgz9GjR0VE5KyzivX/60UhYG7Cyfhx/lmzZo1kZWVJhw4dJCsr63QfVkh+fr6UKlWsH4VMxXo2HzdunIwaNUo6duwokyZNKoiPHz9eunfvLj/96U8lMzNTWrVqJatWrXK237Nnj/To0UMeeOAB59/27t0rw4YNkzZt2khWVpYMHz5cfvjhB/M48vPzZciQIZKRkSEtW7YMTYBjxoyRpk2bSmZmpgwePFh2794tIiJ5eXly4403SkZGhmRkZMj9998vIiJjx46V+fPny8iRIyU7O1tmzJhxMqcIp9Hy5cvlwgsvlGbNmklmZqb87ne/K/i3jRs3Su/evaVJkybStWtX2bFjh4i4//eqadOmctNNN0l2dra88cYbMn36dBkzZoxkZ2fL2LFjT8fbwmnCeMLJGDBggNx6663y+OOPy3333SdXXHGF9OzZUzIyMmTjxo3y8ccfy0UXXSQtW7aUNm3ayOzZs0Xk2OPuvffek6ysLMnOzpaMjAyZNm3a6Xx7OI2Ym3A8qamp0qhRIxk0aFDoL7nPPfecXH/99TG3nzBhgmRlZUlWVpZcdtllsmHDBhERadiwocyfP7+g3/jx46V///4iIrJp0ya5+uqrpU2bNs6YTEtLk1GjRkmbNm1k6NChiXmTRU1QTH3zzTdB7dq1g/z8/GDatGlBu3btCv7tlVdeCZKSkoJVq1YFQRAEo0aNCoYPHx4EQRDMnj07aNasWbB27dqgRYsWwauvvlqwXadOnYJ33303CIIgGDZsWMG/HT16NLjpppuC0aNHO8cxe/bsQESCWbNmBUEQBJMmTQoaNWoUHD16NJgxY0aQnp4e7Ny5s2Cft956axAEQfCrX/0qGDRoUHDkyJEgLy8vyM7ODt566y3nOFB0rV69OqhUqdIx/33EiBHBI488UtDevn17EARBcO+99wapqanBtm3bgiAIgoEDBxb0e+WVV4K+ffsGQfDvsVWiRIlgzpw5BfsYOnRo8NRTTyX2jaBIYDwhkVJTU4MFCxaEYlOmTAkaN24c3HvvvUHNmjWDTZs2BUEQBCtXrgzatm0b7N69OwiCIFi+fHlw7rnnBgcPHjzmuMvKygrmzp0bBEEQHDlypOA6h+hhbsKJ+s/5Z9GiRUHFihWDQYMGBXfeeWdBn2effTYYOnRoEATueGjWrFkQBEGwePHiICUlJVi/fn0QBEHw0EMPBb169QqCIAgefvjh4I477ijYX8eOHYPp06cHQRAEPXr0KBhPhw8fDnr27Bm8/fbbBcd20003BUePHi2Mt14kFNu/RY4bN06GDBkiJUuWlEsvvVRuueUW+e6776Rx48YiItKuXTupW7duwX8/++yzBdtu3rxZOnbsKGPHjpVu3bqZ+586dap8/vnn8uSTT4qIyIEDB6RkyZJm37S0tIL9XH311TJ8+HBZt26dzJo1SwYOHCjJyckiInLbbbfJVVddJSIis2bNkieeeELOOussKV++vAwZMkRmzpzp/Vt2FH0dO3aUX/7yl5KXlyedOnWS7t27F/xbr169pGrVqiLy7/G5ePFicx/16tWTTp06nZLjRdHGeMLJCoKg4L8vvfRSSUlJERGRjz76SFasWCEdO3Ys+PezzjpL1q5de8xx161bN7nzzjvlyiuvlB49epAjcwZjboJl4MCBUrZsWSlXrpy8/PLLsmTJEtm1a9cJ7WP27NnSq1cvqV27toiI3H777fLAAw/IkSNHZMiQIdK8eXN54oknZMOGDbJs2TK55JJLZN++ffLJJ5/I5s2bC/aTl5cn33//fUH7+uuvlxIlSiTkfRZFxfJnmYcPH5YJEybIq6++KmlpadKgQQPZv3+/jBs3rqBPmTJlCv67ZMmSkp+fX9BOTk6WjIwMef/990MXu/8UBIFMnjxZFi5cKAsXLpTvv/9eXnzxRa/jK1GihDlojjeQojzIzhQjRowoSDhfvHixXHHFFfLZZ59Jo0aN5LnnnpPLL7+8oO/xxud/ikoeKU4c4wmJ9uWXX0pGRoaIhMdCEARy8cUXF1zvFi5cKBs2bJDzzz//mOPuySeflFdeeUXKlSsnQ4cOldGjR5+W94RTj7kJPiZNmiQLFy6UuXPnypVXXimlSpWSI0eOFPz7wYMHT3if/3mvXKdOHWnVqpVMmzZNXn31Vbn22mulVKlSBff18+bNK5jPVqxYEfppZtTHW7F8uJs+fbrUq1dPNmzYIDk5OZKTkyPz5s2TCRMmyOHDh2NuX7p0aZkyZYrk5ubKsGHDChLK/1O/fv3kscceK5iIdu7cKStWrDD3l5OTU5Cf8M4770hKSorUqVNHunfvLm+//bbs2bNHRERefPFF6dGjh4iIdO/eXcaNGydBEMi+fftkwoQJBf+WlJRUkJuH4uNPf/pTwUSSmZkpy5cvl5SUFBkyZIiMHj06IZW/GBtnDsYTEmnatGny5z//We6++27n33r27CmzZs2SRYsWFcS++OILEZFjjrulS5dK06ZN5Wc/+5ncdtttVDY8gzA3IR4NGjSQ+fPny5EjR2T//v0yefLkmNt06dJFPvroI8nNzRURkRdeeEG6detW8Eu6G264QV5++WV57bXX5MYbbxSRfz+4denSRf74xz8W7Cc3N1fWr19fCO+qaCqWD3fjxo2TwYMHh2KNGzeW2rVry3vvvee1j7PPPlvefPNNOXLkiAwePNj5v0lPPfWUlC1bVrKzsyUrK0u6desmOTk55r6aNm0q48ePl8zMTHn00Udl4sSJUqJECbnkkkvkhhtukHbt2klmZqbs2bNHHn30URER+f3vfy9nn322ZGZmygUXXCB9+vSRq6++WkREhg8fLo888ggFVYq5d955RzIzM6V58+YycOBAeeGFF056n9ddd528/fbb0rx5c5LMzzCMJ5yogQMHFiyFMG7cOJkxY0aoUuaPGjRoIG+++abccsst0qxZM2ncuLE8/fTTInLscfeb3/xGmjZtKs2bN5cJEybIfffddwrfGYoS5ib4GDBggNSqVUsaN24sl19+uTRv3jzmNhkZGTJmzBjp1auXZGVlyaeffiovvfRSwb/37dtXvvzyS0lJSSlIyxIReeONN2TFihWSkZEhmZmZMmDAANm+fXuhvK+iqERwrN8lAgAAAACKjWL5lzsAAAAAQBgPdwAAAAAQATzcAQAAAEAE8HAHAAAAABHAwx0AAAAARAAPdwAAAAAQATzcAQAAAEAElPLtWKJEicI8DhQjJ7s0YmGOJWvfiVzKMT09PdR+7rnnnD5//etfndiCBQtC7R9++MHpc/jwYSeWkZHhxPr37x9qr1y50ukzZswYJ7Zr1y4ndrol4rMpLnNTjRo1nNj1118far/22mtOn02bNhXWIYmISHZ2dqitx7iIyOTJk52YNV5Pt6iOp7S0NCfWuXNnJ9a3b99Q21q09/XXX3diX3/9dahtjYErrrjCiXXr1s2J7d+/P+br/eUvf3FiRVFRvtadCWrVqhVq5+bmnqYjOXlFbW7S+0rkfZJ1revatWuoffPNNzt9rHuU7777LtS27p2Sk5OdWPv27Z3YvHnzQu3f/OY3Tp8DBw44MR+Ffe95ovvmL3cAAAAAEAE83AEAAABABPBwBwAAAAARwMMdAAAAAERAicAz44/EYPzodCWZJzJhVReRuOaaa5w+VgGBI0eOhNrly5d3+pQtW9aJVa1a9QSP8NiWLVsWah89etTp06hRIye2efPmUPvjjz92+jz++ONObMmSJSd6iN6KWpJ5olSoUMGJWWPszjvvDLWtZPFt27Y5Md3P2q5ixYpOrHTp0k6sTp06ofa0adOcPp9//rkTswoHnW7FcTxdcsklTmzkyJGhtpXkf8455zixgwcPhtrWGLCKNKWkpITaOTk5Tp/8/HwntnHjRie2e/fuUNsac7Vr13Zin3zySag9YsQIp8+pFsWCKvo8i4hUrlzZieliPMOGDXP6WOPEhy6UIiIye/ZsJ6avpWvWrHH69OrVy4nt27cvruMqTKdzbor33qlatWpOTF+zunfv7vSxvvP6M7H6WIWcrDlMs4p7rV+/3onp+cq6V9uxY4cT++c//xlqP/vss06fnTt3xjzORKKgCgAAAACcAXi4AwAAAIAI4OEOAAAAACKAnDucsKKch5CUlOTErIWhs7KyQu2zznL/P8fevXudmM5psX7rrfPyRETOPvvsULtSpUpOHytPwMqni/f8lylTJtS2fm9u5fF8+umnofZ1110X1+tbimOOVLyuuuoqJ6ZzqX772986faz8FJ0jZeUvWDkAeXl5TmzmzJmh9sSJE50+Vg7h1KlTndjpVtTHU/369Z3Yfffd58R0fmy5cuWcPtacpecLK0/uvPPOi3WY5rxjxXR+nfWa1hxp5bXoPDxrQeN77rnHiRWmonyti9ecOXOcmDUu9ZxiXS+sa+TkyZND7WuvvdbpU7JkSSemr60i7hiwck+bNWvmxIqiop5zZ42B9957z4npucn63Hzuiw4dOuT0seYFfe2x7q+sfVn3MtWrVw+1S5Uq5bWdju3fv9/p88ILLzixd99914klCjl3AAAAAHAG4OEOAAAAACKAhzsAAAAAiAAe7gAAAAAgAiio8r/iXeTRWmDxoosucmIffvhhXMegE4+tBPl4+Xym1jkoyknms2bNcmKpqalOTC/QahULsJJt9fn3fS+6+IG16LSVZO6zr3j5jvmaNWuG2j179nT6LF26NK5jKOoFMBJp8ODBTmzLli2hdo0aNZw+1mLOetFhq6CKVZDiq6++cmIvv/xyqF23bl2nz9atW53YRx995MROt6I+nv7f//t/TswqSKDnI6ugjS6QJOLOT1biv3UN0YVRrH1bc6Q17jSrAIJ1DPo8WIutW8WxPvjgg5jHEK+ifK2Lly54IiLSqlUrJ6bHTpUqVZw+ukCFiHt90gtAi7gFzUTcQh0i7jXYWsS8a9euTqwoKupz09tvv+3ErEXMddETXSxOxH6vusiKNZ9YhVF0zJovrXnIKlqnjzXe+zer6Ip1Hvr16xdqWwXN4kVBFQAAAAA4A/BwBwAAAAARwMMdAAAAAEQAD3cAAAAAEAFuxYgzlFWkQieCN2jQwOlz8803O7EDBw44sX379oXaVlLoF1984cR8CqhYSaH6/Vh9fPbtW+TjdGnZsmWobRVP2bZtmxPTidrW+7SKCtSuXTvULleunNPHGks6mdgq1mIVHrA+N524a32Oe/fudWLr16+PuZ1FH5c15u+55x6vfZ3JrGRqnbBuFQz4xS9+4cTq1KkTaluFDVavXu3EdCEh6xissVkUC0MUR+PHj3diI0eOdGK6gI1VbMIq5qXnGYtVzMkqnKDt2bPHiVnXOh/WMegCCOvWrXP6FGbxlDPFqlWrnFjbtm2dmL4+WMUufOaFnJwcJ9ahQwcntmHDBidWtmzZUNu63iI+ulDaueee6/TRhZZE3GIi1n2E9TmVL18+1Lbuk6wiK/r+w7pPsu7V9OtZ21rHbu1fX7ut+3fr9Xr37h1qT5w40elTWPjLHQAAAABEAA93AAAAABABPNwBAAAAQASQc/e/rJwr/dtba7HM7t27OzGd2yTiLrJo/Sb54osvdmJjx44Nta3cC2sxQ+t3w5q1MK7+zbO1CG5R0qVLl1DbWszSiun3aX3+Vo7BqFGjQu3c3Fynj/X516pVK9TeuHGj08f6DbqVm6Lfj/U5tmjRwon9/Oc/D7V9chFF3HN15ZVXOn3IuYvNJ8fRJ/dJxP3sNm3a5PSx5hidMyrizhXWfJKIBXhh51V//vnnTqxPnz6h9r/+9S+nj/Vd1Z+5lWNpzSl6PFk5JdZ4so5B5+ZZ+aAWvf9f//rXXtvhxHz77bdOzCe3XtcNELHHkrVAuWblalr5e3p8WXmfiE/lypVDbSvnzrqP1Dl3Vq6Zda3T9y1Wfp01BnzyOq3xa22nX9PqY71nPYdZ907Wwub6np6cOwAAAADACeHhDgAAAAAigIc7AAAAAIgAHu4AAAAAIAIoqPK/rMRgrXXr1k4sLS3NiVnJnbpYxscff+z0ad68uRMbPXp0qD1//nynz+LFi53Yd999F2q3adPG6WO9n7lz54baVrJ/UaKLe1iJvD7FcqxFMK0FPF966aVQu0ePHk4fq5jJK6+8EmrfcsstTp8lS5Y4sSpVqjgx/X6sIjtPPfWUE7v99ttDbasYgnUedFGd9PR0p0/Dhg2d2LJly5zYmcwqmKMLlVjJ3Nb4TU5OTthx6aRyq3iKNVaQGH/605+c2J133hlqr1271umjFzoXcYteWAWx9u7dG/OYrDFnFdSwxsXZZ58d8/X0guUiIh9++GGoTfGMwmEtFn748GEnpucr/bmK2IXBvv7661Db+vytY/ApimFdkxEfXfjGOv9WkRU9LqzrmlWQSRefW7lypdPHWvBezzvWvq25yRrTuuiJVfzn8ssvd2L6Na3rr1XYzio2c6rwlzsAAAAAiAAe7gAAAAAgAni4AwAAAIAI4OEOAAAAACLgjMySt1alt4oI6NXlW7Vq5fSxkoWtJEpdcMIqQPHll186sRUrVoTaVtJmu3btnNiAAQNCbSu51Hq9m2++OdQ+dOiQ06coadasWai9bt06p4+V8Fu6dOmY+05KSorZ56OPPnJiVnJvkyZNQu177rnH6fPuu+86sd69ezsxXcRAJ7CLiLRs2dKJ6WIz1ji1CnocPXo01LaKO1hjkIIqYdZ3V49DK1ncSnTXn4lPMYJj8UmQtwrt4MRZBUisIlAXXXRRqP3www977V8XULH2XbZsWSd24MCBmMdpxazrgzV+fPq89957MbfDydOFLUTs+wM9f+g5R8Ser7799ttQ2yrEYn3+VrEUPT/6zmmI7a233gq1P/30U6fP4MGDnVhGRkao/cgjjzh9li5dGtcxlStXzonp+cqav6x7Geuape/NJk6c6PT5r//6Lyem75VTUlKcPlbxqnr16jmxU4W/3AEAAABABPBwBwAAAAARwMMdAAAAAERA5HLuEvmb7AcffDDUrlmzptd21u+Gde6DtWi6zrMQcfP8rN+9WzlXOlfPyr244447nJj+jbBeJPx00r/1FnEX8vVdxFyPE+t33Nu3b4/rmKw8FD12rBwaa+z65EJY+W4WnWtRu3Ztp49Pzp3OzxER6dChgxN79dVXvY7rTGHlLOnP0hoDVn5KorYTcb8z1nbWdwgnzpqfLHpxaGvB37p16zoxnQNl5YT75E5ZYyAvL8+JVa9e3Yn5jKc1a9Y4MZwa27Ztc2JpaWlOTOdNWfl11nxizXOadf9j7Utfj6zrIeIzevToUNuaF2bPnu3EFixYEGpbtQmsnDv9+e7Zs8fpY91z7dq1K9S2xoBVM8MaT5UqVQq1mzZt6vSx5lqde2jNhdaxn86aFfzlDgAAAAAigIc7AAAAAIgAHu4AAAAAIAJ4uAMAAACACIhcQRUrsTJeO3fuDLWtgipWcQlrgWydZGwtaGwlLOtCH1bSq1XMon379qG2ldReo0YNJ2Ytyl1UjBo1yonp82MlulpFQvR21rm3ih/oAjdVq1Z1+lSpUsWJ6YVcrUUwrURh67jOOeecUDs5OdnpM3DgQCdWuXLlUNsauzrh2OqnX1/EPS9wWd9BvfCpVbjEpzCKNcYtPvPj6UwCh80aAxUrVnRi+vpgXYusQgb6O23NO1YRDItP0ZgtW7Z47QuJt2nTJq9+esz5LkauWXOOtS/r+qfvm/Q9GeL38ccfh9rdunVz+lxxxRVOrEePHqG2VTjttttuc2L6PqVBgwZOH+u+WI8f6xpp3ZNY85WeH19//XWnj1WESt97Wvu2xuaAAQNCbX1fLiKyY8cOJ5YI/OUOAAAAACKAhzsAAAAAiAAe7gAAAAAgAni4AwAAAIAIiFxBlUQqV65cqG0lD/sUSRAR2b17d6htrWaflpbmxHQyqS6kcKxj0MduFVywirOcd955TqyomDt3rhM799xzQ20rSTcpKcmJlS9fPtRevny508c6Z/PmzQu1rXNoxfS+rKRgnTwuYn/eel/W528lBS9btizU1mPkWMel95+bm+v0mTp1qhNDmE/xAev8W+NJ9/PZ97HocWcVVLGKLyExrM9Of+br1693+mRlZcXcl/VZ+hS4sOa+MmXKODGrKJMuxlKtWjWnz4YNG5yYZs2HPsVacOJ8iij5FqvT/Xyuh8eK6eufVQwI8fnjH/8YalsFbaxr/XfffRdq9+7d2+nzhz/8IebrW69njUM9LqxxaM0L1rVUz3NWARerMMoXX3wRaltFiWbPnu3E9H1lYRVPsfCXOwAAAACIAB7uAAAAACACeLgDAAAAgAiIXM6dT06a9dtu67e3tWrVCrWt3wNbMWvhWL3ooZWXZy1GrXPzrDwpawFHnXNlLU69aNEiJ6bPQ1FanPrPf/5zzJheqFtE5Pzzz3diepHNTp06OX2s30cvWbIk1N61a5fTx1qg1fr9d7z0GLdydqxFiPUYsD7/wYMHn+TRQcQeh9YY0J+llU9wMvl0mpX/onObrLGjc1RF3BwsazskRk5OjhOzxoW+Fljj0NqXzlmpWrWq08fKRbFyXfQ10TpOcueKFmte8GHNV3pOs+7JLFY/vf99+/adwNHheKZMmRJqW4uYW/d/H374Yag9ffp0p4+Vo7127dpQ2ycnTsS9zli5uBZrjtH33dZi5FaNhtTU1FD7rrvuitlHRKRz586h9oIFC5w+CxcudGKJwF/uAAAAACACeLgDAAAAgAjg4Q4AAAAAIoCHOwAAAACIgMgVVLESfHXiplVQZeDAgU5ML5C9detWp0/ZsmWdmJWcrAsSWIuFW8mdujiLtfCjlWCqj8tKkH/++eedWHZ2dsx9F2U+C1CKuEn/Xbt2dfpYY0kXLLAKTfguRK1ZCeVWTO/Lp4CPiJuYbC0Kj8TwLb7kuzBwPNv5FJeyWON39+7dTowCKqeOtVi4z5xi9bE+Xz03WNtZc6u1QHnFihVjHpdVOAGnT7xFm6w5xqd4mPV61pym79WsQh2IT5MmTUJta46xFuueN29eqH3hhRc6fTIyMpyY/nx9i8zpuciniM+xYvo1rXnOes9vvvlmqG0VQVm1apUTW7duXai9bNkyp09h4S93AAAAABABPNwBAAAAQATwcAcAAAAAEcDDHQAAAABEQPGqluHBKgBiFZfQlixZ4sR0AQQrCdxKCrUKtuhEYKsYwfbt252Yfk2d+C5iF/XQye/r1693+gwaNMiJjRkzJtTWybNFjU6atT4j6/PXSbl79uxx+vh8tr4FMfRxxltIw5dPsvKuXbvi2peVhFzY76e48SnsVFToY7UK9KDw+BRGyc/Pd2JWgS8911lFUCy6nzVnWsXDtmzZ4sSqV68eaufl5XkdA04fq/iETx+fok3W2LW2s+7d9LZpaWmxDhOe6tWrF2pb579OnTpOTBcc2b9/v9PH+sz37t0baltFdaztfAoi+tL3ylaBQj1/ibjv0SoaZZ2r5OTkUFsXaRSxC7EkAn+5AwAAAIAI4OEOAAAAACKAhzsAAAAAiICE5tz5Lmipf2trbWf9Fjbe3AQfM2bMcGL79u0Lta1FHvWi1iJ2vo3Oj/BZSFbEPg8+ffS5sl4vKyvLiVmLFRdl+lz7nC8RkZUrV4baVs5dvPmbPotsxpurZ7GOyWeRYOs9W/T39WR+836miHeB1ngXE7bmxnj3ZW1nfea6n8/8DJd1vvW5tHI8Kleu7MR0bkiVKlW8jmHbtm2hdrly5Zw+lSpVcmI+86E1h6WmpsbcLt5rOU6cz3XGGqfx5upZfHLcyblLHP15WnUgrHlf585Zc4V1LdCfr/V5++Rw+o5Dn2uidf9uHZeeHy3WXKvvIWvVquX0IecOAAAAAHBMPNwBAAAAQATwcAcAAAAAEcDDHQAAAABEwEkVVPFZXPBUJ0V37NjRiV1xxRWh9oUXXuj0sRZi1IuKW8mXVtEN6zzo/VtJm9biwbrIilWIwzp2zTp2a3HZAQMGhNrvvfdezH0XJb7FIHRxHKswgPV56PFsff5Wcq/+3OJdENba16FDh5w+VpKz3j8FCwqPVRzJ+u76jAufYiYns0C6T7EfK6bnFCshH7H5FKKxFixfsmSJE1u3bl2obc0D1ueUkpISalvzYU5Ojte+dOGVjRs3On2swgI4NRo2bOjErPsDPS6ta53Fp2Ceb0xfo6pVq+Z1DIjN53Oy5qYdO3aE2mXLlvXaTr+eb1E53c+nYJ2IXVxP39NZY9q659ILt/sWn9HXZaswVmHhL3cAAAAAEAE83AEAAABABPBwBwAAAAARwMMdAAAAAETASRVUsRIIfeiV3K3k6vPPP9+J6X66+IeInSysC05YCZNWUZKqVauG2rm5uU4fK7HSSk6uUaNGqG0lrFvJ73Pnzg21K1So4PSxisjohNbdu3c7fayE07Zt2zqx4sQ3SVefH2ss+xSWsMaSz+v5FsDwKbJiHWdhJjQjtniLCPiea2tfieK7b9+xj5PXoUMHJ7Zq1SontmbNmlDbuj7t2bPHiSUlJYXauiiKiFuESsS+jtWsWdOJaeeee64T09fILVu2OH2sMedTkAb/p3Hjxk5s/fr1TkzfH5x99tle+9fXtpOZT/S9my78IyLSvn17J6bvmxCbdU9ifbc2b94calsFVXz4FnDRRU+scWLFfIqT+d6HWfOczzHE+3qJwNUZAAAAACKAhzsAAAAAiAAe7gAAAAAgAni4AwAAAIAIOKmCKrr4xoMPPuj0qV69uhNLTk4OtX1WdhcR2bVrV6idn5/v9Nm7d68T08mQVqKllSyuk3Kvvvpqp8/8+fOdmLUKvU4MTktLc/pYMjMzY+573bp1TkwXiLGSXq3iLKmpqV7HFTW1a9d2Yjt37nRielxaBTB8k3sTxXo9q1iOPoZTmdx7pinsc6vHne/4svrpfVnHbsV0ojti8y0Ict5554XaTZo0cfpYBVX0tbVatWpOnxUrVjix8uXLh9p169Z1+ujrr4hbiMVXXl6eExs0aFCo/fTTTzt9KJ5y8rp16+bEfK5jPnOHxbdIlDXH6G1Xrlzp9LntttucGAVVYvP5XKzPXN8XWYV2fIq8Wfu27ul9CsH5jjG9f997NX3/bM2FZcqUifn6Pn0Shb/cAQAAAEAE8HAHAAAAABHAwx0AAAAARIB30oT1e+g//elPoba1eKmVT6dj1gLiFr04uLVvK3dOsxZotXLN/vjHP8bct/V7b5/Fzj/55BOnj5VDoRdz1wuri9gLLOrfQfvmZW3dutWJFSfxLrht/dbb4jMGfRar9s1f8Fno0/rNu87xtPbvuygti5ifOJ8FVEX8cud8Fgs/mQXpffZvHZeeR63FsRHmmzPWs2fPUPvbb791+lj5G/ozsHK7N2zY4MTS09NDbes4rYWus7KynJhe5Ni6Zln5zDrvuUGDBk4fK18QJ0bXShCx7wV8FiO35pN4c3GteUiPcX0fJSLSrl27uF4PiWHNQ9b8ocdPvLUJTuZap2PWvbN1XDrnzpqHsrOznZhPvY/Cwl/uAAAAACACeLgDAAAAgAjg4Q4AAAAAIoCHOwAAAACIAO/M1yFDhjgxXYTEWmDSWihbx6pUqeJ1DLoAhFUYxVrQWxc4KVeunNNHJ4GLiLz66quhdr9+/Zw+7733nhOzktj1e27ZsqXTp0uXLk5MJ3daCaClS5d2Yrrwh8Uq8KDPsV5MN6qsAiRWESFdeMXqYyUT+ywUbX22PgnrVh+fIkV6wWMkjlWsJt4E8ngXD46XT+EXEXveQWLoQiWLFi1y+lhziJ73fT8ja1+aNa9ZMV30wrqGWMV3fIrBUFDl5Fnn1Spw47N4tEWPpZOZq/S+rHu3c88914npcW9d3890e/fuDbXLly/v9PEptqWLjYj43cv4FpfyKTpmxaw5Te/LKiTkU9Rs7dq1Tp9WrVo5MT3ufObZROEvdwAAAAAQATzcAQAAAEAE8HAHAAAAABHAwx0AAAAARIB3QZUtW7Y4MV28pGLFik4fK5FVb2cVXbEKgiQlJYXaO3bscPqsWbPGien9HzhwwOmjk8BF3OIZ7777rtNn8eLFTsxKWNZFY6yE0127djkxnfCpj0nETkzVBR2sPlbiqD7vDRs2dPpEkW9yrxZvsYt4i2tY+/c9Bj12rERon9dDbLrojYhf0vfpONfWnKJZiec+yfaIzbpebNy4MdQuU6aM0ycvL8+J6XFnfbY+33vf64xPwRaruFNKSooT27BhQ6hdvXr1mPtGbJUrVw61q1Wr5vSxCsrpMWfNTT7XHqtAk+/1T9+P/O1vf3P6XHXVVU5MF6ybO3eu0+dMYt1P68/J+kyswkeaVTzMul7Een0R+zj1+PG9T7KuwXpfPsXvrH3l5OQ4fazzoF/P6lNYuDoDAAAAQATwcAcAAAAAEcDDHQAAAABEgHfOnf49vIj729T169c7fayFEfVvvq1cs23btjmxrVu3htrWb2qtHAD9O1crf8HKF9S/QbaOqXHjxk5s3759TkznGVqLhlrHrl/T+i2zlR+h+1l5Ftbin7t37w61s7OznT5RFG/+ULw5UonMubP25ZNzZy0Ii8SwcgcsPgu7nurcNmvsWPMO4ycxfvKTnzgxPQ6sa501xvS1zcp3sval6TwtEfs6Y+1Lx1avXu30Of/8852YzvuqVKmS00fnrovYuff4P/oabl1nrHGi+1nzgjU36TFojVPfXCc95ho1auT0scagvi8703PurHOrY9Z5tO77NZ/FwkX86hr4LFBu9bH2bY1pnxx3azv9fLBs2TKnj3X+9HH53uMlAn+5AwAAAIAI4OEOAAAAACKAhzsAAAAAiAAe7gAAAAAgArwLqixcuNCJTZkyJdS+8cYbnT65ublObNWqVaG2tYC4tbC5LoxiFQmxknd1EqW1sLqVRKmTLa3FWPVis9Z21v6t5Euf8+C7+LmO+RZiqVu3bqhtLW5alCVyEWgrUTjeY/BJpPV9vXgXSddjMN73h9h8Fo0Vcb+DpzLh+kd6rFhzoTV/NGjQINS2rhGIzfoe6s/EuvZYBW30NdK6XvgUs7Cuv9b1wrqW1q5dO9SeP3++06djx45OTF9LrWukVeiFgirH17t371DbKgxnfb/1OLHGjTVO9BxmLdxsFbWzFszWx2UVgbPGZWZmphNDmE9hNp+CKr4F3fQ4sLaz5sJ4C7H43If7Xm91cadvvvnG6WO9Hx2joAoAAAAA4ITwcAcAAAAAEcDDHQAAAABEAA93AAAAABAB3gVVLI8++miobSXU33PPPU4sLS0t1LYSfK0iIfv27Qu1reRLq5CBTsy2tvNJyLQSg62YdQy6n29ipe5nFTixkpqrVKkSaltJqVZy8qJFi0Lt119/3ekzYcIE+2CLAN/EWs0qPGAVLPBhnWs95qwk8HiP3Ve8BVUSeQxnilq1ann10wnX1rn2GU++n5GV9K33b41Da7xa8zZOXLVq1ZyYvoZs3brV6ZORkeHEdKEKq0iFdX3Sn2/FihW9trOKgGVlZYXaH3zwgdPHur7r/VvFU6wiKzi++vXrh9rWZ2vdC+i5wipcY22nC7i8//77Tp8DBw44Met6u3fvXiemlS9f3ok1bdo05nZnOp+CKmvXro25H6uokjVf6c/SuqZYfIqgWMdu9dOx0qVLO32sYj96jFmFZqzX09fWUzl/8Zc7AAAAAIgAHu4AAAAAIAJ4uAMAAACACPD+AahPrsaHH37o9LFiXbp0CbV17p6ISGpqqhPTCwn6LoKof+dqLdJr2bJlS6ht5bVYv721foOcl5cX8zgt+jWtxUatBW71uZk5c6bT57vvvnNic+fO9TquM4HPQuC+v//WMZ/v07H2r1nj0tq/xiLmhcfKRbLyc/Vn55sPHG/+pDV/6G19Fytes2aN12vi+KycO/393b59u9NHXw9F3GudXhhcxM6d27lzZ6it89utY/Klr33W64m44846hpo1azqx77//Pq7jOlPonLfOnTt7bac/j7Jly3ptZ33empVvZeW9a9a9mzXXLl68OOa+ziQ++WcWK2dXs/LWrJi+9ui6ECL256vHSrz1KkTcOcx6f1YOp86ht8acT70Pq09h4S93AAAAABABPNwBAAAAQATwcAcAAAAAEcDDHQAAAABEgHdBFSvJPl6zZ88Otdu2beu1XXp6eqhtJaJbi6PWqVMn1M7JyXH6WIUGVq5c6XVcKDriXXA7NzfXiTVs2NCJ6eRe63thxXQxDd/trPejk459F8b0Kd7hsx1i++KLL5yYNZ6Sk5NDbWtxX4tOFrcKFMT7uVlFK6xE92XLlsW1f4RZxWp0kSxrQW+LXoDXKlJhzRfVq1cPta1FiK1CA3o7Efe6rBfRFrHnOl3swOpjLcCN43vppZdC7b/85S9OH6v4xLZt20Jt33tAn3563yJ2gSB9X2Z9/klJSU7smWeeiXkMZxLrWq/nBusa4lNEafLkyU7M+kx0gUJrHvJZ2NzazrdgjB6b1uvt3r3bic2fPz/mcVn70rF4i1LFg7/cAQAAAEAE8HAHAAAAABHAwx0AAAAARAAPdwAAAAAQAd4FVYqCpUuXxrXdkiVLEnwkiBpd2ELELiCgk3mtoj5W0qyO6QIrJ0IXt7CSpdetW+fEypUrF2pbhQ4sPoUOEKYLYoiIvPbaa06sS5cuobY1nqxxqD9zn0R0EXts6vG0evVqp48ugiViv0ecuPPPP9+J6c9AF0o5Fv356u+8iMjBgwed2Ny5c0PtQYMGOX2sQgaffPJJzGOwxpw13+7bty/U9h2HODGZmZlObPHixTG3O3TokNf+a9SoEbNPSkqKEytbtqwT02POKqjSs2dPJ7ZmzZqYx3Amsc6tLjji+z3VHn300biPK0qsAmb6nPqcz0ThL3cAAAAAEAE83AEAAABABPBwBwAAAAARUCLwXOnWWhAQZ6aTXdS6MMeStW+f4x0zZowTK126tBPbtWtXqO2bO6d/e52Xl+f0sY7Tej8+C6lbixfrhZCthbbff/99J1aYErFAelGcm+Idh5YqVao4sXPPPTfUthaNtWzatClmzMrJsuj3WBQWuy+O48lnMV8rH8b63us8Wiv3qE6dOk4sJycn1mGekYryta6wXXTRRaF2kyZNnD5du3Z1YiNHjgy1N27c6PSxrrdWrt5bb70Van/44Yf2wRYDRW1ueuKJJ0JtKz/3gw8+cGL6HsH3mIrC9aEwPfzww06sXr16obaVdx/vmI51PvnLHQAAAABEAA93AAAAABABPNwBAAAAQATwcAcAAAAAEeBdUAUAAAAAUHTxlzsAAAAAiIDIPtxNmTJFWrZsKdnZ2ZKeni5du3Y1S0cnQokSJZwS+YiOtLQ0adSokWRnZ0uTJk3k+eefP27/8ePHS79+/UREZM6cOZKdnV34B4liq7DnqlatWsmcOXNi9ktLS5OFCxcm7HVx6jGWkEjZ2dkF172SJUsWtAcOHOi1/X333Sd33XWX+W/Tp093lk34T1OnTpV58+bFc9g4hfbu3SsVKlSQm266yXub490Xde7cWaZOnXpCx1AY91njx4+XpUuXJnSfp5K7wE4EbNy4UYYPHy5fffWVpKamiojI119/XWTXnMnPzzfXOkLRMWnSJMnOzpY1a9ZIVlaWdOjQQbKysk73YYUwjoqf4jZXoehiLCHRfnxAz8nJkezs7IQ+sPfp00f69Olj/lt+fr5MnTpVsrOzpW3btgl7TSTepEmTpGXLljJlyhR55plnpEKFCqf7kBJi/PjxkpycLOnp6af7UOISyb/cbd68WUqWLBla/LdFixZSokQJSUtLkz/84Q/Srl07qVu3rjz00EMFfTZt2iRXX321tGnTRjIzM+V3v/tdwb/dc8890rp1a8nOzpaOHTvK999/77xuEAQyatQo6dOnj+zfv1+WL18ul112mbRu3VqysrLkueeeK+hbokQJuffee6V169byX//1X4V0JpBoqamp0qhRIxk0aFDo/0g+99xzcv3118fcfsKECZKVlSVZWVly2WWXyYYNG0REpGHDhjJ//vyCfuPHj5f+/fuLyPHHZVpamowaNUratGkjQ4cOTcybxClzvLnqeHNOiRIl5JFHHpE2bdpI3bp15ZVXXin4t7lz50p2drZkZGTIDTfcEFoQ+8knnyzYZ+vWreXzzz8/NW8UhY6xhNNl+fLlcuGFF0qzZs2ca9TGjRuld+/e0qRJE+natavs2LFDRNxfuDRt2lRuuukmyc7OljfeeEOmT58uY8aMkezsbBk7duzpeFvwMG7cOBk1apR07NhRJk2aVBAfP368dO/eXX76059KZmamtGrVSlatWuVsv2fPHunRo4c88MADzr/t3btXhg0bJm3atJGsrCwZPny4/PDDD+Zx5Ofny5AhQyQjI0NatmwZ+h8RY8aMkaZNm0pmZqYMHjxYdu/eLSIieXl5cuONN0pGRoZkZGTI/fffLyIiY8eOlfnz58vIkSMlOztbZsyYcTKn6PQIIujIkSPBgAEDgsqVKwf9+vULRo8eHaxfvz4IgiBITU0Nfv7znwdBEARbt24NkpKSCv6tR48ewZw5c4IgCILDhw8HPXv2DN5+++0gCIJgy5YtBfufOHFi0LNnz4K2iASbNm0KBg4cGNx+++1Bfn5+kJ+fH7Rs2TL47rvvgiAIgn379gWZmZnBF198UbDN/fffX8hnAomQmpoaLFiwIAiCIFi0aFFQsWLFYNCgQcGdd95Z0OfZZ58Nhg4dGgRBELzyyitB3759gyAIgtmzZwfNmjULgiAIFi9eHKSkpBSMt4ceeijo1atXEARB8PDDDwd33HFHwf46duwYTJ8+PQiC44/L1NTU4KabbgqOHj1aGG8dhex4c1WsOefxxx8PgiAIvvvuu6BChQrB4cOHg0OHDgV16tQJZs6cGQRBEHz88ceBiASzZ8929vn5558HjRo1Kmj/5zhH8cNYQmFZvXp1UKlSpWP++4gRI4JHHnmkoL19+/YgCILg3nvvDVJTU4Nt27YFQRAEAwcOLOinr5MlSpQouM4FQRAMHTo0eOqppxL7RpBQ33zzTVC7du0gPz8/mDZtWtCuXbuCf3vllVeCpKSkYNWqVUEQBMGoUaOC4cOHB0Hwf/dFa9euDVq0aBG8+uqrBdt16tQpePfdd4MgCIJhw4YV/NvRo0eDm266KRg9erRzHLNnzw5EJJg1a1YQBEEwadKkoFGjRsHRo0eDGTNmBOnp6cHOnTsL9nnrrbcGQRAEv/rVr4JBgwYFR44cCfLy8oLs7Ozgrbfeco6jOIrkb7jOOussmTx5sixdulT+8Y9/yIcffigPP/xwwV9GBg0aJCIi1apVk3r16snq1aslOTlZPvnkE9m8eXPBfvLy8gr+D+fMmTPl2Weflb1798rRo0cL/u/Tjy677DLp27ev/P73vxcRkW+//Va++eYbueaaawr67N27V7799ltp3bq1iIjceOONhXcSkFADBw6UsmXLSrly5eTll1+WJUuWnHCe5ezZs6VXr15Su3ZtERG5/fbb5YEHHpAjR47IkCFDpHnz5vLEE0/Ihg0bZNmyZXLJJZfIvn37jjsuRUSuv/56fnpVTB1vrvriiy+OO+cMHjxYRETS09OlVKlSsmnTJtmxY4eUKlVKunfvLiIiPXr0kHr16hVss2DBAnn44Ydl+/btUqpUKfn+++/lwIEDUrZs2VP3plEoGEs4XTp27Ci//OUvJS8vTzp16lQwZkREevXqJVWrVhURkXbt2snixYvNfdSrV086dep0So4XiTFu3DgZMmSIlCxZUi699FK55ZZb5LvvvpPGjRuLiBT8Qu7H/3722WcLtt28ebN07NhRxo4dK926dTP3P3XqVPn888/lySefFBGRAwcOSMmSJc2+aWlpBfu5+uqrZfjw4bJu3TqZNWuWDBw4UJKTk0VE5LbbbpOrrrpKRERmzZolTzzxhJx11llSvnx5GTJkiMycOdM7p7Qoi+TD3Y/S09MlPT1dbrnlFunVq5dMnz5dRETKlClT0KdkyZKSn58vwf+uCDFv3rzQv4uIrF27Vn72s5/Jl19+KfXr15dFixZJx44dQ326du0qM2fOlDvvvFOSkpIkCAKpUqXKcX+jHpXfJp8Jfsy5+9HSpUvlyJEjBe2DBw+e8D7/84GsTp060qpVK5k2bZp88803cu2110qpUqUK9muNyx8xjoo/PVe9+eab8vTTTx93zrHmMcuP4+yHH36QAQMGyOzZs6V169ayZ88eqVSpkhw6dIgb8ghhLKGwjRgxQv75z3+KyL9TDa644gpp3769zJw5U5577jl5+umnC37K5ju2uI4VL4cPH5YJEybI2WefLW+++aaIiOzfv1/GjRsnjz/+uIgc/7NPTk6WBg0ayPvvvy9du3Y1/wd1EAQyefJkadiw4QkfX4kSJcx9Hu9/hEfpf5JHMuduw4YN8tlnnxW0d+7cKatXr5b69esfc5sKFSpIly5d5I9//GNBLDc3V9avXy+7d++Ws88+W2rWrClBEIRy5370m9/8RgYMGCDdu3eX7du3S6NGjSQpKSmUv7BixQrn/5iieGrQoIHMnz9fjhw5Ivv375fJkyfH3KZLly7y0UcfSW5uroiIvPDCC9KtW7eC/xN1ww03yMsvvyyvvfZawV91jzcuUfwda66qVKlSzDnHkp6eLvn5+TJ79mwR+ff/mVy5cqWI/Pt/QPzwww/yk5/8REQk9H9RUfwxlnCq/OlPf5KFCxfKwoULJTMzU5YvXy4pKSkyZMgQGT16dEKqXCYlJRXkRqHomT59utSrV082bNggOTk5kpOTI/PmzZMJEybI4cOHY25funRpmTJliuTm5sqwYcPMqr79+vWTxx57rOChcOfOnbJixQpzfzk5OQVz1TvvvCMpKSlSp04d6d69u7z99tuyZ88eERF58cUXpUePHiIi0r17dxk3bpwEQSD79u2TCRMmFPxbcR9/kXy4y8/PlwceeEAaNmwo2dnZ0qFDBxk6dKj07dv3uNu98cYbsmLFCsnIyJDMzEwZMGCAbN++XTIzM+Waa66Rpk2bSuvWrQsuaNpdd90lw4YNk65du8q2bdvk/ffflylTpkhWVlZBsvCBAwcK4y3jFBswYIDUqlVLGjduLJdffrk0b9485jYZGRkyZswY6dWrl2RlZcmnn34qL730UsG/9+3bV7788ktJSUkp+FmDyLHHJYq/Y81Vd955p9eco51zzjkyadIkGTlypGRmZsqbb74pzZo1E5F/X6weeughadOmjbRs2VLOOeecwnxrOMUYSzhd3nnnHcnMzJTmzZvLwIED5YUXXjjpfV533XXy9ttvS/PmzSmoUgSNGzeu4OfcP2rcuLHUrl1b3nvvPa99/PhXvyNHjsjgwYOdv+o+9dRTUrZsWcnOzpasrCzp1q2b5OTkmPtq2rSpjB8/XjIzM+XRRx+ViRMnSokSJeSSSy6RG264Qdq1ayeZmZmyZ88eefTRR0VE5Pe//72cffbZkpmZKRdccIH06dNHrr76ahERGT58uDzyyCPFtqBKieDH3yMCAAAAAIqtSP7lDgAAAADONDzcAQAAAEAE8HAHAAAAABHAwx0AAAAARAAPdwAAAAAQATzcAQAAAEAE8HAHAAAAABHAwx0AAAAAREAp344lSpQozONAMZKIde8Lczz57jsR70NEpFOnTk5s5cqVTmz9+vVx7T8tLc2JtW7dOtT+61//Gte+i4KT/RyYm/Cjoj43oXiJwtxUvnz5UPvBBx90+rRv396Jvfrqq6H2n//858QeWByuuuoqJ3bzzTc7sQ8//DDUfvrppwvrkLwxNyVGo0aNnFivXr2c2I4dO5zYwYMHQ+25c+c6fTZs2HASR3d81ucX77iItR1/uQMAAACACODhDgAAAAAigIc7AAAAAIiAEoHnDz75rS9+VNR/O37WWe7/szh69GjM7erUqePEbrzxRid29913h9pJSUkncHSJceTIkVA7Pz/f6TNq1Cgn9swzz8T1evqc+pxPX1HIa0HRUNTnJhQvxW1ueuGFF5xYx44dQ+2SJUs6fTZv3uzEmjRpEmpv27bN6bNu3TontmzZslB7z549Tp8qVao4MSvv75xzzgm1rWttbm6uE6tQoULM4xw+fLgTW7VqlRNLFOamsHjzz/7+9787MV2DQETk7LPPdmKlS5eOuf+xY8c6sWbNmoXaZcuWdfp8+umnTkzfKx44cMDpY30f9f2dhZw7AAAAADgD8HAHAAAAABHAwx0AAAAARAA5dzhhRe234/Hmg3399deh9vnnn+/0KVOmjBPbv39/qL1v3z6v7Xbu3Blq79q1y+lTs2ZNJ1auXLmYx2D9BlznHYi4a7/MmjXL6TN48GAnpsWb12gpbnktp5r1/nzGvO959Tl/iVoTUsTNrbHWGrLWMtK5PNYxFbW5qbg41WMgXhMmTHBiTz31VKit53URO9fm0KFDMV+vKM9NXbp0cWK//vWvndj27dtD7YoVKzp9rPlcX1OqV6/u9LGuTZs2bQq1v/rqK6dPq1atnJh1zdy9e3eobeUG1qhRw4np61xycrLTZ+/evU6sf//+TixRmJvC4r2H+Oabb5yYlYup8zVFRH744YdQ2xoXVg6c/i4cPnzY6WPl+D377LOh9ogRI2LuW8TOzdPIuQMAAACAMwAPdwAAAAAQATzcAQAAAEAE8HAHAAAAABFAQRWcsNOZGBzvwpeff/65E9NJ3ToRXMROxNevZyXgWsekk8+thGJdKEXEXtBSJ+/6JOBa21WrVs3pM23aNCfWr1+/mPuO97MpykULigKfgio+i54Wts6dOzuxzMxMJ6YLF2VlZTl9rPfco0ePUNsqiBHVogXxfrcSuZ3FZ19WoQGrIEFGRkaoPXnyZKdPw4YNnZier6y5yqe4gqUoz02PPvqoE7MWB9fXsEqVKjl9SpUq5cT0NcUqPmG9P309tIpkWNc+aw7TRU+soi7WNVMXObOKp1hFOB5//PFQ+7PPPnP6xCuqc1O8rDGXn5/vxPR3VxcIErHHgHVfpr8LVjE8XYxHRKRevXqhtjV3WIVR9LUuJycn5jGJJKbYE3+5AwAAAIAI4OEOAAAAACKAhzsAAAAAiAAe7gAAAAAgAtyMRqAI80lK7t+/vxO74IILnNj69etDbStZ2SoGoBPErWOyYjqp26dQxrH66eRzK5nXSmTXCctr1651+ujCFSIil1xySaj94YcfOn0SkTAeZYksOBNvAZUhQ4Y4sXnz5oXaHTp0cPqMGDHCieXm5obaVmGU5cuXO7Gvv/461L7rrrucPgsXLnRiZzJrDPgUV7C2swoNaNY85FN0w9rWKp7SsWNHJzZlypSY2y1dutSJ3XHHHU5Ms/ZV3NWqVcuJ7dmzx4npgg3WufApPmEVebAKUuhrps/1S8QucKKLkFmFM6xiKXrcW+PZ+m7ouS+RBVXOdHocWJ+JpWvXrqF2hQoVnD55eXlOzCqipFn3d9b+9XfBmgsXL14cc1/nnnuu08cq5KfPjXUvFwt/uQMAAACACODhDgAAAAAigIc7AAAAAIgAcu5QZFl5AD65Rjp3Q0Rk27ZtTqxixYqh9q5du5w+Vn6C/r21b16L/h31yeSo6W2t8+KTp2P95nz37t1ObMaMGaF2zZo1nT7Wb8f1ubIWKcXJS09Pd2JWXoC10HirVq1C7cqVKzt9xo8f78T++c9/hto6l05EpGXLlk6sdevWoba1IGyDBg2c2IoVK5zYmSze+cNnDrX6+Oat6fyQ8847z+nzwQcfODGdN2PNob/4xS+c2IYNG0LteHNbizp9/bBy1Ky5W8fKlCnj9Xp6/rDmE4u+pljfb+u645PnafWx9mXlgmpWHlPDhg1jbof46O+gNS4s+nph3WdY927WZ6mPwRon1apVi7mdlec5bdo0J3bxxReH2tY10no/iVisnr/cAQAAAEAE8HAHAAAAABHAwx0AAAAARAAPdwAAAAAQARRUQZHlu1CzTmS1kmutRS5TU1NjbuezELjFd4HORPFdSF2fU6togbUwrU48tgpzvPXWWzFf70wWb1EHvZCviEj79u1DbSsp21rQeNy4cU5s5MiRobZenFxE5KmnnnJiNWrUCLWt9/f99987MV1kRSedi4gcPHjQiVFQJcyaY+JZ7FZEJCUlJdS2iupUrVrVieliPNa+rEIcO3fudGJ6DFeqVMnp89VXXzmxM0XdunVDbevzL1u2rBPTBVWsc299Rvrztq57enFnEbcYhFXAxSoY4VO8zNrOGvM6Zi1+bqldu7ZXP5w4/dn5Xg/1vYa1nTWmZ82a5cTq1asXc1/Vq1d3YgsWLAi1mzdv7vSxCvvo4n5r1qxx+lgScd/EX+4AAAAAIAJ4uAMAAACACODhDgAAAAAigIc7AAAAAIgACqqg2GvXrl3MPuecc44T0wm+vkmsOgnXNzHYSgZPFN+CKj7v2UoM1knxViEFq6BKvEVEosgqXqMT/63zVaFCBSemC45kZGQ4fayiN7fccosT69WrV6j98ccfO30sW7ZsidlHF10REdmxY0eobRUxuPHGG53YZ599FmovWbIk5utHmc94ql+/vtPn6aefdmLJycmh9t69e50+TZs2dWIbNmyI2W/OnDle2+k5+tChQ04fq/BHoljnsyg599xzQ23r/FjFRfScbxV1sN67LkJmXb/Kly/vxHThFeuYfIqniLiFUKx9WedBF+exilJVrFjRiW3fvj3UtoprbN261YkhNj3GfArTibhzmDXmrHtA/VmKuGPYKjpmzVd16tQJtSdOnOj0+e1vf+vEYr2+SOHdI/GXOwAAAACIAB7uAAAAACACeLgDAAAAgAiIXM6dT16T9RvXePNhrN+J+/6WWEvkorSalUdlHWdxzJHSC2xb+XU++XS+C6vqc+mbP6DPrXVM1hiwYj77suhjt/IVrPOnFzYfPHiw0+eee+7xOoYzlfVd9vm+6fEt4o6Jrl27On1ef/11J3brrbfGfL1Esha+TkpKCrXnz5/v9LHGpV4w2dr3mcSad7SVK1c6seuvv96JWfkpiWLlKFkLWy9evDjUfvvtt50+ubm5Tkxfu63vmXV91/N9vNftU6VatWqh9saNG50+1sLvHTp0CLXfeOMNp491XmvWrBlqWwuWW3OTHpfWHGddr3744Qcn5nOttXJ/27ZtG2pbY+K7775zYnpuatSokdOHnLv4+NyjXHTRRU5M5z1+8803Tp8qVao4scqVKzsxvdi5lROu8zVFRBo0aBBqW2OnqOEvdwAAAAAQATzcAQAAAEAE8HAHAAAAABHAwx0AAAAAREDkCqrEWxAk3sUF403Cvu2225zY7373OydmLfAbD5/k++KgWbNmTkwnmlsLU1oJ/DqB2+qjF4sWcYtZWMnaPjFrfPnuy4dViEWPA2vcW4nI+lwV9eIDRVG8c5O1oPQ///nP47aPpWzZsk5Mj3Hf49Rjx9pOF2UQcRcxt97fhx9+6MRq1aoVaqempnodJ8Ks4il6rrAKkMR7DZk9e7YTGzBggBPTxQ46derk9HnsscecmE+hBp8+Rb1Ajy4sUaFCBadPly5dnJi+PrZq1crpY80fWVlZofauXbucPta1SY8la9xYRbusMaevyVbhjLVr1zoxvfj5BRdcEHPfIiLr1q0LtbOzs50+//3f/+3EEJvPdeW6665zYj5zk76miNjFfvR9i1Vo0NpO++tf/+rEnnjiCSd29913h9rWOSishc35yx0AAAAARAAPdwAAAAAQATzcAQAAAEAE8HAHAAAAABEQuYIqFp/E/3gLRPz0pz91Ys2bN3diV111VahtJW1u27bNiU2cODHm6/mwEph/9atfObGHHnoorv2fKqVKuUNWJ9han2/58uWdmE6ytxJbrYRb3c8qXGLty6cQi+++fAoEWPvS49xKTra+C/r16tSpE/P1UXj05+Y7liy6n8/Y8qWLQIiI5OXlhdrW+LbGpS4gQVGf+Pgk8PsWT7HmY/25vPbaa04ffT0UccdhgwYNnD5WQSCfAghNmjRxYs8//3yovX79+pj7OZ3Gjh0bas+cOdPpYxXDGjFiRKh94403On3S09OdmC60pItqidj3FXou8rmGHmv/ujBKxYoVnT6tW7d2YldffXWoPXLkSKePdQ279dZbQ+1Dhw45fRCbNX/7XFcuvvhiJ6bvi63PrVy5ck7MGnfWGNN8CitNmDDBiVnXo2nTpoXaffv2dfokoniKhb/cAQAAAEAE8HAHAAAAABHAwx0AAAAARECxzrnzXfzP5zet1u/7dV5A+/btnT49evRwYitXrnRi+vf81kLbaWlpTuzSSy91YvG45pprnJi1sGdR16JFCyemf1ttfd5W/pH+/bWVu2EtFOvzu23fBcp9+li/X/fp47OddV6svBa90LTOmxKxx9O//vWvmMeAExfvws3WGPcZJ/EutGrlug4dOjTUfv/9950+b775phPTY07n48BPInM8fOY06/O1Fh2uVKlSqL17926nT9euXZ2YvrZOmTIl5jGJuPlpgwYNcvpce+21Xvs6HdasWePErMXhtcWLFzuxDh06ODF9Xn3ngHhz0q38PX2fZOVD+Sxq/fvf/97pg8LjM8dkZWU5sbp16zqxVatWhdrW4vM6P1TEXty+fv36obaVZ+tzbbW+exdeeKETs65jpwp/uQMAAACACODhDgAAAAAigIc7AAAAAIgAHu4AAAAAIAIKvaCKlUyrk7CtRNp4i1ZYkpOTQ+2HH37Y6TNw4EAnphP2N27c6PT54osvnJi1eKIuUrF06VKnj7U444MPPujEtBo1ajgx/X6efPJJp4+1cGnLli1D7a+++irm659K8S4O7rsor8/r6cUqS5cu7fSxknL1gr8ns/C0Zn0XrOPSRQqsghc+C5tb+77rrruc2E9/+lMnFkV6nBTWwqSFQY9VnwIr1nYWvQCtiMiCBQtC7VatWjl9XnzxRSemk+Hnzp0b8/XPdPEWwrG2i3dfFquQgV6gukqVKk4fqziLPoYtW7Y4faz5f86cOaG2dX0vSnwKlVgx/d6tgipWgSx9Xq19W/c6+lrhe52z5h19DFYRJeu+yYfPPOczx8HlU2jJKkZofU/1s4BVPEXfW4nYC97r+xbrO1+9evWYx/WTn/zE6eNzrz5+/Hgndv3118fcLh785Q4AAAAAIoCHOwAAAACIAB7uAAAAACACeLgDAAAAgAhIaEEV3yRszad4iqVbt25O7IorrnBigwYNCrW3b9/u9Pn222+dmE4MTkpKcvpUrVrViR04cMCJ6URgq4jApk2bnJg+9l/+8pder6eTpq0CGGXKlHFie/fudWJFic/xWcna1hjTSbI+xVos8W6XSFYispVkrBPUfYquiLjv59ChQ04fazydKYpTAZVY4i0ikJ2d7cT+53/+x4m99dZbofbll1/u9OnZs6cT04W31q1bd4JHeOZJ5Lj0KZLgq1mzZk5s0aJFoXatWrWcPtdcc40T09fl+++/3+ljFY6aOXNmzOMsSvRnaX1PfT6jffv2eb2evmZa87u+RxJxC5VYY9C6ZlrHrl/TOvZ4i6VZrxelefxUsQrTWGNT32uMGDHC6bNw4UIn1qhRo1DbtwDjnj17nJhmFfyqV6+eE9P3P9a+rcIoa9asCbU7d+7s9LGuf1bhqBPFX+4AAAAAIAJ4uAMAAACACODhDgAAAAAiIKE5d9bvlePN37B+j3vrrbeG2ikpKU4fa3FUnX9mHZO1L833N9o+C7dv3brV6WPl9GnWwr39+/ePud3vfvc7J3b77bc7sbVr14ba1157bcx9n0q/+c1vnJj+zb2VB2DllulFcq3fX/vkjJ4O+nfu1m/OrfGqz4O1CK2V11i2bNlQ28rz7NevnxMrzot7nyn0WPKds0eNGhVqW4tO//nPf3Zi1113Xaht5UDPmDHDiaWmpoba8eZqw+XzPbVyeK2xore15lArZ1fnscQ79/72t791YlZe0F//+te49l+UWe9TXw99Fh4Xca+rVt6ile+mrzHWvq17JGss6euONW6WLVvmxHxY44vr04nzvV7oe9DzzjvP6bNr1y4npu9J09PTnT7WmM7Ly/M6Ls26b9Lj1eqj62qIuPmB1vi95JJLnJhegH3ixIn2wR4Hf7kDAAAAgAjg4Q4AAAAAIoCHOwAAAACIAB7uAAAAACACTqqgSosWLULtiy++2OmjFyAUcRemtBYrrVChghPTyZYbNmxw+lSqVCnm61mLcVqJtDpB0kratJJyrQRTnYxuJWRaRSoOHjwYardp08bpk5ub68T0+bMKzSxfvtyJlStXLtQeNmyY0+d0shaY1EmqVvEUK6YXmNTJ2yLFJ+naOk6r4IQeFz4LnYu4ifrWdjk5OV77QtGi56u0tDSnz3333efE9JiwikRdeeWVTkzPO9ZYsq4J8S5WXJRY31OruIRPMRHrGpLIhcZ99u3z/f7yyy+d2OzZs52YtXC9D120wCoqoud6EbuA1pnA97tl3SdpvkVWNGssWfdXel++hVjq1KkTalv3P0W1WFpR51NcxKIX+d65c6fTR39uIm6Bt0WLFjl9zj//fCdWuXJlJ6bvUaznDJ9CXda8p++dRdyiLp988onT54477oj5evHgL3cAAAAAEAE83AEAAABABPBwBwAAAAARwMMdAAAAAESAd0GVn/3sZ05swIABobZvQQqdsGgl0lqrvet9WcmQVnLnvn37Qm1dmEXETurX+7ISjK33ZxXv0Ene1rmy9q/PzZ49e5w++fn5Tkwnq1p9rGOoWLGiEztdateu7cSspFWdGG/1sZJk9edrJeJb48knodiK6TFmJYdbrIRxHbP2pQvNiLgFh6zkd13ER0QkKSkp1LbG03nnnefEihs9BqxzXxTo47TmIV1oQsSeV9PT00PtMWPGOH2s4kv687777rudPj4FN7Kzs52YVTjp888/j7mvwqLPr/W+fIovWdsV1TGm+RZOmDx5cqi9ePFip88NN9wQcz/WnOYzr1rz/4IFC2K+XhT4fN/atWvnxKzrgE+hGusao+8rrD6+BVX0fGUdg3UfU6NGjVDbKqjiW5zlTOFbPM5nHujdu7cT08VSrPtw67PU9x7Wff///M//ODHr+peamhpqW+PJOi79nn3HyapVq0Ltm2++2Wu7ROAvdwAAAAAQATzcAQAAAEAE8HAHAAAAABHAwx0AAAAARIB3QZUJEyY4sS+//DLUbt++vdMnIyPDiemkRquIh7W6vE6ctpIarSTZ6tWrH7ctYieJ6mRLK0HTKsTiUyhDr1wv4hZ+EXELgViFLKzj0kUxrD7W6+nk5w8++MDp86tf/cqJFYYOHTp49dPjwHqvVkEVfY6qVKni9LESzXWSsTV2fBLbffqcDOs96wR169it76Me51bRFSs5ubjxSZS2Es+1wv5s9XFa594qnmIVKdKFUP7+9787fdq2bevErrrqqpjH6cM6V77v51TRx+hbfCBeusjNjTfe6PSxCt9s3bo15r59CpVYxb2s7/yDDz7oxHQxiyuuuCLmMVl8C7j4FMZauXJlzP34fK+LOp9z1qBBAydm3VfowjRWwROrWIq+VljXUN/PVo9Daw6wrveNGjUKtb/++munT2HP0cVNIs/HAw884MRycnJCbZ8iXSLuGEhLS3P6XHTRRU5s2bJlTkxfNzt37hyzj4jIgQMHQm3ru2Cx5lEfPgW8YuEvdwAAAAAQATzcAQAAAEAE8HAHAAAAABHgnXNn/R59yZIlofa//vUvr33pRb7r1q3r9LF+F65/a1urVi2nj89C476Lo+rFsa08ue3btzsxaxFEHbP66N/1ivjlmVi/OffJH9DvT8TNwzudv0u3fqtv0b/7tz5f63wkJyfH3M46Bp9FzK196ZiVE+e7sLlPfptPnqHVx8o91K9n5WicKU71d8Inv8t3UdX77rvPieXm5obazZo1c/oMHDjQa//xsI69WrVqTswaq4XByqfQn4H1nbe+EzonbdiwYU6fTZs2xTwm6xrZt29fJ6ZzjSzWsev3Z+XXWfkwV199tRO79NJLYx6DtVixvv75Xqd1fr7V57//+79jHlNxy7mLd5F3nRMpYn/ePnmmFn1/Z31vfesl6Ouv7zU63u8BwnzGWKtWrZw+1jVE32+2bt3a6bNz504ntnr16lB7xYoVTh+rTkCLFi2c2N69e0Ptzz77zOlj5ZfrMa33I2KPp927dzsxH4m4x+AvdwAAAAAQATzcAQAAAEAE8HAHAAAAABHAwx0AAAAARIB3QRWrAEj58uVD7Zo1azp9fJJwd+zY4cTmzJnjxHSxFN+CG7oYhJWsaCWO6tezilj4Lmyu91WhQgWnj7W4elJSUqhtJftb50Efg16QVMROCtX7WrNmjdPnVPnHP/7h1c9nUXGfBG6rIIK1SKvPeLLGgN7OOibr+2L10/vyXUBcv2frOK2YPjdRXQDWZ/FQXYhHRCQlJSXUtuZCa07zEe+5vv/++52YNcazsrJC7f79+8f1eta4sehjsLazCqqcKr7XFR86qV+PExG/69GWLVucPtb1onfv3qH2e++953WcPmPszTffdGIfffSRE/NZMNwqHhYvfU51UTARkblz5ybs9YoK3wIn+h7CKgJnjSV9f2AVrfApOGaxrlfW+4n3Wlu/fv2Yx+BTWCgq1zmfAm/W+fApOjN69GgnZt036XNpFfGpU6eOE9OFFK19f//9907s22+/dWJ6rkhNTXX66EKRIm6BHuu8WN8Fq0DMqcJf7gAAAAAgAni4AwAAAIAI4OEOAAAAACKAhzsAAAAAiADvgioWnbhsJTL7KFu2rBOzCofowhJWURK9kvyx9qVZCb4+BTd896VZxUxyc3OdmE7wtZKHrffnU7TAej/79++PeUynymWXXebV74cffjhuW8ROGN+8eXPM7axEZH0ureRa69zqz9JKIPdNGNfHZW1njQudxGyNVZ+xYhV5iQKfBPomTZo4sfPOOy/U3rNnj9PHKmqkv2/xql27thNr3769E9OFnUREOnTokJBjsM6dT0K+td1PfvKThBxTPDp27OjE9PG88847Th+rQECtWrVivt7u3budmC4yZhUgsa63Tz/9dKjtW1BFmzZtmhPLyMhwYv369Ytr/4mkCxzFW6zFt0BJUeF7vHpusgqjWN9BfS9lFY+zttP9rHsy67ti7UvfG1r3Tda1Vhe38C1Ep6+rxeE6p8eB9V6texufudnyy1/+MtS+4IILnD5WMTx9PbI+N2su1Pco1vuzCpjVqFHDiWk333yzE2vbtq0Ty87ODrWtcWjdN23dujXmMRQW/nIHAAAAABHAwx0AAAAARAAPdwAAAAAQASeVc5co1m/kfX43fzoXCETh69Wrl1c//dt5a5FLK8/gtttuC7Vff/11p4+VZ6B/b239dt36jbv+/b6V72blHfjkMll5DVZ+VaVKlUJt67fx1sKeu3btcmI+9KKhOs8xEXxzF+PZl7Wforgg8l/+8hcn1rBhQyfmm8caDys/xScvyNouPT09IccUj3r16jmxF198MdR+8MEHnT55eXlOTOfcWX2s3B+dJ2Ut7mudNz2nWAsMjx071ok99thjoXaXLl2cPjNnznRi1oLYp5rOt7HyXX1EZcFqTX+X9KLmIva9VOXKlUNt65pm5RnpmFVTwcq5s/av8ymtPta+9LVPX/dERLZt2+bEilvepYg7bq1zZNFzhZ5zRER+/vOfO7Ff/OIXobZ1PTz33HOdmO7XokULp49VR8Pn/fjmD/bp0yfUtnKSL7nkkrhezxo7Vg6hz3aJmIv4yx0AAAAARAAPdwAAAAAQATzcAQAAAEAE8HAHAAAAABFQJAqqABafYiYiIuXLlw+1fZNr33333VD72WefdfoMGjTIieniLFWrVnX6WIu/W0VPNOvYreRanWRcrVo1p49VcOFf//pXqP3MM884fTp16hTzuOJNYH7ppZe8tjsRiSyE4LMvKwF6xowZoba1qPijjz7qxCZOnHgCR/d//vCHP4TaVvEh67NdsmRJXK9XmKyiDLqYw6k0fvx4JzZs2LBQu2nTpk4f65j1d3DTpk1OHz1/ibiFJKziD1bBJE0vOHysmF5s1ypodu+998Z8PRF3Ieh4F0v2pc9VvMWfCvs4T5cqVaqE2ta4sYr66CIkVvEc67ur51A9HkTshaitYkP6GKzrv1WYTMesAh/WdyoKrrzySif2yiuvODF9jqzCN9b1UBcJycjIcPp89dVXTiwzMzPUXrFiRcw+1nFaY9X6fPv37+/ErAIqmjWmfVjnyroP1Kzvh3XvdqL4yx0AAAAARAAPdwAAAAAQATzcAQAAAEAE8HAHAAAAABFAQRUUWVaCqi5mIhJ/Ar3261//2ivmw0pa18duFeaw3rNPQZU9e/ac6CGeEH2sVtKxVYShd+/eoXZhFFTp3LmzE/M5Pzt37nRi+/btC7UPHTrk9Dl48GDMWP369Z0+d999txP75JNPQu0tW7Y4fXr06OHERowYEWr/4x//cPrEO3YTyadAjZVQbp3j0yknJyfUbtu2rdNn3bp1TkwXjkhJSXH6WPOAHodWMSbr3Op97dixw+ljjWlt8+bNTsy3GE+iChxZ79maY3TRDevYLXqOLmpjLhZr3Fjq1q0bauu58Vj70oV+Vq1a5fTxKRKWlJTkxKy51zoufc20in5Y41m/nwoVKsQ8Tmu74qBmzZqh9pgxY5w++fn5TkwXp9GFUo5FFzixxkC7du2c2Lx580LtevXqxTwmEZEaNWqE2tZnOWXKFCc2depUJ+bDGoeadT6tec/n3rSwxhx/uQMAAACACODhDgAAAAAigIc7AAAAAIgAcu5QZN18881O7IorrnBi5cqVC7ULa1HIE+GTl1VUrV692olVr1491LZ+S27lGX722WcJO65jSUtLixnTxy9i54LoBVKtnCVrsWOdb/XGG284fRYtWuTEunXrFmq3b9/e6ZOVleXE9Hm18vms3AErP8InB6sw7d+/34n97W9/Ow1Hcmx6AfpBgwY5ferUqePEdD6FtVCzlWeiPztrzFkLQeuYNRdaiz7rPJbBgwc7fSzW/hO1GLhvLoqed6y8VYt17FGkr33W993KZdNj0Fo8+pxzznFiOldPL6IuYl9jrH1pvtd267vhoziOiT59+oTa1vnetGmTE9P3Tda8YF3TdT/r/Fvf3VatWoXaGzZscPp8+eWXTqxly5ahtnW9t+4LLfr6Z30XdL6zz36OxTrvp0rxG8kAAAAAAAcPdwAAAAAQATzcAQAAAEAE8HAHAAAAABFAQRUUWVbRjtTUVCemi0voRW1FRCZOnJiw49KsJGyfmO9ivz79rCIGVkwnOlv7/vjjj52YLm5jLSb/wQcfOLHHHnvMPdgEGz9+fML2VbVq1VDbKpJhJazrflZCuTV2dQEV67zOmDHDib355puhtrWAtuV0F0+xWIWGRo4c6cQefPDBU3E4Jr2At/X59urVy4k98MADoXbr1q2dPlZhn1Pt008/DbVnz559mo7k//gWZtHfodzcXK/tErXYelGnC6P4FiDRhWmsz8OaT/S+rNezClXpAh8ibgGieAv4+BYzS1QxoFPptddeC7Wvuuoqp0/jxo2dmL7WWN8HnwI21jmzimTpQiz169d3+liFz5KTk0PtLl26OH18WYuPa1bhIJ8+pUq5j1M+xVmsQjY+xxkLf7kDAAAAgAjg4Q4AAAAAIoCHOwAAAACIAB7uAAAAACACKKiCYmXt2rVOrHTp0qG2VZTCKoyhlS9f3on5JMT6FjMpCnTyrpW4u3DhQiemE4grVKjg9Hn++edP7uCKgO3btx+3jcTLyclxYsVxLH300UdeMa1hw4ZOrGXLlqF2VlaW06d27dpOrHLlyjFfb8OGDU7s1ltvjbmdVUSmMOc53+I/o0ePDrW///57r+10oZGo0uNLF6gQsQtE6H7W2DrnnHOcWLVq1UJtq2DQ+eef78Rq1KjhxJo3bx5qz5071+ljXe/1WI3yZ33gwIFQu3v37k4f6/5n6NChofbll1/u9GnRooUTsz7zRClTpowTu+yyy0LtOXPmFNrri4gsX748Zh/rO7Ry5Uon9s0338Tcl1VwKBH4yx0AAAAARAAPdwAAAAAQATzcAQAAAEAElAg8V/K0fm+PM1MiFn+NdzxZ2w0ZMiTUthZI3bhxoxObP39+qG0tQpmIxSSLEp1zZ/3eu3///k5s7NixobaVw6B/wy8i8re//S3mMZ3seGJuwo9O59yE6CnKc5O1+LE1n999992hts6JE3EXLBdxF/7etm2b08e6Pupc0Jo1azp9vv76aydm5XKlpaWF2tbnYS2YnZ2dHWrfc889Th/rPsHn+hivqMxNOoezXr16Th8rP1OfbytHbcWKFSd5dMcX7+fbuXPnUNv6vljjadOmTf4Hd4JijSf+cgcAAAAAEcDDHQAAAABEAA93AAAAABABPNwBAAAAQAR4F1QBAAAAABRd/OUOAAAAACKAhzsAAAAAiAAe7gAAAAAgAni4AwAAAIAI4OEOAAAAACKAhzsAAAAAiAAe7gAAAAAgAni4AwAAAIAI4OEOAAAAACLg/wNG7i1oKUqwogAAAABJRU5ErkJggg==",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# viz: a few examples with labels\n",
"imgs = [train_full[i][0].squeeze().numpy() for i in range(12)]\n",
"labels = [CLASSES[train_full[i][1]] for i in range(12)]\n",
"show_grid(imgs, labels, ncols=6)"
]
},
{
"cell_type": "markdown",
"id": "8159c5cd",
"metadata": {},
"source": [
"> **Interpretation.** These are low-resolution clothing silhouettes. A shirt and a coat differ by a few pixels at the collar, which is why FashionMNIST is a more honest benchmark than digit MNIST: a pixel-similarity baseline does poorly, and the model has to find shape.\n"
]
},
{
"cell_type": "markdown",
"id": "16f4fc64",
"metadata": {},
"source": [
"### 4.1 A LeNet-shaped CNN\n",
"\n",
"Two conv-pool blocks then a small fully-connected head, the LeNet template adapted to 28x28 input. We use `nn.Sequential` so `layer_summary` can walk it and print the shape and parameter count after each layer, which is how you catch a flatten-dimension bug before training.\n"
]
},
{
"cell_type": "code",
"execution_count": 23,
"id": "f7198d84",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:53.972446Z",
"iopub.status.busy": "2026-06-10T19:24:53.972331Z",
"iopub.status.idle": "2026-06-10T19:24:53.983791Z",
"shell.execute_reply": "2026-06-10T19:24:53.983559Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"layer summary (dummy forward on one 1x28x28 image):\n",
"0 (1, 8, 28, 28) params=80\n",
"1 (1, 8, 28, 28) params=0\n",
"2 (1, 8, 14, 14) params=0\n",
"3 (1, 16, 14, 14) params=1,168\n",
"4 (1, 16, 14, 14) params=0\n",
"5 (1, 16, 7, 7) params=0\n",
"6 (1, 784) params=0\n",
"7 (1, 64) params=50,240\n",
"8 (1, 64) params=0\n",
"9 (1, 10) params=650\n",
"\n",
"total parameters: 52,138\n"
]
}
],
"source": [
"def make_cnn():\n",
" torch.manual_seed(SEED) # deterministic init so the layer summary and counts reproduce\n",
" return nn.Sequential(\n",
" nn.Conv2d(1, 8, kernel_size=3, padding=1), nn.ReLU(), # (1,28,28) -> (8,28,28)\n",
" nn.MaxPool2d(2), # -> (8,14,14)\n",
" nn.Conv2d(8, 16, kernel_size=3, padding=1), nn.ReLU(), # -> (16,14,14)\n",
" nn.MaxPool2d(2), # -> (16,7,7)\n",
" nn.Flatten(), # -> (16*7*7,) = (784,)\n",
" nn.Linear(16 * 7 * 7, 64), nn.ReLU(), # -> (64,)\n",
" nn.Linear(64, 10), # -> (10,) class logits\n",
" )\n",
"\n",
"cnn = make_cnn()\n",
"print(\"layer summary (dummy forward on one 1x28x28 image):\")\n",
"final_shape = layer_summary(cnn, (1, 28, 28))\n",
"assert tuple(final_shape) == (1, 10), \"the head must produce 10 logits; check the flatten dim 16*7*7\"\n",
"print(\"\\ntotal parameters:\", f\"{n_params(cnn):,}\")"
]
},
{
"cell_type": "markdown",
"id": "b3b555e0",
"metadata": {},
"source": [
"> **Predict:** before the next cell, where do you think most of the parameters live: the conv layers or the linear head? AnswerThe linear head. `Linear(784, 64)` alone is `784*64 + 64 = 50,240` parameters, far more than both convs combined. This is the historical pattern: classic CNNs put most of their weights in the FC head, which is exactly what global average pooling later removed.\n"
]
},
{
"cell_type": "markdown",
"id": "37e25fb4",
"metadata": {},
"source": [
"### Exercise 12.6 — Count the parameters by hand\n",
"`Difficulty 3/5 · ~15 min`\n",
"\n",
"Compute the parameter count of `make_cnn()` from the layer specs, without calling `n_params`. A conv has `C_out*C_in*kH*kW + C_out` (bias); a linear has `in*out + out`; ReLU, pool, and flatten have zero. Return the total as an int. The check compares your hand count to the library's `n_params(cnn)`, so a mismatch points at exactly which formula you got wrong.\n"
]
},
{
"cell_type": "code",
"execution_count": 24,
"id": "da6170a9",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:53.990084Z",
"iopub.status.busy": "2026-06-10T19:24:53.989940Z",
"iopub.status.idle": "2026-06-10T19:24:53.996409Z",
"shell.execute_reply": "2026-06-10T19:24:53.996174Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] 12.6 hand param count: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def hand_param_count():\n",
" \"\"\"Sum the parameters of make_cnn() from the layer arithmetic. Return an int.\"\"\"\n",
" total = 0\n",
" # TODO 1: conv1 = Conv2d(1, 8, 3) -> 8*1*3*3 + 8\n",
" # TODO 2: conv2 = Conv2d(8, 16, 3) -> 16*8*3*3 + 16\n",
" # TODO 3: fc1 = Linear(16*7*7, 64) -> (16*7*7)*64 + 64\n",
" # TODO 4: fc2 = Linear(64, 10) -> 64*10 + 10\n",
" # TODO 5: add them up; ReLU/MaxPool/Flatten contribute 0\n",
" total = None\n",
" attempted(total)\n",
" return int(total)\n",
"\n",
"def _check_params():\n",
" got, ref = hand_param_count(), n_params(make_cnn())\n",
" assert got == ref, (f\"hand count {got} != library {ref}; \"\n",
" f\"diff {ref - got} hints at which layer's formula is off\")\n",
"\n",
"check(\"12.6 hand param count\", _check_params)"
]
},
{
"cell_type": "markdown",
"id": "d281c031",
"metadata": {},
"source": [
"Hint 1 (conceptual)Four layers carry weights: two convs and two linears. A conv with bias is `C_out*C_in*kH*kW + C_out`. A linear is `in_features*out_features + out_features`. Everything else is zero.\n",
"\n",
"Hint 2 (pseudocode)\n",
"\n",
"```python\n",
"conv1 = 8 * 1 * 3 * 3 + 8\n",
"conv2 = 16 * 8 * 3 * 3 + 16\n",
"fc1 = (16 * 7 * 7) * 64 + 64\n",
"fc2 = 64 * 10 + 10\n",
"total = conv1 + conv2 + fc1 + fc2\n",
"```\n",
"\n",
"\n",
"Help — off by a few thousandThe flatten dimension is `16*7*7 = 784`, not `16*14*14`. Two 2x2 pools take 28 -> 14 -> 7. If your `fc1` is ~4x too big, you used the wrong spatial size.\n"
]
},
{
"cell_type": "code",
"execution_count": 25,
"id": "8aa17908",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:54.001025Z",
"iopub.status.busy": "2026-06-10T19:24:54.000941Z",
"iopub.status.idle": "2026-06-10T19:24:54.021470Z",
"shell.execute_reply": "2026-06-10T19:24:54.021180Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] 12.6 hand param count\n",
"hand count 52,138 == library 52,138\n"
]
}
],
"source": [
"# solution\n",
"def hand_param_count():\n",
" conv1 = 8 * 1 * 3 * 3 + 8\n",
" conv2 = 16 * 8 * 3 * 3 + 16\n",
" fc1 = (16 * 7 * 7) * 64 + 64\n",
" fc2 = 64 * 10 + 10\n",
" return conv1 + conv2 + fc1 + fc2\n",
"\n",
"check(\"12.6 hand param count\", _check_params, required=True)\n",
"print(f\"hand count {hand_param_count():,} == library {n_params(make_cnn()):,}\")"
]
},
{
"cell_type": "markdown",
"id": "fa253e68",
"metadata": {},
"source": [
"### 4.2 FLOP accounting\n",
"\n",
"A conv layer's multiply-accumulates (MACs) are `C_out * H_out * W_out * (C_in * kH * kW)`: one output pixel costs `C_in*kH*kW` MACs, and there are `C_out*H_out*W_out` of them. People quote FLOPs as `2x` MACs (a multiply and an add). The headline fact: conv FLOPs scale with spatial size, while parameters do not. A conv on 28x28 and the same conv on 224x224 have identical parameters but ~64x the FLOPs.\n"
]
},
{
"cell_type": "code",
"execution_count": 26,
"id": "51ffaee5",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:54.022808Z",
"iopub.status.busy": "2026-06-10T19:24:54.022726Z",
"iopub.status.idle": "2026-06-10T19:24:54.036584Z",
"shell.execute_reply": "2026-06-10T19:24:54.036201Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"conv1 MACs on 28x28: 56,448 (112,896 FLOPs)\n",
"same conv on 224x224: 3,612,672 MACs -> 64x more compute, 0x more parameters\n"
]
}
],
"source": [
"# FLOP (MAC) count for the first conv layer of our CNN, by the formula\n",
"C_in, C_out, kH, kW = 1, 8, 3, 3\n",
"H_out, W_out = 28, 28 # same padding keeps 28x28\n",
"macs_conv1 = C_out * H_out * W_out * (C_in * kH * kW)\n",
"print(f\"conv1 MACs on 28x28: {macs_conv1:,} ({2*macs_conv1:,} FLOPs)\")\n",
"# the same kernel on 224x224 input: parameters identical, MACs scale with H*W\n",
"macs_big = C_out * 224 * 224 * (C_in * kH * kW)\n",
"print(f\"same conv on 224x224: {macs_big:,} MACs -> {macs_big / macs_conv1:.0f}x more compute, 0x more parameters\")"
]
},
{
"cell_type": "markdown",
"id": "cb5837cf",
"metadata": {},
"source": [
"> **Interpretation.** This is the whole budgeting story of CNNs. You add capacity (parameters) by adding channels; you add compute (FLOPs) by processing larger feature maps. Downsampling early (strided convs, pooling) is how architectures keep FLOPs in check while keeping resolution where it matters.\n"
]
},
{
"cell_type": "markdown",
"id": "7bfaf720",
"metadata": {},
"source": [
"### 4.3 Train it\n",
"\n",
"We train on a subset for speed (`N_TRAIN` images), using the canonical four-comment loop. The four comments (`# forward / # loss / # backward / # update`) are identical in every training loop in this curriculum; learn the shape once.\n",
"\n",
"> **Runtime:** this cell takes ~1-3 min on CPU at full fidelity (`TRAIN_BATCHES` mini-batches). Under `NB_FAST=1` it runs far fewer.\n"
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "02443856",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:24:54.038137Z",
"iopub.status.busy": "2026-06-10T19:24:54.038053Z",
"iopub.status.idle": "2026-06-10T19:25:15.859564Z",
"shell.execute_reply": "2026-06-10T19:25:15.859183Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"trained 300 batches · final loss 0.361 · test accuracy 82.7%\n",
"[ ok ] CNN learned something well above the 10% chance baseline\n"
]
}
],
"source": [
"from torch.utils.data import DataLoader, Subset\n",
"\n",
"def get_loaders():\n",
" torch.manual_seed(SEED)\n",
" g = torch.Generator().manual_seed(SEED) # seeded shuffle so batches are reproducible\n",
" tr = Subset(train_full, range(N_TRAIN))\n",
" te = Subset(test_full, range(N_TEST))\n",
" return (DataLoader(tr, batch_size=BATCH, shuffle=True, generator=g),\n",
" DataLoader(te, batch_size=256, shuffle=False))\n",
"\n",
"@torch.no_grad()\n",
"def accuracy(model, loader):\n",
" model.eval() # eval mode: BN/Dropout use inference behavior\n",
" correct = total = 0\n",
" for xb, yb in loader:\n",
" pred = model(xb).argmax(1)\n",
" correct += int((pred == yb).sum()); total += len(yb)\n",
" return correct / total\n",
"\n",
"def train_cnn(model, n_batches, lr=0.1):\n",
" opt = torch.optim.SGD(model.parameters(), lr=lr, momentum=0.9)\n",
" model.train()\n",
" losses, seen = [], 0\n",
" loader = get_loaders()[0]\n",
" while seen < n_batches:\n",
" for xb, yb in loader:\n",
" logits = model(xb) # forward\n",
" loss = F.cross_entropy(logits, yb) # loss\n",
" opt.zero_grad() # backward\n",
" loss.backward()\n",
" opt.step() # update\n",
" losses.append(float(loss.item())); seen += 1\n",
" if seen >= n_batches:\n",
" break\n",
" return losses\n",
"\n",
"cnn = make_cnn()\n",
"train_loader, test_loader = get_loaders()\n",
"losses = train_cnn(cnn, TRAIN_BATCHES)\n",
"acc = accuracy(cnn, test_loader)\n",
"print(f\"trained {TRAIN_BATCHES} batches · final loss {losses[-1]:.3f} · test accuracy {acc:.1%}\")\n",
"assert acc > 0.5, \"a working CNN should clear 50% on FashionMNIST quickly (chance is 10%)\"\n",
"print(\"[ ok ] CNN learned something well above the 10% chance baseline\")"
]
},
{
"cell_type": "code",
"execution_count": 28,
"id": "05ab028a",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:25:15.861773Z",
"iopub.status.busy": "2026-06-10T19:25:15.861656Z",
"iopub.status.idle": "2026-06-10T19:25:15.976603Z",
"shell.execute_reply": "2026-06-10T19:25:15.974550Z"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAdwAAAEiCAYAAABTO2OcAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAXdBJREFUeJzt3XdYU9cbB/Bvwt5bBEFEcaHioGpxgVvrqLbWqq11Veuodbdaf3V1OOpoba0dWu2w1lFHtdWKe29xbxFRloONzJzfH4eb3JsBCYQg4f08D0+Sm5t7zw2QN++ZMsYYAyGEEELKlLy8C0AIIYRUBhRwCSGEEBOggEsIIYSYAAVcQgghxAQo4BJCCCEmQAGXEEIIMQEKuIQQQogJUMAlhBBCTIACLiGEEGICFHCJ0QwdOhSOjo567SuTyTBnzpyyLRB54WzcuBHu7u7IyMgo76LodP/+fchkMixevLi8i6JUo0YN9OzZ06Tn3L17NxwdHfH48WOTntecUcA1c2vXroVMJtP6M3369PIuXpmKiIiATCZD7dq1tT4fGRmpfC82b96s3C68Z7a2tnj06JHW4zZs2FCyTdsHYkZGBmbPno2GDRvCwcEBHh4eaNKkCSZMmIC4uDjlB7s+P/fv3y/yWrOzs7Fs2TK0bNkSLi4usLW1RZ06dfD+++/j1q1byv3mzJkDmUwGb29vZGVlaRxH23UIZViyZInG/sJ7dfbs2SLLBwAFBQWYPXs2xo8fL/li9sUXX+Dll1+Gl5cXbG1tUbt2bUycOFHrB71CocCiRYsQGBgIW1tbhISEYP369VrPd/36dXTr1g2Ojo5wd3fH4MGDyy14HD9+HHPmzEFKSkq5nL8kunXrhqCgIMyfP7+8i2I2LMu7AMQ05s2bh8DAQMk29aBhSs+fP4elZdn/+dna2uLOnTs4ffo0WrRoIXlu3bp1sLW1RXZ2ttbX5uTkYMGCBfjmm28MPm9eXh7atWuHGzduYMiQIRg/fjwyMjJw9epV/PHHH+jbty+aN2+O3377TfK6JUuW4OHDh1i2bJlku5eXl85zPXnyBN26dcO5c+fQs2dPDBo0CI6Ojrh58yb+/PNP/Pjjj8jNzZW8JikpCStXrsSUKVP0vqYvv/wSY8aMgb29vd6vEduxYwdu3ryJUaNGSbafO3cOTZo0wYABA+Dk5ITr16/jp59+wj///IOoqCg4ODgo9505cyYWLFiAkSNHonnz5ti+fTsGDRoEmUyGAQMGKPd7+PAh2rVrBxcXF3zxxRfIyMjA4sWLcfnyZZw+fRrW1tYluoaSOn78OObOnYuhQ4fC1dXVpOcujffeew9Tp07F3Llz4eTkVN7FqfgYMWtr1qxhANiZM2fK/FxDhgxhDg4OZX4efYWHh7MGDRqwunXrsokTJ0qee/78OXN2dmavv/46A8A2bdqkfE54z5o0acJsbGzYo0ePtB5XLCAggPXo0UP5eOPGjQwAW7dunUa5nj9/zlJTU7WWuUePHiwgIMCg6+zRoweTy+Vs8+bNGs9lZ2ezKVOmKB/Pnj1beW3e3t4sKyuryOtgjCn3B8CWLFkiec6Qv6/evXuzNm3a6HVNmzdvZgDY+vXrldsePnzIrKys2Lhx45TbFAoFa9u2LfPz82P5+fnK7WPGjGF2dnYsJiZGuS0yMpIBYD/88EOR546OjmYA2JdffqlXWfXx5ZdfMgAsOjq6RK/X9nsxhcTERGZhYcFWr15t8nObI6pSruRiYmIwduxY1K1bF3Z2dvDw8MAbb7yhUYWZl5eHuXPnonbt2rC1tYWHhwfatGmDyMhIjWM+evQIffr0gaOjI7y8vDB16lQUFBRI9tHWhnvhwgV0794dzs7OcHR0RMeOHXHy5EnJPkIV5rFjxzB58mR4eXnBwcEBffv21VldOHDgQGzYsAEKhUK5bceOHcjKykL//v11vjcff/wxCgoKsGDBAp376HL37l0AQOvWrTWes7W1hbOzs8HH1ObUqVP4559/MGLECLz++usaz9vY2Ghti5w1axYSExOxcuVKvc7TunVrdOjQAYsWLcLz588NLmd2djZ2796NTp066bV/jRo1AEBSBbt9+3bk5eVh7Nixym0ymQxjxozBw4cPceLECeX2v/76Cz179kT16tWV2zp16oQ6depg48aNepd72bJlCAgIgJ2dHcLDw3HlyhXJ85cuXcLQoUNRs2ZN2NraomrVqhg+fDiePn2q3GfOnDmYNm0aACAwMFBrM8Hvv/+OFi1awN7eHm5ubmjXrh327NmjUZ6jR4+iRYsWsLW1Rc2aNfHrr79q7JOSkoKJEyfC398fNjY2CAoKwsKFCyV//wDw559/IjQ0FE5OTnB2dkajRo3w9ddfS/apUqUKQkJCsH37dr3fM6IbBdxKIjU1FU+ePJH8AMCZM2dw/PhxDBgwAMuXL8fo0aOxb98+RERESNr45syZg7lz56J9+/b49ttvMXPmTFSvXh3nz5+XnKegoABdu3aFh4cHFi9ejPDwcCxZsgQ//vhjkeW7evUq2rZti4sXL+LDDz/EJ598gujoaERERODUqVMa+48fPx4XL17E7NmzMWbMGOzYsQPvv/++1mMPGjQI8fHxOHjwoHLbH3/8gY4dO6JKlSo6yxQYGIh33nkHP/30E+Li4oosv7qAgAAAwK+//gpWhitg/v333wCAwYMHG/S6tm3bGhxA58yZY1CQFjt37hxyc3PRrFkzrc8zxvDkyRMkJCTgyJEj+OCDD2BhYYGIiAjlPhcuXICDgwPq168vea3QVHDhwgUA/AtfUlISXnrpJY3ztGjRQrlfcX799VcsX74c48aNw4wZM3DlyhV06NABiYmJyn0iIyNx7949DBs2DN988w0GDBiAP//8E6+88ory9/7aa69h4MCBAHgA/+233/Dbb78pmwnmzp2LwYMHw8rKCvPmzcPcuXPh7++P/fv3S8pz584d9OvXD507d8aSJUvg5uaGoUOH4urVq8p9srKyEB4ejt9//x3vvPMOli9fjtatW2PGjBmYPHmypNwDBw6Em5sbFi5ciAULFiAiIgLHjh3TeB9CQ0Nx/Phxvd4zUoxyzrBJGROq/LT9MMY0qhQZY+zEiRMMAPv111+V2xo3blxsldaQIUMYADZv3jzJ9qZNm7LQ0FDJNgBs9uzZysd9+vRh1tbW7O7du8ptcXFxzMnJibVr107jejp16sQUCoVy+6RJk5iFhQVLSUlRbhNX/b700ktsxIgRjDHGkpOTmbW1Nfvll1/YgQMHdFYpnzlzht29e5dZWlqyDz74QOtxBepVfllZWaxu3boMAAsICGBDhw5lq1evZomJiUW+h4ZWKfft25cBYMnJyXrtL1QpP378mB06dIgBYEuXLtV5HYzx35VQjdu+fXtWtWpV5d+NvlXKq1atYgDY5cuXtT4fHx8v+dv08/NjGzZskOzTo0cPVrNmTY3XZmZmMgBs+vTpjDHGzpw5o/H3K5g2bRoDwLKzs3WWVahStrOzYw8fPlRuP3XqFAPAJk2apNym7f9n/fr1DAA7fPiwcpuuKuXbt28zuVzO+vbtywoKCiTPif++AwICNI6ZlJTEbGxsJE0Gn376KXNwcGC3bt2SHGv69OnMwsKCPXjwgDHG2IQJE5izs7OkGl6XL774ggEo9m+XFI8y3EpixYoViIyMlPwAgJ2dnXKfvLw8PH36FEFBQXB1dZVkr66urrh69Spu375d7LlGjx4tedy2bVvcu3dP5/4FBQXYs2cP+vTpg5o1ayq3+/j4YNCgQTh69CjS0tIkrxk1ahRkMpnkHAUFBYiJidF6jkGDBmHLli3Izc3F5s2bYWFhgb59+xZ7LTVr1sTgwYPx448/Ij4+vtj9BXZ2djh16pSyKnHt2rUYMWIEfHx8MH78eOTk5Oh9rKII70tJOrS0a9cO7du3NzjLTUhIwPfff2/QuYQqVjc3N63Pu7u7IzIyEjt27MC8efPg6empMXTo+fPnsLGx0Xitra2t8nnxrT77FqVPnz6oVq2a8nGLFi3QsmVL/Pvvv8pt4v+f7OxsPHnyBC+//DIAaNT+aLNt2zYoFArMmjULcrn041j89w0AwcHBaNu2rfKxl5cX6tatK/nf2rRpE9q2bQs3NzdJbVanTp1QUFCAw4cPA+D/z5mZmVqbhNQJvzOhVoyUHAXcSqJFixbo1KmT5AfgHzyzZs1Stvd4enrCy8sLKSkpSE1NVb5+3rx5SElJQZ06ddCoUSNMmzYNly5d0jiPra2tRo9aNzc3JCcn6yzb48ePkZWVhbp162o8V79+fSgUCsTGxkq2i9vmhHMA0HmeAQMGIDU1Fbt27cK6devQs2dPvYPU//73P+Tn5xvcluvi4oJFixbh/v37uH//PlavXo26devi22+/xaeffmrQsXQR2oLT09NL9HpDA2hJgrQY01G9bm1tjU6dOqFnz5745JNPsGLFCowYMQI7d+5U7mNnZ6f1i4rQy1wIfsKtPvsWRdtwsjp16kjaXp89e4YJEybA29sbdnZ28PLyUo4GEP//6HL37l3I5XIEBwcXu6/63zyg+b91+/Zt7N69G15eXpIf4f89KSkJADB27FjUqVMH3bt3h5+fH4YPH47du3drPa/wO1P/AkAMRwG3khs/fjw+//xz9O/fHxs3bsSePXsQGRkJDw8PSSeLdu3a4e7du/j555/RsGFDrFq1Cs2aNcOqVaskx7OwsDBJuXWdR9cHuo+PDyIiIrBkyRIcPnwYgwYN0vtcNWvWxNtvv21wlisWEBCA4cOH49ixY3B1dcW6detKdBx19erVAwBcvny5RK9v164dIiIiDAqgs2fPRkJCAn744Qe9z+Ph4QFA9xcida1atYKPj4/kffLx8UFCQoLG71j4nfj6+ir3E29X39fd3V1r9lsS/fv3x08//YTRo0djy5Yt2LNnjzJwqXdSKi19/uYVCgU6d+6sUZsl/Agd66pUqYKoqCj8/fff6N27Nw4cOIDu3btjyJAhGscXfmeenp5GvZ7KiAJuJbd582YMGTIES5YsUXbIaNOmjdYB+u7u7hg2bBjWr1+P2NhYhISEGGW2KC8vL9jb2+PmzZsaz924cQNyuRz+/v6lPs+gQYNw5MgRODs745VXXjHotUKWu3DhwlKVwc3NDbVq1Spx4FbXq1cvALyXa0kJWa6+ATQ8PBwRERFYuHCh3kFa+GIQHR2td7mys7MlWWKTJk2QlZWF69evS/YTOtU1adIEAFCtWjV4eXlpnYzj9OnTyv2Ko6355NatW8oe1MnJydi3bx+mT5+OuXPnom/fvujcubOkWUSgKzusVasWFAoFrl27pleZilOrVi1kZGRo1GYJP+Is2draGr169cJ3332Hu3fv4r333sOvv/6KO3fuSI4ZHR2trPkipUMBt5KzsLDQyBi++eYbjWE84mEOAODo6IigoCCjtEVaWFigS5cu2L59u6S6LjExEX/88QfatGljlGE0/fr1w+zZs/Hdd98ZPPFBrVq18Pbbb+OHH35AQkJCsftfvHhRa5tXTEwMrl27prX6vCTCwsLQrVs3rFq1Ctu2bdN4Pjc3F1OnTi3yGOIAqmsSEHVCkC6u97kgNDQU1tbWGkEwMzNT64xXf/31F5KTkyU9jV999VVYWVnhu+++U25jjOH7779HtWrV0KpVK+X2119/HTt37pQ0Rezbtw+3bt3CG2+8oVeZt23bJplp7PTp0zh16hS6d+8OQJVxqv//fPXVVxrHEibvUP8i26dPH8jlcsybN08jI9ZVW1OU/v3748SJE/jvv/80nktJSUF+fj4Azf9nuVyOkJAQAJpV8efOnUNYWJjBZSGaaKapSq5nz5747bff4OLiguDgYJw4cQJ79+5VVgEKgoODERERgdDQULi7u+Ps2bPYvHmzzqE4hvrss88QGRmJNm3aYOzYsbC0tMQPP/yAnJwcLFq0yCjncHFxKVVGPnPmTPz222+4efMmGjRoUOS+kZGRmD17Nnr37o2XX34Zjo6OuHfvHn7++Wfk5OQYdR7pX3/9FV26dMFrr72GXr16oWPHjnBwcMDt27fx559/Ij4+vth5gWfPno327dvrfc7w8HCEh4fj0KFDeu1va2uLLl26YO/evZg3b55y++3bt9GpUye8+eabqFevHuRyOc6ePYvff/8dNWrUwIQJE5T7+vn5YeLEifjyyy+Rl5eH5s2bY9u2bThy5AjWrVsnqXL9+OOPsWnTJrRv3x4TJkxARkYGvvzySzRq1AjDhg3Tq8xBQUFo06YNxowZg5ycHHz11Vfw8PDAhx9+CIC3n7dr1w6LFi1CXl4eqlWrhj179mjN4kNDQwHwv6EBAwbAysoKvXr1QlBQEGbOnIlPP/0Ubdu2xWuvvQYbGxucOXMGvr6+Bk+rOG3aNPz999/o2bMnhg4ditDQUGRmZuLy5cvYvHkz7t+/D09PT7z77rt49uwZOnToAD8/P8TExOCbb75BkyZNJMOukpKScOnSJYwbN86gchAdyq1/NDGJ4oZtJCcns2HDhjFPT0/m6OjIunbtym7cuMECAgLYkCFDlPt99tlnrEWLFszV1ZXZ2dmxevXqsc8//5zl5uYq99E105QwFEUMasOCGGPs/PnzrGvXrszR0ZHZ29uz9u3bs+PHj+t1PcLwngMHDii3aRu+o664YUHqhKFPxQ0LunfvHps1axZ7+eWXWZUqVZilpSXz8vJiPXr0YPv379dZnpLMNMUYH56yePFi1rx5c+bo6Misra1Z7dq12fjx49mdO3eU+4mHBakLDw9nAIocFiQmvHdF/X2JbdmyhclkMuXQFMYYe/z4MRs1ahSrV68ec3BwUJZ74sSJWstYUFDAvvjiCxYQEMCsra1ZgwYN2O+//671fFeuXGFdunRh9vb2zNXVlb311lssISGh2HKKZ5pasmQJ8/f3ZzY2Nqxt27bs4sWLkn0fPnzI+vbty1xdXZmLiwt74403WFxcnNa/708//ZRVq1aNyeVyjSFCP//8M2vatCmzsbFhbm5uLDw8nEVGRiqf1zXTVHh4OAsPD5dsS09PZzNmzGBBQUHM2tqaeXp6slatWrHFixcr/183b97MunTpwqpUqcKsra1Z9erV2Xvvvcfi4+Mlx1q5ciWzt7dnaWlpxb5vpHgyxspwVD4hhBQqKChAcHAw+vfvb7Re2qRsNW3aFBERERpze5OSoYBLCDGZDRs2YMyYMXjw4IHeSzmS8rF7927069cP9+7dK3JGNqI/CriEEEKICVAvZUIIIcQEKOASQgghJkABlxBCCDEBCriEEEKICVToiS8UCgXi4uLg5OREE2sTQggpF4wxpKenw9fXV2PVJ7EKHXDj4uKMMscuIYQQUlqxsbHw8/PT+XyFDrjC8mqxsbFGmWuXEEIIMVRaWhr8/f2LXfKzQgdcoRrZ2dmZAi4hhJByVVzTJnWaIoQQQkyAAi4hhBBiAhRwCSGEEBOggEsIIYSYAAVcQgghxAQo4Jaj59nA8QtATm55l4QQQkhZo4Bbjhb+BHzxPfDDhvIuCSGEkLJGAbccnb3Cb/ccLd9yEEIIKXsUcF8AlhblXQJCCCFljQLuC6CIua4JIYSYCfqofwFQhksIIeaPAu4LwIICLiGEmD0KuC8ACriEEGL+KOC+ACjgEkKI+aOA+wKgNlxCCDF/FHBfAJThEkKI+aOA+wKwpN8CIYSYPfqofwFQhksIIeaPAu4LgNpwCSHE/FHAfQFQhksIIeaPAm45YUx1nwIuIYSYPwq45aRAobpPVcqEEGL+KOCWk7w81X3KcAkhxPxRwC0nueKAS78FQggxe/RRX07y8lX3xe25hBBCzBMF3HIiznAVFHAJIcTsUcAtJ/miDLegoPzKQQghxDQo4JaTHFGGS1XKhBBi/ijglhNxL2WFQvd+hBBCzAMF3HIi7jRVQAGXEELMHgXccpJLVcqEEFKpUMAtJ+IMl6qUCSHE/FHALSd5NCyIEEIqlXINuCtXrkRISAicnZ3h7OyMsLAw7Nq1qzyLZDLiKmUaFkQIIeavXAOun58fFixYgHPnzuHs2bPo0KEDXn31VVy9erU8i2USNNMUIYRULpblefJevXpJHn/++edYuXIlTp48iQYNGpRTqUwjl4YFEUJIpVKuAVesoKAAmzZtQmZmJsLCwrTuk5OTg5ycHOXjtLQ0UxXP6GhYECGEVC7l3mnq8uXLcHR0hI2NDUaPHo2tW7ciODhY677z58+Hi4uL8sff39/EpTUecacpqlEmhBDzZ3DA/eWXX/DPP/8oH3/44YdwdXVFq1atEBMTY3AB6tati6ioKJw6dQpjxozBkCFDcO3aNa37zpgxA6mpqcqf2NhYg8/3osgVDwuiTlOEEGL2DA64X3zxBezs7AAAJ06cwIoVK7Bo0SJ4enpi0qRJBhfA2toaQUFBCA0Nxfz589G4cWN8/fXXWve1sbFR9mgWfioqGhZECCGVi8FtuLGxsQgKCgIAbNu2Da+//jpGjRqF1q1bIyIiotQFUigUknZac0VtuIQQUrkYnOE6Ojri6dOnAIA9e/agc+fOAABbW1s8f/7coGPNmDEDhw8fxv3793H58mXMmDEDBw8exFtvvWVosSocSRsuBVxCCDF7Bme4nTt3xrvvvoumTZvi1q1beOWVVwAAV69eRY0aNQw6VlJSEt555x3Ex8fDxcUFISEh+O+//5RB3JxJ2nCpSpkQQsyewQF3xYoV+N///ofY2Fj89ddf8PDwAACcO3cOAwcONOhYq1evNvT0ZiOPZpoihJBKxeCA6+rqim+//VZj+9y5c41SoMoilzpNEUJIpWJwG+7u3btx9OhR5eMVK1agSZMmGDRoEJKTk41aOHMmmdqR2nAJIcTsGRxwp02bppzh6fLly5gyZQpeeeUVREdHY/LkyUYvoLmiDJcQQioXg6uUo6OjlTNB/fXXX+jZsye++OILnD9/XtmBihQvn4YFEUJIpWJwhmttbY2srCwAwN69e9GlSxcAgLu7e4We29jUcmlYECGEVCoGZ7ht2rTB5MmT0bp1a5w+fRobNmwAANy6dQt+fn5GL6C5yhP1TKYqZUIIMX8GZ7jffvstLC0tsXnzZqxcuRLVqlUDAOzatQvdunUzegHNlXgokEJBa+ISQoi5MzjDrV69Onbu3KmxfdmyZUYpUGWRrzb2ljFAJiufshBCCCl7JVoPt6CgANu2bcP169cBAA0aNEDv3r1hYWFh1MKZM/XJLhTsBVgrkRBCSJkxOODeuXMHr7zyCh49eoS6desC4OvU+vv7459//kGtWrWMXkhzpBFwFQDo+wohhJgtg5OqDz74ALVq1UJsbCzOnz+P8+fP48GDBwgMDMQHH3xQFmU0S1oDLiGEELNlcIZ76NAhnDx5Eu7u7sptHh4eWLBgAVq3bm3Uwpkz9bG3FHAJIcS8GZzh2tjYID09XWN7RkYGrK2tjVKoykA94NLkF4QQYt4MDrg9e/bEqFGjcOrUKTDGwBjDyZMnMXr0aPTu3bssymiWtHWaIoQQYr4MDrjLly9HrVq1EBYWBltbW9ja2qJ169YICgrC119/XRZlNDvaxt3SbFOEEGLeSrQ83/bt23H79m3cuHEDAFC/fn0EBQUZvXDmSn0MLkBVyoQQYu5KNA4XAGrXro3atWsbsyyVhrYOUjTTFCGEmDe9Aq4hy+4tXbq0xIWpLMQZrlzOAzBluIQQYt70CrgXLlzQ62AymptQL+LgamUJ5ORSGy4hhJg7vQLugQMHyroclYrQQ1kuBywKu61RhksIIeaNpu8tB+KAKy+czpGGBRFCiHmjgFsOhIBraQHIC2vhqUqZEELMGwXcciBUH1vIeZYr3kYIIcQ8UcAtB0KGa2GhCrg0LIgQQsybwQE3MzOzLMpRqQjDgizkqkXnKcMlhBDzZnDA9fb2xvDhw3H06NGyKE+loKxStlD1UqY2XEIIMW8GB9zff/8dz549Q4cOHVCnTh0sWLAAcXFxZVE2syWpUqYMlxBCKgWDA26fPn2wbds2PHr0CKNHj8Yff/yBgIAA9OzZE1u2bEF+fn5ZlNOsiDNcGhZECCGVQ4k7TXl5eWHy5Mm4dOkSli5dir1796Jfv37w9fXFrFmzkJWVZcxymhXlsCC5KsOlBegJIcS8lXjxgsTERPzyyy9Yu3YtYmJi0K9fP4wYMQIPHz7EwoULcfLkSezZs8eYZTUb4iplFN6ngEsIIebN4IC7ZcsWrFmzBv/99x+Cg4MxduxYvP3223B1dVXu06pVK9SvX9+Y5SxzBQWFAdAE8sUBtxAFXEIIMW8GB9xhw4ZhwIABOHbsGJo3b651H19fX8ycObPUhTOl5b8BT5KBpvWB89cAL3cg8zkQnwSEtwBeaghU8wasrXhwLE1wVogmvhCyXWrDJYQQ82ZwwI2Pj4e9vX2R+9jZ2WH27NklLpSpPc8Gjp7jq/ZcvKH5/K/b+I9Mxlf3KSgAWjUD3ugG1PQ3/HziDNeiMPhShksIIebN4IBrb2+PgoICbN26FdevXwcA1K9fH3369IGlZYmbhMuVnS3w3RzgvyPAjXtA43rA0xTeoSmgGnDoNHA/DsjIBHLz+GuOnAWOnQe6tgHe7g24OOl/vgLRxBfUaYoQQioHgyPk1atX0atXLyQmJqJu3boAgIULF8LLyws7duxAw4YNjV5IU/D2AN7po/257u341Iup6UB2DpDxHPjrPx50dx0GIo8BgX7AqDeB+rWKP1e+lqkdqUqZEELMm8HDgt599100bNgQDx8+xPnz53H+/HnExsYiJCQEo0aNKosyvhBkMsDVGajqBQRVBz4aCSyYyquU8wuA2zHArOU8Qy6OQsviBZThEkKIeTM44EZFRWH+/Plwc3NTbnNzc8Pnn3+OCxcuGLVwL7qGtYGvZwKrP+fV0M+zgaVriw+ekokvqEqZEEIqBYMDbp06dZCYmKixPSkpCUFBQUYpVEUikwHensD/xgCODkBcInAiqujX5IvWwxV6O1PAJYQQ82ZwwJ0/fz4++OADbN68GQ8fPsTDhw+xefNmTJw4EQsXLkRaWprypzKxswV6hPP7f/1X9HJ7ClHAFVYLojZcQggxbwZ3murZsycAoH///pAVRgtWGF169eqlfCyTyVAgdMetJHq1B7ZEArfuA+eu8rG72ggZrtyC2nAJIaSyMDjgHjhwoCzKYRZcnYGeEcDWSOD3v4HQBqoMVkzSaYracAkhpFIwOOCGh4eXRTnMRr+ufKjQnRg+iUYTLTNc0rAgQgipfEo0U0VKSgpWr16tnPiiQYMGGD58OFxcXIxauIrIxQlo3QzYdwK4fEt7wJVMfEFVyoQQUikY3Gnq7NmzqFWrFpYtW4Znz57h2bNnWLp0KWrVqoXz58+XRRkrHGHyi+t3tT+fr2UBegq4hBBi3gzOcCdNmoTevXvjp59+Uk7lmJ+fj3fffRcTJ07E4cOHjV7Iiia4MODejNa+ClGBqA2XhgURQkjlUKIM96OPPpLMm2xpaYkPP/wQZ8+eNehY8+fPR/PmzeHk5IQqVaqgT58+uHnzpqFFeuH4VQUc7PhiCNGPNJ8v0DIsqIACLiGEmDWDA66zszMePHigsT02NhZOTgbM4A/g0KFDGDduHE6ePInIyEjk5eWhS5cuyMzMNLRYLxS5HKhXk9+/oaVaWTLTVOFvoKhxu4QQQio+g6uU33zzTYwYMQKLFy9Gq1atAADHjh3DtGnTMHDgQIOOtXv3bsnjtWvXokqVKjh37hzatWtnaNFeKPVq8rG4N6OBnu2lzwnVx5bUhksIIZWGwQF38eLFkMlkeOedd5Cfnw8AsLKywpgxY7BgwYJSFSY1NRUA4O7urvX5nJwc5OTkKB+/yLNZBfrx25g4zeeUE1/IeTsuYNxhQSlpQOZzoJq38Y5JCCGkdAyqUi4oKMDJkycxZ84cJCcnIyoqClFRUXj27BmWLVsGGxubEhdEoVBg4sSJaN26tc4l/ubPnw8XFxflj79/CVZ/N5GAavw2NkHVZiuQTO1YBsOCZi4Dxs0D0jKMd0xCCCGlY1DAtbCwQJcuXZCSkgJ7e3s0atQIjRo1gr29fakLMm7cOFy5cgV//vmnzn1mzJiB1NRU5U9sbGypz1tWvD0AG2sgLw+Ifyx9TjK1YxlUKcc/BvLzgeRU4x2TEEJI6Rjcaaphw4a4d0+PRV8N8P7772Pnzp04cOAA/Pz8dO5nY2MDZ2dnyc+LSi4Hqvvw+/fVeiqLJ74w9rAgxoDcPH4/N984xySEEFJ6Bgfczz77DFOnTsXOnTsRHx8vWR3I0DZVxhjef/99bN26Ffv370dgYKChxXmh1SisVn6g1o6bX8SwoNw84MxlPqSoJIRgC/AslxBCyIvB4E5Tr7zyCgCgd+/eytWCgJKtEDRu3Dj88ccf2L59O5ycnJCQkAAAcHFxgZ2dnaFFe+FU9+W3GhluYXAVd5piDMjOBeZ9C1y6CbQIAWaNM/yc4kCdRwGXEEJeGOW6WtDKlSsBABEREZLta9aswdChQ412nvIidJx6EC/drm3iC4UC+HIVD7YAcPoSsHk30LyR6jj6EGe44vuEEELKl8EBNzAwEP7+/pLsFuAZrqGdmJiZz/bg7cFvn6ZIt4snvhAy3F2HgaxswNISaNEIOH4BWLsV+H0H8PMXgLue60KIM1yqUiaEkBeHwW24gYGBePz4scb2Z8+emV0bbGm5Fvbpep4NZKuGD0sz3MLfQFY2v32nDzB5OBDRkj/OzwduRet/TqpSJoSQF5PBAVdoq1WXkZEBW1tboxTKXNjbAlZW/H6KqD+ZuA1XrvZWvtoBsLUGpg4HOvGJvHDPgIoD9YDLGPDN78D6nYaXnxBCiPHoXaU8efJkAIBMJsMnn3wiGXtbUFCAU6dOoUmTJkYvYEUmk/Gq4MQnQHIaUNWLbxdnuBairzz2ttKVhYRezvce6n9OcbttXj4QGw/8d4Q/7t2RL6pACCHE9PQOuBcuXADAM9zLly/D2tpa+Zy1tTUaN26MqVOnGr+EFZybsyrgCgrE6+GKAq6D2vwhtQon0oo2IOBKMtw8ICVd9fhODNC4nv7HIoQQYjx6B1yhd/KwYcPw9ddfv9CTTrxIhHZcccDN1xFwHdUCbo3COUASnwAZWZrPa6Oe4T5JVj2+fZ8CLiGElBeD23DXrFlDwdYAroUrFqaIplkUzzQlbg5Xr+51cgCqFPZ01jfLVW/DfZaienzrvn7HIIQQYnwGDwvKzMzEggULsG/fPiQlJUGhNiehsad9rOiE4TzJOjpNidtwHR00X1/TH0h6Cly5BTSqU/z5stWGBaWJlhamgEsIIeXH4ID77rvv4tChQxg8eDB8fHy09lgmKtqqlLUNCwIAJy1Vxq2aAiejgMjjwJuvSKugtckVBdxctQz3STJf0MBNzzG9hBBCjMfggLtr1y78888/aN26dVmUx+wIwU3bsCALC+mwIHstPYhbNwN+3Miz3PPXgJe0r1yopN5pSn3SjXsPgVAKuIQQYnIGt+G6ubnpXCCeaHIrJsMtrkrZxhro8DK/v/9k8eeTLF5QADxJ4ffdXfltfJI+pSaEEGJsBgfcTz/9FLNmzUJWVlZZlMfsiAOuMJOlMsOVq/VS1jFGtkUjfnv7fvHnE2e4ubmqNXEb1ua3cZqThBFCCDEBg6uUlyxZgrt378Lb2xs1atSAlTCVUqHz588brXDmwKUw4Obl8ekbHex0DwtSH4crqFk4Hjf+MZD5vOjJK8QBN+kZXxRBLgcaBAGHzwBxiSW/FkIIISVncMDt06dPGRTDfNla8xmksrJ5tulgp1ps3sJCOizISUuVMgA4OwKebrzTU/RDVbaqjSTgPuW3rk6AX1V+nzJcQggpHwYH3NmzZ5dFOcyaq0thwE3jgU+c4YrbcIvKXGv684B7L7bogCtuw014wm89XAHfKvx+4mPehiyeQpIQQkjZM7gNFwBSUlKwatUqzJgxA8+ePQPAq5IfPXpUzCsrJ/fCamWhp7J44ouiZpoSE6Z5LG4hA3GGK7QZu7nwoGtlxduPHz/Tu+iEEEKMxOAM99KlS+jUqRNcXFxw//59jBw5Eu7u7tiyZQsePHiAX3/9tSzKWaG5qnWcEqqULdWqlHW14QJAzer89u6Dos+lbdF5ezse2H08gQfxwKMk1UIKhBBCTMPgDHfy5MkYOnQobt++LVmO75VXXsHhw4eNWjhzoQy4qaoeygAPguKMtMgMtzDgPoiTzialLkfLc7aF60z4FFYr09AgQggxPYMD7pkzZ/Dee+9pbK9WrRoSEhKMUihz4yaqUhaqkwGe4WY9Vz22sYZOXm6841SBArhZxOyZ2jJcu8LvRd6e/JaqlAkhxPQMDrg2NjZIS0vT2H7r1i14eVE9pTbi+ZTFAdfCAnieo98xZDKgYeFcyldv695PW4ZrZ8NvhXG+mc819yGEEFK2DA64vXv3xrx585CXx1MpmUyGBw8e4KOPPsLrr79u9AKaA3EbrrhK2UJuWPBrEMRvrxQVcLVkuELNv9BGnEUBlxBCTM7ggLtkyRJkZGSgSpUqeP78OcLDwxEUFAQnJyd8/vnnZVHGCk8ScEUZrlwOhNTl9y31GKYjZLjX7/Gl97QpMsMtDLgZFHAJIcTkDO6l7OLigsjISBw7dgwXL15ERkYGmjVrhk6dOpVF+cyCuA1XCJTCWrgvNwbmjAcC/Yo/jp834OIEpKbz4UF1AzX30RpwhQy3sEqZMlxCCDE9gwOuoHXr1ggICICPjw8saBaFIgkZrkKhWsRAeMtksuJXABLIZDwwR13nvZXVA65CwaeQVCdkuELAzaBpsAkhxORKNPGFIDg4GDExMcYqi9mytODTMwI8UAKAk2PJjiVM0fhQS4fwXB3VzHZqbbiZFHAJIcTkShVwmTCVESmWUK0c/ZDfOuuYN7k4/oUBN1ZLwNVWnQwAtuoZLlUpE0KIyZUq4BL9CdXKMYUZrotTyY6jzHC1rPqTqyPgChmusMB9Xp728bqEEELKTqkC7scff0yL0etJGIt7v3C66dIG3ITHmj2VhQzXQu23qt6GC1DHKUIIMbVSBdwZM2bAyckJUVFRSE5ONlaZzJK7K78VFjAoacB1d+EZq0LB18cVEwKu+pzMQoYrl/OlAgGa/IIQQkzN4IA7ceJErF69GgBQUFCA8PBwNGvWDP7+/jh48KCxy2c2vNykj11K2GlKJuPDgwDNjlN3Chc2EJbiE9iKpoykjlOEEFI+DA64mzdvRuPGjQEAO3bswL1793Djxg1MmjQJM2fONHoBzYWQ4QpKmuECqmrl2Hjp9lMX+a36MCPxqC0Hmt6REELKhcEB98mTJ6halX/i//vvv+jfvz/q1KmD4cOH4/Lly0YvoLnwVM9wSxFwhZWDrt1RbcvJ5eNzAaBFiO7XKjNcCriEEGJSBgdcb29vXLt2DQUFBdi9ezc6d+4MAMjKyqIJMIrg4Sp9XNIqZUA1HeS1u6qpIi/e4D2PvdyLnrVKffILhYIH6ufZJS8PIYSQ4hk809SwYcPQv39/+Pj4QCaTKad0PHXqFOrVq2f0ApoLN2feaUlYfL40GW6NajxTzcwC7sYCdWoAdwrnH2lST7qovToh4G6JBLKygaSnwM4DwOtdgWGvlbxMhBBCimZwhjtnzhysWrUKo0aNwrFjx2Bjw8ecWFhYYPr06UYvoLmQywE3F9Vj51JkuHK5auWgdX8DjxJVk1kI4311EQJuXCLw82YebAHgr/9KXh5CCCHFK9Fcyv369ZM8TklJwZAhQ4xSIHPm6Qo8TebjZB3ti929SI3qAKcvAeeuAlMWAs3q8+3qx1VfhUh9yJDAw037dkIIIcZhcIa7cOFCbNiwQfm4f//+8PDwgJ+fHy5dumTUwpkboR3X2bHoal99tAkFqnjw+xmZqqke1QOqerO6roD7NBnI1jFTFSGEkNIzOOB+//338Pf3BwBERkYiMjISu3btQrdu3TB16lSjF9CcCFlkadpvBV7uwM9fqGawEibB0Mhw1eowhFmnAGD+ZGD156ognKA2kQYhhBDjMbhKOSEhQRlwd+7cif79+6NLly6oUaMGWrZsafQCmhNPV35rjIArcHIAnqUC2Tn8cXFVyuIFDoKDeAbs6wXcjuFtwTWqSfdnjL/G1gaEEEJKweAM183NDbGxsQCA3bt3K3spM8ZQIIxRIVo1qgtYWwFN6hvvmE5qqw4VV6XcJpSPCe7fXfVctcKJNOKSNI//5Srg7Wn8ua9+AU5GGaXYhBBS6Ric4b722msYNGgQateujadPn6J79+4AgAsXLiAoKMjoBTQndWoAG7/WzDpLQ723s5NawLVS+w17ugFrF0i3+Xrx2zgtKxAdPstvl64BbtwD7sUCLzcpcXEJIaTSMjjgLlu2DDVq1EBsbCwWLVoER0f+iR8fH4+xY8cavYDmxpjBFtAMuOoZbi3/4o/hWzg3c5xaG654CT+hKjo1w7DyEUII4QwOuFZWVlo7R02aNMkoBSKGEVcpy2SqcbZfzQT2HAXe6lX8MYTFDtQz3CeiBaDSMvltRmbJy0oIIZVZicbh3r17F1999RWuX+eT9wYHB2PixImoWbOmUQtHiuckynDtbfmkGAAQVB0IGqTfMYSAm5zG18kVFqp//Ey1z9PC4JuTyzNfa6vSlZsQQiobgztN/ffffwgODsbp06cREhKCkJAQnDp1CsHBwYiMjCyLMpIiOIsyXEcH3fsVxdFe1XNaXK38RMcSx2lUrUwIIQYzOMOdPn06Jk2ahAULFmhs/+ijj5SLGRDTEFcpC9XJJeHjBaSm82rloMLViJKead83I4t3viooAH7cyPfv3Lrk5yaEkMrA4Az3+vXrGDFihMb24cOH49q1awYd6/Dhw+jVqxd8fX0hk8mwbds2Q4tT6Yk7TakPETJEtcKOU9EPgfPX+CILujLc9MIM9/It4J+DwKpNfLxucS7fBFaso5WJCCGVk8EB18vLC1FRURrbo6KiUKVKFYOOlZmZicaNG2PFihWGFoMUEgfc0mS4Qjvupt3ArK+BI2elbbhiQgcqYYWizOfA05Tiz7H+H2DXYeDkxZKXkxBCKiqDq5RHjhyJUaNG4d69e2jVqhUA4NixY1i4cCEmT55s0LG6d++uHMdLSkZSpVyKBRGEDFew7yTwREfATS8MuHdjVdti4ng1c1FS0vht0tOSlZEQQioygwPuJ598AicnJyxZsgQzZswAAPj6+mLOnDn44IMPjF5AsZycHOTk5Cgfp6Wllen5KgJHez4ciDHNSS8M4atWOeHsCFy/q33fdLUMFwAexAFVPYE53wB9OwMRLXivZ3EgF16nK3MmhBBzZlDAzc/Pxx9//IFBgwZh0qRJSE9PBwA4ORlxcuAizJ8/H3PnzjXJuSoKuZxnthmZpctwfbykj2Me6W5rzcjkHafiRT2aH8TzqSLjHwN7j/Oq4+iHwHezgeq+/AtBehbfV1fbMCGEmDOD2nAtLS0xevRoZGfzT2InJyeTBVsAmDFjBlJTU5U/wpzOlZ3QjluaNXbtbIG6garH9x8VHlNLR6y0DD7Fo1hsvGqsbmwCD7YAcKpwxcbsHCA/n9+ngEsIqYwM7jTVokULXLhwoSzKUiwbGxs4OztLfohqLG5peikDwOeTgKW8lUDZ6ziwGg/GYmmZwM1ofl+oMn4QBzxJ4ffFmbEwl7OQ3QLAYwq4hJBKyOA23LFjx2LKlCl4+PAhQkND4eAg/ZQPCQkxWuGIfvp3Bw6eBkIblu44tjZATX9VmzAA+FUFEp9Kg2hGFnC6MHPt3g5Y8xeQlQ3cvq95zOTUwteIpoTMzOLHUw/khBBizgwOuAMGDAAASQcpmUwGxhhkMplBS/RlZGTgzp07ysfR0dGIioqCu7s7qlevbmjRKq0WIfzHGCwtADdnvsYuAPj78Gw26akqEMfGqzpAtW4G7D4CPEzQvrzf08LjqM9O9TgZqO5jnDITQkhFYHDAjY6ONtrJz549i/bt2ysfC8OKhgwZgrVr1xrtPMQwHq6qgOtXVVVV7enGexgLwbN2AODlzquVHyZoP9azFH6bkSXdfuQM0DEMqFrYWYsx4M9/AGcnoEe4Ma+GEEJeDAYH3ICAAKOdPCIiAkyfKYqISXm5A7cLh/z4V1V1yvKtIh3S07Ixv/XzBk7pOJbQrpuutsrQ+n94ZjxzDK+ebhoMrNvBe113fJlXbxNCiDkxuNPU/Pnz8fPPP2ts//nnn7Fw4UKjFIqUL4/CCSxsbXi2K/R+9quq2sfNGejSht9XnzRDLFlHlTLAx+lOXQhs3AUs/IlvUyj4ECNCCDE3BgfcH374AfXq1dPY3qBBA3z//fdGKRQpX16FAbeaN884I1oC9WryKuABPYBubYHv5gDuLoX7iQKxbxX+utqFFSHPs/mPUKVsoeMvLkU0h0nMI8PLrFDwpQUJIeRFZXCVckJCAnx8NHu7eHl5IT6eUhNz0LAOYGUFtCzsiNUgCFj8Eb9fp4bm/n6iDNfbE5g7nnewenMi7738JEVVpfxqRz5BRqdWvM32ZrRmZ6uYOH6bnqmaSas4i1cDJy4C388FvD0MuFhCCDERgwOuv78/jh07hsDAQMn2Y8eOwdfX12gFI+WnTg1gwzL9F5l3duQTZGRkAp6uPCsGAHdXICsB+HgJrz4GAJ8qfCgRAEwZzm+/+B44Lhraff8RcCsamLGUZ8yz3y9+nuZLt4C8PD7dJAVcQsiLyOAq5ZEjR2LixIlYs2YNYmJiEBMTg59//hmTJk3CyJEjy6KMpBzoG2wBnoFWK5yL2c1Ftd3Dld8mi6qLtU3O0bUtvw0qrIZ+EMc7UOXk8hmrPl4G5OXrPn9OrqpKOoWm1yaEvKAMznCnTZuGp0+fYuzYscjNzQUA2Nra4qOPPlIuZkAqn/q1ePVwTX/VNm1DssXLCQpCGwCrPwdsbYG3pvAhScKwJACISwRu3AMa1dF+bnHPaSG4/7wZOHqefxmYPkrVplxR5eUDqenFZ/qEkBeXwRmuTCbDwoUL8fjxY5w8eRIXL17Es2fPMGvWrLIoH6kg3ukDLP8fnwhD0P5lzf10zffs7Qm4OPJbQYeXgXbN+f2o67rPnaQWcJ9nA1si+WQdiU+Az1aqekubgq5FH0rjy1XAsBl8nmpCSMVkcMAVODo6onnz5mjYsCFsbGjQZGVnbaWaFlLQtQ2wfikwX7RMcnHzPY8eALRvCQzqCbz7BtC0Pt9eZMAVra+bnCqdq9nfhy+q8PsO1baCAmD/ybJZJvDcVeDNScCWPcY97s37fHKQew+Me1xCiOmUOOASog8nB6BBbaBhbT60SGjX1aV5I96ZalAvXv3cpDDg3o7RnK1KoBFwCwNpgC8P2gBw7opqfugTUcDSNcDK9SW9Kt0u3eBDlM5fM94xFQpVhv7UgEx95wFg34nSnZsx4N9DwK37pTsOIaQEbbiEGEouBxZMLdlrhakjHyUCF2/wKmshcArZtLhK+Vmaavk/L3ce6K2s+Lab0YCrk2qcr7bFFkpLKEvC46L3M0RKOg+6gGqqzGJfkwZ8/ye//3ITwM4G2L6Pt7XXq6n/uc9cBr77g9/f8b1+Q7QIIdpRhkteeC8VroJ0+hIPPP/7Chg6g3ciAqQZbmoab7cFeMC1sQaCa/HHUxcCY+YC1+7yx8lpxu/VLJw76RmQr/86HkUSrx/8TM8MVzyz1+37wKWbwOrNwIp1hp1bGBMNaF+cghCiPwq45IX3cuGczacvAXuP80z3aTKw6zDfLg64+QV8HC+g6tHbWDQxWl4ef73g3kPjljWxsCwKhfHaiJ+mqO7rm+GmigLuzWjgbmHb76NEVQ2BPsTv7QUd1eQKBfDPIX5sQohuFHDJCy84iE+skZ4JLP9Ntf3fQ0B2rirrEybcENobvdz5bWgD3ceOjjWsLHn5wLRFwOzlqmpeQXaOKusGgHhRtXJBgWGBTuypOMNVy8jjkoDpi3lnLbE0tYArfAnJzdM+r7Uu4lWgdLVLH78ArPwDeG8Wfw8IIdpRwCUvPAsLoHlD1WNfb9WavRv/5YHPxlq1iIJQTSwE3FrVgY9H8zmh1Rma4d64B1y/ywOcegASZ4OAKuDefwT0mwCs3WLYuQTqGa44cP+9D7hyG/jrP+lrxEH11n1VwAWAhCeaqzfpIh6GdOE68MMGzSFWQhU9wFeBIoRoRwGXVAh9OwM1qgE9IoBFU4HOrfl2YfhNozqqxRQEVdxV91s15cv+qVPPcGMeAXe0DL2JecQ7IR07r9q25Gdg825V8EpUC7hCx6nIY7wq+6890g5e+hIH3Owc6Tjf84XDpW7ck7YZiwNuShqfsUuwaBXw1lTVUKv8AiBTtPBDXj7/Sc9UfXlxd+XXsGM/bwe/Lgqy4g5if+83Xts14XLzDNs/67n091mWTl8q2d90ZUUBl1QINf2Bb2cBYwYCrs5Aq8IJNoQP92YNpNNKymQ8SIgF+qnuC+27sQl8+Mzs5TwAjZsHTJ4vbQMFgKVr+X47D6i2pWcCa7cCf+zkj4UOUwIhwxV3dCrJ+FxxwAVUQ4MSn/BZuAD+oXxP9OWhqGrjxCe8VmDOt/zx7OU8AG/ezT+sR88GPvhMFaQ93IAf5gIz3uO/h4xMvvCEQNyxKi9Pd0e0ggJg/g/Ar9uKvt68fN4zesnPmtX2lc2PG4EBk/RvH8/LB4bPBMbMMfyLz+VbvHNdUe7Fqr5gHTgFzFvB50on+qGASyqkWv5AFdEiBaENAH/RMoGWloCV2qA3V2fAxYnfDw4C2r7Eq2e//5NXEQsBSKHgiycIGFN1OhKIp4oUOhMJVcq+hVXbwgeTOBDuPiKt3tUl8zkfkqNQaAZcoUpXvUr72h3VfSHgFjUEKD+fX+fFG/z+2q3AR4t5QI6NV/Vo9vMG7Gz5kKzxg/m2m/d52Z5nq75oWBXOv52cpr29+m4sryHYuEt3AFEo+NrI/x7iH+j6vFemkpunfbpSgJf7ZJT+VfX6On+Vn/fqbf32f5TIvxA9S9H8uylKdi7/4jX7G93t8KkZwOQFfFERAPjvCL9NeKJ9/7KW+AT4crX2GqkXFQVcUiHJZEBYE37f25OvKvRqRyC8Bd/WRHPJZgC8WhoAqnoCw1/nbb+CfNECCeKAq565AsCS6cAfS3g5HibwDzfhg0dob46N58FRyHRrB/BzLPqJL7jAGLDmL+Db3zU/yNftAOZ+C3z1i+qD06MwKxceCx2lXJ357bU7/AP3l62qITxd22h/HwQ/bZI+Flc9C0FRPD92oB8PrBmZ/BwPClfkdHcBqheu2rnvODBoCu9MdfmmaiUocdvvnqPay3P1Dg9cgrsPeFCf9AXw9S9FX4sxpKbz9/xmtHR7bh4w/lP+oy3rPnSGTyH600b9znP8Au/ZLcaYNCtlTDUkTL25QpcHotoGfXu0A/zLYm4er6F4kszPvfuItOkgNp7//T5+xn8nkqaOXP3PZSz7TgCHTvN+DBUFBVxSYb0SDvhVBV7rwgOfrQ0wbQTw3Rzgw3e1v6bDyzw4tGzMO1V9Mhbo350fS0w8s9IN0YevXA70jOC3zo68QxbAM5FLt/j9lxvzLLdAAWzbyz+gXZyAOeN5Z68H8XwSiv+O8nbd3UeAnQel5488xm/3n1S12QpZ9ZKfeTZ6vjDgvtmd3164znsKb9qtChjOjnyN4PYtgeH9VMe3t+W3wgdqvZp8khCAZ7NC8GxQG3i9i+p1lhaqctyMVk0iUt1X1Ya+5xjP9E5GAfO+49XIT1OkVev7TmpfAUoI4IK7sTyY3Y4BIo9LA9Lf+zWDVmlt3cuHnqlXe1+/y7+APIiX9j4X3LzHb89fK743ekEBX7955R+abeuDp6kCWVa2KttU75Cni7h6X98x2+rHf5rCr/fb34EFP6muR/zFM/6x9EtAUjlkuULbsb7vzYuAZpoiFVY1bx5M1AnBQpuOYfxH0KQ+/0nP5FWhLo48AApzF1+5pWp37d0R6N+ND1ESNK7L1+Bdu5VnfW7OPEi1aARsS+QBFeCZtYsTMOIN/mGr/oH++9+8ylZoW7axlnaOCmvKq9GF7G/zbn7r5c47kh06wztOqXN25F9Kpgzn1bM/F27v1QHY8K9qvxrVgN4dgPk/Al1aAx3CeOZev6ZquJWgXiDPpm/c45k6wKfRzCosr9DJ52a06hqEbF+QksarJHtEADsO8Pe6V3tVm7SLE88278VKax6SnvLajORU4McNfFvLEOOtonTmEr+9dZ9/UYp+yIOwQhToH8SresSLtwnXFZfEg90vW4H331bVqgjiH6veo4s3eBbfuB5w8iLPME9f4mtGi4eD6dsxSRxwDalSFo8Zf5Ks6sPwNJnPTV7FXRpwj1+QZvoJT/iXLmNISePvQccwPkKhuDIbcp3ljQIuIeBzPs8czbOufSd58Nx7HPj6V9U+9QJV1beCpsE8qArjb1uH8gDVvBHPbgVCtWy7l4Dte3nGBvAP2pxcHrx+3MCHL2U9V3U86hHBe2C3bsaDl5sL8N06nj0DfNpGuRz4aCTv6KTehiheDlHcazukLnDkrKrqubov/1k5R7WPS5D296puYbvwkXP8fQKAFiHSCUUAaTvto0RVZy9XZ359woISQuC8eEOVwbYN5Vl/9EPpB2r8Yx5wH4lmvbpwTdVrfdNuHiTe7c+zcUMkPlEFrOfZ/EvCV79Is1CAV9sKzRnKbaLM/Nod/gUsNR2YvgT4c6nufVcVVun7evNgC/COS93bSRfh0DeLe1DCgJuoluHeF13zrWj+tyNuqz1yVvp6Y7bjfvM7cOoi/1LSs73u/YT350kK/8JWEaYdpSplQkSsLHkmCUgXN3B04EFKXeN6QKdWqsftXuK3wUGAgx2/b2mpyqrlcmDSML7fhHd4NfO4t/j24xd4RykhmLg6817ZbUL5h4m9HW+TnfuB6nytmvBbL3fgq495T2IxccC1t+OdywL9NOdULqpWQF3D2rzaWQi2r3bk74P6sCyxR0mqNsWBPXlGnJGpmu8Z4FmNUE3ePIS3FYs7ZQGqjmjxooArdB6LTeBZ5c6DwIYSjAc+c1n6+JdtmsFWOI9YWoa0Z/a1O6ovYBmZmsN6YtWqzQFVZg/wgKtQSDPcpym6O2wBwNkrwKhZ0uk3hfc7L7/4am5xQH+WIu2IdPoS78AmPrbwZcqiMIKUNuDei+Xt9N/+ruqbECX6Apeeyb/ACjUmjKky3Lw86YQzLzLKcAlR068r7wCTm8ezpOWfAJ6uPGCpk8mA8W/zdX7z81VBzMoSmDmGVzd3DFP1jgZ4cPtwpOpxoB/QpyNfw/fXbfz8AODjpb18jevxDCgziwd2gbentOc2oAr6gjnjVeWuV5O3EQM8AOrLxYlX5e85ytsY336Vb1fP/sUeJfBFGACeLU0ZznsjP0rk70dIXWk7tr8PUMNXVRMgENpPxR/+F67zALX7sGrbhl08aNepof91HTzNbx0deKA8XVi9LJdLq0/FWSSg2e58+Rb/kiVUhV+8wWs8BMWtaZyazo8pnkNboeCPxetFi23eLQ3aAK/WTk7l46ar+wCzxulej1occGPipO/v/pOqvxN1oQ35+5TwmAfBG/f435K2/xVdbkbz+dGfZ0t/31dvqzLXX7bypp4nyXwFsLRMVY0AwLPcov7+XhSU4RKi5uUmwMAe/H63tvzDqqgPEAsL/iEweqC0vTOkLu/QJQ62urzRnXf6in6o6gik3k4okMl4VvzhSM02LplMGmTV219lMlXVW6O6/L6nm+EfVh6uPFMd9rpq+JVbERlunCjDdXNRjav+ZCwwfwrwkiggWVnxLzg92/Pe5PVr8SwfUAVccceljExgaySwt3ApQr+qPED9IVoDWaCe6SWn8qk6x3/Kg4WlJTC4t+p5L3fgp894Fv/+23xbbII0AAsZa72a/P1MeCJtdz4umiwF0AzY2ly4xoOImK523OwcaW9ih8Kg+iSZZ/8ZmTzrnvW1NEs+fIaP81UopMe+XNj5z9qq+HI2C+a3iU+A9Tv5ezlsBg+QusZjP4gDhkznXy4LCoBvfuPB1s5Wul96Jv9/yM0Dzlzh205EFfbeVnsv1B+LpWYAn3/Pe7mX9zKTlOESosVbvYF2LQBfHVmmsTk58Ori7ftU42l1ZbjFcXXWb6Yh/6rApxP4/sZo/3IrImiLA6RHYWC2suS9xQFVD2mBXC7t4Hb2CnD0nKp6WcjAfL15ZremcNpMHy/gf2OAsXP5a27HqHpVX74FLFvL21/ffYNf8x87pcGqfQse3Ndu5QHg80mAtwcwsj8PTD9s4JlV4lPV70fIcOvX4u3x6tXQVwt/nzejeZWp8PyM93j7aNIzVZtoSF1epXzsvGbthK523Mu3eJu+tyfvh1Cg4NWzz1KlHelu3efvR72avBp80Sq+vXFdzek6Ab5K141o3cOLLC1UzSwxcar278znvC191xFg1WfSrJoxYOYy3kFx4y7eDHH/Ea9VWDAFmDSfv78ebrxK/YPPpDUMwhhx9YVBxLUB6n7bBpwoHJp24DSwYpbuL7NljTJcQnTwr1p0L0lj69NJOi64pB8Khqx326S+Zi/akioq4Arkcu0Zv62N6n5enubzQnBLeMI/tIUA/r/RfNiSTMZnG5sznme47Zrz57cXjtG8eofPipT0lG/bGsmrs/8rHA9sIedl69uZl++nT/nsWr5VpGX3K/ydiCczESalqFENaCCq4m9Uh9/GJQEZWbw3tjgYhzXhNQTivgEDe/Lz3LinmqNa6IGta7KQc4XZX7NgXnMglPF5tuYKT3cKq2x3H1Ftu3ZHextv02D+hePLD7Wft6oX/xFnpg1r86YUp8JqefH7BPB+CsmizHdT4Rzgb/fi799nE/kXka6tVfuoj3s+EaUZYH/bzjugCe3l+QV82Nj8H/gwNUF+Ps++ywsFXEJeEF7uPDsTlDQQDu/Hew3PHFP8vsZkZ6v68BVXZTuJhlG5OmtWcwvGDOK3owdoPlfFgwfV7Byg7/s8mMhk/AN/2OvAX98A8z5QfUnpUTiu+swlHlxmLuWvEdpA12wB5q7gH+YvNQSWfcwzLGFoi6uzZhUnoPoyI2StjxJ5UJHLebW4uE29fi3V+e4+kE6b2Lie6n2oG8hvraz42s1CoBY6CAntv3/9xydDWbOFV2u/N4v3pBc6jQmrYol/D0JnJqEX950HPBiJ28uFcvmqfcFr1ZR/6axfizetAKpOUgD/MmJtBXwxmV+PhxtvVglrwl8D8OpjhYJn9ut3qtrJBULHMKHJoEEQ75HfMYy/l8J2gLfrAzz7FzJcoTyZz3mnqq/W8hqLiZ/z3u/HzvPztwnlma3QOVF95jhToSplQl4gTYP5h3/SU56plYSLI+8gUx7cnHmgqOXPqy/lcj77lzAHta217te+0g4IDdbs+AXw6ufqPrzaUtw+KrQzqrc31g1UDT/69ne+rU0oMHEIz4R2H+FV0Y4OvCe4rs5I6kLq8nWYLxcGqaPn+G3jevx9Fwdcv6pAUHVeDXrwFK+atbLiH/werqr9avoDI/rxTNbCAohooRpiJZMBb/fmAeXwGd6T+sxlXhX9KBFYtZlnkhZy6brPHq6qpRU93fhkLJHHgNv3+dhycTWx0FHJz1vV8creVloT8d4AHpAb1eHV1YBqKtXaATwTFqvuyztTxcTxqnQhoxZqMrzcVUHT11uzD4G3J7BoGs+8c3L5ilhThgNTF/IvOEImW9Nf2tHq8Fn+A/Ae+j0jeLt893b8i1+zYN7UcOOeatIaU6IMl5AXTO0A/i2/ImpQmwcVYYpNNxdgSB/V8zZFBFwhY9WVAf9vLDB1hOq9Edp/tZHLpc83qM3HKtva8CwstAG/P3W4/sEWUGWf0Q95xil0cGtbmIl5uvFe53I5ULsGEFTYfhx5nN8G1+KZofr70Lczn9sbANq/DAx+lc+KNnYQD3zTRkhrP4SOTcLQrOAgaUYuZM0Az8qFcsQm8OkQAc0mgFr+qmFkvTpIn7OyBF7rzL9ACGVXz4jFhGFmsfHSzD47h5ezs2goXYNauo8jk/GOdX8s4ecWhsEJXyZCRF8y3n2D13AIv/vvZgODevGZ5IRaFqEG5GY0H0a14EfTLpBBGS4hxGg+GAyM6s/vH7/AZ4GyswW+mgl8v55/+JWUjxf/afcS/xAP9C96/5cbqybYH9pX1THM0oK39ebkStuO9eHqzD+0H4k6anm48ZnABJ+M4xmkf1XgiVoWpW0stzpLC+DNV6TbZDLee/7dN1STZYgJ1cmC99/mXw7OX+XH8nDlq2c9S1EF/z6dVNcA8Pfz80n8ve2lY8IJmYz3HI+JK3oomfBcTJxmP4jgWvzLiPKxjglWBHK56ktYlzaqDLZRHeCtXjy7r+nPazD6dOIBVNeXNqEvwOnL/MsKK2LfskABlxBiNDKZKtNaNE21Pag6sPgj45xDLuedvYrTNJhnid4eqjZFcTkNDbbK49ZXdWAa0pdPhSlup67irprVKyhANSZXWHGpNIQMW10ztYBrZcknZBFPylK/pmo9Z7mcBy9xwK3lz7N98TKW2kwYwqt1xVm0umpV+Xucnqk5A1lIXWl1bnEBV/21YU35bGwz3uNNCe/0ke5TVACtWhhwhZqBGsVcq7FRwCWEmCVLC2DyMOMft29nICePB1r1QK7OyQGY8z6fKaplY90TT+gr0I+Ps83M4m22d2P5mOXigiTAV8eKS+LV4a2a8rIJw28c7LW3nWtTp0bxE4rYWvNMWOhN7uzIq9Jv3+ed1NxdeG1HQYG0J3hx5HI+9Kmk1Ifa1TDS/M/6ooBLCCEG8Pbk03LqS59sXF9yOc+wj54DwpvzyU+sLPQbR+3tCSydwTsNCVmljycPuDX9jD8XcaCfKuCGNuBjmVMzVJ2t1DNTU/By59cpDIOiDJcQQohOowfwqunWzQxvf7SylC684OPFewDXLKY9vCTe6cM7VlWrwstqbyed27s8WFvxjm1CD2ljjUHXFwVcQgipQFydVT2aS+vVTkBeQdGr8pSUX1XeWe1FU9WTB1xrq5LP5lZSNCyIEEIqqRrV+NAoUwee8lS1cBhYgK9peygDFHAJIYRUIgGF1cjioUmmQlXKhBBCKo1ubfnCEC1CTH9uCriEEEIqDVsb1dzSpkZVyoQQQogJUMAlhBBCTIACLiGEEGICFHAJIYQQE6CASwghhJgABVxCCCHEBCr0sCBWOAN1WlpaOZeEEEJIZSXEICEm6VKhA256ejoAwN+/DGbeJoQQQgyQnp4OFxcXnc/LWHEh+QWmUCgQFxcHJycnyEq5tlRaWhr8/f0RGxsLZ2dnI5Ww4qjM11+Zrx2g66/M11+Zrx0w3vUzxpCeng5fX1/Ii5iguUJnuHK5HH5+xl3Q0NnZuVL+4Qkq8/VX5msH6Por8/VX5msHjHP9RWW2Auo0RQghhJgABVxCCCHEBCjgFrKxscHs2bNhY2NT3kUpF5X5+ivztQN0/ZX5+ivztQOmv/4K3WmKEEIIqSgowyWEEEJMgAIuIYQQYgIUcAkhhBAToIBbaMWKFahRowZsbW3RsmVLnD59uryLZHRz5syBTCaT/NSrV0/5fHZ2NsaNGwcPDw84Ojri9ddfR2JiYjmWuHQOHz6MXr16wdfXFzKZDNu2bZM8zxjDrFmz4OPjAzs7O3Tq1Am3b9+W7PPs2TO89dZbcHZ2hqurK0aMGIGMjAwTXkXJFXf9Q4cO1fh76Natm2Sfinr98+fPR/PmzeHk5IQqVaqgT58+uHnzpmQfff7eHzx4gB49esDe3h5VqlTBtGnTkJ+fb8pLMZg+1x4REaHxux89erRkn4p47QCwcuVKhISEKMfWhoWFYdeuXcrny/P3TgEXwIYNGzB58mTMnj0b58+fR+PGjdG1a1ckJSWVd9GMrkGDBoiPj1f+HD16VPncpEmTsGPHDmzatAmHDh1CXFwcXnvttXIsbelkZmaicePGWLFihdbnFy1ahOXLl+P777/HqVOn4ODggK5duyI7O1u5z1tvvYWrV68iMjISO3fuxOHDhzFq1ChTXUKpFHf9ANCtWzfJ38P69eslz1fU6z906BDGjRuHkydPIjIyEnl5eejSpQsyMzOV+xT3915QUIAePXogNzcXx48fxy+//IK1a9di1qxZ5XFJetPn2gFg5MiRkt/9okWLlM9V1GsHAD8/PyxYsADnzp3D2bNn0aFDB7z66qu4evUqgHL+vTPCWrRowcaNG6d8XFBQwHx9fdn8+fPLsVTGN3v2bNa4cWOtz6WkpDArKyu2adMm5bbr168zAOzEiRMmKmHZAcC2bt2qfKxQKFjVqlXZl19+qdyWkpLCbGxs2Pr16xljjF27do0BYGfOnFHus2vXLiaTydijR49MVnZjUL9+xhgbMmQIe/XVV3W+xpyuPykpiQFghw4dYozp9/f+77//MrlczhISEpT7rFy5kjk7O7OcnBzTXkApqF87Y4yFh4ezCRMm6HyNuVy7wM3Nja1atarcf++VPsPNzc3FuXPn0KlTJ+U2uVyOTp064cSJE+VYsrJx+/Zt+Pr6ombNmnjrrbfw4MEDAMC5c+eQl5cneR/q1auH6tWrm+X7EB0djYSEBMn1uri4oGXLlsrrPXHiBFxdXfHSSy8p9+nUqRPkcjlOnTpl8jKXhYMHD6JKlSqoW7cuxowZg6dPnyqfM6frT01NBQC4u7sD0O/v/cSJE2jUqBG8vb2V+3Tt2hVpaWnKbKkiUL92wbp16+Dp6YmGDRtixowZyMrKUj5nLtdeUFCAP//8E5mZmQgLCyv333uFnkvZGJ48eYKCggLJmwsA3t7euHHjRjmVqmy0bNkSa9euRd26dREfH4+5c+eibdu2uHLlChISEmBtbQ1XV1fJa7y9vZGQkFA+BS5DwjVp+70LzyUkJKBKlSqS5y0tLeHu7m4W70m3bt3w2muvITAwEHfv3sXHH3+M7t2748SJE7CwsDCb61coFJg4cSJat26Nhg0bAoBef+8JCQla/z6E5yoCbdcOAIMGDUJAQAB8fX1x6dIlfPTRR7h58ya2bNkCoOJf++XLlxEWFobs7Gw4Ojpi69atCA4ORlRUVLn+3it9wK1MunfvrrwfEhKCli1bIiAgABs3boSdnV05loyUhwEDBijvN2rUCCEhIahVqxYOHjyIjh07lmPJjGvcuHG4cuWKpL9CZaHr2sXt8I0aNYKPjw86duyIu3fvolatWqYuptHVrVsXUVFRSE1NxebNmzFkyBAcOnSovItFnaY8PT1hYWGh0UstMTERVatWLadSmYarqyvq1KmDO3fuoGrVqsjNzUVKSopkH3N9H4RrKur3XrVqVY2Oc/n5+Xj27JlZvic1a9aEp6cn7ty5A8A8rv/999/Hzp07ceDAAcnKYvr8vVetWlXr34fw3ItO17Vr07JlSwCQ/O4r8rVbW1sjKCgIoaGhmD9/Pho3boyvv/663H/vlT7gWltbIzQ0FPv27VNuUygU2LdvH8LCwsqxZGUvIyMDd+/ehY+PD0JDQ2FlZSV5H27evIkHDx6Y5fsQGBiIqlWrSq43LS0Np06dUl5vWFgYUlJScO7cOeU++/fvh0KhUH5AmZOHDx/i6dOn8PHxAVCxr58xhvfffx9bt27F/v37ERgYKHlen7/3sLAwXL58WfKlIzIyEs7OzggODjbNhZRAcdeuTVRUFABIfvcV8dp1USgUyMnJKf/fe6m6XJmJP//8k9nY2LC1a9eya9eusVGjRjFXV1dJLzVzMGXKFHbw4EEWHR3Njh07xjp16sQ8PT1ZUlISY4yx0aNHs+rVq7P9+/ezs2fPsrCwMBYWFlbOpS659PR0duHCBXbhwgUGgC1dupRduHCBxcTEMMYYW7BgAXN1dWXbt29nly5dYq+++ioLDAxkz58/Vx6jW7durGnTpuzUqVPs6NGjrHbt2mzgwIHldUkGKer609PT2dSpU9mJEydYdHQ027t3L2vWrBmrXbs2y87OVh6jol7/mDFjmIuLCzt48CCLj49X/mRlZSn3Ke7vPT8/nzVs2JB16dKFRUVFsd27dzMvLy82Y8aM8rgkvRV37Xfu3GHz5s1jZ8+eZdHR0Wz79u2sZs2arF27dspjVNRrZ4yx6dOns0OHDrHo6Gh26dIlNn36dCaTydiePXsYY+X7e6eAW+ibb75h1atXZ9bW1qxFixbs5MmT5V0ko3vzzTeZj48Ps7a2ZtWqVWNvvvkmu3PnjvL558+fs7FjxzI3Nzdmb2/P+vbty+Lj48uxxKVz4MABBkDjZ8iQIYwxPjTok08+Yd7e3szGxoZ17NiR3bx5U3KMp0+fsoEDBzJHR0fm7OzMhg0bxtLT08vhagxX1PVnZWWxLl26MC8vL2ZlZcUCAgLYyJEjNb5kVtTr13bdANiaNWuU++jz937//n3WvXt3Zmdnxzw9PdmUKVNYXl6eia/GMMVd+4MHD1i7du2Yu7s7s7GxYUFBQWzatGksNTVVcpyKeO2MMTZ8+HAWEBDArK2tmZeXF+vYsaMy2DJWvr93Wi2IEEIIMYFK34ZLCCGEmAIFXEIIIcQEKOASQgghJkABlxBCCDEBCriEEEKICVDAJYQQQkyAAi4hhBBiAhRwCSGEEBOggEtIBXDw4EHIZDKNSdeLMnToUPTp06fIfWrUqIGvvvqqVGUriZJcDyEVHQVcQiqAVq1aIT4+Hi4uLnq/5uuvv8batWvLrlCF1q5dq7G+KCFEE62HS0gFYG1tbfDSYIYEZ0JI2aMMl5ByEBERgfHjx2PixIlwc3ODt7c3fvrpJ2RmZmLYsGFwcnJCUFAQdu3aBUCzClbIKv/77z/Ur18fjo6O6NatG+Lj45Xn0KdKGQDS09MxcOBAODg4oFq1alixYoXk+aVLl6JRo0ZwcHCAv78/xo4di4yMDGW5hg0bhtTUVMhkMshkMsyZMwcAkJOTg48++gj+/v6wsbFBUFAQVq9eLTn2uXPn8NJLL8He3h6tWrXCzZs3S/iOEvLio4BLSDn55Zdf4OnpidOnT2P8+PEYM2YM3njjDbRq1Qrnz59Hly5dMHjwYGRlZWl9fVZWFhYvXozffvsNhw8fxoMHDzB16lSDy/Hll1+icePGuHDhAqZPn44JEyYgMjJS+bxcLsfy5ctx9epV/PLLL9i/fz8+/PBDALyq+6uvvoKzszPi4+MRHx+vLMM777yD9evXY/ny5bh+/Tp++OEHODo6Ss49c+ZMLFmyBGfPnoWlpSWGDx9ucPkJqTBKvd4QIcRg4eHhrE2bNsrH+fn5zMHBgQ0ePFi5LT4+ngFgJ06cUC61l5yczBhjbM2aNQyAZHnFFStWMG9vb+XjIUOGsFdffbXIcgQEBLBu3bpJtr355puse/fuOl+zadMm5uHhoXy8Zs0a5uLiItnn5s2bDACLjIzUegzhevbu3avc9s8//zAAkvWICTEnlOESUk5CQkKU9y0sLODh4YFGjRopt3l7ewMAkpKStL7e3t4etWrVUj728fHRue+6devg6Oio/Dly5IjyubCwMMm+YWFhuH79uvLx3r170bFjR1SrVg1OTk4YPHgwnj59qjPzBoCoqChYWFggPDxc5z6A9D3w8fEBoPt6CanoKOASUk6srKwkj2UymWSbTCYDACgUCr1fz3Qsb927d29ERUUpf1566SW9ynj//n307NkTISEh+Ouvv3Du3DllG29ubq7O19nZ2el1fEOul5CKjnopE1IJODk5wcnJSetzJ0+e1Hhcv359ALxTk0KhwJIlSyCX8+/nGzdulOxvbW2NgoICybZGjRpBoVDg0KFD6NSpk7Eug5AKjTJcQiq5Y8eOYdGiRbh16xZWrFiBTZs2YcKECQCAoKAg5OXl4ZtvvsG9e/fw22+/4fvvv5e8vkaNGsjIyMC+ffvw5MkTZGVloUaNGhgyZAiGDx+Obdu2ITo6GgcPHtQI1oRUJhRwCankpkyZgrNnz6Jp06b47LPPsHTpUnTt2hUA0LhxYyxduhQLFy5Ew4YNsW7dOsyfP1/y+latWmH06NF488034eXlhUWLFgEAVq5ciX79+mHs2LGoV68eRo4ciczMTJNfHyEvChnT1ehDCCGEEKOhDJcQQggxAQq4hBBCiAlQwCWEEEJMgAIuIYQQYgIUcAkhhBAToIBLCCGEmAAFXEIIIcQEKOASQgghJkABlxBCCDEBCriEEEKICVDAJYQQQkyAAi4hhBBiAv8HhMhMaGGW8LwAAAAASUVORK5CYII=",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# viz: training loss curve\n",
"plt.figure(figsize=(5, 3))\n",
"plt.plot(losses, color=\"#1E40FF\", alpha=0.8)\n",
"plt.xlabel(\"mini-batch\"); plt.ylabel(\"cross-entropy loss\")\n",
"plt.title(f\"FashionMNIST CNN ({TRAIN_BATCHES} batches)\"); plt.tight_layout(); plt.show()"
]
},
{
"cell_type": "markdown",
"id": "b28bd78c",
"metadata": {},
"source": [
"> **Interpretation.** The loss falls fast then flattens; that plateau is where a longer run, augmentation, or a bigger model would buy the next few points. At this budget the model already clears 50%, far above 10% chance, which is the signal the architecture and loop are wired correctly.\n"
]
},
{
"cell_type": "markdown",
"id": "a45a137a",
"metadata": {},
"source": [
"### 4.4 A deliberate failure: the gradients that never zero\n",
"\n",
"This is the most common training-loop bug, and you will write it at least once. We remove the `opt.zero_grad()` call. PyTorch *accumulates* gradients across `backward()` calls by default, so without zeroing, every step adds the new gradient to the stale sum. The update direction is wrong, and the loss refuses to fall (or blows up). We run it broken first, observe the symptom, then fix the one line.\n"
]
},
{
"cell_type": "code",
"execution_count": 29,
"id": "98d93792",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:25:15.977724Z",
"iopub.status.busy": "2026-06-10T19:25:15.977555Z",
"iopub.status.idle": "2026-06-10T19:25:18.142403Z",
"shell.execute_reply": "2026-06-10T19:25:18.142118Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"BROKEN: first loss 2.30 -> last loss 2.89 · test acc 10.0%\n",
"Symptom: the loss does not settle and accuracy is near 10% (chance). The model learned nothing.\n"
]
}
],
"source": [
"# BROKEN ON PURPOSE: no opt.zero_grad(), so gradients accumulate across steps\n",
"def train_broken(model, n_batches, lr=0.1):\n",
" opt = torch.optim.SGD(model.parameters(), lr=lr, momentum=0.9)\n",
" model.train()\n",
" losses, seen = [], 0\n",
" loader = get_loaders()[0]\n",
" while seen < n_batches:\n",
" for xb, yb in loader:\n",
" logits = model(xb)\n",
" loss = F.cross_entropy(logits, yb)\n",
" # opt.zero_grad() <-- the missing line: gradients accumulate every step\n",
" loss.backward()\n",
" opt.step()\n",
" losses.append(float(loss.item())); seen += 1\n",
" if seen >= n_batches:\n",
" break\n",
" return losses\n",
"\n",
"cnn_broken = make_cnn()\n",
"broken_losses = train_broken(cnn_broken, min(TRAIN_BATCHES, 40))\n",
"broken_acc = accuracy(cnn_broken, test_loader)\n",
"print(f\"BROKEN: first loss {broken_losses[0]:.2f} -> last loss {broken_losses[-1]:.2f} · test acc {broken_acc:.1%}\")\n",
"print(\"Symptom: the loss does not settle and accuracy is near 10% (chance). The model learned nothing.\")"
]
},
{
"cell_type": "markdown",
"id": "3a3992a2",
"metadata": {},
"source": [
"> **Common confusion:** the code raises no error. It runs, produces a loss, produces an accuracy, and is completely broken. This is why a 10%-accuracy model is a more dangerous bug than a crash: nothing tells you except the number. The fix is one line, `opt.zero_grad()` before `loss.backward()`, which is already in our working `train_cnn`. Re-confirm it works:\n"
]
},
{
"cell_type": "code",
"execution_count": 30,
"id": "db22d7fc",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:25:18.143715Z",
"iopub.status.busy": "2026-06-10T19:25:18.143631Z",
"iopub.status.idle": "2026-06-10T19:25:20.692535Z",
"shell.execute_reply": "2026-06-10T19:25:20.692209Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"FIXED: first loss 2.30 -> last loss 1.38 · test acc 41.8%\n",
"[ ok ] one line (opt.zero_grad) is the difference between 10% and a working model\n"
]
}
],
"source": [
"cnn_fixed = make_cnn()\n",
"fixed_losses = train_cnn(cnn_fixed, min(TRAIN_BATCHES, 40)) # this version HAS zero_grad\n",
"fixed_acc = accuracy(cnn_fixed, test_loader)\n",
"print(f\"FIXED: first loss {fixed_losses[0]:.2f} -> last loss {fixed_losses[-1]:.2f} · test acc {fixed_acc:.1%}\")\n",
"assert fixed_acc > broken_acc + 0.15, \\\n",
" \"zeroing gradients must recover real accuracy well above the broken (accumulating) run\"\n",
"print(\"[ ok ] one line (opt.zero_grad) is the difference between 10% and a working model\")"
]
},
{
"cell_type": "markdown",
"id": "23851285",
"metadata": {},
"source": [
"> **Key takeaways**\n",
"> - Most of a classic CNN's parameters live in the FC head; conv FLOPs scale with spatial size, parameters do not.\n",
"> - The four-comment loop trains the model; `opt.zero_grad()` is load-bearing, and omitting it fails silently at ~10% accuracy.\n",
"> - A model that runs without error can still be totally broken. Always check the number against the chance baseline.\n"
]
},
{
"cell_type": "markdown",
"id": "6e46f49e",
"metadata": {},
"source": [
"## Part 5 — Load a pretrained net and verify outputs\n",
"\n",
"> **Objectives**\n",
"> - Save the trained CNN's weights to disk (a checkpoint) and reload them into a fresh module.\n",
"> - Verify the reloaded model reproduces the original's outputs bit-identically (in `eval` mode).\n",
"> - See the eval-mode footgun and why a checkpoint without `eval()` can silently disagree.\n",
"\n",
"\"Pretrained\" means weights someone already trained, saved, and shipped. You load them into the same architecture and use them without training. The contract that makes this work is exact reproducibility: the reloaded model, given the same input, must produce the same output. We test that contract here on our own trained CNN, which is the honest small-scale version of loading a ResNet checkpoint. (The torchvision ImageNet ResNet-18 is a 224x224 RGB model and a ~45MB download; it is the GPU-fenced optional cell at the end.)\n"
]
},
{
"cell_type": "code",
"execution_count": 31,
"id": "14855dbd",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:25:20.696277Z",
"iopub.status.busy": "2026-06-10T19:25:20.696084Z",
"iopub.status.idle": "2026-06-10T19:25:20.723227Z",
"shell.execute_reply": "2026-06-10T19:25:20.722946Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"checkpoint size: 207 KB (52,138 parameters)\n",
"loaded state_dict into a fresh make_cnn()\n"
]
}
],
"source": [
"import io\n",
"# save the trained CNN's weights to an in-memory checkpoint (a real file on disk works identically)\n",
"buffer = io.BytesIO()\n",
"torch.save(cnn.state_dict(), buffer)\n",
"ckpt_bytes = buffer.getvalue()\n",
"print(f\"checkpoint size: {len(ckpt_bytes) / 1024:.0f} KB ({n_params(cnn):,} parameters)\")\n",
"\n",
"# reload into a FRESH module of the same architecture\n",
"cnn_reloaded = make_cnn()\n",
"cnn_reloaded.load_state_dict(torch.load(io.BytesIO(ckpt_bytes)))\n",
"print(\"loaded state_dict into a fresh make_cnn()\")"
]
},
{
"cell_type": "markdown",
"id": "49e4f804",
"metadata": {},
"source": [
"### Exercise 12.7 — Verify the reload is exact\n",
"`Difficulty 2/5 · ~10 min`\n",
"\n",
"Confirm the reloaded model reproduces the original's outputs. Put both models in `eval()` mode, run the same input batch through both under `torch.no_grad()`, and return whether the logits are exactly equal. Exact equality is the right check here: loading the same weights into the same architecture and running on CPU is deterministic, so any difference means the checkpoint did not round-trip.\n"
]
},
{
"cell_type": "code",
"execution_count": 32,
"id": "ba645f1e",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:25:20.724487Z",
"iopub.status.busy": "2026-06-10T19:25:20.724404Z",
"iopub.status.idle": "2026-06-10T19:25:20.774434Z",
"shell.execute_reply": "2026-06-10T19:25:20.774172Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] 12.7 checkpoint round-trip: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 32,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def reload_matches(model_a, model_b, x):\n",
" \"\"\"Return True iff model_a(x) and model_b(x) are exactly equal, both in eval mode.\"\"\"\n",
" # TODO 1: put both models in eval() mode (matters once BN/Dropout are present)\n",
" # TODO 2: under torch.no_grad(), compute logits from both on x\n",
" out_a = None\n",
" out_b = None\n",
" attempted(out_a, out_b)\n",
" # TODO 3: return torch.equal(out_a, out_b) (exact equality, not allclose)\n",
" raise NotImplementedError\n",
"\n",
"def _check_reload():\n",
" xb = next(iter(test_loader))[0][:16] # a fixed batch\n",
" assert reload_matches(cnn, cnn_reloaded, xb), \\\n",
" \"reloaded checkpoint must reproduce the original logits exactly\"\n",
" # negative control: a FRESH untrained model must NOT match (proves the check has teeth)\n",
" assert not reload_matches(cnn, make_cnn(), xb), \\\n",
" \"an untrained model should not match; if it does, your check is trivially true\"\n",
"\n",
"check(\"12.7 checkpoint round-trip\", _check_reload)"
]
},
{
"cell_type": "markdown",
"id": "76a34fc0",
"metadata": {},
"source": [
"Hint 1 (conceptual)Call `.eval()` on both models. Wrap the two forward passes in `with torch.no_grad():`. Compare with `torch.equal`, which is exact, unlike `torch.allclose`.\n",
"\n",
"Hint 2 (pseudocode)\n",
"\n",
"```python\n",
"model_a.eval(); model_b.eval()\n",
"with torch.no_grad():\n",
" out_a = model_a(x)\n",
" out_b = model_b(x)\n",
"return torch.equal(out_a, out_b)\n",
"```\n",
"\n",
"\n",
"Help — the positive case passes but the negative control failsIf a fresh `make_cnn()` matches, you are probably comparing a model to itself, or `make_cnn()` is not re-seeding per call in a way that still differs after `load_state_dict`. The fresh model has different (untrained) weights, so its logits must differ.\n"
]
},
{
"cell_type": "code",
"execution_count": 33,
"id": "d701e56e",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:25:20.779463Z",
"iopub.status.busy": "2026-06-10T19:25:20.779302Z",
"iopub.status.idle": "2026-06-10T19:25:20.893207Z",
"shell.execute_reply": "2026-06-10T19:25:20.892750Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] 12.7 checkpoint round-trip\n",
"reloaded == original : True\n",
"fresh == original : False (expected False)\n"
]
}
],
"source": [
"# solution\n",
"def reload_matches(model_a, model_b, x):\n",
" model_a.eval(); model_b.eval()\n",
" with torch.no_grad():\n",
" out_a = model_a(x)\n",
" out_b = model_b(x)\n",
" return torch.equal(out_a, out_b)\n",
"\n",
"check(\"12.7 checkpoint round-trip\", _check_reload, required=True)\n",
"xb = next(iter(test_loader))[0][:16]\n",
"print(\"reloaded == original :\", reload_matches(cnn, cnn_reloaded, xb))\n",
"print(\"fresh == original :\", reload_matches(cnn, make_cnn(), xb), \"(expected False)\")"
]
},
{
"cell_type": "markdown",
"id": "b65fef39",
"metadata": {},
"source": [
"> **Caveat (the eval-mode footgun):** our small CNN has no BatchNorm, so `train()` vs `eval()` does not change its output. The moment you add BatchNorm or Dropout, it does: in `train()` mode BatchNorm uses the current batch's statistics, so a batch of size 1 has degenerate variance and the output is garbage. Forgetting `model.eval()` at inference is the single most common deployment bug for CNNs. The habit of always calling `eval()` before inference is what the exercise drills.\n"
]
},
{
"cell_type": "markdown",
"id": "6c9bb23c",
"metadata": {},
"source": [
"> **Key takeaways**\n",
"> - A checkpoint is a `state_dict` saved and reloaded into the same architecture; the contract is exact output reproduction.\n",
"> - `torch.equal` (exact) is the right check for a reload on the same device; `allclose` would hide a real round-trip bug.\n",
"> - Always `model.eval()` before inference. With BatchNorm or Dropout, skipping it silently corrupts outputs.\n"
]
},
{
"cell_type": "markdown",
"id": "58e29501",
"metadata": {},
"source": [
"### 5.1 Optional: a real pretrained ResNet (GPU-fenced, off by default)\n",
"\n",
"Loading the torchvision ImageNet ResNet-18 is a ~45MB download and the model expects 224x224 RGB; training it to the draft's \">93% CIFAR-10\" number wants a GPU. Per the runtime budget this stays off the canonical path: the cell below prints what it would do and skips unless you opt in with a GPU available. It never errors on CPU.\n"
]
},
{
"cell_type": "code",
"execution_count": 34,
"id": "6dbe70ce",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:25:20.894403Z",
"iopub.status.busy": "2026-06-10T19:25:20.894295Z",
"iopub.status.idle": "2026-06-10T19:25:20.900799Z",
"shell.execute_reply": "2026-06-10T19:25:20.900574Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"skipped: needs GPU and RUN_PRETRAINED_RESNET=True.\n",
"It would download ResNet-18 ImageNet weights (~45MB), run a 224x224x3 forward pass,\n",
"and produce 1000-class logits. The same load_state_dict / eval() contract from 12.7 applies.\n"
]
}
],
"source": [
"# gpu-only: load a real pretrained ResNet-18 and run one forward pass.\n",
"RUN_PRETRAINED_RESNET = False # set True only with a GPU; this is the explicit opt-in flag\n",
"if RUN_PRETRAINED_RESNET and device == \"cuda\":\n",
" from torchvision.models import resnet18, ResNet18_Weights\n",
" weights = ResNet18_Weights.IMAGENET1K_V1\n",
" model = resnet18(weights=weights).to(device).eval()\n",
" dummy = torch.randn(1, 3, 224, 224, device=device)\n",
" with torch.no_grad():\n",
" logits = model(dummy)\n",
" print(\"pretrained ResNet-18 output shape:\", tuple(logits.shape), \"(1000 ImageNet classes)\")\n",
"else:\n",
" print(\"skipped: needs GPU and RUN_PRETRAINED_RESNET=True.\")\n",
" print(\"It would download ResNet-18 ImageNet weights (~45MB), run a 224x224x3 forward pass,\")\n",
" print(\"and produce 1000-class logits. The same load_state_dict / eval() contract from 12.7 applies.\")"
]
},
{
"cell_type": "markdown",
"id": "a9687261",
"metadata": {},
"source": [
"## Safety lens\n",
"\n",
"CNNs have a famous brittleness, and it is cheap to demonstrate. A trained classifier's decision boundary in pixel space is closer to every input than intuition expects, so a tiny, near-invisible perturbation can flip a confident prediction. The Fast Gradient Sign Method (FGSM) takes one gradient step on the *input* in the direction that increases the loss: `x_adv = x + epsilon * sign(grad)`. We run it on our FashionMNIST CNN and watch a correctly-classified image flip.\n"
]
},
{
"cell_type": "code",
"execution_count": 35,
"id": "b3e1f2be",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:25:20.904547Z",
"iopub.status.busy": "2026-06-10T19:25:20.904460Z",
"iopub.status.idle": "2026-06-10T19:25:20.925770Z",
"shell.execute_reply": "2026-06-10T19:25:20.925091Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"true label : Ankle boot\n",
"clean pred : Ankle boot\n",
"adv pred : Bag (epsilon=0.15)\n"
]
}
],
"source": [
"# a 10-line FGSM attack on our trained CNN: perturb one correctly-classified image\n",
"cnn.eval()\n",
"xb, yb = next(iter(test_loader))\n",
"x0, y0 = xb[:1].clone(), yb[:1]\n",
"x0.requires_grad_(True)\n",
"logits = cnn(x0)\n",
"orig_pred = int(logits.argmax(1))\n",
"loss = F.cross_entropy(logits, y0)\n",
"cnn.zero_grad()\n",
"loss.backward()\n",
"epsilon = 0.15 # L-infinity budget; small but visible on FashionMNIST\n",
"x_adv = (x0 + epsilon * x0.grad.sign()).clamp(0, 1).detach()\n",
"with torch.no_grad():\n",
" adv_pred = int(cnn(x_adv).argmax(1))\n",
"print(f\"true label : {CLASSES[int(y0)]}\")\n",
"print(f\"clean pred : {CLASSES[orig_pred]}\")\n",
"print(f\"adv pred : {CLASSES[adv_pred]} (epsilon={epsilon})\")"
]
},
{
"cell_type": "code",
"execution_count": 36,
"id": "defeb70c",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:25:20.926836Z",
"iopub.status.busy": "2026-06-10T19:25:20.926703Z",
"iopub.status.idle": "2026-06-10T19:25:21.006753Z",
"shell.execute_reply": "2026-06-10T19:25:21.006190Z"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAArIAAAD/CAYAAAANIDUbAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAKZtJREFUeJzt3Xt0VOW9xvFnEiBXEiByKSAQBCJFlBYEA4EEVCAUEBABL+VyqLQVUHvKcVVOXcAq1aq1pWJLUTwoBHq4WKWcgxVUpEeLSLmpYBBUKCLXAOESLkLe84crU4Zkv3uSyZC88P2sxXJlfvvd8+49e/b+Zc/kMWCMMQIAAAAcE1PVEwAAAAAqgkYWAAAATqKRBQAAgJNoZAEAAOAkGlkAAAA4iUYWAAAATqKRBQAAgJNoZAEAAOAkGlkAAAA46YptZN955x0FAgG98847VT2VaiMQCGjChAnWZXbt2qVAIKCXXnqpUp6zZH2//vWvK2V9gIvCee9VlpycHOXk5FyW5wIuh5deekmBQEC7du2q6qmgGrpiG9krRefOnRUIBDRr1qyqnopztm3bpqlTp3Lyg68//OEPlfbL2+XAsQ1UrpycHAUCgeC/WrVqKT09XePGjdOePXuqenqwoJGtxnbs2KH169erRYsWWrBgQVVPxznbtm3TtGnTuNjDl4uNrNexvXLlSq1cufLyTwpwXNOmTTV//nzNnz9ff/zjH3XnnXdq4cKFysrKUlFRUVVPDx5qVPUE4C0vL08NGjTQM888o6FDh2rXrl1q0aJFVU8LuGIUFRUpMTExKus2xujMmTNKSEiIyvq91KpV67I+H3ClSE1N1X333RfyWHp6uiZMmKD33ntPt99+exXNDDbO3pHdu3evxo4dq8aNGysuLk7p6en68Y9/rHPnzlnHrVu3Tn379lVqaqoSExOVnZ2t9957L2SZ3bt364EHHlBGRoYSEhKUlpamu+66q9Tdj5Lv7bz33nv693//d9WvX19JSUkaPHiwDh06FLJsYWGh8vPzVVhYGPY2Lly4UEOHDlX//v2VmpqqhQsXllpm6tSpCgQC2rlzp0aPHq06deooNTVVY8aMCes3yOnTpysmJkYzZ860Lpefn6+hQ4eqXr16io+PV6dOnfSXv/wl7G2RpN/+9rdq3ry5EhISlJ2drY8//rjUMm+//ba6d++upKQk1alTR3fccYc++eSTUstt2rRJubm5SklJUXJysm699Va9//77wfpLL72ku+66S5LUs2fP4MdFfGfaHSXHdn5+voYNG6aUlBSlpaXpoYce0pkzZ0KWzcvLU8eOHZWQkKB69eppxIgRpT4OzMnJ0Q033KANGzaoR48eSkxM1OTJk9WiRQtt3bpVa9asCR4nJd8xLZnDpcr6zl6LFi3Uv39/vfHGG+rUqZMSEhI0e/bskHELFixQRkaG4uPj1bFjR/3tb38LqYdz7vE7tsv6juzBgwc1duxYNWzYUPHx8brpppv08ssvhyxz8ffZn3/+eV133XWKi4vTzTffrPXr15f5GgHhCPeaKklbt25Vr169lJCQoKZNm2r69OkqLi4OWaZ///5q2bJlmc+VmZmpTp06SarYdfdSjRo1kiTVqPGv+37l2Z4PP/xQ2dnZIdszd+5cvvNbiZy8I/vVV1+pc+fOOnbsmMaNG6frr79ee/fu1dKlS1VUVOR5R+Ltt99Wbm6uOnbsqClTpigmJkZz585Vr1699H//93/q3LmzJGn9+vX6+9//rhEjRqhp06batWuXZs2apZycHG3btq3UHZyJEyeqbt26mjJlinbt2qUZM2ZowoQJWrRoUXCZV199VWPGjNHcuXM1evRo321ct26ddu7cqblz56pWrVoaMmSIFixYoMmTJ5e5/LBhw5Senq4nnnhCGzdu1Jw5c9SgQQM9+eSTns/x85//XI8//rhmz56t+++/33O5rVu3qlu3bmrSpIl+9rOfKSkpSYsXL9agQYP0yiuvaPDgwb7bM2/ePJ04cULjx4/XmTNn9Lvf/U69evXSRx99pIYNG0qS3nzzTeXm5qply5aaOnWqTp8+rZkzZ6pbt27auHFj8G701q1b1b17d6WkpOiRRx5RzZo1NXv2bOXk5GjNmjXq0qWLevTooQcffFDPPvusJk+erLZt20pS8L9wx7Bhw9SiRQs98cQTev/99/Xss8/q6NGjmjdvniTpl7/8pR577DENGzZMP/jBD3To0CHNnDlTPXr00KZNm1SnTp3gugoKCpSbm6sRI0bovvvuU8OGDZWTk6OJEycqOTlZ//mf/ylJwWOyvLZv3667775bP/zhD3X//fcrIyMjWFuzZo0WLVqkBx98UHFxcfrDH/6gvn376oMPPtANN9wgKbxzT3mP7dOnTysnJ0c7d+7UhAkTlJ6eriVLlmj06NE6duyYHnrooZDlFy5cqBMnTuiHP/yhAoGAnnrqKQ0ZMkSff/65atasWaH9gqtbuNfU/fv3q2fPnjp//nzwWvP888+X+lRj+PDhGjlypNavX6+bb745+Pju3bv1/vvv6+mnn5ZU/uvuhQsXdPjwYUnS119/rU8++URTpkxRq1at1K1bt3Jvz969e4O/bD766KNKSkrSnDlzFBcXF9H+xCWMg0aOHGliYmLM+vXrS9WKi4uNMcasXr3aSDKrV68OPt66dWvTp0+f4DLGGFNUVGTS09PN7bffHvLYpdauXWskmXnz5gUfmzt3rpFkbrvttpB1/uQnPzGxsbHm2LFjpZadO3duWNs4YcIEc+211wbXu3LlSiPJbNq0KWS5KVOmGEnm3/7t30IeHzx4sElLSwt5TJIZP368McaYn/70pyYmJsa89NJLIct88cUXpeZ56623mvbt25szZ84EHysuLjZdu3Y1rVu3tm5HyfoSEhLMl19+GXx83bp1RpL5yU9+EnysQ4cOpkGDBqagoCD42JYtW0xMTIwZOXJk8LFBgwaZWrVqmc8++yz42FdffWVq165tevToEXxsyZIlIccA3FJybA8cODDk8QceeMBIMlu2bDG7du0ysbGx5pe//GXIMh999JGpUaNGyOPZ2dlGkvnjH/9Y6rnatWtnsrOzPedwqZL38xdffBF8rHnz5kaS+etf/1pqeUlGkvnHP/4RfGz37t0mPj7eDB48OPhYuOce27GdnZ0dsi0zZswwkkxeXl7wsXPnzpnMzEyTnJxsjh8/boz513s1LS3NHDlyJLjssmXLjCSzfPnyUs8FhCPc4/rhhx82ksy6deuCjx08eNCkpqaGvN8KCwtNXFyc+elPfxqyzqeeesoEAgGze/duY0z5rrsl54dL/7Vt29Z8/vnnFdqeiRMnmkAgEHLdLigoMPXq1St1/kDFOffVguLiYr322msaMGBA8OODi5X1MaAkbd68WTt27NA999yjgoICHT58WIcPH9apU6d066236m9/+1vw44uLf/v7+uuvVVBQoFatWqlOnTrauHFjqXWPGzcu5Hm7d++uCxcuaPfu3cHHRo8eLWNMWL8Vnj9/XosWLdLw4cOD6+3Vq5caNGjg+UdfP/rRj0J+7t69uwoKCnT8+PGQx40xmjBhgn73u98pLy9Po0aNss7lyJEjevvttzVs2DCdOHEiuN8KCgrUp08f7dixQ3v37vXdpkGDBqlJkybBnzt37qwuXbpoxYoVkqR9+/Zp8+bNGj16tOrVqxdc7sYbb9Ttt98eXO7ChQtauXKlBg0aFPLR0re+9S3dc889evfdd0ttM9w2fvz4kJ8nTpwoSVqxYoX+/Oc/q7i4WMOGDQsem4cPH1ajRo3UunVrrV69OmRsXFycxowZE7W5pqenq0+fPmXWMjMz1bFjx+DPzZo10x133KE33nhDFy5ckFT+c084VqxYoUaNGunuu+8OPlazZk09+OCDOnnypNasWROy/PDhw1W3bt3gz927d5ckff755xV6fiDc43rFihW65ZZbgp+OSlL9+vV17733hqwvJSVFubm5Wrx4sYwxwccXLVqkW265Rc2aNZNUvuuu9M3Xg1atWqVVq1bp9ddf14wZM1RYWKjc3NyQrwuGuz1//etflZmZqQ4dOgQfq1evXqntQWSc+2rBoUOHdPz48eBHceHasWOHJFkbt8LCQtWtW1enT5/WE088oblz52rv3r0hb5SyvmtT8qYpUXIROHr0aLnmWGLlypU6dOiQOnfurJ07dwYf79mzp/70pz/pySefVExM6O8gtjmkpKQEH583b55OnjypWbNmhVzYvOzcuVPGGD322GN67LHHylzm4MGDIU1qWVq3bl3qsTZt2mjx4sWSFGz6L/4otkTbtm31xhtv6NSpUzpx4oSKioo8lysuLtaePXvUrl07322DGy49dq677jrFxMRo165diomJkTGmzONLUqmPwps0aRLVP4ZKT0/3rHm9B4qKinTo0CE1atSo3OeecOzevVutW7cudc4o+SrCxb9wS5V/PgPCPa53796tLl26lBpf1vl++PDheu2117R27Vp17dpVn332mTZs2KAZM2ZUeJ5JSUm67bbbgj/37dtXWVlZ6tSpk371q1/pmWeeKff2ZGZmlnqeVq1aVXiOKM25RraiSu62Pv300yG/HV0sOTlZ0jd3fObOnauHH35YmZmZSk1NVSAQ0IgRI0p96VySYmNjy1zfxQd3eZTcdR02bFiZ9TVr1qhnz54VmkO3bt20efNmPffccxo2bFjI3c+ylGzvpEmTPO808abE5XTxpx/FxcUKBAJ6/fXXy3wPlLynS5Q3QcDrE56SO6iXijShoLznnmio7PMZEI3jesCAAUpMTNTixYvVtWtXLV68WDExMcE/hKwsHTt2VGpqasgfZlaH9yn+xblGtn79+kpJSSnzL95trrvuOknffCRx8W9cZVm6dKlGjRoV/O1Lks6cOaNjx46Ve77lderUKS1btkzDhw/X0KFDS9UffPBBLViwoFQjG65WrVrpqaeeUk5Ojvr27au33npLtWvX9ly+5OP7mjVr+u43m5I74hf79NNPg3/A1bx5c0nf/LHMpfLz83XNNdcoKSlJ8fHxSkxM9FwuJiZG1157rSTvJgRu2bFjR8idzp07d6q4uFgtWrRQbGysjDFKT09XmzZtKvwcXsdKyd3IY8eOhfzR2KV3McPh9R5ITExU/fr1JYV/7inPsd28eXN9+OGHKi4uDrkrm5+fH6wD0RTucd28efMy3ydlne+TkpLUv39/LVmyRL/5zW+0aNEide/eXY0bN670+V+4cEEnT54M/lye7bn4U9USZT2GinPuO7IxMTEaNGiQli9frn/84x+l6l53DTp27KjrrrtOv/71r0MOyBIXf/+l5OJ4sZkzZ3rehQlHuDEgr776qk6dOqXx48dr6NChpf71799fr7zyis6ePVvhudx4441asWKFPvnkEw0YMECnT5/2XLZBgwbKycnR7NmztW/fvlL1S2PGvLz22msh36X94IMPtG7dOuXm5kr65juuHTp00MsvvxxyMvj444+1cuVK9evXT9I3r03v3r21bNmykOiSAwcOBIOrS75KkZSUJEmX5RcQRM/vf//7kJ9LouJyc3M1ZMgQxcbGatq0aaXes8YYFRQUhPUcSUlJZR4nJb8AX3w35tSpU6Wiq8Kxdu3akO/P7dmzR8uWLVPv3r2Dd0HDPfeU59ju16+f9u/fH5Kicv78ec2cOVPJycnKzs4u97YA5RHucd2vXz+9//77+uCDD4KPHTp0yPNvQ4YPH66vvvpKc+bM0ZYtWzR8+PCQemXEb61evVonT57UTTfdVO7t6dOnj9auXavNmzcHHzty5Aj/g6NK5twdWUl6/PHHtXLlSmVnZ2vcuHFq27at9u3bpyVLlujdd98NuXNSIiYmRnPmzFFubq7atWunMWPGqEmTJtq7d69Wr16tlJQULV++XNI3GXXz589Xamqqvv3tb2vt2rV68803lZaWVuE5hxsDsmDBAqWlpalr165l1gcOHKgXXnhB//u//6shQ4ZUeD633HKLli1bpn79+mno0KF67bXXPKN1fv/73ysrK0vt27fX/fffr5YtW+rAgQNau3atvvzyS23ZssX3+Vq1aqWsrCz9+Mc/1tmzZzVjxgylpaXpkUceCS7z9NNPKzc3V5mZmRo7dmwwfis1NVVTp04NLjd9+nStWrVKWVlZeuCBB1SjRg3Nnj1bZ8+e1VNPPRVcrkOHDoqNjdWTTz6pwsJCxcXFBf9oDu744osvNHDgQPXt21dr165VXl6e7rnnnuCFZfr06Xr00Ue1a9cuDRo0SLVr19YXX3yhV199VePGjdOkSZN8n6Njx46aNWuWpk+frlatWqlBgwbq1auXevfurWbNmmns2LH6j//4D8XGxuq//uu/VL9+ff3zn/8s13bccMMN6tOnT0j8liRNmzYtuEy4557yHNvjxo3T7NmzNXr0aG3YsEEtWrTQ0qVL9d5772nGjBnWT2SAyhDucf3II49o/vz56tu3rx566KFg/FbJpwqX6tevn2rXrq1JkyYpNjZWd955Z0i9vPFbhYWFysvLk/TNL3vbt2/XrFmzlJCQoJ/97GcV2p68vDzdfvvtmjhxYjB+q1mzZjpy5AifGlaWy5qRUIl2795tRo4caerXr2/i4uJMy5Ytzfjx483Zs2eNMaXjt0ps2rTJDBkyxKSlpZm4uDjTvHlzM2zYMPPWW28Flzl69KgZM2aMueaaa0xycrLp06ePyc/PN82bNzejRo0KLlcS7XFpDFhZzx1ODMiBAwdMjRo1zPe//33PZYqKikxiYmIwsqckHujQoUMhy5UVD6SL4rdKLFu2zNSoUcMMHz7cXLhwocz4LWOM+eyzz8zIkSNNo0aNTM2aNU2TJk1M//79zdKlSz3nasy/In2efvpp88wzz5hrr73WxMXFme7du5stW7aUWv7NN9803bp1MwkJCSYlJcUMGDDAbNu2rdRyGzduNH369DHJyckmMTHR9OzZ0/z9738vtdwLL7xgWrZsaWJjY4nickzJsb1t2zYzdOhQU7t2bVO3bl0zYcIEc/r06ZBlX3nlFZOVlWWSkpJMUlKSuf7668348ePN9u3bg8tkZ2ebdu3alflc+/fvN9/73vdM7dq1jaSQ+KoNGzaYLl26mFq1aplmzZqZ3/zmN57xW9/73vfKXH/Jey8vL8+0bt3axMXFme985zuljsdwzz3GeB/bl8ZvGfPNuaVkvbVq1TLt27cv9R6/+L1a1vynTJlS5rYBfspzXH/44YcmOzvbxMfHmyZNmphf/OIX5sUXX/SMq7r33nuDMZiXiiR+KxAImHr16pmBAweaDRs2VHh7Nm3aZLp3727i4uJM06ZNzRNPPGGeffZZI8ns37/fd17wFzCGb/ADqH6mTp2qadOm6dChQ7rmmmuqejoAUCkefvhhzZ49WydPnvT840qEz7nvyAIAALjg0r9BKSgo0Pz585WVlUUTW0mc/I4sAABAdZeZmamcnBy1bdtWBw4c0Isvvqjjx4975rKj/GhkAQAAoqBfv35aunSpnn/+eQUCAX33u9/Viy++qB49elT11K4YfEcWAAAATuI7sgAAAHASjSwAAACcRCMLAAAAJ4X9x178HyiAyne1fUX94v+LVXUyZcqUqK37atzm6srvtYhkn2RkZFR4bFX69NNPq3oKZWrTpk3U1n01bnN15fdahHON5I4sAAAAnEQjCwAAACfRyAIAAMBJNLIAAABwEo0sAAAAnEQjCwAAACeFHb8FAJI9wsgvvshWj2ZMVVVGTfk9d3WN57Kpzq+Vi/szUrYII79IJ1s9mjFVVRk15ffc1TWey6Y6v1bR3p/ckQUAAICTaGQBAADgJBpZAAAAOIlGFgAAAE6ikQUAAICTaGQBAADgJOK3AISIJL7Ib6wtWqkqI7L8RBI55qc6bzeqh0jii/zG2qKVqjIiy08kkWN+qvN2ozTuyAIAAMBJNLIAAABwEo0sAAAAnEQjCwAAACfRyAIAAMBJNLIAAABwEo0sAAAAnBQwxpiwFgwEoj0X4KoT5tuvWolmpircEEnWcHUWyfGbkZFRiTMpn2hmqsINkWQNV2fhXCO5IwsAAAAn0cgCAADASTSyAAAAcBKNLAAAAJxEIwsAAAAn0cgCAADASTSyAAAAcFKNqp4AAESbX+5pVebfVlUmayTb7Dc2km2K5rpRffjlnlZl/m1VZbJGss1+YyPZpmiuuzJwRxYAAABOopEFAACAk2hkAQAA4CQaWQAAADiJRhYAAABOopEFAACAk4jfAq5AtoiiSKOmqjKqKloiieeKZhzUlbiv/Vyp8Vq2iKJIo6aqMqoqWiKJ54pmHNSVuK/9VHW8lh/uyAIAAMBJNLIAAABwEo0sAAAAnEQjCwAAACfRyAIAAMBJNLIAAABwEo0sAAAAnESOLHAFuhrzR69EkeTbonq5GvNHr0SR5NsiOrgjCwAAACfRyAIAAMBJNLIAAABwEo0sAAAAnEQjCwAAACfRyAIAAMBJxG/hsomNjbXWi4uLPWvGmIieOy4uzrN29uxZ69hWrVpZ6zt37qzQnKLJFttEZFNp1XWfVOW8HnvsMc9ajRqRXTps2+UXOTZ9+nRr/ec//3mFnjfabLFNRDaVVl33STTnFRNjv7eYn58ftee+4YYbPGsff/xx1J63MvYnd2QBAADgJBpZAAAAOIlGFgAAAE6ikQUAAICTaGQBAADgJBpZAAAAOIlGFgAAAE4iR9ZBgUAgorotr7VJkybWsZmZmdb666+/7lk7deqUdWw0+WXF2tx5553W+pNPPlnhdUdLNPMyyagNdf78eWv9woUL1rot47h169bWsTt27LDWbfzyWiPNio3kuW389nd1PQajmT9KRu3lY9vXkvToo49a67Zr5ObNmysypUoRSVbs4MGDrfWtW7dWeN3h4I4sAAAAnEQjCwAAACfRyAIAAMBJNLIAAABwEo0sAAAAnEQjCwAAACfRyAIAAMBJ5MhegWw5sX66d+9urXfp0sVab9y4sWft2WefrdCcKkODBg08a3369LGOPX78eGVPx2lVldPplz1qm1dVZovacmL97Ny5sxJnUn2sWLHCs7Zu3Trr2OqaE1uVqior1i9T1TYvV/NtO3ToYK1nZWVZ67Zr5MSJEysypaj7xS9+Ya3Xrl3bWidHFgAAACgDjSwAAACcRCMLAAAAJ9HIAgAAwEk0sgAAAHASjSwAAACcFDDGmLAWDASiPReEqUYNe2ra+fPnrfVOnTp51gYMGGAde/r0aWu9SZMmnrWmTZtaxx45csSzlpCQYB27e/duaz0tLc2zlpKSYh375ZdfWuuTJk2y1m3CfPtVK7YYrKqMRvKL57Lxm7dt3YWFhdaxqamp1vrUqVM9a1V5fETznB/JdvnNy7Y/o3l8ZmRkRG3dfmwxWFUZc+UXz2XjN+9I1u3Hdk2YPXu2dWwk18hrr73WOvbll1/2rEV6jRw9erRnze8aOWvWLGv9+eef96z5vc7bt2+31iXuyAIAAMBRNLIAAABwEo0sAAAAnEQjCwAAACfRyAIAAMBJNLIAAABwEo0sAAAAnGQPJEWViYnx/h3DLyc2KSnJWr/rrrs8a2fPnrWOjY+Pt9Zr167tWfPLf7Rts9/Ydu3aWet79uzxrB09etQ61i+392pTVVmxVZUTK9nfF345sTt27LDWo5kVG60sWL85V2XueFVmGVeVqsqKdTUn1o8trzXSa+Q777zjWatbt651rO0aOXDgQOvYt99+21o/d+6cZ82W8S5JsbGx1nq0j0/uyAIAAMBJNLIAAABwEo0sAAAAnEQjCwAAACfRyAIAAMBJNLIAAABwEo0sAAAAnHTFB2Ta8gz9shBtmW1+4/3W7Ze7duHCBWvd5kc/+pG1vn//fs/amTNnrGNbtGhhrdsy9A4cOGAda9snxcXF1rGnTp2y1m0ZeSkpKdaxcXFx1rott9dvXhXll3tqy9KMZGxV8stvfPzxxyu87khzZm3y8vIqPNYvjzWSDNpo5tdGkjPrNzaa846EX+6pLUszkrGRiuS18JORkVHhdV9//fXWen5+foXmJEmzZ8+21m3XqqVLl1rHjh8/3lpPSEjwrHXu3Nk6NpJrZK9evaz1F154wbN2zz33WMfed9991vratWs9a0VFRdax4eCOLAAAAJxEIwsAAAAn0cgCAADASTSyAAAAcBKNLAAAAJxEIwsAAAAnORG/VVVRLX5xFjbRjNe6++67rfVGjRpZ6xs3bvSs1axZ0zq2Tp061npBQYFn7ciRI9ax11xzjWetdu3a1rF++9vGL2YtMTHRWm/durVnbfPmzRWZkq9IIrKqa7yWH78YtGiy7TO/iKxI9nek8Ue28X7zruh6q5ptu6ZOnWodG8lrFUlEVqTxWtU1zsy2br/zrt/117bPkpOTrWP9rpGbNm3yrGVnZ1vH1q1b11rPzMz0rL344ovWsbZr5KhRo6xjI7lG+o31u0Zu377ds3b69OkKzeli3JEFAACAk2hkAQAA4CQaWQAAADiJRhYAAABOopEFAACAk2hkAQAA4CQaWQAAADjJiRzZSHLubFl1fjl2flmvtnlFkhMrSWPGjPGsZWRkWMfu2bPHWrdl0fllSyYkJFjre/fu9az5ZcHacgOLioqsY+Pj4631SHIW/fTp08ezFq0c2atRVebfRpK5Gs11RzMD1C9ztbqy7ZNp06ZdxplcPlV1jfS7FtnmFUlOuyTVr1/fs+aXy/vPf/7TWl+7dq1nze/92rt3b2vdlrXul3/bqlUrz1qk10ibSM8zzz33nGdt7NixEa1b4o4sAAAAHEUjCwAAACfRyAIAAMBJNLIAAABwEo0sAAAAnEQjCwAAACfRyAIAAMBJlyVH1i+Lzo8tw8wv082WVRdpjp1N48aNrfUhQ4ZY67a81h07dljH+mXRxcXFedbS0tKsY8+dO2et216rxMRE61gbv1zes2fPVnj8qVOnrGP9jpNu3bpZ69Hgl4dpy1ydPHlyRM/9+OOPV3jdtmPvSs2JjYRffmMkuajRzImtrvszmj799FNr3ZZt6jfWjy3Pdfv27RVer18eayT8rpG1atWy1iO5Rnbq1Mlat2Wujho1yjr2o48+stYPHjzoWfvOd75jHWvjd41csmSJtW67zp08ebLCYyUpKyvLWo8Ud2QBAADgJBpZAAAAOIlGFgAAAE6ikQUAAICTaGQBAADgJBpZAAAAOCns+K3Y2Fhr3Rb9EM2YK794Gpv69etb682bN7fWr7/+es/at771LetYvxir48ePe9bq1KljHZuSkmKt16xZ07Nmi0eS/F9L2z6zPa8kHTt2zLP29ddfRzQvWwTc6dOnrWP9jv0TJ0541tq1a2cdW1F+kWG2WCa/19iPbd22aC6/sa6K5Bzkxy/GKpoRWjbRjNfyW3ck+7sqI94ijdiysUVs+UVo2ebld41s0aKFtW6LBfO7Rv75z3+21m3XyO9///vWsX7XyLp163rWVq1aZR3rF6EVrWuk3/7yu47ZrpF+0V0jRoyw1m3XyMGDB1vHhoM7sgAAAHASjSwAAACcRCMLAAAAJ9HIAgAAwEk0sgAAAHASjSwAAACcRCMLAAAAJ4WdI2vLifXTsGFDa90vrzUpKalCNUlKSEjwrKWnp1vHJiYmWuu2bNOTJ09ax9oy2yQpNTXVs2bbJkk6f/68tW7brqKiIuvYs2fPWuu1atXyrO3bt8861rbNfq/F0aNHrfXk5GTPmi0zUPLPbG3UqJFnLS0tzTq2ovyOe5vly5db6xs2bLDWI8nxtOWeRjPjszrnnkYzC9a2TyPZJ1WZnXs1sp0bJXuOp9+5Mz4+3rM2ZswY69hIrpH//d//bR0byTXynXfesY7t16+ftW7L//bLgD9y5Ii13r59e8/a3LlzrWNt2zx8+PCI5mW7Rr777rsVHivZe8DKuEZyRxYAAABOopEFAACAk2hkAQAA4CQaWQAAADiJRhYAAABOopEFAACAk2hkAQAA4KSwc2T93HbbbZ61xo0bW8fasuYkqUGDBp41v6y54uLiCj/viRMnrHVbdpotW1Tyz0qMi4vzrPllpvrtE9u8Y2NjrWP9MlVt+6ywsNA61vY6R8q2z2zHiOSf22vLzvXL9K2oSDJXo5lbWpWimT9qy02tytxTv9fSxdfaL6O2uubftmnTxlq/9dZbPWsbN260jvU7j9x4442eNb9zen5+foWf1+8aactz9btG+hk6dKhnbf78+dax0bxGbtq0yVr/05/+5FmL5BoZ6XnIljPbtWtX69j/+Z//sdZbtmzpWauMayR3ZAEAAOAkGlkAAAA4iUYWAAAATqKRBQAAgJNoZAEAAOAkGlkAAAA4Kez4rd69e1vrY8eO9azZ4j0kad++fdb68ePHPWt+URjnzp2r8Fg/tugRWySTJF24cMFaT0lJ8az5xWz4xUXZ4qZq1qxpHesXmdKwYUPPWrt27axjbc8d6Wtliw1LTEy0jj1z5kyF133w4EH7xKIkmpFQtnX7xRtVVYxVpLFLkcxt2rRp1no0I7Ii2e5IttlvbHWNM4uE3zXSdm60XeMk6auvvrLWFy5c6Fm74447rGNt0Urz5s2zjvV7rWzn9CFDhljH+sUizpkzx1q32blzp7Vui6KyRWNK/tfIH/zgB9a6zRtvvOFZa9WqlXXsJ598Yq336tXLs/bWW29ZxzZp0sRaP3nypGftwIED1rHh4I4sAAAAnEQjCwAAACfRyAIAAMBJNLIAAABwEo0sAAAAnEQjCwAAACfRyAIAAMBJYefIfvDBB9b6Lbfc4llr3769dWy3bt3CnUYp58+ft9ZtWa+2rLhw6oWFhZ41vxxZv/y9tLQ0z1pGRoZ1rF8uqi2j1i938qabbrLWP/zwQ8/arl27rGNvu+02z5pfdl8keZl+x9DevXutdVsGZHJycoXm5Kdfv34VHhvNTNVI8kOvVNHMifVbdySZvy+//HKF1hsOV7NibZ577jlrfcSIEZ61yZMnW8fWrVvXWl++fLlnze/81rdvX8/asWPHrGP9rpG28X7bFBNjv89myy6N9Br57W9/27Pm97656667rPWlS5d61vx6pTFjxnjW/HJib775Zmvd1it997vftY6N5Bpp60fCxR1ZAAAAOIlGFgAAAE6ikQUAAICTaGQBAADgJBpZAAAAOIlGFgAAAE4KmDDzcKIZl+IXUdSlSxfPWps2baxju3bt6llr0KCBdaxfLERSUpJnLdIYouLiYs+aX+RJfn6+tb5q1SrP2uuvv24de+bMGWs9En/5y188a82aNbOOPXz4sLVuixax1ST/+JqzZ8961iZNmmQde/LkSWvdy7Rp0yo0TpKmTJlS4bFVKZJzUDQjxyJVXSPJrsSIrHvvvddaz8vLq/C6I9lfftexSK6RflFUtvjL+vXrW8euWbPGWrddIwcOHGgd6/e+WLZsmWct0mtkfHy8Z23FihXWsdG8Ro4bN86z5uo18re//a11bDjnR+7IAgAAwEk0sgAAAHASjSwAAACcRCMLAAAAJ9HIAgAAwEk0sgAAAHASjSwAAACcVC1yZIGrVUUzRK/GHFm/bY5ku6K5bldFcoy5KpLXOZo5stXVp59+aq1Hsl3RXLer/PbJlYgcWQAAAFyxaGQBAADgJBpZAAAAOIlGFgAAAE6ikQUAAICTaGQBAADgJBpZAAAAOKlGVU8AQPldjbmmkbgaM1H9sE8q19WYaxqJqzET1Q/7pGK4IwsAAAAn0cgCAADASTSyAAAAcBKNLAAAAJxEIwsAAAAn0cgCAADAScRvAQ6qrtFJkcSC+W0TkWPlV12PkytRdY1OiiQWzG+biBwrv+p6nLiMO7IAAABwEo0sAAAAnEQjCwAAACfRyAIAAMBJNLIAAABwEo0sAAAAnEQjCwAAACcFjDEmrAUDgWjPBbjqhPn2gyLLRPXLoI0kw/ZKzWolt7d8MjIyqnoKl10kmah+GbSRZNheqVmtV2Nu7/bt232X4Y4sAAAAnEQjCwAAACfRyAIAAMBJNLIAAABwEo0sAAAAnEQjCwAAACfRyAIAAMBJ5MgCVYgc2X+JZh4rmajlF83c3ivRlZgjG8081qsxEzVS0cztra7IkQUAAMAVi0YWAAAATqKRBQAAgJNoZAEAAOAkGlkAAAA4iUYWAAAATqpR1RMAUPls0UlXYzQSys92nEQzKi2a/OYdzfeGLTrJ1WgkXF624ySaUWnRVBnz5o4sAAAAnEQjCwAAACfRyAIAAMBJNLIAAABwEo0sAAAAnEQjCwAAACfRyAIAAMBJAWOMqepJAAAAAOXFHVkAAAA4iUYWAAAATqKRBQAAgJNoZAEAAOAkGlkAAAA4iUYWAAAATqKRBQAAgJNoZAEAAOAkGlkAAAA46f8BRlpg/RWOFZEAAAAASUVORK5CYII=",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# viz: clean vs adversarial image and the perturbation\n",
"fig, ax = plt.subplots(1, 3, figsize=(7, 2.6))\n",
"ax[0].imshow(x0.detach().squeeze().numpy(), cmap=\"gray\"); ax[0].set_title(f\"clean: {CLASSES[orig_pred]}\")\n",
"ax[1].imshow((x_adv - x0.detach()).squeeze().numpy(), cmap=\"gray\"); ax[1].set_title(\"perturbation\")\n",
"ax[2].imshow(x_adv.squeeze().numpy(), cmap=\"gray\"); ax[2].set_title(f\"adv: {CLASSES[adv_pred]}\")\n",
"for a in ax: a.axis(\"off\")\n",
"plt.tight_layout(); plt.show()"
]
},
{
"cell_type": "markdown",
"id": "e6eda545",
"metadata": {},
"source": [
"> **Interpretation.** Whether or not this particular image flips at `epsilon=0.15` (it often does on a lightly-trained model), the perturbation panel shows the attack is structured noise, not random: it concentrates on the pixels the model leans on. Habits worth keeping: generate at least one adversarial example for any model you deploy; check that your first conv layer's filters look like edge detectors, not noise; and test on stylized or color-jittered inputs, because ImageNet CNNs classify by texture more than shape (Geirhos et al. 2019) and quietly fail on distribution shift. The same gradient-on-the-input move underlies image prompt-injection against vision-language models, treated in Ch 24.\n"
]
},
{
"cell_type": "markdown",
"id": "c49a0a32",
"metadata": {},
"source": [
"## Test yourself\n",
"\n",
"Three parts: concept self-checks with folded answers, two auto-checked problems, and a capstone with a rubric and a folded reference. Every answer is in this notebook; if unsure, re-run that section.\n"
]
},
{
"cell_type": "markdown",
"id": "a46ad5a7",
"metadata": {},
"source": [
"### Part A — Concepts\n",
"\n",
"1. What does deep-learning \"convolution\" actually compute, and how does it differ from the textbook convolution? AnswerCross-correlation: the kernel slides and dots against the patch with no flip. Textbook convolution flips the kernel first. The network learns whatever orientation it needs, so the flip is wasted work and PyTorch skips it.\n",
"2. A `nn.Conv2d(16, 32, kernel_size=3, stride=2, padding=1)` runs on a `(N, 16, 28, 28)` input. What is the output spatial size? Answer`floor((28 - 3 + 2*1)/2) + 1 = floor(27/2) + 1 = 13 + 1 = 14`, so `(N, 32, 14, 14)`. Stride 2 with same-style padding halves the spatial dims.\n",
"3. In Part 1 you asserted `nn.Conv2d` equals `corr2d`. What single bug would break that assert, and what would the symptom be? AnswerFlipping the kernel (true convolution). The output would be the corr2d result computed with `K` reversed, so it would disagree everywhere the kernel is not symmetric. The assert in cell 1.2 is the guard.\n",
"4. Look at the FashionMNIST CNN's layer summary you printed in Part 4. Which layer holds the most parameters, and why is that the historical motivation for global average pooling? Answer`Linear(784, 64)` with ~50k parameters dwarfs both convs. Classic CNNs put most weight in the FC head. Global average pooling replaces that head with a single small classifier, cutting parameters ~10x with no accuracy hit, which is what ResNet does.\n",
"5. The deliberate-failure run in Part 4 produced ~10% accuracy with no error. What was the bug and why is a silent 10% worse than a crash? AnswerThe missing `opt.zero_grad()` let gradients accumulate across steps, so the update direction was wrong. A crash tells you where to look; a silent 10% model passes every type check and only the accuracy number reveals it, which is why you always compare against the chance baseline.\n",
"6. Why must you compute the receptive-field growth with the *current* jump before updating the jump? AnswerA layer's filter spans `(k-1)` steps at the resolution it sees, which is the jump accumulated by all *earlier* strides, not including its own. Updating jump first would double-count this layer's stride and overstate the receptive field (you would get 4 instead of 5 for two 3x3 convs).\n",
"7. Two stacked 3x3 convs vs one 5x5 conv: same receptive field. Give two reasons VGG prefers the stack. AnswerFewer parameters (`2*9=18` vs `25` per channel pair) and an extra nonlinearity between the two convs, which adds representational power. The cost is more activation memory and compute.\n"
]
},
{
"cell_type": "markdown",
"id": "5c6dac65",
"metadata": {},
"source": [
"### Part B — Auto-checked problems\n"
]
},
{
"cell_type": "markdown",
"id": "14c16fef",
"metadata": {},
"source": [
"**Problem B.1 — `same_padding(k)`** · `Difficulty 1/5 · ~5 min`\n",
"\n",
"For an odd kernel size `k` and stride 1, return the padding that keeps the spatial size unchanged. Then prove it: the check builds a `Conv2d` with your padding and asserts the output size equals the input size for `k` in `{1, 3, 5, 7}`.\n"
]
},
{
"cell_type": "code",
"execution_count": 37,
"id": "36332e30",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:25:21.012121Z",
"iopub.status.busy": "2026-06-10T19:25:21.012030Z",
"iopub.status.idle": "2026-06-10T19:25:21.022489Z",
"shell.execute_reply": "2026-06-10T19:25:21.021875Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] B.1 same_padding: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 37,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def same_padding(k):\n",
" \"\"\"Padding that preserves spatial size for odd kernel k at stride 1.\"\"\"\n",
" # TODO: for odd k at stride 1, P = (k - 1) // 2 keeps the size\n",
" p = None\n",
" attempted(p)\n",
" return p\n",
"\n",
"def _check_same_pad():\n",
" for k in (1, 3, 5, 7):\n",
" p = same_padding(k)\n",
" conv = nn.Conv2d(1, 1, kernel_size=k, padding=p)\n",
" out = conv(torch.zeros(1, 1, 20, 20)).shape[-1]\n",
" assert out == 20, f\"k={k}, padding={p} gave size {out}, expected 20 (same padding)\"\n",
"\n",
"check(\"B.1 same_padding\", _check_same_pad)"
]
},
{
"cell_type": "markdown",
"id": "6b752d0e",
"metadata": {},
"source": [
"Hint 1Derive it from the output formula with stride 1 and `H_out = H_in`: `H = H - k + 2P + 1`, so `P = (k-1)/2`. For odd `k` this is an integer.\n",
"Solution\n",
"\n",
"```python\n",
"def same_padding(k):\n",
" return (k - 1) // 2\n",
"```\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 38,
"id": "2974af61",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:25:21.023472Z",
"iopub.status.busy": "2026-06-10T19:25:21.023384Z",
"iopub.status.idle": "2026-06-10T19:25:21.031839Z",
"shell.execute_reply": "2026-06-10T19:25:21.028977Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] B.1 same_padding\n",
"same_padding: {1: 0, 3: 1, 5: 2, 7: 3}\n"
]
}
],
"source": [
"# solution\n",
"def same_padding(k):\n",
" return (k - 1) // 2\n",
"\n",
"check(\"B.1 same_padding\", _check_same_pad, required=True)\n",
"print(\"same_padding:\", {k: same_padding(k) for k in (1, 3, 5, 7)})"
]
},
{
"cell_type": "markdown",
"id": "33d0f7c0",
"metadata": {},
"source": [
"**Problem B.2 — A 1x1 conv is a per-pixel linear layer** · `Difficulty 3/5 · ~15 min`\n",
"\n",
"A 1x1 convolution mixes channels at each spatial location independently: it is a linear layer applied to the channel vector at every pixel. Prove this property. Fill in `apply_1x1_as_matmul(x, conv)` that reproduces a given `nn.Conv2d(C_in, C_out, kernel_size=1)`'s output using only a matrix multiply over channels (no conv call). The check asserts your matmul version matches the real conv to `1e-5`, which is the property that 1x1 conv == per-pixel matmul.\n"
]
},
{
"cell_type": "code",
"execution_count": 39,
"id": "f5e9748e",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:25:21.032792Z",
"iopub.status.busy": "2026-06-10T19:25:21.032709Z",
"iopub.status.idle": "2026-06-10T19:25:21.040794Z",
"shell.execute_reply": "2026-06-10T19:25:21.040522Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ -- ] B.2 1x1 conv == matmul: not attempted yet — fill in the TODO above, then re-run.\n"
]
},
{
"data": {
"text/plain": [
"False"
]
},
"execution_count": 39,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def apply_1x1_as_matmul(x, conv):\n",
" \"\"\"x: (N, C_in, H, W). conv: nn.Conv2d(C_in, C_out, kernel_size=1).\n",
" Reproduce conv(x) with a matmul over channels, no convolution.\"\"\"\n",
" N, C_in, H, W = x.shape\n",
" W_mat = conv.weight.detach().view(conv.out_channels, C_in) # (C_out, C_in)\n",
" b = conv.bias.detach() # (C_out,)\n",
" # TODO 1: reshape x to (N, C_in, H*W) so each column is one pixel's channel vector\n",
" # TODO 2: for each pixel, out_pixel = W_mat @ in_pixel + b (batched matmul)\n",
" # result shape (N, C_out, H*W); use torch.einsum or W_mat @ ...\n",
" out = None\n",
" attempted(out)\n",
" # TODO 3: reshape back to (N, C_out, H, W)\n",
" raise NotImplementedError\n",
"\n",
"def _check_1x1():\n",
" torch.manual_seed(SEED)\n",
" conv = nn.Conv2d(4, 6, kernel_size=1)\n",
" x = torch.randn(2, 4, 5, 5)\n",
" ref = conv(x)\n",
" got = apply_1x1_as_matmul(x, conv)\n",
" check_close(got.detach().numpy(), ref.detach().numpy(), atol=1e-5,\n",
" msg=\"1x1 conv must equal a per-pixel matmul W @ channel_vector + b\")\n",
"\n",
"check(\"B.2 1x1 conv == matmul\", _check_1x1)"
]
},
{
"cell_type": "markdown",
"id": "b8782f2d",
"metadata": {},
"source": [
"Hint 1 (conceptual)Flatten the spatial dims into one axis so `x` is `(N, C_in, P)` with `P = H*W`. The weight is `(C_out, C_in)`. A 1x1 conv computes, for every pixel, `W_mat @ channel_vector + bias`. `torch.einsum(\"oc,ncp->nop\", W_mat, x_flat)` does all pixels at once.\n",
"\n",
"Hint 2 (pseudocode)\n",
"\n",
"```python\n",
"x_flat = x.reshape(N, C_in, H * W) # (N, C_in, P)\n",
"out = torch.einsum(\"oc,ncp->nop\", W_mat, x_flat) # (N, C_out, P)\n",
"out = out + b.view(1, -1, 1)\n",
"return out.reshape(N, conv.out_channels, H, W)\n",
"```\n",
"\n",
"\n",
"Help — shapes line up but values are off by the biasAdd the bias broadcast over channels: `b.view(1, -1, 1)` before reshaping back. Forgetting the bias makes every value off by a per-channel constant.\n"
]
},
{
"cell_type": "code",
"execution_count": 40,
"id": "68e3f88b",
"metadata": {
"execution": {
"iopub.execute_input": "2026-06-10T19:25:21.041785Z",
"iopub.status.busy": "2026-06-10T19:25:21.041708Z",
"iopub.status.idle": "2026-06-10T19:25:21.046369Z",
"shell.execute_reply": "2026-06-10T19:25:21.045863Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ ok ] B.2 1x1 conv == matmul\n",
"verified: a 1x1 conv is a per-pixel linear layer over channels\n"
]
}
],
"source": [
"# solution\n",
"def apply_1x1_as_matmul(x, conv):\n",
" N, C_in, H, W = x.shape\n",
" W_mat = conv.weight.detach().view(conv.out_channels, C_in)\n",
" b = conv.bias.detach()\n",
" x_flat = x.reshape(N, C_in, H * W) # (N, C_in, P)\n",
" out = torch.einsum(\"oc,ncp->nop\", W_mat, x_flat) # (N, C_out, P)\n",
" out = out + b.view(1, -1, 1)\n",
" return out.reshape(N, conv.out_channels, H, W)\n",
"\n",
"check(\"B.2 1x1 conv == matmul\", _check_1x1, required=True)\n",
"print(\"verified: a 1x1 conv is a per-pixel linear layer over channels\")"
]
},
{
"cell_type": "markdown",
"id": "aee6f1dd",
"metadata": {},
"source": [
"### Part C — Capstone: a deeper CNN with global average pooling\n",
"\n",
"Build and train a slightly deeper CNN on FashionMNIST that replaces the big FC head with global average pooling, the modern design. Deliverables:\n",
"\n",
"1. An architecture with three conv blocks (channels growing `16 -> 32 -> 64`), each `conv -> ReLU -> 2x2 pool`, then `AdaptiveAvgPool2d((1,1))` -> `Flatten` -> a single `Linear(64, 10)` head.\n",
"2. A layer summary (use `layer_summary`) showing the spatial size halving each block and the head reaching `(1, 10)`.\n",
"3. The total parameter count, and a one-line comparison to the Part 4 CNN's count (global average pooling should make it *smaller* despite being deeper).\n",
"4. A training run reaching test accuracy above the Part 4 model's, or an honest note on why it did not at this budget.\n",
"\n",
"Self-assessment (pass / partial / fail): (a) the layer summary ends at `(1, 10)`; (b) parameters are fewer than the Part 4 CNN despite more conv layers; (c) the model trains above chance and you report the accuracy; (d) you can name where the parameters went (the convs, not a fat FC head); (e) the notebook still runs top to bottom.\n",
"\n",
"My solution (reference)\n",
"\n",
"```python\n",
"def make_gap_cnn():\n",
" torch.manual_seed(SEED)\n",
" return nn.Sequential(\n",
" nn.Conv2d(1, 16, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2), # -> (16,14,14)\n",
" nn.Conv2d(16, 32, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2), # -> (32,7,7)\n",
" nn.Conv2d(32, 64, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2), # -> (64,3,3)\n",
" nn.AdaptiveAvgPool2d((1, 1)), # -> (64,1,1)\n",
" nn.Flatten(), # -> (64,)\n",
" nn.Linear(64, 10), # -> (10,)\n",
" )\n",
"\n",
"gap_cnn = make_gap_cnn()\n",
"layer_summary(gap_cnn, (1, 28, 28))\n",
"print(\"GAP CNN params:\", f\"{n_params(gap_cnn):,}\", \"vs Part 4 CNN:\", f\"{n_params(make_cnn()):,}\")\n",
"losses = train_cnn(gap_cnn, TRAIN_BATCHES)\n",
"print(\"GAP CNN test accuracy:\", f\"{accuracy(gap_cnn, test_loader):.1%}\")\n",
"```\n",
"\n",
"The GAP model has roughly `(16*9+16) + (32*16*9+32) + (64*32*9+64) + (64*10+10)` ~= 24k parameters, well under the Part 4 CNN's ~58k, because the `AdaptiveAvgPool2d` collapses each channel to one number before the classifier, so there is no `784->64` head. Whether it beats the Part 4 accuracy depends on the training budget; at the FAST budget it may not, and saying so is the honest answer the rubric rewards.\n"
]
},
{
"cell_type": "markdown",
"id": "ada6647a",
"metadata": {},
"source": [
"## Reflection\n",
"\n",
"Write ~150 words on the dumbest bug you hit in this notebook and how you found it. Common candidates: a kernel flip that broke the `corr2d`-vs-`nn.Conv2d` assert; using `H` twice in `conv_out_shape` so only the non-square case failed; the wrong flatten dimension (`16*14*14` instead of `16*7*7`); adding the gradient instead of subtracting it in the kernel-recovery loop; or forgetting `model.eval()` and watching the checkpoint check disagree. Nobody grades this. Writing it is the point: the act of naming the bug and the diagnostic that caught it is what turns a one-time fix into a habit you keep. The best engineers are not the ones who avoid these bugs; they are the ones who recognize the symptom in five seconds because they have written it down before.\n"
]
},
{
"cell_type": "markdown",
"id": "52c61f7a",
"metadata": {},
"source": [
"## Going further\n",
"\n",
"- d2l.ai, *Convolutional Neural Networks* and *Modern CNNs* chapters: the conv-layer, padding/stride, pooling, LeNet, and the AlexNet -> VGG -> GoogLeNet -> ResNet lineage, with code that mirrors this notebook's from-scratch path.\n",
"- ARENA 3.0, *CNNs & ResNets* (Callum McDougall): the most rigorous from-scratch CNN curriculum online; builds ResNet-34 module by module with reference tests, the natural next step after this notebook.\n",
"- Olah, *Understanding Convolutions* and the Distill *Circuits* thread (Zoom In, Early Vision, Curve Detectors): what a trained CNN actually detects, labeled by hand. The proof-of-concept for the mech-interp techniques later applied to transformers.\n",
"- fastbook ch13-14: convolutions, pooling, BatchNorm, and ResNet with skip connections, plus class activation maps for making a prediction visible to a user.\n",
"- Karpathy, *Recurse Center: reproducing LeCun 1989*: re-running the 1989 ConvNet paper on modern hardware. Short and clarifying.\n",
"- lucidrains, *vit-pytorch*: a clean 200-line Vision Transformer; the patch embedding is a single strided conv, the bridge to Ch 15.\n",
"\n",
"## What this enables\n",
"\n",
"- **Ch 13 — Sequences and Time Series**: 1D convolutions are the WaveNet trick for sequences; the output-shape arithmetic transfers directly.\n",
"- **Ch 15 — Transformers from Scratch**: the ViT patch embedding is one strided `nn.Conv2d(3, d_model, kernel_size=16, stride=16)`; everything after it is the transformer.\n",
"- **Ch 16 — Vision Transformers**: a tiny ViT on FashionMNIST patches, built on the conv and shape machinery here.\n",
"- **Ch 22 — Mech-Interp**: the Circuits work on vision is the original demonstration of the feature-visualization and ablation techniques used on language models.\n",
"\n",
"This notebook stopped at a LeNet-shaped CNN on a subset of FashionMNIST. The draft's full target was a from-scratch ResNet-18 reaching >93% on CIFAR-10 in under 30 minutes on a T4. That run wants a GPU and is the chapter's capstone-at-scale; the residual block, `make_layer`, and the FLOP accounting you would need are all foreshadowed here, and the skip-connection idea is what Ch 22 later rhymes with transformer residual streams.\n"
]
},
{
"cell_type": "markdown",
"id": "7e9bea7f",
"metadata": {},
"source": [
"---\n",
"*Built top-to-bottom. If every check above printed `[ ok ]`, you've reproduced the chapter. Total running time and verified-on stamp written by CI.*\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "obvix-nb",
"language": "python",
"name": "obvix-nb"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
},
"obvix": {
"title": "Ch 12 — CNNs & Computer Vision"
}
},
"nbformat": 4,
"nbformat_minor": 5
}