Files
Python/d2l/d2l-zh/pytorch/chapter_optimization/sgd.ipynb
T
2025-12-16 09:23:53 +08:00

3651 lines
235 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"cells": [
{
"cell_type": "markdown",
"id": "e7304c91",
"metadata": {
"origin_pos": 0
},
"source": [
"# 随机梯度下降\n",
":label:`sec_sgd`\n",
"\n",
"在前面的章节中,我们一直在训练过程中使用随机梯度下降,但没有解释它为什么起作用。为了澄清这一点,我们刚在 :numref:`sec_gd`中描述了梯度下降的基本原则。本节继续更详细地说明*随机梯度下降*stochastic gradient descent)。\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "31394483",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T06:58:28.061787Z",
"iopub.status.busy": "2023-08-18T06:58:28.060862Z",
"iopub.status.idle": "2023-08-18T06:58:30.450459Z",
"shell.execute_reply": "2023-08-18T06:58:30.447423Z"
},
"origin_pos": 2,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import math\n",
"import torch\n",
"from d2l import torch as d2l"
]
},
{
"cell_type": "markdown",
"id": "cedec3af",
"metadata": {
"origin_pos": 5
},
"source": [
"## 随机梯度更新\n",
"\n",
"在深度学习中,目标函数通常是训练数据集中每个样本的损失函数的平均值。给定$n$个样本的训练数据集,我们假设$f_i(\\mathbf{x})$是关于索引$i$的训练样本的损失函数,其中$\\mathbf{x}$是参数向量。然后我们得到目标函数\n",
"\n",
"$$f(\\mathbf{x}) = \\frac{1}{n} \\sum_{i = 1}^n f_i(\\mathbf{x}).$$\n",
"\n",
"$\\mathbf{x}$的目标函数的梯度计算为\n",
"\n",
"$$\\nabla f(\\mathbf{x}) = \\frac{1}{n} \\sum_{i = 1}^n \\nabla f_i(\\mathbf{x}).$$\n",
"\n",
"如果使用梯度下降法,则每个自变量迭代的计算代价为$\\mathcal{O}(n)$,它随$n$线性增长。因此,当训练数据集较大时,每次迭代的梯度下降计算代价将较高。\n",
"\n",
"随机梯度下降(SGD)可降低每次迭代时的计算代价。在随机梯度下降的每次迭代中,我们对数据样本随机均匀采样一个索引$i$,其中$i\\in\\{1,\\ldots, n\\}$,并计算梯度$\\nabla f_i(\\mathbf{x})$以更新$\\mathbf{x}$\n",
"\n",
"$$\\mathbf{x} \\leftarrow \\mathbf{x} - \\eta \\nabla f_i(\\mathbf{x}),$$\n",
"\n",
"其中$\\eta$是学习率。我们可以看到,每次迭代的计算代价从梯度下降的$\\mathcal{O}(n)$降至常数$\\mathcal{O}(1)$。此外,我们要强调,随机梯度$\\nabla f_i(\\mathbf{x})$是对完整梯度$\\nabla f(\\mathbf{x})$的无偏估计,因为\n",
"\n",
"$$\\mathbb{E}_i \\nabla f_i(\\mathbf{x}) = \\frac{1}{n} \\sum_{i = 1}^n \\nabla f_i(\\mathbf{x}) = \\nabla f(\\mathbf{x}).$$\n",
"\n",
"这意味着,平均而言,随机梯度是对梯度的良好估计。\n",
"\n",
"现在,我们将把它与梯度下降进行比较,方法是向梯度添加均值为0、方差为1的随机噪声,以模拟随机梯度下降。\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "b3ab49ac",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T06:58:30.457362Z",
"iopub.status.busy": "2023-08-18T06:58:30.456559Z",
"iopub.status.idle": "2023-08-18T06:58:30.463210Z",
"shell.execute_reply": "2023-08-18T06:58:30.461955Z"
},
"origin_pos": 6,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"def f(x1, x2): # 目标函数\n",
" return x1 ** 2 + 2 * x2 ** 2\n",
"\n",
"def f_grad(x1, x2): # 目标函数的梯度\n",
" return 2 * x1, 4 * x2"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "4f115cb7",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T06:58:30.468107Z",
"iopub.status.busy": "2023-08-18T06:58:30.467724Z",
"iopub.status.idle": "2023-08-18T06:58:30.484992Z",
"shell.execute_reply": "2023-08-18T06:58:30.481595Z"
},
"origin_pos": 7,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"def sgd(x1, x2, s1, s2, f_grad):\n",
" g1, g2 = f_grad(x1, x2)\n",
" # 模拟有噪声的梯度\n",
" g1 += torch.normal(0.0, 1, (1,)).item()\n",
" g2 += torch.normal(0.0, 1, (1,)).item()\n",
" eta_t = eta * lr()\n",
" return (x1 - eta_t * g1, x2 - eta_t * g2, 0, 0)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "f9f129c4",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T06:58:30.492182Z",
"iopub.status.busy": "2023-08-18T06:58:30.491793Z",
"iopub.status.idle": "2023-08-18T06:58:30.693380Z",
"shell.execute_reply": "2023-08-18T06:58:30.692264Z"
},
"origin_pos": 9,
"tab": [
"pytorch"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 50, x1: 0.020569, x2: 0.227895\n"
]
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"245.120313pt\" height=\"180.65625pt\" viewBox=\"0 0 245.120313 180.65625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T06:58:30.650679</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 180.65625 \n",
"L 245.120313 180.65625 \n",
"L 245.120313 0 \n",
"L 0 0 \n",
"L 0 180.65625 \n",
"z\n",
"\" style=\"fill: none\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 237.920313 143.1 \n",
"L 237.920313 7.2 \n",
"L 42.620312 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path id=\"m77e0f57915\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m77e0f57915\" x=\"88.39375\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(81.022656 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
"L 4684 2272 \n",
"L 4684 1741 \n",
"L 678 1741 \n",
"L 678 2272 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
"L 825 1625 \n",
"L 2419 1625 \n",
"L 2419 4116 \n",
"z\n",
"M 2253 4666 \n",
"L 3047 4666 \n",
"L 3047 1625 \n",
"L 3713 1625 \n",
"L 3713 1100 \n",
"L 3047 1100 \n",
"L 3047 0 \n",
"L 2419 0 \n",
"L 2419 1100 \n",
"L 313 1100 \n",
"L 313 1709 \n",
"L 2253 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-34\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use xlink:href=\"#m77e0f57915\" x=\"149.425\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(142.053907 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use xlink:href=\"#m77e0f57915\" x=\"210.456251\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(207.275001 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- x1 -->\n",
" <g transform=\"translate(134.129687 171.376563)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"59.179688\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_4\">\n",
" <defs>\n",
" <path id=\"m953b37fac9\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m953b37fac9\" x=\"42.620312\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 3 -->\n",
" <g transform=\"translate(20.878125 146.899219)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
"Q 3050 2419 3304 2112 \n",
"Q 3559 1806 3559 1356 \n",
"Q 3559 666 3084 287 \n",
"Q 2609 -91 1734 -91 \n",
"Q 1441 -91 1130 -33 \n",
"Q 819 25 488 141 \n",
"L 488 750 \n",
"Q 750 597 1062 519 \n",
"Q 1375 441 1716 441 \n",
"Q 2309 441 2620 675 \n",
"Q 2931 909 2931 1356 \n",
"Q 2931 1769 2642 2001 \n",
"Q 2353 2234 1838 2234 \n",
"L 1294 2234 \n",
"L 1294 2753 \n",
"L 1863 2753 \n",
"Q 2328 2753 2575 2939 \n",
"Q 2822 3125 2822 3475 \n",
"Q 2822 3834 2567 4026 \n",
"Q 2313 4219 1838 4219 \n",
"Q 1578 4219 1281 4162 \n",
"Q 984 4106 628 3988 \n",
"L 628 4550 \n",
"Q 988 4650 1302 4700 \n",
"Q 1616 4750 1894 4750 \n",
"Q 2613 4750 3031 4423 \n",
"Q 3450 4097 3450 3541 \n",
"Q 3450 3153 3228 2886 \n",
"Q 3006 2619 2597 2516 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-33\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use xlink:href=\"#m953b37fac9\" x=\"42.620312\" y=\"108.253846\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(20.878125 112.053065)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#m953b37fac9\" x=\"42.620312\" y=\"73.407692\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(20.878125 77.206911)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_7\">\n",
" <g>\n",
" <use xlink:href=\"#m953b37fac9\" x=\"42.620312\" y=\"38.561538\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(29.257812 42.360757)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- x2 -->\n",
" <g transform=\"translate(14.798437 81.290625)rotate(-90)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"59.179688\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <path d=\"M 57.878125 108.253846 \n",
"L 87.400638 75.857417 \n",
"L 110.760487 57.616075 \n",
"L 128.278904 46.158217 \n",
"L 147.661304 48.341081 \n",
"L 156.556485 43.074892 \n",
"L 163.991016 46.322122 \n",
"L 172.095645 43.273967 \n",
"L 179.447472 42.602955 \n",
"L 189.065731 45.011982 \n",
"L 191.848485 37.095222 \n",
"L 191.694983 37.595448 \n",
"L 195.968429 39.221728 \n",
"L 195.736427 37.764877 \n",
"L 202.972315 40.607961 \n",
"L 203.222036 47.563469 \n",
"L 203.206943 43.478522 \n",
"L 202.592157 42.071803 \n",
"L 205.787854 43.472704 \n",
"L 201.002728 41.882756 \n",
"L 202.290898 39.363537 \n",
"L 197.109957 36.813163 \n",
"L 200.709465 35.990325 \n",
"L 199.023737 31.240715 \n",
"L 196.977777 41.619325 \n",
"L 195.459851 41.087079 \n",
"L 197.824094 38.948063 \n",
"L 197.631354 36.636445 \n",
"L 204.187023 41.846721 \n",
"L 202.711551 45.727514 \n",
"L 203.685962 46.347417 \n",
"L 207.353191 44.528727 \n",
"L 207.174359 41.146072 \n",
"L 203.51082 41.889386 \n",
"L 210.3926 41.057406 \n",
"L 211.956556 36.838642 \n",
"L 206.973915 43.391492 \n",
"L 207.429482 45.667178 \n",
"L 211.690155 39.620429 \n",
"L 212.650319 35.959256 \n",
"L 216.048187 36.184795 \n",
"L 215.444246 40.85764 \n",
"L 213.363914 39.55584 \n",
"L 219.40783 36.489414 \n",
"L 220.476765 39.417051 \n",
"L 216.218325 42.372611 \n",
"L 212.133628 40.948595 \n",
"L 212.966232 39.350876 \n",
"L 209.645915 38.889188 \n",
"L 205.439777 37.586697 \n",
"L 211.083939 30.620279 \n",
"\" clip-path=\"url(#p62b913f3b2)\" style=\"fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" <defs>\n",
" <path id=\"ma83c4a791f\" d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" style=\"stroke: #ff7f0e\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p62b913f3b2)\">\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"57.878125\" y=\"108.253846\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"87.400638\" y=\"75.857417\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"110.760487\" y=\"57.616075\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"128.278904\" y=\"46.158217\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"147.661304\" y=\"48.341081\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"156.556485\" y=\"43.074892\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"163.991016\" y=\"46.322122\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"172.095645\" y=\"43.273967\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"179.447472\" y=\"42.602955\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"189.065731\" y=\"45.011982\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"191.848485\" y=\"37.095222\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"191.694983\" y=\"37.595448\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"195.968429\" y=\"39.221728\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"195.736427\" y=\"37.764877\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"202.972315\" y=\"40.607961\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"203.222036\" y=\"47.563469\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"203.206943\" y=\"43.478522\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"202.592157\" y=\"42.071803\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"205.787854\" y=\"43.472704\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"201.002728\" y=\"41.882756\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"202.290898\" y=\"39.363537\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"197.109957\" y=\"36.813163\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"200.709465\" y=\"35.990325\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"199.023737\" y=\"31.240715\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"196.977777\" y=\"41.619325\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"195.459851\" y=\"41.087079\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"197.824094\" y=\"38.948063\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"197.631354\" y=\"36.636445\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"204.187023\" y=\"41.846721\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"202.711551\" y=\"45.727514\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"203.685962\" y=\"46.347417\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"207.353191\" y=\"44.528727\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"207.174359\" y=\"41.146072\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"203.51082\" y=\"41.889386\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"210.3926\" y=\"41.057406\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"211.956556\" y=\"36.838642\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"206.973915\" y=\"43.391492\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"207.429482\" y=\"45.667178\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"211.690155\" y=\"39.620429\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"212.650319\" y=\"35.959256\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"216.048187\" y=\"36.184795\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"215.444246\" y=\"40.85764\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"213.363914\" y=\"39.55584\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"219.40783\" y=\"36.489414\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"220.476765\" y=\"39.417051\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"216.218325\" y=\"42.372611\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"212.133628\" y=\"40.948595\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"212.966232\" y=\"39.350876\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"209.645915\" y=\"38.889188\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"205.439777\" y=\"37.586697\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma83c4a791f\" x=\"211.083939\" y=\"30.620279\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_1\"/>\n",
" <g id=\"PathCollection_2\">\n",
" <path d=\"M 133.389338 7.2 \n",
"L 131.354961 10.684614 \n",
"L 131.115628 11.149219 \n",
"L 129.618631 14.16923 \n",
"L 128.121641 17.653845 \n",
"L 128.064069 17.812225 \n",
"L 126.898922 21.138461 \n",
"L 125.900228 24.623076 \n",
"L 125.123465 28.107691 \n",
"L 125.012502 28.804598 \n",
"L 124.58421 31.592307 \n",
"L 124.262994 35.076923 \n",
"L 124.155921 38.561539 \n",
"L 124.262994 42.046154 \n",
"L 124.58421 45.530769 \n",
"L 125.012502 48.318478 \n",
"L 125.123465 49.015384 \n",
"L 125.900228 52.500001 \n",
"L 126.898922 55.984615 \n",
"L 128.064069 59.310851 \n",
"L 128.121641 59.469231 \n",
"L 129.618631 62.953845 \n",
"L 131.115628 65.973855 \n",
"L 131.354961 66.438459 \n",
"L 133.389338 69.923076 \n",
"L 134.167188 71.115182 \n",
"L 135.724107 73.407692 \n",
"L 137.218755 75.398907 \n",
"L 138.387435 76.892308 \n",
"L 140.270314 79.089128 \n",
"L 141.423126 80.37692 \n",
"L 143.321874 82.328309 \n",
"L 144.883135 83.861536 \n",
"L 146.373441 85.216671 \n",
"L 148.829573 87.346153 \n",
"L 149.425 87.826789 \n",
"L 152.476564 90.169895 \n",
"L 153.383783 90.830769 \n",
"L 155.528127 92.292059 \n",
"L 158.57969 94.25918 \n",
"L 158.672164 94.315385 \n",
"L 161.631253 96.004895 \n",
"L 164.682813 97.641606 \n",
"L 164.998499 97.800001 \n",
"L 167.734376 99.094286 \n",
"L 170.785939 100.438353 \n",
"L 172.861005 101.284618 \n",
"L 173.837499 101.66133 \n",
"L 176.889062 102.744387 \n",
"L 179.940626 103.733262 \n",
"L 182.992189 104.627962 \n",
"L 183.530712 104.769234 \n",
"L 186.04375 105.394675 \n",
"L 189.095313 106.064794 \n",
"L 192.146877 106.645562 \n",
"L 195.198438 107.136982 \n",
"L 198.250001 107.539054 \n",
"L 201.301564 107.851774 \n",
"L 204.353126 108.075148 \n",
"L 207.404689 108.20917 \n",
"L 210.456251 108.253846 \n",
"L 213.507813 108.20917 \n",
"L 216.559376 108.075148 \n",
"L 219.610939 107.851774 \n",
"L 222.662501 107.539054 \n",
"L 225.714063 107.136982 \n",
"L 228.765626 106.645562 \n",
"L 231.817188 106.064794 \n",
"L 234.868751 105.394675 \n",
"L 237.38179 104.769234 \n",
"L 237.920313 104.627962 \n",
"\" clip-path=\"url(#p62b913f3b2)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_3\">\n",
" <path d=\"M 94.741 7.2 \n",
"L 94.496869 7.814946 \n",
"L 93.387217 10.684614 \n",
"L 92.198295 14.16923 \n",
"L 91.44531 16.715693 \n",
"L 91.174923 17.653845 \n",
"L 90.325119 21.138461 \n",
"L 89.629825 24.623076 \n",
"L 89.089044 28.107691 \n",
"L 88.702769 31.592307 \n",
"L 88.471007 35.076923 \n",
"L 88.393757 38.561206 \n",
"L 88.39375 38.561539 \n",
"L 88.393757 38.561871 \n",
"L 88.471007 42.046154 \n",
"L 88.702769 45.530769 \n",
"L 89.089044 49.015384 \n",
"L 89.629825 52.500001 \n",
"L 90.325119 55.984615 \n",
"L 91.174923 59.469231 \n",
"L 91.44531 60.407382 \n",
"L 92.198295 62.953845 \n",
"L 93.387217 66.438459 \n",
"L 94.496869 69.30813 \n",
"L 94.741 69.923076 \n",
"L 96.287127 73.407692 \n",
"L 97.548436 75.979666 \n",
"L 98.008263 76.892308 \n",
"L 99.931161 80.37692 \n",
"L 100.599996 81.491996 \n",
"L 102.061308 83.861536 \n",
"L 103.651563 86.249145 \n",
"L 104.403397 87.346153 \n",
"L 106.703122 90.470284 \n",
"L 106.976403 90.830769 \n",
"L 109.754682 94.25917 \n",
"L 109.801639 94.315385 \n",
"L 112.806249 97.694405 \n",
"L 112.903126 97.800001 \n",
"L 115.857816 100.836595 \n",
"L 116.308047 101.284618 \n",
"L 118.909375 103.733262 \n",
"L 120.04725 104.769234 \n",
"L 121.960942 106.422193 \n",
"L 124.155924 108.253846 \n",
"L 125.012502 108.93377 \n",
"L 128.064069 111.271016 \n",
"L 128.697415 111.738466 \n",
"L 131.115628 113.440255 \n",
"L 133.748347 115.223078 \n",
"L 134.167188 115.494103 \n",
"L 137.218755 117.391282 \n",
"L 139.426266 118.70769 \n",
"L 140.270314 119.189606 \n",
"L 143.321874 120.857774 \n",
"L 145.876677 122.192311 \n",
"L 146.373441 122.441211 \n",
"L 149.425 123.899059 \n",
"L 152.476564 125.285791 \n",
"L 153.383788 125.676923 \n",
"L 155.528127 126.565159 \n",
"L 158.57969 127.760863 \n",
"L 161.631253 128.888239 \n",
"L 162.418734 129.161535 \n",
"L 164.682813 129.917634 \n",
"L 167.734376 130.870972 \n",
"L 170.785939 131.758564 \n",
"L 173.837499 132.580406 \n",
"L 174.102859 132.646155 \n",
"L 176.889062 133.311398 \n",
"L 179.940626 133.976642 \n",
"L 182.992189 134.57853 \n",
"L 186.04375 135.117062 \n",
"L 189.095313 135.592236 \n",
"L 192.146877 136.004057 \n",
"L 193.256511 136.130768 \n",
"L 195.198438 136.344737 \n",
"L 198.250001 136.619839 \n",
"L 201.301564 136.833805 \n",
"L 204.353126 136.986639 \n",
"L 207.404689 137.078339 \n",
"L 210.456251 137.108907 \n",
"L 213.507813 137.078339 \n",
"L 216.559376 136.986639 \n",
"L 219.610939 136.833805 \n",
"L 222.662501 136.619839 \n",
"L 225.714063 136.344737 \n",
"L 227.655991 136.130768 \n",
"L 228.765626 136.004057 \n",
"L 231.817188 135.592236 \n",
"L 234.868751 135.117062 \n",
"L 237.920313 134.57853 \n",
"\" clip-path=\"url(#p62b913f3b2)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_4\">\n",
" <path d=\"M 66.101283 7.2 \n",
"L 65.009145 10.684614 \n",
"L 64.045489 14.16923 \n",
"L 63.981244 14.437284 \n",
"L 63.226219 17.653845 \n",
"L 62.534115 21.138461 \n",
"L 61.967845 24.623076 \n",
"L 61.527415 28.107691 \n",
"L 61.21282 31.592307 \n",
"L 61.024065 35.076923 \n",
"L 60.961145 38.561539 \n",
"L 61.024065 42.046154 \n",
"L 61.21282 45.530769 \n",
"L 61.527415 49.015384 \n",
"L 61.967845 52.500001 \n",
"L 62.534115 55.984615 \n",
"L 63.226219 59.469231 \n",
"L 63.981244 62.685792 \n",
"L 64.045489 62.953845 \n",
"L 65.009145 66.438459 \n",
"L 66.101283 69.923076 \n",
"L 67.032818 72.582402 \n",
"L 67.328125 73.407692 \n",
"L 68.706252 76.892308 \n",
"L 70.084378 80.073917 \n",
"L 70.218509 80.37692 \n",
"L 71.895191 83.861536 \n",
"L 73.135938 86.249145 \n",
"L 73.718821 87.346153 \n",
"L 75.70748 90.830769 \n",
"L 76.187512 91.617635 \n",
"L 77.871125 94.315385 \n",
"L 79.239071 96.374484 \n",
"L 80.208385 97.800001 \n",
"L 82.290631 100.687261 \n",
"L 82.731817 101.284618 \n",
"L 85.342191 104.627962 \n",
"L 85.455214 104.769234 \n",
"L 88.39375 108.253846 \n",
"L 88.393757 108.253854 \n",
"L 91.44531 111.610969 \n",
"L 91.564212 111.738466 \n",
"L 94.496869 114.736843 \n",
"L 94.98513 115.223078 \n",
"L 97.548436 117.662305 \n",
"L 98.677098 118.70769 \n",
"L 100.599996 120.412926 \n",
"L 102.66303 122.192311 \n",
"L 103.651563 123.010127 \n",
"L 106.703122 125.463574 \n",
"L 106.976403 125.676923 \n",
"L 109.754682 127.760856 \n",
"L 111.679511 129.161535 \n",
"L 112.806249 129.950509 \n",
"L 115.857816 132.021554 \n",
"L 116.808306 132.646155 \n",
"L 118.909375 133.97664 \n",
"L 121.960942 135.845668 \n",
"L 122.442758 136.130768 \n",
"L 125.012502 137.597976 \n",
"L 128.064069 139.279152 \n",
"L 128.697416 139.615388 \n",
"L 131.115628 140.855672 \n",
"L 134.167188 142.361734 \n",
"L 135.724104 143.1 \n",
"\" clip-path=\"url(#p62b913f3b2)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_5\">\n",
" <path d=\"M 42.620312 71.115188 \n",
"L 43.320213 73.407692 \n",
"L 44.496043 76.892308 \n",
"L 45.671872 80.073903 \n",
"L 45.785952 80.37692 \n",
"L 47.211915 83.861536 \n",
"L 48.723432 87.281609 \n",
"L 48.7525 87.346153 \n",
"L 50.438126 90.830769 \n",
"L 51.775006 93.41615 \n",
"L 52.249024 94.315385 \n",
"L 54.204398 97.800001 \n",
"L 54.826565 98.845393 \n",
"L 56.307021 101.284618 \n",
"L 57.878125 103.733263 \n",
"L 58.556253 104.769234 \n",
"L 60.929685 108.209162 \n",
"L 60.961151 108.253846 \n",
"L 63.540821 111.738466 \n",
"L 63.981244 112.305719 \n",
"L 66.294014 115.223078 \n",
"L 67.032818 116.113596 \n",
"L 69.231252 118.70769 \n",
"L 70.084378 119.671522 \n",
"L 72.364666 122.192311 \n",
"L 73.135938 123.010127 \n",
"L 75.70748 125.676923 \n",
"L 76.187512 126.155214 \n",
"L 79.239071 129.127391 \n",
"L 79.274952 129.161535 \n",
"L 82.290631 131.922942 \n",
"L 83.09947 132.646155 \n",
"L 85.342191 134.578536 \n",
"L 87.188189 136.130768 \n",
"L 88.393757 137.108915 \n",
"L 91.44531 139.52368 \n",
"L 91.564212 139.615388 \n",
"L 94.496869 141.800646 \n",
"L 96.287127 143.1 \n",
"\" clip-path=\"url(#p62b913f3b2)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_6\">\n",
" <path d=\"M 42.620312 115.494102 \n",
"L 44.943982 118.70769 \n",
"L 45.671872 119.671517 \n",
"L 47.611182 122.192311 \n",
"L 48.723432 123.579037 \n",
"L 50.438126 125.676923 \n",
"L 51.775006 127.248424 \n",
"L 53.434101 129.161535 \n",
"L 54.826565 130.706605 \n",
"L 56.609163 132.646155 \n",
"L 57.878125 133.97664 \n",
"L 59.974143 136.130768 \n",
"L 60.929685 137.078341 \n",
"L 63.540815 139.615388 \n",
"L 63.981244 140.028815 \n",
"L 67.032818 142.834235 \n",
"L 67.32812 143.1 \n",
"\" clip-path=\"url(#p62b913f3b2)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_7\">\n",
" <path d=\"M 42.620312 142.361735 \n",
"L 43.320214 143.1 \n",
"\" clip-path=\"url(#p62b913f3b2)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_8\"/>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 42.620312 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 237.920313 143.1 \n",
"L 237.920313 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 237.920313 143.1 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 42.620312 7.2 \n",
"L 237.920313 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p62b913f3b2\">\n",
" <rect x=\"42.620312\" y=\"7.2\" width=\"195.3\" height=\"135.9\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 252x180 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"def constant_lr():\n",
" return 1\n",
"\n",
"eta = 0.1\n",
"lr = constant_lr # 常数学习速度\n",
"d2l.show_trace_2d(f, d2l.train_2d(sgd, steps=50, f_grad=f_grad))"
]
},
{
"cell_type": "markdown",
"id": "d3eb8c3f",
"metadata": {
"origin_pos": 10
},
"source": [
"正如我们所看到的,随机梯度下降中变量的轨迹比我们在 :numref:`sec_gd`中观察到的梯度下降中观察到的轨迹嘈杂得多。这是由于梯度的随机性质。也就是说,即使我们接近最小值,我们仍然受到通过$\\eta \\nabla f_i(\\mathbf{x})$的瞬间梯度所注入的不确定性的影响。即使经过50次迭代,质量仍然不那么好。更糟糕的是,经过额外的步骤,它不会得到改善。这给我们留下了唯一的选择:改变学习率$\\eta$。但是,如果我们选择的学习率太小,我们一开始就不会取得任何有意义的进展。另一方面,如果我们选择的学习率太大,我们将无法获得一个好的解决方案,如上所示。解决这些相互冲突的目标的唯一方法是在优化过程中*动态*降低学习率。\n",
"\n",
"这也是在`sgd`步长函数中添加学习率函数`lr`的原因。在上面的示例中,学习率调度的任何功能都处于休眠状态,因为我们将相关的`lr`函数设置为常量。\n",
"\n",
"## 动态学习率\n",
"\n",
"用与时间相关的学习率$\\eta(t)$取代$\\eta$增加了控制优化算法收敛的复杂性。特别是,我们需要弄清$\\eta$的衰减速度。如果太快,我们将过早停止优化。如果减少的太慢,我们会在优化上浪费太多时间。以下是随着时间推移调整$\\eta$时使用的一些基本策略(稍后我们将讨论更高级的策略):\n",
"\n",
"$$\n",
"\\begin{aligned}\n",
" \\eta(t) & = \\eta_i \\text{ if } t_i \\leq t \\leq t_{i+1} && \\text{分段常数} \\\\\n",
" \\eta(t) & = \\eta_0 \\cdot e^{-\\lambda t} && \\text{指数衰减} \\\\\n",
" \\eta(t) & = \\eta_0 \\cdot (\\beta t + 1)^{-\\alpha} && \\text{多项式衰减}\n",
"\\end{aligned}\n",
"$$\n",
"\n",
"在第一个*分段常数*piecewise constant)场景中,我们会降低学习率,例如,每当优化进度停顿时。这是训练深度网络的常见策略。或者,我们可以通过*指数衰减*(exponential decay)来更积极地减低它。不幸的是,这往往会导致算法收敛之前过早停止。一个受欢迎的选择是$\\alpha = 0.5$的*多项式衰减*polynomial decay)。在凸优化的情况下,有许多证据表明这种速率表现良好。\n",
"\n",
"让我们看看指数衰减在实践中是什么样子。\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "942ccd1d",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T06:58:30.699948Z",
"iopub.status.busy": "2023-08-18T06:58:30.699052Z",
"iopub.status.idle": "2023-08-18T06:58:30.961037Z",
"shell.execute_reply": "2023-08-18T06:58:30.959877Z"
},
"origin_pos": 11,
"tab": [
"pytorch"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 1000, x1: -0.998659, x2: 0.023408\n"
]
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"245.120313pt\" height=\"180.65625pt\" viewBox=\"0 0 245.120313 180.65625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T06:58:30.863881</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 180.65625 \n",
"L 245.120313 180.65625 \n",
"L 245.120313 0 \n",
"L 0 0 \n",
"L 0 180.65625 \n",
"z\n",
"\" style=\"fill: none\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 237.920313 143.1 \n",
"L 237.920313 7.2 \n",
"L 42.620312 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path id=\"m0f1907eb46\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m0f1907eb46\" x=\"88.39375\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(81.022656 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
"L 4684 2272 \n",
"L 4684 1741 \n",
"L 678 1741 \n",
"L 678 2272 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
"L 825 1625 \n",
"L 2419 1625 \n",
"L 2419 4116 \n",
"z\n",
"M 2253 4666 \n",
"L 3047 4666 \n",
"L 3047 1625 \n",
"L 3713 1625 \n",
"L 3713 1100 \n",
"L 3047 1100 \n",
"L 3047 0 \n",
"L 2419 0 \n",
"L 2419 1100 \n",
"L 313 1100 \n",
"L 313 1709 \n",
"L 2253 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-34\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use xlink:href=\"#m0f1907eb46\" x=\"149.425\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(142.053907 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use xlink:href=\"#m0f1907eb46\" x=\"210.456251\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(207.275001 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- x1 -->\n",
" <g transform=\"translate(134.129687 171.376563)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"59.179688\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_4\">\n",
" <defs>\n",
" <path id=\"m689d47602c\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m689d47602c\" x=\"42.620312\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 3 -->\n",
" <g transform=\"translate(20.878125 146.899219)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
"Q 3050 2419 3304 2112 \n",
"Q 3559 1806 3559 1356 \n",
"Q 3559 666 3084 287 \n",
"Q 2609 -91 1734 -91 \n",
"Q 1441 -91 1130 -33 \n",
"Q 819 25 488 141 \n",
"L 488 750 \n",
"Q 750 597 1062 519 \n",
"Q 1375 441 1716 441 \n",
"Q 2309 441 2620 675 \n",
"Q 2931 909 2931 1356 \n",
"Q 2931 1769 2642 2001 \n",
"Q 2353 2234 1838 2234 \n",
"L 1294 2234 \n",
"L 1294 2753 \n",
"L 1863 2753 \n",
"Q 2328 2753 2575 2939 \n",
"Q 2822 3125 2822 3475 \n",
"Q 2822 3834 2567 4026 \n",
"Q 2313 4219 1838 4219 \n",
"Q 1578 4219 1281 4162 \n",
"Q 984 4106 628 3988 \n",
"L 628 4550 \n",
"Q 988 4650 1302 4700 \n",
"Q 1616 4750 1894 4750 \n",
"Q 2613 4750 3031 4423 \n",
"Q 3450 4097 3450 3541 \n",
"Q 3450 3153 3228 2886 \n",
"Q 3006 2619 2597 2516 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-33\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use xlink:href=\"#m689d47602c\" x=\"42.620312\" y=\"108.253846\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(20.878125 112.053065)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#m689d47602c\" x=\"42.620312\" y=\"73.407692\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(20.878125 77.206911)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_7\">\n",
" <g>\n",
" <use xlink:href=\"#m689d47602c\" x=\"42.620312\" y=\"38.561538\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(29.257812 42.360757)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- x2 -->\n",
" <g transform=\"translate(14.798437 81.290625)rotate(-90)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"59.179688\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <path d=\"M 57.878125 108.253846 \n",
"L 82.596409 87.583485 \n",
"L 100.640528 76.976615 \n",
"L 114.592395 67.629962 \n",
"L 125.211498 59.600105 \n",
"L 136.299108 55.39061 \n",
"L 142.381629 54.627041 \n",
"L 147.298046 51.073973 \n",
"L 151.045314 48.891111 \n",
"L 154.515538 45.993472 \n",
"L 158.655773 44.572998 \n",
"L 162.254888 43.024081 \n",
"L 164.968403 41.096695 \n",
"L 166.643032 40.384609 \n",
"L 168.07896 39.373924 \n",
"L 169.335179 39.988006 \n",
"L 170.700142 38.773663 \n",
"L 171.384089 37.213028 \n",
"L 172.176154 37.302523 \n",
"L 172.816819 37.213524 \n",
"L 173.030974 37.400593 \n",
"L 173.904222 37.108041 \n",
"L 174.820948 37.596368 \n",
"L 176.427067 37.313364 \n",
"L 176.720426 37.373783 \n",
"L 176.918716 37.262486 \n",
"L 177.340568 37.412928 \n",
"L 177.521401 37.283087 \n",
"L 177.603278 37.428856 \n",
"L 178.342743 37.555498 \n",
"L 178.522068 37.440463 \n",
"L 178.528149 37.532278 \n",
"L 178.940166 37.607181 \n",
"L 179.124402 37.562345 \n",
"L 179.404637 37.705815 \n",
"L 179.601022 37.661835 \n",
"L 179.981553 37.745851 \n",
"L 179.981553 37.745851 \n",
"\" clip-path=\"url(#p3d519d4fbe)\" style=\"fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" <defs>\n",
" <path id=\"m2a43ec4593\" d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" style=\"stroke: #ff7f0e\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p3d519d4fbe)\">\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"57.878125\" y=\"108.253846\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"82.596409\" y=\"87.583485\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"100.640528\" y=\"76.976615\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"114.592395\" y=\"67.629962\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"125.211498\" y=\"59.600105\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"136.299108\" y=\"55.39061\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"142.381629\" y=\"54.627041\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"147.298046\" y=\"51.073973\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"151.045314\" y=\"48.891111\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"154.515538\" y=\"45.993472\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"158.655773\" y=\"44.572998\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"162.254888\" y=\"43.024081\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"164.968403\" y=\"41.096695\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"166.643032\" y=\"40.384609\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"168.07896\" y=\"39.373924\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"169.224485\" y=\"40.020079\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"169.335179\" y=\"39.988006\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"170.700142\" y=\"38.773663\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"171.384089\" y=\"37.213028\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"172.176154\" y=\"37.302523\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"172.816819\" y=\"37.213524\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"173.030974\" y=\"37.400593\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"173.904222\" y=\"37.108041\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"174.820948\" y=\"37.596368\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"175.688551\" y=\"37.397451\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"176.427067\" y=\"37.313364\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"176.720426\" y=\"37.373783\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"176.918716\" y=\"37.262486\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"177.340568\" y=\"37.412928\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"177.521401\" y=\"37.283087\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"177.603278\" y=\"37.428856\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"177.95057\" y=\"37.542481\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"178.21557\" y=\"37.651474\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"178.270794\" y=\"37.642879\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"178.342743\" y=\"37.555498\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"178.522068\" y=\"37.440463\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"178.528149\" y=\"37.532278\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"178.686621\" y=\"37.601726\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"178.866507\" y=\"37.69121\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"178.940166\" y=\"37.607181\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.124402\" y=\"37.562345\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.238156\" y=\"37.659775\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.274675\" y=\"37.703635\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.322466\" y=\"37.716478\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.353502\" y=\"37.708097\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.404637\" y=\"37.705815\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.45951\" y=\"37.658671\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.483651\" y=\"37.626821\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.538767\" y=\"37.661181\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.601022\" y=\"37.661835\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.645658\" y=\"37.672424\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.649331\" y=\"37.697945\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.697185\" y=\"37.707602\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.715931\" y=\"37.708995\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.744399\" y=\"37.728401\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.781819\" y=\"37.730834\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.786657\" y=\"37.742861\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.802143\" y=\"37.732775\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.819322\" y=\"37.733215\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.836672\" y=\"37.72408\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.849717\" y=\"37.729406\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.863038\" y=\"37.74072\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.881128\" y=\"37.737887\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.896759\" y=\"37.752007\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.90067\" y=\"37.751624\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.913036\" y=\"37.74862\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.920499\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.926362\" y=\"37.746834\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.929565\" y=\"37.748183\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.939269\" y=\"37.755297\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.943574\" y=\"37.754576\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.948806\" y=\"37.751884\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.952457\" y=\"37.749466\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.954007\" y=\"37.746016\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.958702\" y=\"37.746807\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.958523\" y=\"37.744857\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.958693\" y=\"37.744466\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.958927\" y=\"37.744273\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.961661\" y=\"37.744603\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.963823\" y=\"37.742809\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.965917\" y=\"37.744027\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.965829\" y=\"37.742584\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.967384\" y=\"37.74208\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.969793\" y=\"37.742437\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.970426\" y=\"37.742246\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.972127\" y=\"37.743022\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.973717\" y=\"37.743047\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.973972\" y=\"37.743931\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.974296\" y=\"37.744163\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.975028\" y=\"37.744002\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.975984\" y=\"37.744132\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.976809\" y=\"37.744564\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.977299\" y=\"37.744905\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.97776\" y=\"37.745494\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.977783\" y=\"37.745086\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.978068\" y=\"37.745076\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.978476\" y=\"37.745249\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.978734\" y=\"37.745526\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.979105\" y=\"37.745854\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.979267\" y=\"37.745698\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.979555\" y=\"37.745815\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.979595\" y=\"37.745808\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.979859\" y=\"37.745704\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.979963\" y=\"37.745514\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.979926\" y=\"37.745457\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.979999\" y=\"37.745408\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.980217\" y=\"37.745391\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.98042\" y=\"37.745535\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.980518\" y=\"37.74561\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.980606\" y=\"37.745681\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.980699\" y=\"37.745696\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.980768\" y=\"37.745738\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.980826\" y=\"37.745754\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.98088\" y=\"37.745801\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.980939\" y=\"37.745767\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981005\" y=\"37.745765\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981089\" y=\"37.7458\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981121\" y=\"37.745798\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981146\" y=\"37.745801\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981187\" y=\"37.745818\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981235\" y=\"37.745803\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981278\" y=\"37.745788\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981322\" y=\"37.745788\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981358\" y=\"37.745789\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981375\" y=\"37.745802\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981392\" y=\"37.745825\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981408\" y=\"37.745818\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981428\" y=\"37.745823\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981434\" y=\"37.745843\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981443\" y=\"37.745846\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981449\" y=\"37.745847\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981448\" y=\"37.745843\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981463\" y=\"37.745836\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981468\" y=\"37.745835\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.98148\" y=\"37.745834\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981488\" y=\"37.745833\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981496\" y=\"37.745835\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981504\" y=\"37.745831\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981508\" y=\"37.745834\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981515\" y=\"37.745839\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981516\" y=\"37.745841\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981518\" y=\"37.745842\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981521\" y=\"37.745843\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981524\" y=\"37.745844\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981529\" y=\"37.745844\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981531\" y=\"37.745845\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981531\" y=\"37.745845\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981532\" y=\"37.745845\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981536\" y=\"37.745848\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981537\" y=\"37.745848\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981538\" y=\"37.745847\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.98154\" y=\"37.745848\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981541\" y=\"37.745848\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981542\" y=\"37.745848\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981542\" y=\"37.745849\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981543\" y=\"37.74585\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981544\" y=\"37.74585\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981545\" y=\"37.74585\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981545\" y=\"37.74585\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981546\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981547\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981548\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981548\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981549\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981549\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981549\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.98155\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.98155\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.98155\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981551\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981551\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981551\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981551\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981552\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981552\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981552\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981552\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981552\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981552\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981552\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981552\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981552\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981552\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#m2a43ec4593\" x=\"179.981553\" y=\"37.745851\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_1\"/>\n",
" <g id=\"PathCollection_2\">\n",
" <path d=\"M 133.389338 7.2 \n",
"L 131.354961 10.684614 \n",
"L 131.115628 11.149219 \n",
"L 129.618631 14.16923 \n",
"L 128.121641 17.653845 \n",
"L 128.064069 17.812225 \n",
"L 126.898922 21.138461 \n",
"L 125.900228 24.623076 \n",
"L 125.123465 28.107691 \n",
"L 125.012502 28.804598 \n",
"L 124.58421 31.592307 \n",
"L 124.262994 35.076923 \n",
"L 124.155921 38.561539 \n",
"L 124.262994 42.046154 \n",
"L 124.58421 45.530769 \n",
"L 125.012502 48.318478 \n",
"L 125.123465 49.015384 \n",
"L 125.900228 52.500001 \n",
"L 126.898922 55.984615 \n",
"L 128.064069 59.310851 \n",
"L 128.121641 59.469231 \n",
"L 129.618631 62.953845 \n",
"L 131.115628 65.973855 \n",
"L 131.354961 66.438459 \n",
"L 133.389338 69.923076 \n",
"L 134.167188 71.115182 \n",
"L 135.724107 73.407692 \n",
"L 137.218755 75.398907 \n",
"L 138.387435 76.892308 \n",
"L 140.270314 79.089128 \n",
"L 141.423126 80.37692 \n",
"L 143.321874 82.328309 \n",
"L 144.883135 83.861536 \n",
"L 146.373441 85.216671 \n",
"L 148.829573 87.346153 \n",
"L 149.425 87.826789 \n",
"L 152.476564 90.169895 \n",
"L 153.383783 90.830769 \n",
"L 155.528127 92.292059 \n",
"L 158.57969 94.25918 \n",
"L 158.672164 94.315385 \n",
"L 161.631253 96.004895 \n",
"L 164.682813 97.641606 \n",
"L 164.998499 97.800001 \n",
"L 167.734376 99.094286 \n",
"L 170.785939 100.438353 \n",
"L 172.861005 101.284618 \n",
"L 173.837499 101.66133 \n",
"L 176.889062 102.744387 \n",
"L 179.940626 103.733262 \n",
"L 182.992189 104.627962 \n",
"L 183.530712 104.769234 \n",
"L 186.04375 105.394675 \n",
"L 189.095313 106.064794 \n",
"L 192.146877 106.645562 \n",
"L 195.198438 107.136982 \n",
"L 198.250001 107.539054 \n",
"L 201.301564 107.851774 \n",
"L 204.353126 108.075148 \n",
"L 207.404689 108.20917 \n",
"L 210.456251 108.253846 \n",
"L 213.507813 108.20917 \n",
"L 216.559376 108.075148 \n",
"L 219.610939 107.851774 \n",
"L 222.662501 107.539054 \n",
"L 225.714063 107.136982 \n",
"L 228.765626 106.645562 \n",
"L 231.817188 106.064794 \n",
"L 234.868751 105.394675 \n",
"L 237.38179 104.769234 \n",
"L 237.920313 104.627962 \n",
"\" clip-path=\"url(#p3d519d4fbe)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_3\">\n",
" <path d=\"M 94.741 7.2 \n",
"L 94.496869 7.814946 \n",
"L 93.387217 10.684614 \n",
"L 92.198295 14.16923 \n",
"L 91.44531 16.715693 \n",
"L 91.174923 17.653845 \n",
"L 90.325119 21.138461 \n",
"L 89.629825 24.623076 \n",
"L 89.089044 28.107691 \n",
"L 88.702769 31.592307 \n",
"L 88.471007 35.076923 \n",
"L 88.393757 38.561206 \n",
"L 88.39375 38.561539 \n",
"L 88.393757 38.561871 \n",
"L 88.471007 42.046154 \n",
"L 88.702769 45.530769 \n",
"L 89.089044 49.015384 \n",
"L 89.629825 52.500001 \n",
"L 90.325119 55.984615 \n",
"L 91.174923 59.469231 \n",
"L 91.44531 60.407382 \n",
"L 92.198295 62.953845 \n",
"L 93.387217 66.438459 \n",
"L 94.496869 69.30813 \n",
"L 94.741 69.923076 \n",
"L 96.287127 73.407692 \n",
"L 97.548436 75.979666 \n",
"L 98.008263 76.892308 \n",
"L 99.931161 80.37692 \n",
"L 100.599996 81.491996 \n",
"L 102.061308 83.861536 \n",
"L 103.651563 86.249145 \n",
"L 104.403397 87.346153 \n",
"L 106.703122 90.470284 \n",
"L 106.976403 90.830769 \n",
"L 109.754682 94.25917 \n",
"L 109.801639 94.315385 \n",
"L 112.806249 97.694405 \n",
"L 112.903126 97.800001 \n",
"L 115.857816 100.836595 \n",
"L 116.308047 101.284618 \n",
"L 118.909375 103.733262 \n",
"L 120.04725 104.769234 \n",
"L 121.960942 106.422193 \n",
"L 124.155924 108.253846 \n",
"L 125.012502 108.93377 \n",
"L 128.064069 111.271016 \n",
"L 128.697415 111.738466 \n",
"L 131.115628 113.440255 \n",
"L 133.748347 115.223078 \n",
"L 134.167188 115.494103 \n",
"L 137.218755 117.391282 \n",
"L 139.426266 118.70769 \n",
"L 140.270314 119.189606 \n",
"L 143.321874 120.857774 \n",
"L 145.876677 122.192311 \n",
"L 146.373441 122.441211 \n",
"L 149.425 123.899059 \n",
"L 152.476564 125.285791 \n",
"L 153.383788 125.676923 \n",
"L 155.528127 126.565159 \n",
"L 158.57969 127.760863 \n",
"L 161.631253 128.888239 \n",
"L 162.418734 129.161535 \n",
"L 164.682813 129.917634 \n",
"L 167.734376 130.870972 \n",
"L 170.785939 131.758564 \n",
"L 173.837499 132.580406 \n",
"L 174.102859 132.646155 \n",
"L 176.889062 133.311398 \n",
"L 179.940626 133.976642 \n",
"L 182.992189 134.57853 \n",
"L 186.04375 135.117062 \n",
"L 189.095313 135.592236 \n",
"L 192.146877 136.004057 \n",
"L 193.256511 136.130768 \n",
"L 195.198438 136.344737 \n",
"L 198.250001 136.619839 \n",
"L 201.301564 136.833805 \n",
"L 204.353126 136.986639 \n",
"L 207.404689 137.078339 \n",
"L 210.456251 137.108907 \n",
"L 213.507813 137.078339 \n",
"L 216.559376 136.986639 \n",
"L 219.610939 136.833805 \n",
"L 222.662501 136.619839 \n",
"L 225.714063 136.344737 \n",
"L 227.655991 136.130768 \n",
"L 228.765626 136.004057 \n",
"L 231.817188 135.592236 \n",
"L 234.868751 135.117062 \n",
"L 237.920313 134.57853 \n",
"\" clip-path=\"url(#p3d519d4fbe)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_4\">\n",
" <path d=\"M 66.101283 7.2 \n",
"L 65.009145 10.684614 \n",
"L 64.045489 14.16923 \n",
"L 63.981244 14.437284 \n",
"L 63.226219 17.653845 \n",
"L 62.534115 21.138461 \n",
"L 61.967845 24.623076 \n",
"L 61.527415 28.107691 \n",
"L 61.21282 31.592307 \n",
"L 61.024065 35.076923 \n",
"L 60.961145 38.561539 \n",
"L 61.024065 42.046154 \n",
"L 61.21282 45.530769 \n",
"L 61.527415 49.015384 \n",
"L 61.967845 52.500001 \n",
"L 62.534115 55.984615 \n",
"L 63.226219 59.469231 \n",
"L 63.981244 62.685792 \n",
"L 64.045489 62.953845 \n",
"L 65.009145 66.438459 \n",
"L 66.101283 69.923076 \n",
"L 67.032818 72.582402 \n",
"L 67.328125 73.407692 \n",
"L 68.706252 76.892308 \n",
"L 70.084378 80.073917 \n",
"L 70.218509 80.37692 \n",
"L 71.895191 83.861536 \n",
"L 73.135938 86.249145 \n",
"L 73.718821 87.346153 \n",
"L 75.70748 90.830769 \n",
"L 76.187512 91.617635 \n",
"L 77.871125 94.315385 \n",
"L 79.239071 96.374484 \n",
"L 80.208385 97.800001 \n",
"L 82.290631 100.687261 \n",
"L 82.731817 101.284618 \n",
"L 85.342191 104.627962 \n",
"L 85.455214 104.769234 \n",
"L 88.39375 108.253846 \n",
"L 88.393757 108.253854 \n",
"L 91.44531 111.610969 \n",
"L 91.564212 111.738466 \n",
"L 94.496869 114.736843 \n",
"L 94.98513 115.223078 \n",
"L 97.548436 117.662305 \n",
"L 98.677098 118.70769 \n",
"L 100.599996 120.412926 \n",
"L 102.66303 122.192311 \n",
"L 103.651563 123.010127 \n",
"L 106.703122 125.463574 \n",
"L 106.976403 125.676923 \n",
"L 109.754682 127.760856 \n",
"L 111.679511 129.161535 \n",
"L 112.806249 129.950509 \n",
"L 115.857816 132.021554 \n",
"L 116.808306 132.646155 \n",
"L 118.909375 133.97664 \n",
"L 121.960942 135.845668 \n",
"L 122.442758 136.130768 \n",
"L 125.012502 137.597976 \n",
"L 128.064069 139.279152 \n",
"L 128.697416 139.615388 \n",
"L 131.115628 140.855672 \n",
"L 134.167188 142.361734 \n",
"L 135.724104 143.1 \n",
"\" clip-path=\"url(#p3d519d4fbe)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_5\">\n",
" <path d=\"M 42.620312 71.115188 \n",
"L 43.320213 73.407692 \n",
"L 44.496043 76.892308 \n",
"L 45.671872 80.073903 \n",
"L 45.785952 80.37692 \n",
"L 47.211915 83.861536 \n",
"L 48.723432 87.281609 \n",
"L 48.7525 87.346153 \n",
"L 50.438126 90.830769 \n",
"L 51.775006 93.41615 \n",
"L 52.249024 94.315385 \n",
"L 54.204398 97.800001 \n",
"L 54.826565 98.845393 \n",
"L 56.307021 101.284618 \n",
"L 57.878125 103.733263 \n",
"L 58.556253 104.769234 \n",
"L 60.929685 108.209162 \n",
"L 60.961151 108.253846 \n",
"L 63.540821 111.738466 \n",
"L 63.981244 112.305719 \n",
"L 66.294014 115.223078 \n",
"L 67.032818 116.113596 \n",
"L 69.231252 118.70769 \n",
"L 70.084378 119.671522 \n",
"L 72.364666 122.192311 \n",
"L 73.135938 123.010127 \n",
"L 75.70748 125.676923 \n",
"L 76.187512 126.155214 \n",
"L 79.239071 129.127391 \n",
"L 79.274952 129.161535 \n",
"L 82.290631 131.922942 \n",
"L 83.09947 132.646155 \n",
"L 85.342191 134.578536 \n",
"L 87.188189 136.130768 \n",
"L 88.393757 137.108915 \n",
"L 91.44531 139.52368 \n",
"L 91.564212 139.615388 \n",
"L 94.496869 141.800646 \n",
"L 96.287127 143.1 \n",
"\" clip-path=\"url(#p3d519d4fbe)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_6\">\n",
" <path d=\"M 42.620312 115.494102 \n",
"L 44.943982 118.70769 \n",
"L 45.671872 119.671517 \n",
"L 47.611182 122.192311 \n",
"L 48.723432 123.579037 \n",
"L 50.438126 125.676923 \n",
"L 51.775006 127.248424 \n",
"L 53.434101 129.161535 \n",
"L 54.826565 130.706605 \n",
"L 56.609163 132.646155 \n",
"L 57.878125 133.97664 \n",
"L 59.974143 136.130768 \n",
"L 60.929685 137.078341 \n",
"L 63.540815 139.615388 \n",
"L 63.981244 140.028815 \n",
"L 67.032818 142.834235 \n",
"L 67.32812 143.1 \n",
"\" clip-path=\"url(#p3d519d4fbe)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_7\">\n",
" <path d=\"M 42.620312 142.361735 \n",
"L 43.320214 143.1 \n",
"\" clip-path=\"url(#p3d519d4fbe)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_8\"/>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 42.620312 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 237.920313 143.1 \n",
"L 237.920313 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 237.920313 143.1 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 42.620312 7.2 \n",
"L 237.920313 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p3d519d4fbe\">\n",
" <rect x=\"42.620312\" y=\"7.2\" width=\"195.3\" height=\"135.9\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 252x180 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"def exponential_lr():\n",
" # 在函数外部定义,而在内部更新的全局变量\n",
" global t\n",
" t += 1\n",
" return math.exp(-0.1 * t)\n",
"\n",
"t = 1\n",
"lr = exponential_lr\n",
"d2l.show_trace_2d(f, d2l.train_2d(sgd, steps=1000, f_grad=f_grad))"
]
},
{
"cell_type": "markdown",
"id": "75292fbd",
"metadata": {
"origin_pos": 12
},
"source": [
"正如预期的那样,参数的方差大大减少。但是,这是以未能收敛到最优解$\\mathbf{x} = (0, 0)$为代价的。即使经过1000个迭代步骤,我们仍然离最优解很远。事实上,该算法根本无法收敛。另一方面,如果我们使用多项式衰减,其中学习率随迭代次数的平方根倒数衰减,那么仅在50次迭代之后,收敛就会更好。\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "d1bf1cd1",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T06:58:30.970973Z",
"iopub.status.busy": "2023-08-18T06:58:30.970266Z",
"iopub.status.idle": "2023-08-18T06:58:31.111573Z",
"shell.execute_reply": "2023-08-18T06:58:31.110527Z"
},
"origin_pos": 13,
"tab": [
"pytorch"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 50, x1: -0.174174, x2: -0.000615\n"
]
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"245.120313pt\" height=\"180.65625pt\" viewBox=\"0 0 245.120313 180.65625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T06:58:31.078318</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 180.65625 \n",
"L 245.120313 180.65625 \n",
"L 245.120313 0 \n",
"L 0 0 \n",
"L 0 180.65625 \n",
"z\n",
"\" style=\"fill: none\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 237.920313 143.1 \n",
"L 237.920313 7.2 \n",
"L 42.620312 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path id=\"m77d496a090\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m77d496a090\" x=\"88.39375\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(81.022656 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
"L 4684 2272 \n",
"L 4684 1741 \n",
"L 678 1741 \n",
"L 678 2272 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
"L 825 1625 \n",
"L 2419 1625 \n",
"L 2419 4116 \n",
"z\n",
"M 2253 4666 \n",
"L 3047 4666 \n",
"L 3047 1625 \n",
"L 3713 1625 \n",
"L 3713 1100 \n",
"L 3047 1100 \n",
"L 3047 0 \n",
"L 2419 0 \n",
"L 2419 1100 \n",
"L 313 1100 \n",
"L 313 1709 \n",
"L 2253 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-34\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use xlink:href=\"#m77d496a090\" x=\"149.425\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(142.053907 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use xlink:href=\"#m77d496a090\" x=\"210.456251\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(207.275001 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- x1 -->\n",
" <g transform=\"translate(134.129687 171.376563)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"59.179688\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_4\">\n",
" <defs>\n",
" <path id=\"mc81484b457\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#mc81484b457\" x=\"42.620312\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 3 -->\n",
" <g transform=\"translate(20.878125 146.899219)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
"Q 3050 2419 3304 2112 \n",
"Q 3559 1806 3559 1356 \n",
"Q 3559 666 3084 287 \n",
"Q 2609 -91 1734 -91 \n",
"Q 1441 -91 1130 -33 \n",
"Q 819 25 488 141 \n",
"L 488 750 \n",
"Q 750 597 1062 519 \n",
"Q 1375 441 1716 441 \n",
"Q 2309 441 2620 675 \n",
"Q 2931 909 2931 1356 \n",
"Q 2931 1769 2642 2001 \n",
"Q 2353 2234 1838 2234 \n",
"L 1294 2234 \n",
"L 1294 2753 \n",
"L 1863 2753 \n",
"Q 2328 2753 2575 2939 \n",
"Q 2822 3125 2822 3475 \n",
"Q 2822 3834 2567 4026 \n",
"Q 2313 4219 1838 4219 \n",
"Q 1578 4219 1281 4162 \n",
"Q 984 4106 628 3988 \n",
"L 628 4550 \n",
"Q 988 4650 1302 4700 \n",
"Q 1616 4750 1894 4750 \n",
"Q 2613 4750 3031 4423 \n",
"Q 3450 4097 3450 3541 \n",
"Q 3450 3153 3228 2886 \n",
"Q 3006 2619 2597 2516 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-33\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use xlink:href=\"#mc81484b457\" x=\"42.620312\" y=\"108.253846\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(20.878125 112.053065)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#mc81484b457\" x=\"42.620312\" y=\"73.407692\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(20.878125 77.206911)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_7\">\n",
" <g>\n",
" <use xlink:href=\"#mc81484b457\" x=\"42.620312\" y=\"38.561538\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(29.257812 42.360757)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- x2 -->\n",
" <g transform=\"translate(14.798437 81.290625)rotate(-90)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"59.179688\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <path d=\"M 57.878125 108.253846 \n",
"L 83.663584 81.452284 \n",
"L 105.595364 61.831746 \n",
"L 122.492211 57.281995 \n",
"L 136.162485 47.082281 \n",
"L 144.958803 44.447027 \n",
"L 152.30774 41.341499 \n",
"L 163.117756 38.723601 \n",
"L 166.921635 43.64529 \n",
"L 174.009953 43.372005 \n",
"L 177.689178 39.808621 \n",
"L 180.889004 39.506494 \n",
"L 183.293617 39.518352 \n",
"L 184.700141 35.587444 \n",
"L 183.703597 35.597902 \n",
"L 189.398794 35.116817 \n",
"L 190.730206 38.334074 \n",
"L 190.277528 38.520339 \n",
"L 190.208634 35.480749 \n",
"L 193.75327 36.564134 \n",
"L 198.002783 37.131033 \n",
"L 197.251952 34.665131 \n",
"L 197.761344 36.693542 \n",
"L 195.859652 38.926952 \n",
"L 196.020178 40.622099 \n",
"L 197.061723 36.914479 \n",
"L 198.612711 37.219028 \n",
"L 199.313443 40.428419 \n",
"L 202.038139 37.563546 \n",
"L 200.73491 34.832031 \n",
"L 199.422368 35.699944 \n",
"L 198.309283 36.820336 \n",
"L 199.098643 35.652396 \n",
"L 199.970878 36.664534 \n",
"L 202.412641 35.107895 \n",
"L 205.216255 35.160076 \n",
"L 205.458612 34.365201 \n",
"L 205.547441 32.705303 \n",
"L 206.989338 33.133019 \n",
"L 207.592873 33.872911 \n",
"L 206.439407 35.807999 \n",
"L 206.278081 37.629671 \n",
"L 207.648851 37.091839 \n",
"L 208.506868 35.855871 \n",
"L 209.309226 38.261012 \n",
"L 208.513026 38.47571 \n",
"L 206.119335 37.970899 \n",
"L 207.465565 38.059905 \n",
"L 206.907543 38.426665 \n",
"L 206.372028 38.563807 \n",
"L 205.141235 38.582955 \n",
"\" clip-path=\"url(#p9362706432)\" style=\"fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" <defs>\n",
" <path id=\"ma05db63bbb\" d=\"M 0 3 \n",
"C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
"C 2.683901 1.55874 3 0.795609 3 0 \n",
"C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
"C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
"C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
"C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
"C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
"C -1.55874 2.683901 -0.795609 3 0 3 \n",
"z\n",
"\" style=\"stroke: #ff7f0e\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#p9362706432)\">\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"57.878125\" y=\"108.253846\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"83.663584\" y=\"81.452284\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"105.595364\" y=\"61.831746\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"122.492211\" y=\"57.281995\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"136.162485\" y=\"47.082281\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"144.958803\" y=\"44.447027\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"152.30774\" y=\"41.341499\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"163.117756\" y=\"38.723601\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"166.921635\" y=\"43.64529\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"174.009953\" y=\"43.372005\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"177.689178\" y=\"39.808621\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"180.889004\" y=\"39.506494\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"183.293617\" y=\"39.518352\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"184.700141\" y=\"35.587444\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"183.703597\" y=\"35.597902\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"189.398794\" y=\"35.116817\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"190.730206\" y=\"38.334074\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"190.277528\" y=\"38.520339\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"190.208634\" y=\"35.480749\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"193.75327\" y=\"36.564134\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"198.002783\" y=\"37.131033\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"197.251952\" y=\"34.665131\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"197.761344\" y=\"36.693542\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"195.859652\" y=\"38.926952\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"196.020178\" y=\"40.622099\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"197.061723\" y=\"36.914479\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"198.612711\" y=\"37.219028\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"199.313443\" y=\"40.428419\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"202.038139\" y=\"37.563546\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"200.73491\" y=\"34.832031\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"199.422368\" y=\"35.699944\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"198.309283\" y=\"36.820336\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"199.098643\" y=\"35.652396\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"199.970878\" y=\"36.664534\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"202.412641\" y=\"35.107895\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"205.216255\" y=\"35.160076\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"205.458612\" y=\"34.365201\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"205.547441\" y=\"32.705303\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"206.989338\" y=\"33.133019\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"207.592873\" y=\"33.872911\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"206.439407\" y=\"35.807999\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"206.278081\" y=\"37.629671\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"207.648851\" y=\"37.091839\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"208.506868\" y=\"35.855871\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"209.309226\" y=\"38.261012\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"208.513026\" y=\"38.47571\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"206.119335\" y=\"37.970899\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"207.465565\" y=\"38.059905\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"206.907543\" y=\"38.426665\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"206.372028\" y=\"38.563807\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" <use xlink:href=\"#ma05db63bbb\" x=\"205.141235\" y=\"38.582955\" style=\"fill: #ff7f0e; stroke: #ff7f0e\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"PathCollection_1\"/>\n",
" <g id=\"PathCollection_2\">\n",
" <path d=\"M 133.389338 7.2 \n",
"L 131.354961 10.684614 \n",
"L 131.115628 11.149219 \n",
"L 129.618631 14.16923 \n",
"L 128.121641 17.653845 \n",
"L 128.064069 17.812225 \n",
"L 126.898922 21.138461 \n",
"L 125.900228 24.623076 \n",
"L 125.123465 28.107691 \n",
"L 125.012502 28.804598 \n",
"L 124.58421 31.592307 \n",
"L 124.262994 35.076923 \n",
"L 124.155921 38.561539 \n",
"L 124.262994 42.046154 \n",
"L 124.58421 45.530769 \n",
"L 125.012502 48.318478 \n",
"L 125.123465 49.015384 \n",
"L 125.900228 52.500001 \n",
"L 126.898922 55.984615 \n",
"L 128.064069 59.310851 \n",
"L 128.121641 59.469231 \n",
"L 129.618631 62.953845 \n",
"L 131.115628 65.973855 \n",
"L 131.354961 66.438459 \n",
"L 133.389338 69.923076 \n",
"L 134.167188 71.115182 \n",
"L 135.724107 73.407692 \n",
"L 137.218755 75.398907 \n",
"L 138.387435 76.892308 \n",
"L 140.270314 79.089128 \n",
"L 141.423126 80.37692 \n",
"L 143.321874 82.328309 \n",
"L 144.883135 83.861536 \n",
"L 146.373441 85.216671 \n",
"L 148.829573 87.346153 \n",
"L 149.425 87.826789 \n",
"L 152.476564 90.169895 \n",
"L 153.383783 90.830769 \n",
"L 155.528127 92.292059 \n",
"L 158.57969 94.25918 \n",
"L 158.672164 94.315385 \n",
"L 161.631253 96.004895 \n",
"L 164.682813 97.641606 \n",
"L 164.998499 97.800001 \n",
"L 167.734376 99.094286 \n",
"L 170.785939 100.438353 \n",
"L 172.861005 101.284618 \n",
"L 173.837499 101.66133 \n",
"L 176.889062 102.744387 \n",
"L 179.940626 103.733262 \n",
"L 182.992189 104.627962 \n",
"L 183.530712 104.769234 \n",
"L 186.04375 105.394675 \n",
"L 189.095313 106.064794 \n",
"L 192.146877 106.645562 \n",
"L 195.198438 107.136982 \n",
"L 198.250001 107.539054 \n",
"L 201.301564 107.851774 \n",
"L 204.353126 108.075148 \n",
"L 207.404689 108.20917 \n",
"L 210.456251 108.253846 \n",
"L 213.507813 108.20917 \n",
"L 216.559376 108.075148 \n",
"L 219.610939 107.851774 \n",
"L 222.662501 107.539054 \n",
"L 225.714063 107.136982 \n",
"L 228.765626 106.645562 \n",
"L 231.817188 106.064794 \n",
"L 234.868751 105.394675 \n",
"L 237.38179 104.769234 \n",
"L 237.920313 104.627962 \n",
"\" clip-path=\"url(#p9362706432)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_3\">\n",
" <path d=\"M 94.741 7.2 \n",
"L 94.496869 7.814946 \n",
"L 93.387217 10.684614 \n",
"L 92.198295 14.16923 \n",
"L 91.44531 16.715693 \n",
"L 91.174923 17.653845 \n",
"L 90.325119 21.138461 \n",
"L 89.629825 24.623076 \n",
"L 89.089044 28.107691 \n",
"L 88.702769 31.592307 \n",
"L 88.471007 35.076923 \n",
"L 88.393757 38.561206 \n",
"L 88.39375 38.561539 \n",
"L 88.393757 38.561871 \n",
"L 88.471007 42.046154 \n",
"L 88.702769 45.530769 \n",
"L 89.089044 49.015384 \n",
"L 89.629825 52.500001 \n",
"L 90.325119 55.984615 \n",
"L 91.174923 59.469231 \n",
"L 91.44531 60.407382 \n",
"L 92.198295 62.953845 \n",
"L 93.387217 66.438459 \n",
"L 94.496869 69.30813 \n",
"L 94.741 69.923076 \n",
"L 96.287127 73.407692 \n",
"L 97.548436 75.979666 \n",
"L 98.008263 76.892308 \n",
"L 99.931161 80.37692 \n",
"L 100.599996 81.491996 \n",
"L 102.061308 83.861536 \n",
"L 103.651563 86.249145 \n",
"L 104.403397 87.346153 \n",
"L 106.703122 90.470284 \n",
"L 106.976403 90.830769 \n",
"L 109.754682 94.25917 \n",
"L 109.801639 94.315385 \n",
"L 112.806249 97.694405 \n",
"L 112.903126 97.800001 \n",
"L 115.857816 100.836595 \n",
"L 116.308047 101.284618 \n",
"L 118.909375 103.733262 \n",
"L 120.04725 104.769234 \n",
"L 121.960942 106.422193 \n",
"L 124.155924 108.253846 \n",
"L 125.012502 108.93377 \n",
"L 128.064069 111.271016 \n",
"L 128.697415 111.738466 \n",
"L 131.115628 113.440255 \n",
"L 133.748347 115.223078 \n",
"L 134.167188 115.494103 \n",
"L 137.218755 117.391282 \n",
"L 139.426266 118.70769 \n",
"L 140.270314 119.189606 \n",
"L 143.321874 120.857774 \n",
"L 145.876677 122.192311 \n",
"L 146.373441 122.441211 \n",
"L 149.425 123.899059 \n",
"L 152.476564 125.285791 \n",
"L 153.383788 125.676923 \n",
"L 155.528127 126.565159 \n",
"L 158.57969 127.760863 \n",
"L 161.631253 128.888239 \n",
"L 162.418734 129.161535 \n",
"L 164.682813 129.917634 \n",
"L 167.734376 130.870972 \n",
"L 170.785939 131.758564 \n",
"L 173.837499 132.580406 \n",
"L 174.102859 132.646155 \n",
"L 176.889062 133.311398 \n",
"L 179.940626 133.976642 \n",
"L 182.992189 134.57853 \n",
"L 186.04375 135.117062 \n",
"L 189.095313 135.592236 \n",
"L 192.146877 136.004057 \n",
"L 193.256511 136.130768 \n",
"L 195.198438 136.344737 \n",
"L 198.250001 136.619839 \n",
"L 201.301564 136.833805 \n",
"L 204.353126 136.986639 \n",
"L 207.404689 137.078339 \n",
"L 210.456251 137.108907 \n",
"L 213.507813 137.078339 \n",
"L 216.559376 136.986639 \n",
"L 219.610939 136.833805 \n",
"L 222.662501 136.619839 \n",
"L 225.714063 136.344737 \n",
"L 227.655991 136.130768 \n",
"L 228.765626 136.004057 \n",
"L 231.817188 135.592236 \n",
"L 234.868751 135.117062 \n",
"L 237.920313 134.57853 \n",
"\" clip-path=\"url(#p9362706432)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_4\">\n",
" <path d=\"M 66.101283 7.2 \n",
"L 65.009145 10.684614 \n",
"L 64.045489 14.16923 \n",
"L 63.981244 14.437284 \n",
"L 63.226219 17.653845 \n",
"L 62.534115 21.138461 \n",
"L 61.967845 24.623076 \n",
"L 61.527415 28.107691 \n",
"L 61.21282 31.592307 \n",
"L 61.024065 35.076923 \n",
"L 60.961145 38.561539 \n",
"L 61.024065 42.046154 \n",
"L 61.21282 45.530769 \n",
"L 61.527415 49.015384 \n",
"L 61.967845 52.500001 \n",
"L 62.534115 55.984615 \n",
"L 63.226219 59.469231 \n",
"L 63.981244 62.685792 \n",
"L 64.045489 62.953845 \n",
"L 65.009145 66.438459 \n",
"L 66.101283 69.923076 \n",
"L 67.032818 72.582402 \n",
"L 67.328125 73.407692 \n",
"L 68.706252 76.892308 \n",
"L 70.084378 80.073917 \n",
"L 70.218509 80.37692 \n",
"L 71.895191 83.861536 \n",
"L 73.135938 86.249145 \n",
"L 73.718821 87.346153 \n",
"L 75.70748 90.830769 \n",
"L 76.187512 91.617635 \n",
"L 77.871125 94.315385 \n",
"L 79.239071 96.374484 \n",
"L 80.208385 97.800001 \n",
"L 82.290631 100.687261 \n",
"L 82.731817 101.284618 \n",
"L 85.342191 104.627962 \n",
"L 85.455214 104.769234 \n",
"L 88.39375 108.253846 \n",
"L 88.393757 108.253854 \n",
"L 91.44531 111.610969 \n",
"L 91.564212 111.738466 \n",
"L 94.496869 114.736843 \n",
"L 94.98513 115.223078 \n",
"L 97.548436 117.662305 \n",
"L 98.677098 118.70769 \n",
"L 100.599996 120.412926 \n",
"L 102.66303 122.192311 \n",
"L 103.651563 123.010127 \n",
"L 106.703122 125.463574 \n",
"L 106.976403 125.676923 \n",
"L 109.754682 127.760856 \n",
"L 111.679511 129.161535 \n",
"L 112.806249 129.950509 \n",
"L 115.857816 132.021554 \n",
"L 116.808306 132.646155 \n",
"L 118.909375 133.97664 \n",
"L 121.960942 135.845668 \n",
"L 122.442758 136.130768 \n",
"L 125.012502 137.597976 \n",
"L 128.064069 139.279152 \n",
"L 128.697416 139.615388 \n",
"L 131.115628 140.855672 \n",
"L 134.167188 142.361734 \n",
"L 135.724104 143.1 \n",
"\" clip-path=\"url(#p9362706432)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_5\">\n",
" <path d=\"M 42.620312 71.115188 \n",
"L 43.320213 73.407692 \n",
"L 44.496043 76.892308 \n",
"L 45.671872 80.073903 \n",
"L 45.785952 80.37692 \n",
"L 47.211915 83.861536 \n",
"L 48.723432 87.281609 \n",
"L 48.7525 87.346153 \n",
"L 50.438126 90.830769 \n",
"L 51.775006 93.41615 \n",
"L 52.249024 94.315385 \n",
"L 54.204398 97.800001 \n",
"L 54.826565 98.845393 \n",
"L 56.307021 101.284618 \n",
"L 57.878125 103.733263 \n",
"L 58.556253 104.769234 \n",
"L 60.929685 108.209162 \n",
"L 60.961151 108.253846 \n",
"L 63.540821 111.738466 \n",
"L 63.981244 112.305719 \n",
"L 66.294014 115.223078 \n",
"L 67.032818 116.113596 \n",
"L 69.231252 118.70769 \n",
"L 70.084378 119.671522 \n",
"L 72.364666 122.192311 \n",
"L 73.135938 123.010127 \n",
"L 75.70748 125.676923 \n",
"L 76.187512 126.155214 \n",
"L 79.239071 129.127391 \n",
"L 79.274952 129.161535 \n",
"L 82.290631 131.922942 \n",
"L 83.09947 132.646155 \n",
"L 85.342191 134.578536 \n",
"L 87.188189 136.130768 \n",
"L 88.393757 137.108915 \n",
"L 91.44531 139.52368 \n",
"L 91.564212 139.615388 \n",
"L 94.496869 141.800646 \n",
"L 96.287127 143.1 \n",
"\" clip-path=\"url(#p9362706432)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_6\">\n",
" <path d=\"M 42.620312 115.494102 \n",
"L 44.943982 118.70769 \n",
"L 45.671872 119.671517 \n",
"L 47.611182 122.192311 \n",
"L 48.723432 123.579037 \n",
"L 50.438126 125.676923 \n",
"L 51.775006 127.248424 \n",
"L 53.434101 129.161535 \n",
"L 54.826565 130.706605 \n",
"L 56.609163 132.646155 \n",
"L 57.878125 133.97664 \n",
"L 59.974143 136.130768 \n",
"L 60.929685 137.078341 \n",
"L 63.540815 139.615388 \n",
"L 63.981244 140.028815 \n",
"L 67.032818 142.834235 \n",
"L 67.32812 143.1 \n",
"\" clip-path=\"url(#p9362706432)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_7\">\n",
" <path d=\"M 42.620312 142.361735 \n",
"L 43.320214 143.1 \n",
"\" clip-path=\"url(#p9362706432)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"PathCollection_8\"/>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 42.620312 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 237.920313 143.1 \n",
"L 237.920313 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 237.920313 143.1 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 42.620312 7.2 \n",
"L 237.920313 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p9362706432\">\n",
" <rect x=\"42.620312\" y=\"7.2\" width=\"195.3\" height=\"135.9\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 252x180 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"def polynomial_lr():\n",
" # 在函数外部定义,而在内部更新的全局变量\n",
" global t\n",
" t += 1\n",
" return (1 + 0.1 * t) ** (-0.5)\n",
"\n",
"t = 1\n",
"lr = polynomial_lr\n",
"d2l.show_trace_2d(f, d2l.train_2d(sgd, steps=50, f_grad=f_grad))"
]
},
{
"cell_type": "markdown",
"id": "d5763565",
"metadata": {
"origin_pos": 14
},
"source": [
"关于如何设置学习率,还有更多的选择。例如,我们可以从较小的学习率开始,然后使其迅速上涨,再让它降低,尽管这会更慢。我们甚至可以在较小和较大的学习率之间切换。现在,让我们专注于可以进行全面理论分析的学习率计划,即凸环境下的学习率。对一般的非凸问题,很难获得有意义的收敛保证,因为总的来说,最大限度地减少非线性非凸问题是NP困难的。有关的研究调查,请参阅例如2015年Tibshirani的优秀[讲义笔记](https://www.stat.cmu.edu/~ryantibs/convexopt-F15/lectures/26-nonconvex.pdf)。\n",
"\n",
"## 凸目标的收敛性分析\n",
"\n",
"以下对凸目标函数的随机梯度下降的收敛性分析是可选读的,主要用于传达对问题的更多直觉。我们只限于最简单的证明之一 :cite:`Nesterov.Vial.2000`。存在着明显更先进的证明技术,例如,当目标函数表现特别好时。\n",
"\n",
"假设所有$\\boldsymbol{\\xi}$的目标函数$f(\\boldsymbol{\\xi}, \\mathbf{x})$在$\\mathbf{x}$中都是凸的。更具体地说,我们考虑随机梯度下降更新:\n",
"\n",
"$$\\mathbf{x}_{t+1} = \\mathbf{x}_{t} - \\eta_t \\partial_\\mathbf{x} f(\\boldsymbol{\\xi}_t, \\mathbf{x}),$$\n",
"\n",
"其中$f(\\boldsymbol{\\xi}_t, \\mathbf{x})$是训练样本$f(\\boldsymbol{\\xi}_t, \\mathbf{x})$的目标函数:$\\boldsymbol{\\xi}_t$从第$t$步的某个分布中提取,$\\mathbf{x}$是模型参数。用\n",
"\n",
"$$R(\\mathbf{x}) = E_{\\boldsymbol{\\xi}}[f(\\boldsymbol{\\xi}, \\mathbf{x})]$$\n",
"\n",
"表示期望风险,$R^*$表示对于$\\mathbf{x}$的最低风险。最后让$\\mathbf{x}^*$表示最小值(我们假设它存在于定义$\\mathbf{x}$的域中)。在这种情况下,我们可以跟踪时间$t$处的当前参数$\\mathbf{x}_t$和风险最小化器$\\mathbf{x}^*$之间的距离,看看它是否随着时间的推移而改善:\n",
"\n",
"$$\\begin{aligned} &\\|\\mathbf{x}_{t+1} - \\mathbf{x}^*\\|^2 \\\\ =& \\|\\mathbf{x}_{t} - \\eta_t \\partial_\\mathbf{x} f(\\boldsymbol{\\xi}_t, \\mathbf{x}) - \\mathbf{x}^*\\|^2 \\\\ =& \\|\\mathbf{x}_{t} - \\mathbf{x}^*\\|^2 + \\eta_t^2 \\|\\partial_\\mathbf{x} f(\\boldsymbol{\\xi}_t, \\mathbf{x})\\|^2 - 2 \\eta_t \\left\\langle \\mathbf{x}_t - \\mathbf{x}^*, \\partial_\\mathbf{x} f(\\boldsymbol{\\xi}_t, \\mathbf{x})\\right\\rangle. \\end{aligned}$$\n",
":eqlabel:`eq_sgd-xt+1-xstar`\n",
"\n",
"我们假设随机梯度$\\partial_\\mathbf{x} f(\\boldsymbol{\\xi}_t, \\mathbf{x})$的$L_2$范数受到某个常数$L$的限制,因此我们有\n",
"\n",
"$$\\eta_t^2 \\|\\partial_\\mathbf{x} f(\\boldsymbol{\\xi}_t, \\mathbf{x})\\|^2 \\leq \\eta_t^2 L^2.$$\n",
":eqlabel:`eq_sgd-L`\n",
"\n",
"我们最感兴趣的是$\\mathbf{x}_t$和$\\mathbf{x}^*$之间的距离如何变化的*期望*。事实上,对于任何具体的步骤序列,距离可能会增加,这取决于我们遇到的$\\boldsymbol{\\xi}_t$。因此我们需要点积的边界。因为对于任何凸函数$f$,所有$\\mathbf{x}$和$\\mathbf{y}$都满足$f(\\mathbf{y}) \\geq f(\\mathbf{x}) + \\langle f'(\\mathbf{x}), \\mathbf{y} - \\mathbf{x} \\rangle$,按凸性我们有\n",
"\n",
"$$f(\\boldsymbol{\\xi}_t, \\mathbf{x}^*) \\geq f(\\boldsymbol{\\xi}_t, \\mathbf{x}_t) + \\left\\langle \\mathbf{x}^* - \\mathbf{x}_t, \\partial_{\\mathbf{x}} f(\\boldsymbol{\\xi}_t, \\mathbf{x}_t) \\right\\rangle.$$\n",
":eqlabel:`eq_sgd-f-xi-xstar`\n",
"\n",
"将不等式 :eqref:`eq_sgd-L`和 :eqref:`eq_sgd-f-xi-xstar`代入 :eqref:`eq_sgd-xt+1-xstar`我们在时间$t+1$时获得参数之间距离的边界,如下所示:\n",
"\n",
"$$\\|\\mathbf{x}_{t} - \\mathbf{x}^*\\|^2 - \\|\\mathbf{x}_{t+1} - \\mathbf{x}^*\\|^2 \\geq 2 \\eta_t (f(\\boldsymbol{\\xi}_t, \\mathbf{x}_t) - f(\\boldsymbol{\\xi}_t, \\mathbf{x}^*)) - \\eta_t^2 L^2.$$\n",
":eqlabel:`eqref_sgd-xt-diff`\n",
"\n",
"这意味着,只要当前损失和最优损失之间的差异超过$\\eta_t L^2/2$,我们就会取得进展。由于这种差异必然会收敛到零,因此学习率$\\eta_t$也需要*消失*。\n",
"\n",
"接下来,我们根据 :eqref:`eqref_sgd-xt-diff`取期望。得到\n",
"\n",
"$$E\\left[\\|\\mathbf{x}_{t} - \\mathbf{x}^*\\|^2\\right] - E\\left[\\|\\mathbf{x}_{t+1} - \\mathbf{x}^*\\|^2\\right] \\geq 2 \\eta_t [E[R(\\mathbf{x}_t)] - R^*] - \\eta_t^2 L^2.$$\n",
"\n",
"最后一步是对$t \\in \\{1, \\ldots, T\\}$的不等式求和。在求和过程中抵消中间项,然后舍去低阶项,可以得到\n",
"\n",
"$$\\|\\mathbf{x}_1 - \\mathbf{x}^*\\|^2 \\geq 2 \\left (\\sum_{t=1}^T \\eta_t \\right) [E[R(\\mathbf{x}_t)] - R^*] - L^2 \\sum_{t=1}^T \\eta_t^2.$$\n",
":eqlabel:`eq_sgd-x1-xstar`\n",
"\n",
"请注意,我们利用了给定的$\\mathbf{x}_1$,因而可以去掉期望。最后定义\n",
"\n",
"$$\\bar{\\mathbf{x}} \\stackrel{\\mathrm{def}}{=} \\frac{\\sum_{t=1}^T \\eta_t \\mathbf{x}_t}{\\sum_{t=1}^T \\eta_t}.$$\n",
"\n",
"因为有\n",
"\n",
"$$E\\left(\\frac{\\sum_{t=1}^T \\eta_t R(\\mathbf{x}_t)}{\\sum_{t=1}^T \\eta_t}\\right) = \\frac{\\sum_{t=1}^T \\eta_t E[R(\\mathbf{x}_t)]}{\\sum_{t=1}^T \\eta_t} = E[R(\\mathbf{x}_t)],$$\n",
"\n",
"根据詹森不等式(令 :eqref:`eq_jensens-inequality`中$i=t$$\\alpha_i = \\eta_t/\\sum_{t=1}^T \\eta_t$)和$R$的凸性使其满足的$E[R(\\mathbf{x}_t)] \\geq E[R(\\bar{\\mathbf{x}})]$,因此,\n",
"\n",
"$$\\sum_{t=1}^T \\eta_t E[R(\\mathbf{x}_t)] \\geq \\sum_{t=1}^T \\eta_t E\\left[R(\\bar{\\mathbf{x}})\\right].$$\n",
"\n",
"将其代入不等式 :eqref:`eq_sgd-x1-xstar`得到边界\n",
"\n",
"$$\n",
"\\left[E[\\bar{\\mathbf{x}}]\\right] - R^* \\leq \\frac{r^2 + L^2 \\sum_{t=1}^T \\eta_t^2}{2 \\sum_{t=1}^T \\eta_t},\n",
"$$\n",
"\n",
"其中$r^2 \\stackrel{\\mathrm{def}}{=} \\|\\mathbf{x}_1 - \\mathbf{x}^*\\|^2$是初始选择参数与最终结果之间距离的边界。简而言之,收敛速度取决于随机梯度标准的限制方式($L$)以及初始参数值与最优结果的距离($r$)。请注意,边界由$\\bar{\\mathbf{x}}$而不是$\\mathbf{x}_T$表示。因为$\\bar{\\mathbf{x}}$是优化路径的平滑版本。只要知道$r, L$和$T$,我们就可以选择学习率$\\eta = r/(L \\sqrt{T})$。这个就是上界$rL/\\sqrt{T}$。也就是说,我们将按照速度$\\mathcal{O}(1/\\sqrt{T})$收敛到最优解。\n",
"\n",
"## 随机梯度和有限样本\n",
"\n",
"到目前为止,在谈论随机梯度下降时,我们进行得有点快而松散。我们假设从分布$p(x, y)$中采样得到样本$x_i$(通常带有标签$y_i$),并且用它来以某种方式更新模型参数。特别是,对于有限的样本数量,我们仅仅讨论了由某些允许我们在其上执行随机梯度下降的函数$\\delta_{x_i}$和$\\delta_{y_i}$组成的离散分布$p(x, y) = \\frac{1}{n} \\sum_{i=1}^n \\delta_{x_i}(x) \\delta_{y_i}(y)$。\n",
"\n",
"但是,这不是我们真正做的。在本节的简单示例中,我们只是将噪声添加到其他非随机梯度上,也就是说,我们假装有成对的$(x_i, y_i)$。事实证明,这种做法在这里是合理的(有关详细讨论,请参阅练习)。更麻烦的是,在以前的所有讨论中,我们显然没有这样做。相反,我们遍历了所有实例*恰好一次*。要了解为什么这更可取,可以反向考虑一下,即我们*有替换地*从离散分布中采样$n$个观测值。随机选择一个元素$i$的概率是$1/n$。因此选择它*至少*一次就是\n",
"\n",
"$$P(\\mathrm{choose~} i) = 1 - P(\\mathrm{omit~} i) = 1 - (1-1/n)^n \\approx 1-e^{-1} \\approx 0.63.$$\n",
"\n",
"类似的推理表明,挑选一些样本(即训练示例)*恰好一次*的概率是\n",
"\n",
"$${n \\choose 1} \\frac{1}{n} \\left(1-\\frac{1}{n}\\right)^{n-1} = \\frac{n}{n-1} \\left(1-\\frac{1}{n}\\right)^{n} \\approx e^{-1} \\approx 0.37.$$\n",
"\n",
"这导致与*无替换*采样相比,方差增加并且数据效率降低。因此,在实践中我们执行后者(这是本书中的默认选择)。最后一点注意,重复采用训练数据集的时候,会以*不同的*随机顺序遍历它。\n",
"\n",
"## 小结\n",
"\n",
"* 对于凸问题,我们可以证明,对于广泛的学习率选择,随机梯度下降将收敛到最优解。\n",
"* 对于深度学习而言,情况通常并非如此。但是,对凸问题的分析使我们能够深入了解如何进行优化,即逐步降低学习率,尽管不是太快。\n",
"* 如果学习率太小或太大,就会出现问题。实际上,通常只有经过多次实验后才能找到合适的学习率。\n",
"* 当训练数据集中有更多样本时,计算梯度下降的每次迭代的代价更高,因此在这些情况下,首选随机梯度下降。\n",
"* 随机梯度下降的最优性保证在非凸情况下一般不可用,因为需要检查的局部最小值的数量可能是指数级的。\n",
"\n",
"## 练习\n",
"\n",
"1. 尝试不同的随机梯度下降学习率计划和不同的迭代次数进行实验。特别是,根据迭代次数的函数来绘制与最优解$(0, 0)$的距离。\n",
"1. 证明对于函数$f(x_1, x_2) = x_1^2 + 2 x_2^2$而言,向梯度添加正态噪声等同于最小化损失函数$f(\\mathbf{x}, \\mathbf{w}) = (x_1 - w_1)^2 + 2 (x_2 - w_2)^2$,其中$\\mathbf{x}$是从正态分布中提取的。\n",
"1. 从$\\{(x_1, y_1), \\ldots, (x_n, y_n)\\}$分别使用替换方法以及不替换方法进行采样时,比较随机梯度下降的收敛性。\n",
"1. 如果某些梯度(或者更确切地说与之相关的某些坐标)始终比所有其他梯度都大,将如何更改随机梯度下降求解器?\n",
"1. 假设$f(x) = x^2 (1 + \\sin x)$。$f$有多少局部最小值?请试着改变$f$以尽量减少它需要评估所有局部最小值的方式。\n"
]
},
{
"cell_type": "markdown",
"id": "7507d8a1",
"metadata": {
"origin_pos": 16,
"tab": [
"pytorch"
]
},
"source": [
"[Discussions](https://discuss.d2l.ai/t/3838)\n"
]
}
],
"metadata": {
"language_info": {
"name": "python"
},
"required_libs": []
},
"nbformat": 4,
"nbformat_minor": 5
}