Files
Python/d2l/d2l-zh/pytorch/chapter_optimization/optimization-intro.ipynb
T
2025-12-16 09:23:53 +08:00

6383 lines
232 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"cells": [
{
"cell_type": "markdown",
"id": "12918bab",
"metadata": {
"origin_pos": 0
},
"source": [
"# 优化和深度学习\n",
"\n",
"本节将讨论优化与深度学习之间的关系以及在深度学习中使用优化的挑战。对于深度学习问题,我们通常会先定义*损失函数*。一旦我们有了损失函数,我们就可以使用优化算法来尝试最小化损失。在优化中,损失函数通常被称为优化问题的*目标函数*。按照传统惯例,大多数优化算法都关注的是*最小化*。如果我们需要最大化目标,那么有一个简单的解决方案:在目标函数前加负号即可。\n",
"\n",
"## 优化的目标\n",
"\n",
"尽管优化提供了一种最大限度地减少深度学习损失函数的方法,但本质上,优化和深度学习的目标是根本不同的。前者主要关注的是最小化目标,后者则关注在给定有限数据量的情况下寻找合适的模型。在 :numref:`sec_model_selection`中,我们详细讨论了这两个目标之间的区别。例如,训练误差和泛化误差通常不同:由于优化算法的目标函数通常是基于训练数据集的损失函数,因此优化的目标是减少训练误差。但是,深度学习(或更广义地说,统计推断)的目标是减少泛化误差。为了实现后者,除了使用优化算法来减少训练误差之外,我们还需要注意过拟合。\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "3a6091f3",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:07:06.516685Z",
"iopub.status.busy": "2023-08-18T07:07:06.515981Z",
"iopub.status.idle": "2023-08-18T07:07:08.496750Z",
"shell.execute_reply": "2023-08-18T07:07:08.495844Z"
},
"origin_pos": 2,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import numpy as np\n",
"import torch\n",
"from mpl_toolkits import mplot3d\n",
"from d2l import torch as d2l"
]
},
{
"cell_type": "markdown",
"id": "9f760af7",
"metadata": {
"origin_pos": 5
},
"source": [
"为了说明上述不同的目标,引入两个概念*风险*和*经验风险*。如 :numref:`subsec_empirical-risk-and-risk`所述,经验风险是训练数据集的平均损失,而风险则是整个数据群的预期损失。下面我们定义了两个函数:风险函数`f`和经验风险函数`g`。假设我们只有有限的训练数据。因此,这里的`g`不如`f`平滑。\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "d5f7a949",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:07:08.500835Z",
"iopub.status.busy": "2023-08-18T07:07:08.500449Z",
"iopub.status.idle": "2023-08-18T07:07:08.505447Z",
"shell.execute_reply": "2023-08-18T07:07:08.504568Z"
},
"origin_pos": 6,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"def f(x):\n",
" return x * torch.cos(np.pi * x)\n",
"\n",
"def g(x):\n",
" return f(x) + 0.2 * torch.cos(5 * np.pi * x)"
]
},
{
"cell_type": "markdown",
"id": "46724f04",
"metadata": {
"origin_pos": 7
},
"source": [
"下图说明,训练数据集的最低经验风险可能与最低风险(泛化误差)不同。\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "66a76e85",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:07:08.509166Z",
"iopub.status.busy": "2023-08-18T07:07:08.508582Z",
"iopub.status.idle": "2023-08-18T07:07:08.781243Z",
"shell.execute_reply": "2023-08-18T07:07:08.779982Z"
},
"origin_pos": 8,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"261.023438pt\" height=\"180.65625pt\" viewBox=\"0 0 261.023438 180.65625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T07:07:08.726154</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 180.65625 \n",
"L 261.023438 180.65625 \n",
"L 261.023438 0 \n",
"L 0 0 \n",
"L 0 180.65625 \n",
"z\n",
"\" style=\"fill: none\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 58.523438 143.1 \n",
"L 253.823438 143.1 \n",
"L 253.823438 7.2 \n",
"L 58.523438 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 85.334594 143.1 \n",
"L 85.334594 7.2 \n",
"\" clip-path=\"url(#p90bc1b5a97)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_2\">\n",
" <defs>\n",
" <path id=\"m46357cc473\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m46357cc473\" x=\"85.334594\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 0.6 -->\n",
" <g transform=\"translate(77.383032 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
"L 1344 794 \n",
"L 1344 0 \n",
"L 684 0 \n",
"L 684 794 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
"Q 1688 2584 1439 2293 \n",
"Q 1191 2003 1191 1497 \n",
"Q 1191 994 1439 701 \n",
"Q 1688 409 2113 409 \n",
"Q 2538 409 2786 701 \n",
"Q 3034 994 3034 1497 \n",
"Q 3034 2003 2786 2293 \n",
"Q 2538 2584 2113 2584 \n",
"z\n",
"M 3366 4563 \n",
"L 3366 3988 \n",
"Q 3128 4100 2886 4159 \n",
"Q 2644 4219 2406 4219 \n",
"Q 1781 4219 1451 3797 \n",
"Q 1122 3375 1075 2522 \n",
"Q 1259 2794 1537 2939 \n",
"Q 1816 3084 2150 3084 \n",
"Q 2853 3084 3261 2657 \n",
"Q 3669 2231 3669 1497 \n",
"Q 3669 778 3244 343 \n",
"Q 2819 -91 2113 -91 \n",
"Q 1303 -91 875 529 \n",
"Q 447 1150 447 2328 \n",
"Q 447 3434 972 4092 \n",
"Q 1497 4750 2381 4750 \n",
"Q 2619 4750 2861 4703 \n",
"Q 3103 4656 3366 4563 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-36\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 121.202363 143.1 \n",
"L 121.202363 7.2 \n",
"\" clip-path=\"url(#p90bc1b5a97)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use xlink:href=\"#m46357cc473\" x=\"121.202363\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 0.8 -->\n",
" <g transform=\"translate(113.2508 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
"Q 1584 2216 1326 1975 \n",
"Q 1069 1734 1069 1313 \n",
"Q 1069 891 1326 650 \n",
"Q 1584 409 2034 409 \n",
"Q 2484 409 2743 651 \n",
"Q 3003 894 3003 1313 \n",
"Q 3003 1734 2745 1975 \n",
"Q 2488 2216 2034 2216 \n",
"z\n",
"M 1403 2484 \n",
"Q 997 2584 770 2862 \n",
"Q 544 3141 544 3541 \n",
"Q 544 4100 942 4425 \n",
"Q 1341 4750 2034 4750 \n",
"Q 2731 4750 3128 4425 \n",
"Q 3525 4100 3525 3541 \n",
"Q 3525 3141 3298 2862 \n",
"Q 3072 2584 2669 2484 \n",
"Q 3125 2378 3379 2068 \n",
"Q 3634 1759 3634 1313 \n",
"Q 3634 634 3220 271 \n",
"Q 2806 -91 2034 -91 \n",
"Q 1263 -91 848 271 \n",
"Q 434 634 434 1313 \n",
"Q 434 1759 690 2068 \n",
"Q 947 2378 1403 2484 \n",
"z\n",
"M 1172 3481 \n",
"Q 1172 3119 1398 2916 \n",
"Q 1625 2713 2034 2713 \n",
"Q 2441 2713 2670 2916 \n",
"Q 2900 3119 2900 3481 \n",
"Q 2900 3844 2670 4047 \n",
"Q 2441 4250 2034 4250 \n",
"Q 1625 4250 1398 4047 \n",
"Q 1172 3844 1172 3481 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-38\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 157.070131 143.1 \n",
"L 157.070131 7.2 \n",
"\" clip-path=\"url(#p90bc1b5a97)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#m46357cc473\" x=\"157.070131\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 1.0 -->\n",
" <g transform=\"translate(149.118568 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 192.937899 143.1 \n",
"L 192.937899 7.2 \n",
"\" clip-path=\"url(#p90bc1b5a97)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use xlink:href=\"#m46357cc473\" x=\"192.937899\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 1.2 -->\n",
" <g transform=\"translate(184.986337 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 228.805667 143.1 \n",
"L 228.805667 7.2 \n",
"\" clip-path=\"url(#p90bc1b5a97)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use xlink:href=\"#m46357cc473\" x=\"228.805667\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 1.4 -->\n",
" <g transform=\"translate(220.854105 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
"L 825 1625 \n",
"L 2419 1625 \n",
"L 2419 4116 \n",
"z\n",
"M 2253 4666 \n",
"L 3047 4666 \n",
"L 3047 1625 \n",
"L 3713 1625 \n",
"L 3713 1100 \n",
"L 3047 1100 \n",
"L 3047 0 \n",
"L 2419 0 \n",
"L 2419 1100 \n",
"L 313 1100 \n",
"L 313 1709 \n",
"L 2253 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-34\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- x -->\n",
" <g transform=\"translate(153.214063 171.376563)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 58.523438 141.197161 \n",
"L 253.823438 141.197161 \n",
"\" clip-path=\"url(#p90bc1b5a97)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_12\">\n",
" <defs>\n",
" <path id=\"m28b64b8249\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m28b64b8249\" x=\"58.523438\" y=\"141.197161\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- 1.25 -->\n",
" <g transform=\"translate(20.878125 144.996379)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
"L 4684 2272 \n",
"L 4684 1741 \n",
"L 678 1741 \n",
"L 678 2272 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
"L 3169 4666 \n",
"L 3169 4134 \n",
"L 1269 4134 \n",
"L 1269 2991 \n",
"Q 1406 3038 1543 3061 \n",
"Q 1681 3084 1819 3084 \n",
"Q 2600 3084 3056 2656 \n",
"Q 3513 2228 3513 1497 \n",
"Q 3513 744 3044 326 \n",
"Q 2575 -91 1722 -91 \n",
"Q 1428 -91 1123 -41 \n",
"Q 819 9 494 109 \n",
"L 494 744 \n",
"Q 775 591 1075 516 \n",
"Q 1375 441 1709 441 \n",
"Q 2250 441 2565 725 \n",
"Q 2881 1009 2881 1497 \n",
"Q 2881 1984 2565 2268 \n",
"Q 2250 2553 1709 2553 \n",
"Q 1456 2553 1204 2497 \n",
"Q 953 2441 691 2322 \n",
"L 691 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"179.199219\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"242.822266\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_13\">\n",
" <path d=\"M 58.523438 115.633183 \n",
"L 253.823438 115.633183 \n",
"\" clip-path=\"url(#p90bc1b5a97)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use xlink:href=\"#m28b64b8249\" x=\"58.523438\" y=\"115.633183\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 1.00 -->\n",
" <g transform=\"translate(20.878125 119.432402)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"179.199219\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"242.822266\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_15\">\n",
" <path d=\"M 58.523438 90.069206 \n",
"L 253.823438 90.069206 \n",
"\" clip-path=\"url(#p90bc1b5a97)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use xlink:href=\"#m28b64b8249\" x=\"58.523438\" y=\"90.069206\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 0.75 -->\n",
" <g transform=\"translate(20.878125 93.868424)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-37\" d=\"M 525 4666 \n",
"L 3525 4666 \n",
"L 3525 4397 \n",
"L 1831 0 \n",
"L 1172 0 \n",
"L 2766 4134 \n",
"L 525 4134 \n",
"L 525 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-37\" x=\"179.199219\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"242.822266\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_17\">\n",
" <path d=\"M 58.523438 64.505228 \n",
"L 253.823438 64.505228 \n",
"\" clip-path=\"url(#p90bc1b5a97)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_18\">\n",
" <g>\n",
" <use xlink:href=\"#m28b64b8249\" x=\"58.523438\" y=\"64.505228\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 0.50 -->\n",
" <g transform=\"translate(20.878125 68.304447)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"179.199219\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"242.822266\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_19\">\n",
" <path d=\"M 58.523438 38.941251 \n",
"L 253.823438 38.941251 \n",
"\" clip-path=\"url(#p90bc1b5a97)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_20\">\n",
" <g>\n",
" <use xlink:href=\"#m28b64b8249\" x=\"58.523438\" y=\"38.941251\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- 0.25 -->\n",
" <g transform=\"translate(20.878125 42.74047)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"179.199219\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"242.822266\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_6\">\n",
" <g id=\"line2d_21\">\n",
" <path d=\"M 58.523438 13.377273 \n",
"L 253.823438 13.377273 \n",
"\" clip-path=\"url(#p90bc1b5a97)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_22\">\n",
" <g>\n",
" <use xlink:href=\"#m28b64b8249\" x=\"58.523438\" y=\"13.377273\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_12\">\n",
" <!-- 0.00 -->\n",
" <g transform=\"translate(29.257812 17.176492)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"159.033203\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_13\">\n",
" <!-- risk -->\n",
" <g transform=\"translate(14.798437 84.094531)rotate(-90)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
"Q 2534 3019 2420 3045 \n",
"Q 2306 3072 2169 3072 \n",
"Q 1681 3072 1420 2755 \n",
"Q 1159 2438 1159 1844 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1341 3275 1631 3429 \n",
"Q 1922 3584 2338 3584 \n",
"Q 2397 3584 2469 3576 \n",
"Q 2541 3569 2628 3553 \n",
"L 2631 2963 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
"L 1178 3500 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 3500 \n",
"z\n",
"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 4134 \n",
"L 603 4134 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
"L 2834 2853 \n",
"Q 2591 2978 2328 3040 \n",
"Q 2066 3103 1784 3103 \n",
"Q 1356 3103 1142 2972 \n",
"Q 928 2841 928 2578 \n",
"Q 928 2378 1081 2264 \n",
"Q 1234 2150 1697 2047 \n",
"L 1894 2003 \n",
"Q 2506 1872 2764 1633 \n",
"Q 3022 1394 3022 966 \n",
"Q 3022 478 2636 193 \n",
"Q 2250 -91 1575 -91 \n",
"Q 1294 -91 989 -36 \n",
"Q 684 19 347 128 \n",
"L 347 722 \n",
"Q 666 556 975 473 \n",
"Q 1284 391 1588 391 \n",
"Q 1994 391 2212 530 \n",
"Q 2431 669 2431 922 \n",
"Q 2431 1156 2273 1281 \n",
"Q 2116 1406 1581 1522 \n",
"L 1381 1569 \n",
"Q 847 1681 609 1914 \n",
"Q 372 2147 372 2553 \n",
"Q 372 3047 722 3315 \n",
"Q 1072 3584 1716 3584 \n",
"Q 2034 3584 2315 3537 \n",
"Q 2597 3491 2834 3397 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6b\" d=\"M 581 4863 \n",
"L 1159 4863 \n",
"L 1159 1991 \n",
"L 2875 3500 \n",
"L 3609 3500 \n",
"L 1753 1863 \n",
"L 3688 0 \n",
"L 2938 0 \n",
"L 1159 1709 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-72\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"41.113281\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"68.896484\"/>\n",
" <use xlink:href=\"#DejaVuSans-6b\" x=\"120.996094\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_23\">\n",
" <path d=\"M 67.40071 13.377276 \n",
"L 69.194097 15.015363 \n",
"L 70.987484 16.716035 \n",
"L 72.78087 18.47753 \n",
"L 74.574268 20.297953 \n",
"L 76.367654 22.175271 \n",
"L 78.161041 24.107348 \n",
"L 79.954428 26.091942 \n",
"L 81.747815 28.126682 \n",
"L 83.541201 30.209078 \n",
"L 85.334588 32.336558 \n",
"L 87.127975 34.506422 \n",
"L 88.921372 36.715883 \n",
"L 90.714759 38.962031 \n",
"L 92.508145 41.241888 \n",
"L 94.301532 43.552353 \n",
"L 96.09493 45.890309 \n",
"L 97.888316 48.252453 \n",
"L 99.681703 50.635472 \n",
"L 101.47509 53.035965 \n",
"L 103.268487 55.450449 \n",
"L 105.061874 57.87538 \n",
"L 106.85526 60.307155 \n",
"L 108.648647 62.742112 \n",
"L 110.442034 65.176524 \n",
"L 112.235421 67.606656 \n",
"L 114.028807 70.028701 \n",
"L 115.822194 72.438806 \n",
"L 117.615591 74.833125 \n",
"L 119.408978 77.207715 \n",
"L 121.202365 79.558693 \n",
"L 122.995751 81.882116 \n",
"L 124.789138 84.174046 \n",
"L 126.582525 86.430522 \n",
"L 128.375912 88.647594 \n",
"L 130.169298 90.821344 \n",
"L 131.962696 92.947846 \n",
"L 133.756082 95.023156 \n",
"L 135.549469 97.043399 \n",
"L 137.342856 99.004722 \n",
"L 139.136242 100.903304 \n",
"L 140.929629 102.735366 \n",
"L 142.723016 104.497154 \n",
"L 144.516403 106.184998 \n",
"L 146.3098 107.795265 \n",
"L 148.103187 109.324397 \n",
"L 149.896573 110.768883 \n",
"L 151.68996 112.125303 \n",
"L 153.483357 113.390323 \n",
"L 155.276744 114.560671 \n",
"L 157.070131 115.633183 \n",
"L 158.863518 116.604776 \n",
"L 160.656904 117.472486 \n",
"L 162.450291 118.233425 \n",
"L 164.243678 118.884851 \n",
"L 166.037086 119.424094 \n",
"L 167.830451 119.84863 \n",
"L 169.623838 120.156058 \n",
"L 171.417224 120.344086 \n",
"L 173.210611 120.410594 \n",
"L 175.003998 120.353534 \n",
"L 176.797385 120.171064 \n",
"L 178.590771 119.861429 \n",
"L 180.384179 119.42307 \n",
"L 182.177566 118.854535 \n",
"L 183.970953 118.154557 \n",
"L 185.764339 117.322014 \n",
"L 187.557726 116.355932 \n",
"L 189.351113 115.255548 \n",
"L 191.1445 114.020191 \n",
"L 192.937886 112.649399 \n",
"L 194.731294 111.142874 \n",
"L 196.524681 109.500498 \n",
"L 198.318068 107.722309 \n",
"L 200.111454 105.808532 \n",
"L 201.904841 103.759564 \n",
"L 203.698228 101.575977 \n",
"L 205.491615 99.258546 \n",
"L 207.285001 96.808123 \n",
"L 209.078409 94.225897 \n",
"L 210.871775 91.513161 \n",
"L 212.665161 88.67131 \n",
"L 214.458548 85.70202 \n",
"L 216.251935 82.607113 \n",
"L 218.045321 79.388589 \n",
"L 219.838708 76.04863 \n",
"L 221.632095 72.589576 \n",
"L 223.425503 69.01392 \n",
"L 225.21889 65.324447 \n",
"L 227.012276 61.524047 \n",
"L 228.805663 57.615626 \n",
"L 230.59905 53.602455 \n",
"L 232.392436 49.487901 \n",
"L 234.185823 45.275486 \n",
"L 235.97921 40.968899 \n",
"L 237.772618 36.571916 \n",
"L 239.566005 32.088663 \n",
"L 241.359391 27.523209 \n",
"L 243.152778 22.879915 \n",
"L 244.946165 18.163058 \n",
"\" clip-path=\"url(#p90bc1b5a97)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_24\">\n",
" <path d=\"M 67.40071 13.377273 \n",
"L 69.194097 18.214622 \n",
"L 70.987484 23.035789 \n",
"L 72.78087 27.762165 \n",
"L 74.574268 32.318866 \n",
"L 76.367654 36.636436 \n",
"L 78.161041 40.652697 \n",
"L 79.954428 44.314078 \n",
"L 81.747815 47.576909 \n",
"L 83.541201 50.408469 \n",
"L 85.334588 52.78774 \n",
"L 87.127975 54.70582 \n",
"L 88.921372 56.166112 \n",
"L 90.714759 57.184166 \n",
"L 92.508145 57.787247 \n",
"L 94.301532 58.01353 \n",
"L 96.09493 57.911206 \n",
"L 97.888316 57.537087 \n",
"L 99.681703 56.955244 \n",
"L 101.47509 56.235241 \n",
"L 103.268487 55.450437 \n",
"L 105.061874 54.676095 \n",
"L 106.85526 53.987395 \n",
"L 108.648647 53.457472 \n",
"L 110.442034 53.155621 \n",
"L 112.235421 53.145485 \n",
"L 114.028807 53.483345 \n",
"L 115.822194 54.216674 \n",
"L 117.615591 55.382893 \n",
"L 119.408978 57.008322 \n",
"L 121.202365 59.107512 \n",
"L 122.995751 61.682719 \n",
"L 124.789138 64.723811 \n",
"L 126.582525 68.208384 \n",
"L 128.375912 72.102238 \n",
"L 130.169298 76.360154 \n",
"L 131.962696 80.926937 \n",
"L 133.756082 85.73851 \n",
"L 135.549469 90.723636 \n",
"L 137.342856 95.805453 \n",
"L 139.136242 100.903286 \n",
"L 140.929629 105.934618 \n",
"L 142.723016 110.816899 \n",
"L 144.516403 115.469626 \n",
"L 146.3098 119.816156 \n",
"L 148.103187 123.785563 \n",
"L 149.896573 127.314233 \n",
"L 151.68996 130.347434 \n",
"L 153.483357 132.840552 \n",
"L 155.276744 134.760064 \n",
"L 157.070131 136.08437 \n",
"L 158.863518 136.804169 \n",
"L 160.656904 136.922727 \n",
"L 162.450291 136.455575 \n",
"L 164.243678 135.430214 \n",
"L 166.037086 133.885247 \n",
"L 167.830451 131.869545 \n",
"L 169.623838 129.440717 \n",
"L 171.417224 126.663868 \n",
"L 173.210611 123.609919 \n",
"L 175.003998 120.353582 \n",
"L 176.797385 116.971849 \n",
"L 178.590771 113.541715 \n",
"L 180.384179 110.138436 \n",
"L 182.177566 106.833638 \n",
"L 183.970953 103.693391 \n",
"L 185.764339 100.776664 \n",
"L 187.557726 98.133794 \n",
"L 189.351113 95.805331 \n",
"L 191.1445 93.820804 \n",
"L 192.937886 92.198219 \n",
"L 194.731294 90.94348 \n",
"L 196.524681 90.050269 \n",
"L 198.318068 89.500171 \n",
"L 200.111454 89.263182 \n",
"L 201.904841 89.298399 \n",
"L 203.698228 89.55508 \n",
"L 205.491615 89.973875 \n",
"L 207.285001 90.488336 \n",
"L 209.078409 91.02664 \n",
"L 210.871775 91.513137 \n",
"L 212.665161 91.870555 \n",
"L 214.458548 92.021764 \n",
"L 216.251935 91.891735 \n",
"L 218.045321 91.409474 \n",
"L 219.838708 90.509759 \n",
"L 221.632095 89.134896 \n",
"L 223.425503 87.236052 \n",
"L 225.21889 84.774677 \n",
"L 227.012276 81.72344 \n",
"L 228.805663 78.066807 \n",
"L 230.59905 73.801851 \n",
"L 232.392436 68.938136 \n",
"L 234.185823 63.497627 \n",
"L 235.97921 57.51428 \n",
"L 237.772618 51.03309 \n",
"L 239.566005 44.109569 \n",
"L 241.359391 36.807854 \n",
"L 243.152778 29.199678 \n",
"L 244.946165 21.362327 \n",
"\" clip-path=\"url(#p90bc1b5a97)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 58.523438 143.1 \n",
"L 58.523438 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 253.823438 143.1 \n",
"L 253.823438 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 58.523438 143.1 \n",
"L 253.823438 143.1 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 58.523438 7.2 \n",
"L 253.823438 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_7\">\n",
" <path d=\"M 138.684558 129.980712 \n",
"Q 146.927992 132.717372 154.110335 135.101771 \n",
"\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
" <path d=\"M 150.944208 131.943347 \n",
"L 154.110335 135.101771 \n",
"L 149.68392 135.739618 \n",
"\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
" </g>\n",
" <g id=\"text_14\">\n",
" <!-- min of -->\n",
" <g transform=\"translate(67.40071 114.660962)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-6d\" d=\"M 3328 2828 \n",
"Q 3544 3216 3844 3400 \n",
"Q 4144 3584 4550 3584 \n",
"Q 5097 3584 5394 3201 \n",
"Q 5691 2819 5691 2113 \n",
"L 5691 0 \n",
"L 5113 0 \n",
"L 5113 2094 \n",
"Q 5113 2597 4934 2840 \n",
"Q 4756 3084 4391 3084 \n",
"Q 3944 3084 3684 2787 \n",
"Q 3425 2491 3425 1978 \n",
"L 3425 0 \n",
"L 2847 0 \n",
"L 2847 2094 \n",
"Q 2847 2600 2669 2842 \n",
"Q 2491 3084 2119 3084 \n",
"Q 1678 3084 1418 2786 \n",
"Q 1159 2488 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1356 3278 1631 3431 \n",
"Q 1906 3584 2284 3584 \n",
"Q 2666 3584 2933 3390 \n",
"Q 3200 3197 3328 2828 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
"Q 1497 3097 1228 2736 \n",
"Q 959 2375 959 1747 \n",
"Q 959 1119 1226 758 \n",
"Q 1494 397 1959 397 \n",
"Q 2419 397 2687 759 \n",
"Q 2956 1122 2956 1747 \n",
"Q 2956 2369 2687 2733 \n",
"Q 2419 3097 1959 3097 \n",
"z\n",
"M 1959 3584 \n",
"Q 2709 3584 3137 3096 \n",
"Q 3566 2609 3566 1747 \n",
"Q 3566 888 3137 398 \n",
"Q 2709 -91 1959 -91 \n",
"Q 1206 -91 779 398 \n",
"Q 353 888 353 1747 \n",
"Q 353 2609 779 3096 \n",
"Q 1206 3584 1959 3584 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
"L 2375 4384 \n",
"L 1825 4384 \n",
"Q 1516 4384 1395 4259 \n",
"Q 1275 4134 1275 3809 \n",
"L 1275 3500 \n",
"L 2222 3500 \n",
"L 2222 3053 \n",
"L 1275 3053 \n",
"L 1275 0 \n",
"L 697 0 \n",
"L 697 3053 \n",
"L 147 3053 \n",
"L 147 3500 \n",
"L 697 3500 \n",
"L 697 3744 \n",
"Q 697 4328 969 4595 \n",
"Q 1241 4863 1831 4863 \n",
"L 2375 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-6d\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"97.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"125.195312\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"188.574219\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"220.361328\"/>\n",
" <use xlink:href=\"#DejaVuSans-66\" x=\"281.542969\"/>\n",
" </g>\n",
" <!-- empirical risk -->\n",
" <g transform=\"translate(67.40071 125.858774)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
"L 3597 1613 \n",
"L 953 1613 \n",
"Q 991 1019 1311 708 \n",
"Q 1631 397 2203 397 \n",
"Q 2534 397 2845 478 \n",
"Q 3156 559 3463 722 \n",
"L 3463 178 \n",
"Q 3153 47 2828 -22 \n",
"Q 2503 -91 2169 -91 \n",
"Q 1331 -91 842 396 \n",
"Q 353 884 353 1716 \n",
"Q 353 2575 817 3079 \n",
"Q 1281 3584 2069 3584 \n",
"Q 2775 3584 3186 3129 \n",
"Q 3597 2675 3597 1894 \n",
"z\n",
"M 3022 2063 \n",
"Q 3016 2534 2758 2815 \n",
"Q 2500 3097 2075 3097 \n",
"Q 1594 3097 1305 2825 \n",
"Q 1016 2553 972 2059 \n",
"L 3022 2063 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
"L 1159 -1331 \n",
"L 581 -1331 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2969 \n",
"Q 1341 3281 1617 3432 \n",
"Q 1894 3584 2278 3584 \n",
"Q 2916 3584 3314 3078 \n",
"Q 3713 2572 3713 1747 \n",
"Q 3713 922 3314 415 \n",
"Q 2916 -91 2278 -91 \n",
"Q 1894 -91 1617 61 \n",
"Q 1341 213 1159 525 \n",
"z\n",
"M 3116 1747 \n",
"Q 3116 2381 2855 2742 \n",
"Q 2594 3103 2138 3103 \n",
"Q 1681 3103 1420 2742 \n",
"Q 1159 2381 1159 1747 \n",
"Q 1159 1113 1420 752 \n",
"Q 1681 391 2138 391 \n",
"Q 2594 391 2855 752 \n",
"Q 3116 1113 3116 1747 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
"L 3122 2828 \n",
"Q 2878 2963 2633 3030 \n",
"Q 2388 3097 2138 3097 \n",
"Q 1578 3097 1268 2742 \n",
"Q 959 2388 959 1747 \n",
"Q 959 1106 1268 751 \n",
"Q 1578 397 2138 397 \n",
"Q 2388 397 2633 464 \n",
"Q 2878 531 3122 666 \n",
"L 3122 134 \n",
"Q 2881 22 2623 -34 \n",
"Q 2366 -91 2075 -91 \n",
"Q 1284 -91 818 406 \n",
"Q 353 903 353 1747 \n",
"Q 353 2603 823 3093 \n",
"Q 1294 3584 2113 3584 \n",
"Q 2378 3584 2631 3529 \n",
"Q 2884 3475 3122 3366 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
"Q 1497 1759 1228 1600 \n",
"Q 959 1441 959 1056 \n",
"Q 959 750 1161 570 \n",
"Q 1363 391 1709 391 \n",
"Q 2188 391 2477 730 \n",
"Q 2766 1069 2766 1631 \n",
"L 2766 1759 \n",
"L 2194 1759 \n",
"z\n",
"M 3341 1997 \n",
"L 3341 0 \n",
"L 2766 0 \n",
"L 2766 531 \n",
"Q 2569 213 2275 61 \n",
"Q 1981 -91 1556 -91 \n",
"Q 1019 -91 701 211 \n",
"Q 384 513 384 1019 \n",
"Q 384 1609 779 1909 \n",
"Q 1175 2209 1959 2209 \n",
"L 2766 2209 \n",
"L 2766 2266 \n",
"Q 2766 2663 2505 2880 \n",
"Q 2244 3097 1772 3097 \n",
"Q 1472 3097 1187 3025 \n",
"Q 903 2953 641 2809 \n",
"L 641 3341 \n",
"Q 956 3463 1253 3523 \n",
"Q 1550 3584 1831 3584 \n",
"Q 2591 3584 2966 3190 \n",
"Q 3341 2797 3341 1997 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-65\"/>\n",
" <use xlink:href=\"#DejaVuSans-6d\" x=\"61.523438\"/>\n",
" <use xlink:href=\"#DejaVuSans-70\" x=\"158.935547\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"222.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"250.195312\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"291.308594\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"319.091797\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"374.072266\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"435.351562\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"463.134766\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"494.921875\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"536.035156\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"563.818359\"/>\n",
" <use xlink:href=\"#DejaVuSans-6b\" x=\"615.917969\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"patch_8\">\n",
" <path d=\"M 174.552997 70.586852 \n",
"Q 174.769518 94.666846 174.975987 117.62885 \n",
"\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
" <path d=\"M 176.93994 113.611029 \n",
"L 174.975987 117.62885 \n",
"L 172.940102 113.646994 \n",
"\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
" </g>\n",
" <g id=\"text_15\">\n",
" <!-- min of risk -->\n",
" <g transform=\"translate(148.103189 64.505228)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-6d\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"97.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"125.195312\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"188.574219\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"220.361328\"/>\n",
" <use xlink:href=\"#DejaVuSans-66\" x=\"281.542969\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"316.748047\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"348.535156\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"389.648438\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"417.431641\"/>\n",
" <use xlink:href=\"#DejaVuSans-6b\" x=\"469.53125\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p90bc1b5a97\">\n",
" <rect x=\"58.523438\" y=\"7.2\" width=\"195.3\" height=\"135.9\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 252x180 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"def annotate(text, xy, xytext): #@save\n",
" d2l.plt.gca().annotate(text, xy=xy, xytext=xytext,\n",
" arrowprops=dict(arrowstyle='->'))\n",
"\n",
"x = torch.arange(0.5, 1.5, 0.01)\n",
"d2l.set_figsize((4.5, 2.5))\n",
"d2l.plot(x, [f(x), g(x)], 'x', 'risk')\n",
"annotate('min of\\nempirical risk', (1.0, -1.2), (0.5, -1.1))\n",
"annotate('min of risk', (1.1, -1.05), (0.95, -0.5))"
]
},
{
"cell_type": "markdown",
"id": "1fbcc94f",
"metadata": {
"origin_pos": 10
},
"source": [
"## 深度学习中的优化挑战\n",
"\n",
"本章将关注优化算法在最小化目标函数方面的性能,而不是模型的泛化误差。在 :numref:`sec_linear_regression`中,我们区分了优化问题中的解析解和数值解。在深度学习中,大多数目标函数都很复杂,没有解析解。相反,我们必须使用数值优化算法。本章中的优化算法都属于此类别。\n",
"\n",
"深度学习优化存在许多挑战。其中最令人烦恼的是局部最小值、鞍点和梯度消失。\n",
"\n",
"### 局部最小值\n",
"\n",
"对于任何目标函数$f(x)$,如果在$x$处对应的$f(x)$值小于在$x$附近任意其他点的$f(x)$值,那么$f(x)$可能是局部最小值。如果$f(x)$在$x$处的值是整个域中目标函数的最小值,那么$f(x)$是全局最小值。\n",
"\n",
"例如,给定函数\n",
"\n",
"$$f(x) = x \\cdot \\text{cos}(\\pi x) \\text{ for } -1.0 \\leq x \\leq 2.0,$$\n",
"\n",
"我们可以近似该函数的局部最小值和全局最小值。\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "0adadab1",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:07:08.784743Z",
"iopub.status.busy": "2023-08-18T07:07:08.784448Z",
"iopub.status.idle": "2023-08-18T07:07:09.024530Z",
"shell.execute_reply": "2023-08-18T07:07:09.023594Z"
},
"origin_pos": 11,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"245.120313pt\" height=\"180.65625pt\" viewBox=\"0 0 245.120313 180.65625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T07:07:08.979118</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 180.65625 \n",
"L 245.120313 180.65625 \n",
"L 245.120313 0 \n",
"L 0 0 \n",
"L 0 180.65625 \n",
"z\n",
"\" style=\"fill: none\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 237.920313 143.1 \n",
"L 237.920313 7.2 \n",
"L 42.620312 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 51.497585 143.1 \n",
"L 51.497585 7.2 \n",
"\" clip-path=\"url(#pdea116b94b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_2\">\n",
" <defs>\n",
" <path id=\"m43eafd2f07\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m43eafd2f07\" x=\"51.497585\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(44.126491 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
"L 4684 2272 \n",
"L 4684 1741 \n",
"L 678 1741 \n",
"L 678 2272 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 110.877336 143.1 \n",
"L 110.877336 7.2 \n",
"\" clip-path=\"url(#pdea116b94b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use xlink:href=\"#m43eafd2f07\" x=\"110.877336\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(107.696086 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 170.257086 143.1 \n",
"L 170.257086 7.2 \n",
"\" clip-path=\"url(#pdea116b94b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#m43eafd2f07\" x=\"170.257086\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(167.075836 157.698438)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 229.636837 143.1 \n",
"L 229.636837 7.2 \n",
"\" clip-path=\"url(#pdea116b94b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use xlink:href=\"#m43eafd2f07\" x=\"229.636837\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(226.455587 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- x -->\n",
" <g transform=\"translate(137.310937 171.376563)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 42.620312 135.021357 \n",
"L 237.920313 135.021357 \n",
"\" clip-path=\"url(#pdea116b94b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_10\">\n",
" <defs>\n",
" <path id=\"m56630524c8\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m56630524c8\" x=\"42.620312\" y=\"135.021357\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(20.878125 138.820576)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 42.620312 94.32435 \n",
"L 237.920313 94.32435 \n",
"\" clip-path=\"url(#pdea116b94b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_12\">\n",
" <g>\n",
" <use xlink:href=\"#m56630524c8\" x=\"42.620312\" y=\"94.32435\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(29.257812 98.123569)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_13\">\n",
" <path d=\"M 42.620312 53.627344 \n",
"L 237.920313 53.627344 \n",
"\" clip-path=\"url(#pdea116b94b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use xlink:href=\"#m56630524c8\" x=\"42.620312\" y=\"53.627344\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(29.257812 57.426562)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_15\">\n",
" <path d=\"M 42.620312 12.930337 \n",
"L 237.920313 12.930337 \n",
"\" clip-path=\"url(#pdea116b94b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use xlink:href=\"#m56630524c8\" x=\"42.620312\" y=\"12.930337\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(29.257812 16.729555)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- f(x) -->\n",
" <g transform=\"translate(14.798437 83.771094)rotate(-90)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
"L 2375 4384 \n",
"L 1825 4384 \n",
"Q 1516 4384 1395 4259 \n",
"Q 1275 4134 1275 3809 \n",
"L 1275 3500 \n",
"L 2222 3500 \n",
"L 2222 3053 \n",
"L 1275 3053 \n",
"L 1275 0 \n",
"L 697 0 \n",
"L 697 3053 \n",
"L 147 3053 \n",
"L 147 3500 \n",
"L 697 3500 \n",
"L 697 3744 \n",
"Q 697 4328 969 4595 \n",
"Q 1241 4863 1831 4863 \n",
"L 2375 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
"Q 1566 4138 1362 3434 \n",
"Q 1159 2731 1159 2009 \n",
"Q 1159 1288 1364 580 \n",
"Q 1569 -128 1984 -844 \n",
"L 1484 -844 \n",
"Q 1016 -109 783 600 \n",
"Q 550 1309 550 2009 \n",
"Q 550 2706 781 3412 \n",
"Q 1013 4119 1484 4856 \n",
"L 1984 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
"L 1013 4856 \n",
"Q 1481 4119 1714 3412 \n",
"Q 1947 2706 1947 2009 \n",
"Q 1947 1309 1714 600 \n",
"Q 1481 -109 1013 -844 \n",
"L 513 -844 \n",
"Q 928 -128 1133 580 \n",
"Q 1338 1288 1338 2009 \n",
"Q 1338 2731 1133 3434 \n",
"Q 928 4138 513 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-66\"/>\n",
" <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
" <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
" <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_17\">\n",
" <path d=\"M 51.497585 53.627344 \n",
"L 53.278976 55.023449 \n",
"L 55.06037 56.74677 \n",
"L 56.841761 58.760566 \n",
"L 59.216953 61.829964 \n",
"L 62.185941 66.14782 \n",
"L 65.748726 71.77755 \n",
"L 73.468093 84.141828 \n",
"L 76.437078 88.454205 \n",
"L 78.812269 91.569977 \n",
"L 81.187462 94.324352 \n",
"L 83.562652 96.670666 \n",
"L 85.344043 98.141793 \n",
"L 87.125435 99.354777 \n",
"L 88.906828 100.30456 \n",
"L 90.68822 100.990367 \n",
"L 92.469613 101.415635 \n",
"L 94.251006 101.5879 \n",
"L 96.032398 101.518633 \n",
"L 97.813791 101.223018 \n",
"L 99.595183 100.719694 \n",
"L 101.970373 99.763546 \n",
"L 104.345563 98.536364 \n",
"L 107.314551 96.722919 \n",
"L 118.596703 89.468868 \n",
"L 120.971893 88.369314 \n",
"L 122.753286 87.739436 \n",
"L 124.534679 87.303077 \n",
"L 126.31607 87.081006 \n",
"L 128.097463 87.090741 \n",
"L 129.878855 87.346256 \n",
"L 131.660248 87.857731 \n",
"L 133.441642 88.631356 \n",
"L 135.223035 89.66917 \n",
"L 137.004426 90.968974 \n",
"L 138.785818 92.524285 \n",
"L 140.567211 94.324351 \n",
"L 142.942399 97.078721 \n",
"L 145.31759 100.194493 \n",
"L 148.286578 104.506873 \n",
"L 151.849363 110.108182 \n",
"L 160.162528 123.398939 \n",
"L 163.131516 127.622777 \n",
"L 165.506707 130.589314 \n",
"L 167.288098 132.510512 \n",
"L 169.069492 134.128717 \n",
"L 170.850883 135.408043 \n",
"L 172.038477 136.056232 \n",
"L 173.226071 136.530108 \n",
"L 174.413672 136.821424 \n",
"L 175.601266 136.922727 \n",
"L 176.78886 136.827391 \n",
"L 177.976453 136.529701 \n",
"L 179.164054 136.024838 \n",
"L 180.351641 135.309005 \n",
"L 181.539235 134.3794 \n",
"L 182.726829 133.234251 \n",
"L 184.508227 131.111227 \n",
"L 186.289618 128.504387 \n",
"L 188.071009 125.421788 \n",
"L 189.852406 121.877232 \n",
"L 191.633797 117.890338 \n",
"L 194.008985 111.930849 \n",
"L 196.384173 105.305589 \n",
"L 199.353165 96.229053 \n",
"L 202.915953 84.456373 \n",
"L 210.041516 59.727847 \n",
"L 214.198102 45.849641 \n",
"L 217.167094 36.763402 \n",
"L 219.542275 30.220144 \n",
"L 221.323665 25.832172 \n",
"L 223.105063 21.954302 \n",
"L 224.886454 18.640958 \n",
"L 226.667852 15.942226 \n",
"L 227.855446 14.507057 \n",
"L 229.04304 13.377273 \n",
"L 229.04304 13.377273 \n",
"\" clip-path=\"url(#pdea116b94b)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 42.620312 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 237.920313 143.1 \n",
"L 237.920313 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 237.920313 143.1 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 42.620312 7.2 \n",
"L 237.920313 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_7\">\n",
" <path d=\"M 99.270201 123.522993 \n",
"Q 96.476691 114.960624 94.029955 107.46115 \n",
"\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
" <path d=\"M 93.369247 111.884211 \n",
"L 94.029955 107.46115 \n",
"L 97.171978 110.643553 \n",
"\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- local minimum -->\n",
" <g transform=\"translate(65.154928 135.021357)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
"Q 1497 3097 1228 2736 \n",
"Q 959 2375 959 1747 \n",
"Q 959 1119 1226 758 \n",
"Q 1494 397 1959 397 \n",
"Q 2419 397 2687 759 \n",
"Q 2956 1122 2956 1747 \n",
"Q 2956 2369 2687 2733 \n",
"Q 2419 3097 1959 3097 \n",
"z\n",
"M 1959 3584 \n",
"Q 2709 3584 3137 3096 \n",
"Q 3566 2609 3566 1747 \n",
"Q 3566 888 3137 398 \n",
"Q 2709 -91 1959 -91 \n",
"Q 1206 -91 779 398 \n",
"Q 353 888 353 1747 \n",
"Q 353 2609 779 3096 \n",
"Q 1206 3584 1959 3584 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
"L 3122 2828 \n",
"Q 2878 2963 2633 3030 \n",
"Q 2388 3097 2138 3097 \n",
"Q 1578 3097 1268 2742 \n",
"Q 959 2388 959 1747 \n",
"Q 959 1106 1268 751 \n",
"Q 1578 397 2138 397 \n",
"Q 2388 397 2633 464 \n",
"Q 2878 531 3122 666 \n",
"L 3122 134 \n",
"Q 2881 22 2623 -34 \n",
"Q 2366 -91 2075 -91 \n",
"Q 1284 -91 818 406 \n",
"Q 353 903 353 1747 \n",
"Q 353 2603 823 3093 \n",
"Q 1294 3584 2113 3584 \n",
"Q 2378 3584 2631 3529 \n",
"Q 2884 3475 3122 3366 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
"Q 1497 1759 1228 1600 \n",
"Q 959 1441 959 1056 \n",
"Q 959 750 1161 570 \n",
"Q 1363 391 1709 391 \n",
"Q 2188 391 2477 730 \n",
"Q 2766 1069 2766 1631 \n",
"L 2766 1759 \n",
"L 2194 1759 \n",
"z\n",
"M 3341 1997 \n",
"L 3341 0 \n",
"L 2766 0 \n",
"L 2766 531 \n",
"Q 2569 213 2275 61 \n",
"Q 1981 -91 1556 -91 \n",
"Q 1019 -91 701 211 \n",
"Q 384 513 384 1019 \n",
"Q 384 1609 779 1909 \n",
"Q 1175 2209 1959 2209 \n",
"L 2766 2209 \n",
"L 2766 2266 \n",
"Q 2766 2663 2505 2880 \n",
"Q 2244 3097 1772 3097 \n",
"Q 1472 3097 1187 3025 \n",
"Q 903 2953 641 2809 \n",
"L 641 3341 \n",
"Q 956 3463 1253 3523 \n",
"Q 1550 3584 1831 3584 \n",
"Q 2591 3584 2966 3190 \n",
"Q 3341 2797 3341 1997 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6d\" d=\"M 3328 2828 \n",
"Q 3544 3216 3844 3400 \n",
"Q 4144 3584 4550 3584 \n",
"Q 5097 3584 5394 3201 \n",
"Q 5691 2819 5691 2113 \n",
"L 5691 0 \n",
"L 5113 0 \n",
"L 5113 2094 \n",
"Q 5113 2597 4934 2840 \n",
"Q 4756 3084 4391 3084 \n",
"Q 3944 3084 3684 2787 \n",
"Q 3425 2491 3425 1978 \n",
"L 3425 0 \n",
"L 2847 0 \n",
"L 2847 2094 \n",
"Q 2847 2600 2669 2842 \n",
"Q 2491 3084 2119 3084 \n",
"Q 1678 3084 1418 2786 \n",
"Q 1159 2488 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1356 3278 1631 3431 \n",
"Q 1906 3584 2284 3584 \n",
"Q 2666 3584 2933 3390 \n",
"Q 3200 3197 3328 2828 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
"L 1178 3500 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 3500 \n",
"z\n",
"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 4134 \n",
"L 603 4134 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-75\" d=\"M 544 1381 \n",
"L 544 3500 \n",
"L 1119 3500 \n",
"L 1119 1403 \n",
"Q 1119 906 1312 657 \n",
"Q 1506 409 1894 409 \n",
"Q 2359 409 2629 706 \n",
"Q 2900 1003 2900 1516 \n",
"L 2900 3500 \n",
"L 3475 3500 \n",
"L 3475 0 \n",
"L 2900 0 \n",
"L 2900 538 \n",
"Q 2691 219 2414 64 \n",
"Q 2138 -91 1772 -91 \n",
"Q 1169 -91 856 284 \n",
"Q 544 659 544 1381 \n",
"z\n",
"M 1991 3584 \n",
"L 1991 3584 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-6c\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"27.783203\"/>\n",
" <use xlink:href=\"#DejaVuSans-63\" x=\"88.964844\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"143.945312\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"205.224609\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"233.007812\"/>\n",
" <use xlink:href=\"#DejaVuSans-6d\" x=\"264.794922\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"362.207031\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"389.990234\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"453.369141\"/>\n",
" <use xlink:href=\"#DejaVuSans-6d\" x=\"481.152344\"/>\n",
" <use xlink:href=\"#DejaVuSans-75\" x=\"578.564453\"/>\n",
" <use xlink:href=\"#DejaVuSans-6d\" x=\"641.943359\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"patch_8\">\n",
" <path d=\"M 185.774179 67.827331 \n",
"Q 181.129909 99.418634 176.648254 129.903793 \n",
"\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
" <path d=\"M 179.208777 126.237224 \n",
"L 176.648254 129.903793 \n",
"L 175.251313 125.655434 \n",
"\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
" </g>\n",
" <g id=\"text_12\">\n",
" <!-- global minimum -->\n",
" <g transform=\"translate(146.505186 61.766745)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-67\" d=\"M 2906 1791 \n",
"Q 2906 2416 2648 2759 \n",
"Q 2391 3103 1925 3103 \n",
"Q 1463 3103 1205 2759 \n",
"Q 947 2416 947 1791 \n",
"Q 947 1169 1205 825 \n",
"Q 1463 481 1925 481 \n",
"Q 2391 481 2648 825 \n",
"Q 2906 1169 2906 1791 \n",
"z\n",
"M 3481 434 \n",
"Q 3481 -459 3084 -895 \n",
"Q 2688 -1331 1869 -1331 \n",
"Q 1566 -1331 1297 -1286 \n",
"Q 1028 -1241 775 -1147 \n",
"L 775 -588 \n",
"Q 1028 -725 1275 -790 \n",
"Q 1522 -856 1778 -856 \n",
"Q 2344 -856 2625 -561 \n",
"Q 2906 -266 2906 331 \n",
"L 2906 616 \n",
"Q 2728 306 2450 153 \n",
"Q 2172 0 1784 0 \n",
"Q 1141 0 747 490 \n",
"Q 353 981 353 1791 \n",
"Q 353 2603 747 3093 \n",
"Q 1141 3584 1784 3584 \n",
"Q 2172 3584 2450 3431 \n",
"Q 2728 3278 2906 2969 \n",
"L 2906 3500 \n",
"L 3481 3500 \n",
"L 3481 434 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-62\" d=\"M 3116 1747 \n",
"Q 3116 2381 2855 2742 \n",
"Q 2594 3103 2138 3103 \n",
"Q 1681 3103 1420 2742 \n",
"Q 1159 2381 1159 1747 \n",
"Q 1159 1113 1420 752 \n",
"Q 1681 391 2138 391 \n",
"Q 2594 391 2855 752 \n",
"Q 3116 1113 3116 1747 \n",
"z\n",
"M 1159 2969 \n",
"Q 1341 3281 1617 3432 \n",
"Q 1894 3584 2278 3584 \n",
"Q 2916 3584 3314 3078 \n",
"Q 3713 2572 3713 1747 \n",
"Q 3713 922 3314 415 \n",
"Q 2916 -91 2278 -91 \n",
"Q 1894 -91 1617 61 \n",
"Q 1341 213 1159 525 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 4863 \n",
"L 1159 4863 \n",
"L 1159 2969 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-67\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"63.476562\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"91.259766\"/>\n",
" <use xlink:href=\"#DejaVuSans-62\" x=\"152.441406\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"215.917969\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"277.197266\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"304.980469\"/>\n",
" <use xlink:href=\"#DejaVuSans-6d\" x=\"336.767578\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"434.179688\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"461.962891\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"525.341797\"/>\n",
" <use xlink:href=\"#DejaVuSans-6d\" x=\"553.125\"/>\n",
" <use xlink:href=\"#DejaVuSans-75\" x=\"650.537109\"/>\n",
" <use xlink:href=\"#DejaVuSans-6d\" x=\"713.916016\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"pdea116b94b\">\n",
" <rect x=\"42.620312\" y=\"7.2\" width=\"195.3\" height=\"135.9\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 252x180 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"x = torch.arange(-1.0, 2.0, 0.01)\n",
"d2l.plot(x, [f(x), ], 'x', 'f(x)')\n",
"annotate('local minimum', (-0.3, -0.25), (-0.77, -1.0))\n",
"annotate('global minimum', (1.1, -0.95), (0.6, 0.8))"
]
},
{
"cell_type": "markdown",
"id": "407b7125",
"metadata": {
"origin_pos": 13
},
"source": [
"深度学习模型的目标函数通常有许多局部最优解。当优化问题的数值解接近局部最优值时,随着目标函数解的梯度接近或变为零,通过最终迭代获得的数值解可能仅使目标函数*局部*最优,而不是*全局*最优。只有一定程度的噪声可能会使参数跳出局部最小值。事实上,这是小批量随机梯度下降的有利特性之一。在这种情况下,小批量上梯度的自然变化能够将参数从局部极小值中跳出。\n",
"\n",
"### 鞍点\n",
"\n",
"除了局部最小值之外,鞍点是梯度消失的另一个原因。*鞍点*(saddle point)是指函数的所有梯度都消失但既不是全局最小值也不是局部最小值的任何位置。考虑这个函数$f(x) = x^3$。它的一阶和二阶导数在$x=0$时消失。这时优化可能会停止,尽管它不是最小值。\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "26f5d6d0",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:07:09.028686Z",
"iopub.status.busy": "2023-08-18T07:07:09.027907Z",
"iopub.status.idle": "2023-08-18T07:07:09.220430Z",
"shell.execute_reply": "2023-08-18T07:07:09.219542Z"
},
"origin_pos": 14,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"245.120313pt\" height=\"180.65625pt\" viewBox=\"0 0 245.120313 180.65625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T07:07:09.183352</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 180.65625 \n",
"L 245.120313 180.65625 \n",
"L 245.120313 0 \n",
"L 0 0 \n",
"L 0 180.65625 \n",
"z\n",
"\" style=\"fill: none\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 237.920313 143.1 \n",
"L 237.920313 7.2 \n",
"L 42.620312 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 51.497585 143.1 \n",
"L 51.497585 7.2 \n",
"\" clip-path=\"url(#pdad0cc75ae)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_2\">\n",
" <defs>\n",
" <path id=\"ma1ffcd3ef6\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#ma1ffcd3ef6\" x=\"51.497585\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(44.126491 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
"L 4684 2272 \n",
"L 4684 1741 \n",
"L 678 1741 \n",
"L 678 2272 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 95.995193 143.1 \n",
"L 95.995193 7.2 \n",
"\" clip-path=\"url(#pdad0cc75ae)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use xlink:href=\"#ma1ffcd3ef6\" x=\"95.995193\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(88.624099 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 140.4928 143.1 \n",
"L 140.4928 7.2 \n",
"\" clip-path=\"url(#pdad0cc75ae)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#ma1ffcd3ef6\" x=\"140.4928\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(137.31155 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 184.990408 143.1 \n",
"L 184.990408 7.2 \n",
"\" clip-path=\"url(#pdad0cc75ae)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use xlink:href=\"#ma1ffcd3ef6\" x=\"184.990408\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(181.809158 157.698438)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 229.488015 143.1 \n",
"L 229.488015 7.2 \n",
"\" clip-path=\"url(#pdad0cc75ae)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use xlink:href=\"#ma1ffcd3ef6\" x=\"229.488015\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(226.306765 157.698438)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- x -->\n",
" <g transform=\"translate(137.310937 171.376563)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 42.620312 113.583786 \n",
"L 237.920313 113.583786 \n",
"\" clip-path=\"url(#pdad0cc75ae)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_12\">\n",
" <defs>\n",
" <path id=\"m1a4cf6a310\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m1a4cf6a310\" x=\"42.620312\" y=\"113.583786\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- 5 -->\n",
" <g transform=\"translate(20.878125 117.383005)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
"L 3169 4666 \n",
"L 3169 4134 \n",
"L 1269 4134 \n",
"L 1269 2991 \n",
"Q 1406 3038 1543 3061 \n",
"Q 1681 3084 1819 3084 \n",
"Q 2600 3084 3056 2656 \n",
"Q 3513 2228 3513 1497 \n",
"Q 3513 744 3044 326 \n",
"Q 2575 -91 1722 -91 \n",
"Q 1428 -91 1123 -41 \n",
"Q 819 9 494 109 \n",
"L 494 744 \n",
"Q 775 591 1075 516 \n",
"Q 1375 441 1709 441 \n",
"Q 2250 441 2565 725 \n",
"Q 2881 1009 2881 1497 \n",
"Q 2881 1984 2565 2268 \n",
"Q 2250 2553 1709 2553 \n",
"Q 1456 2553 1204 2497 \n",
"Q 953 2441 691 2322 \n",
"L 691 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_13\">\n",
" <path d=\"M 42.620312 74.685551 \n",
"L 237.920313 74.685551 \n",
"\" clip-path=\"url(#pdad0cc75ae)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use xlink:href=\"#m1a4cf6a310\" x=\"42.620312\" y=\"74.685551\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(29.257812 78.48477)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_15\">\n",
" <path d=\"M 42.620312 35.787316 \n",
"L 237.920313 35.787316 \n",
"\" clip-path=\"url(#pdad0cc75ae)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use xlink:href=\"#m1a4cf6a310\" x=\"42.620312\" y=\"35.787316\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 5 -->\n",
" <g transform=\"translate(29.257812 39.586535)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-35\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- f(x) -->\n",
" <g transform=\"translate(14.798437 83.771094)rotate(-90)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
"L 2375 4384 \n",
"L 1825 4384 \n",
"Q 1516 4384 1395 4259 \n",
"Q 1275 4134 1275 3809 \n",
"L 1275 3500 \n",
"L 2222 3500 \n",
"L 2222 3053 \n",
"L 1275 3053 \n",
"L 1275 0 \n",
"L 697 0 \n",
"L 697 3053 \n",
"L 147 3053 \n",
"L 147 3500 \n",
"L 697 3500 \n",
"L 697 3744 \n",
"Q 697 4328 969 4595 \n",
"Q 1241 4863 1831 4863 \n",
"L 2375 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
"Q 1566 4138 1362 3434 \n",
"Q 1159 2731 1159 2009 \n",
"Q 1159 1288 1364 580 \n",
"Q 1569 -128 1984 -844 \n",
"L 1484 -844 \n",
"Q 1016 -109 783 600 \n",
"Q 550 1309 550 2009 \n",
"Q 550 2706 781 3412 \n",
"Q 1013 4119 1484 4856 \n",
"L 1984 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
"L 1013 4856 \n",
"Q 1481 4119 1714 3412 \n",
"Q 1947 2706 1947 2009 \n",
"Q 1947 1309 1714 600 \n",
"Q 1481 -109 1013 -844 \n",
"L 513 -844 \n",
"Q 928 -128 1133 580 \n",
"Q 1338 1288 1338 2009 \n",
"Q 1338 2731 1133 3434 \n",
"Q 928 4138 513 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-66\"/>\n",
" <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
" <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
" <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_17\">\n",
" <path d=\"M 51.497585 136.922727 \n",
"L 54.61242 130.613872 \n",
"L 57.72725 124.746459 \n",
"L 60.842079 119.304473 \n",
"L 63.956914 114.271881 \n",
"L 67.071749 109.632695 \n",
"L 70.186578 105.370909 \n",
"L 73.301413 101.470493 \n",
"L 76.416243 97.915453 \n",
"L 79.531078 94.68977 \n",
"L 82.645907 91.777438 \n",
"L 85.760742 89.16244 \n",
"L 88.875572 86.828774 \n",
"L 91.990407 84.760421 \n",
"L 95.105241 82.941374 \n",
"L 98.220074 81.355626 \n",
"L 101.779882 79.808472 \n",
"L 105.339689 78.521221 \n",
"L 108.899497 77.469973 \n",
"L 112.904283 76.539659 \n",
"L 116.909067 75.843762 \n",
"L 121.35883 75.304088 \n",
"L 126.253566 74.940475 \n",
"L 132.038255 74.738912 \n",
"L 140.937776 74.685543 \n",
"L 151.172226 74.578005 \n",
"L 156.956915 74.291489 \n",
"L 161.851651 73.825185 \n",
"L 166.301412 73.167649 \n",
"L 170.306195 72.345722 \n",
"L 173.866006 71.403513 \n",
"L 177.425814 70.23725 \n",
"L 180.985622 68.823036 \n",
"L 184.100454 67.36341 \n",
"L 187.215286 65.679639 \n",
"L 190.330121 63.755708 \n",
"L 193.444956 61.575608 \n",
"L 196.559785 59.123332 \n",
"L 199.674615 56.382869 \n",
"L 202.78945 53.3382 \n",
"L 205.904285 49.973322 \n",
"L 209.019114 46.272228 \n",
"L 212.133949 42.218897 \n",
"L 215.248779 37.797332 \n",
"L 218.363614 32.991506 \n",
"L 221.478443 27.785426 \n",
"L 224.593278 22.163063 \n",
"L 227.708108 16.108425 \n",
"L 229.04304 13.377273 \n",
"L 229.04304 13.377273 \n",
"\" clip-path=\"url(#pdad0cc75ae)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 42.620312 143.1 \n",
"L 42.620312 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 237.920313 143.1 \n",
"L 237.920313 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 42.620313 143.1 \n",
"L 237.920313 143.1 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 42.620313 7.2 \n",
"L 237.920313 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_7\">\n",
" <path d=\"M 146.222705 102.027439 \n",
"Q 143.574932 90.111821 141.169683 79.287615 \n",
"\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
" <path d=\"M 140.084981 83.626212 \n",
"L 141.169683 79.287615 \n",
"L 143.989739 82.758535 \n",
"\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- saddle point -->\n",
" <g transform=\"translate(117.354044 113.583786)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
"L 2834 2853 \n",
"Q 2591 2978 2328 3040 \n",
"Q 2066 3103 1784 3103 \n",
"Q 1356 3103 1142 2972 \n",
"Q 928 2841 928 2578 \n",
"Q 928 2378 1081 2264 \n",
"Q 1234 2150 1697 2047 \n",
"L 1894 2003 \n",
"Q 2506 1872 2764 1633 \n",
"Q 3022 1394 3022 966 \n",
"Q 3022 478 2636 193 \n",
"Q 2250 -91 1575 -91 \n",
"Q 1294 -91 989 -36 \n",
"Q 684 19 347 128 \n",
"L 347 722 \n",
"Q 666 556 975 473 \n",
"Q 1284 391 1588 391 \n",
"Q 1994 391 2212 530 \n",
"Q 2431 669 2431 922 \n",
"Q 2431 1156 2273 1281 \n",
"Q 2116 1406 1581 1522 \n",
"L 1381 1569 \n",
"Q 847 1681 609 1914 \n",
"Q 372 2147 372 2553 \n",
"Q 372 3047 722 3315 \n",
"Q 1072 3584 1716 3584 \n",
"Q 2034 3584 2315 3537 \n",
"Q 2597 3491 2834 3397 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
"Q 1497 1759 1228 1600 \n",
"Q 959 1441 959 1056 \n",
"Q 959 750 1161 570 \n",
"Q 1363 391 1709 391 \n",
"Q 2188 391 2477 730 \n",
"Q 2766 1069 2766 1631 \n",
"L 2766 1759 \n",
"L 2194 1759 \n",
"z\n",
"M 3341 1997 \n",
"L 3341 0 \n",
"L 2766 0 \n",
"L 2766 531 \n",
"Q 2569 213 2275 61 \n",
"Q 1981 -91 1556 -91 \n",
"Q 1019 -91 701 211 \n",
"Q 384 513 384 1019 \n",
"Q 384 1609 779 1909 \n",
"Q 1175 2209 1959 2209 \n",
"L 2766 2209 \n",
"L 2766 2266 \n",
"Q 2766 2663 2505 2880 \n",
"Q 2244 3097 1772 3097 \n",
"Q 1472 3097 1187 3025 \n",
"Q 903 2953 641 2809 \n",
"L 641 3341 \n",
"Q 956 3463 1253 3523 \n",
"Q 1550 3584 1831 3584 \n",
"Q 2591 3584 2966 3190 \n",
"Q 3341 2797 3341 1997 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-64\" d=\"M 2906 2969 \n",
"L 2906 4863 \n",
"L 3481 4863 \n",
"L 3481 0 \n",
"L 2906 0 \n",
"L 2906 525 \n",
"Q 2725 213 2448 61 \n",
"Q 2172 -91 1784 -91 \n",
"Q 1150 -91 751 415 \n",
"Q 353 922 353 1747 \n",
"Q 353 2572 751 3078 \n",
"Q 1150 3584 1784 3584 \n",
"Q 2172 3584 2448 3432 \n",
"Q 2725 3281 2906 2969 \n",
"z\n",
"M 947 1747 \n",
"Q 947 1113 1208 752 \n",
"Q 1469 391 1925 391 \n",
"Q 2381 391 2643 752 \n",
"Q 2906 1113 2906 1747 \n",
"Q 2906 2381 2643 2742 \n",
"Q 2381 3103 1925 3103 \n",
"Q 1469 3103 1208 2742 \n",
"Q 947 2381 947 1747 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
"L 3597 1613 \n",
"L 953 1613 \n",
"Q 991 1019 1311 708 \n",
"Q 1631 397 2203 397 \n",
"Q 2534 397 2845 478 \n",
"Q 3156 559 3463 722 \n",
"L 3463 178 \n",
"Q 3153 47 2828 -22 \n",
"Q 2503 -91 2169 -91 \n",
"Q 1331 -91 842 396 \n",
"Q 353 884 353 1716 \n",
"Q 353 2575 817 3079 \n",
"Q 1281 3584 2069 3584 \n",
"Q 2775 3584 3186 3129 \n",
"Q 3597 2675 3597 1894 \n",
"z\n",
"M 3022 2063 \n",
"Q 3016 2534 2758 2815 \n",
"Q 2500 3097 2075 3097 \n",
"Q 1594 3097 1305 2825 \n",
"Q 1016 2553 972 2059 \n",
"L 3022 2063 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
"L 1159 -1331 \n",
"L 581 -1331 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2969 \n",
"Q 1341 3281 1617 3432 \n",
"Q 1894 3584 2278 3584 \n",
"Q 2916 3584 3314 3078 \n",
"Q 3713 2572 3713 1747 \n",
"Q 3713 922 3314 415 \n",
"Q 2916 -91 2278 -91 \n",
"Q 1894 -91 1617 61 \n",
"Q 1341 213 1159 525 \n",
"z\n",
"M 3116 1747 \n",
"Q 3116 2381 2855 2742 \n",
"Q 2594 3103 2138 3103 \n",
"Q 1681 3103 1420 2742 \n",
"Q 1159 2381 1159 1747 \n",
"Q 1159 1113 1420 752 \n",
"Q 1681 391 2138 391 \n",
"Q 2594 391 2855 752 \n",
"Q 3116 1113 3116 1747 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
"Q 1497 3097 1228 2736 \n",
"Q 959 2375 959 1747 \n",
"Q 959 1119 1226 758 \n",
"Q 1494 397 1959 397 \n",
"Q 2419 397 2687 759 \n",
"Q 2956 1122 2956 1747 \n",
"Q 2956 2369 2687 2733 \n",
"Q 2419 3097 1959 3097 \n",
"z\n",
"M 1959 3584 \n",
"Q 2709 3584 3137 3096 \n",
"Q 3566 2609 3566 1747 \n",
"Q 3566 888 3137 398 \n",
"Q 2709 -91 1959 -91 \n",
"Q 1206 -91 779 398 \n",
"Q 353 888 353 1747 \n",
"Q 353 2609 779 3096 \n",
"Q 1206 3584 1959 3584 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
"L 1178 3500 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 3500 \n",
"z\n",
"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 4134 \n",
"L 603 4134 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
"L 1172 3500 \n",
"L 2356 3500 \n",
"L 2356 3053 \n",
"L 1172 3053 \n",
"L 1172 1153 \n",
"Q 1172 725 1289 603 \n",
"Q 1406 481 1766 481 \n",
"L 2356 481 \n",
"L 2356 0 \n",
"L 1766 0 \n",
"Q 1100 0 847 248 \n",
"Q 594 497 594 1153 \n",
"L 594 3053 \n",
"L 172 3053 \n",
"L 172 3500 \n",
"L 594 3500 \n",
"L 594 4494 \n",
"L 1172 4494 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-73\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"52.099609\"/>\n",
" <use xlink:href=\"#DejaVuSans-64\" x=\"113.378906\"/>\n",
" <use xlink:href=\"#DejaVuSans-64\" x=\"176.855469\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"240.332031\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"268.115234\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"329.638672\"/>\n",
" <use xlink:href=\"#DejaVuSans-70\" x=\"361.425781\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"424.902344\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"486.083984\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"513.867188\"/>\n",
" <use xlink:href=\"#DejaVuSans-74\" x=\"577.246094\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"pdad0cc75ae\">\n",
" <rect x=\"42.620312\" y=\"7.2\" width=\"195.3\" height=\"135.9\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 252x180 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"x = torch.arange(-2.0, 2.0, 0.01)\n",
"d2l.plot(x, [x**3], 'x', 'f(x)')\n",
"annotate('saddle point', (0, -0.2), (-0.52, -5.0))"
]
},
{
"cell_type": "markdown",
"id": "1539e180",
"metadata": {
"origin_pos": 16
},
"source": [
"如下例所示,较高维度的鞍点甚至更加隐蔽。考虑这个函数$f(x, y) = x^2 - y^2$。它的鞍点为$(0, 0)$。这是关于$y$的最大值,也是关于$x$的最小值。此外,它看起来像个马鞍,这就是鞍点的名字由来。\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "52623051",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:07:09.225145Z",
"iopub.status.busy": "2023-08-18T07:07:09.224532Z",
"iopub.status.idle": "2023-08-18T07:07:09.358354Z",
"shell.execute_reply": "2023-08-18T07:07:09.357475Z"
},
"origin_pos": 18,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"167.228315pt\" height=\"169.322863pt\" viewBox=\"0 0 167.228315 169.322863\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T07:07:09.320886</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 169.322863 \n",
"L 167.228315 169.322863 \n",
"L 167.228315 0 \n",
"L 0 0 \n",
"L 0 169.322863 \n",
"z\n",
"\" style=\"fill: none\"/>\n",
" </g>\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 7.242206 143.1 \n",
"L 143.142206 143.1 \n",
"L 143.142206 7.2 \n",
"L 7.242206 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"pane3d_1\">\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 17.503593 109.591415 \n",
"L 62.382211 71.973267 \n",
"L 61.758355 17.721174 \n",
"L 14.732064 52.038819 \n",
"\" style=\"fill: #f2f2f2; opacity: 0.5; stroke: #f2f2f2; stroke-linejoin: miter\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"pane3d_2\">\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 62.382211 71.973267 \n",
"L 134.396381 92.904944 \n",
"L 136.966312 36.784198 \n",
"L 61.758355 17.721174 \n",
"\" style=\"fill: #e6e6e6; opacity: 0.5; stroke: #e6e6e6; stroke-linejoin: miter\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"pane3d_3\">\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 17.503593 109.591415 \n",
"L 93.84213 134.523581 \n",
"L 134.396381 92.904944 \n",
"L 62.382211 71.973267 \n",
"\" style=\"fill: #ececec; opacity: 0.5; stroke: #ececec; stroke-linejoin: miter\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"axis3d_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 17.503593 109.591415 \n",
"L 93.84213 134.523581 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- x -->\n",
" <g transform=\"translate(35.285103 160.043176)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"Line3DCollection_1\">\n",
" <path d=\"M 22.127055 111.101438 \n",
"L 66.761892 73.246268 \n",
"L 66.323213 18.878232 \n",
"\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
" <path d=\"M 54.666604 121.728855 \n",
"L 97.519476 82.186283 \n",
"L 98.41427 27.012379 \n",
"\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
" <path d=\"M 88.738736 132.856813 \n",
"L 129.601877 91.511371 \n",
"L 131.949299 35.512532 \n",
"\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
" </g>\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_2\">\n",
" <path d=\"M 22.515733 110.771797 \n",
"L 21.348032 111.762134 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(7.2 134.466744)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
"L 4684 2272 \n",
"L 4684 1741 \n",
"L 678 1741 \n",
"L 678 2272 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 55.040462 121.383877 \n",
"L 53.917247 122.420325 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(43.997457 145.539064)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_4\">\n",
" <path d=\"M 89.095932 132.495402 \n",
"L 88.02274 133.58126 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(78.152612 157.13687)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"axis3d_2\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 134.396381 92.904944 \n",
"L 93.84213 134.523581 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- y -->\n",
" <g transform=\"translate(139.069861 143.587338)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-79\" d=\"M 2059 -325 \n",
"Q 1816 -950 1584 -1140 \n",
"Q 1353 -1331 966 -1331 \n",
"L 506 -1331 \n",
"L 506 -850 \n",
"L 844 -850 \n",
"Q 1081 -850 1212 -737 \n",
"Q 1344 -625 1503 -206 \n",
"L 1606 56 \n",
"L 191 3500 \n",
"L 800 3500 \n",
"L 1894 763 \n",
"L 2988 3500 \n",
"L 3597 3500 \n",
"L 2059 -325 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-79\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"Line3DCollection_2\">\n",
" <path d=\"M 17.984048 49.665669 \n",
"L 20.596109 106.999207 \n",
"L 96.64817 131.643894 \n",
"\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
" <path d=\"M 39.291291 34.116614 \n",
"L 40.898687 89.981183 \n",
"L 115.027621 112.782056 \n",
"\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
" <path d=\"M 59.005785 19.729874 \n",
"L 59.745943 74.18304 \n",
"L 132.024016 95.339573 \n",
"\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_6\">\n",
" <path d=\"M 96.007277 131.436212 \n",
"L 97.931614 132.059794 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(102.165104 152.673329)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 114.404171 112.590293 \n",
"L 116.276069 113.166059 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(124.250678 133.264539)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_6\">\n",
" <g id=\"line2d_8\">\n",
" <path d=\"M 131.417248 95.161966 \n",
"L 133.239002 95.695212 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(140.798291 115.317846)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"axis3d_3\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 134.396381 92.904944 \n",
"L 136.966312 36.784198 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"Line3DCollection_3\">\n",
" <path d=\"M 134.445618 91.829732 \n",
"L 62.370234 70.931703 \n",
"L 17.450581 108.490593 \n",
"\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
" <path d=\"M 135.653195 65.459319 \n",
"L 62.076788 45.41288 \n",
"L 16.149355 81.469798 \n",
"\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
" <path d=\"M 136.912659 37.955845 \n",
"L 61.771353 18.851469 \n",
"L 14.790021 53.242344 \n",
"\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
" </g>\n",
" <g id=\"xtick_7\">\n",
" <g id=\"line2d_10\">\n",
" <path d=\"M 133.840682 91.654333 \n",
"L 135.656928 92.180948 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(145.286127 96.834718)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_8\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 135.035054 65.290902 \n",
"L 136.890981 65.796563 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(151.057056 70.567824)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_9\">\n",
" <g id=\"line2d_12\">\n",
" <path d=\"M 136.280723 37.795178 \n",
"L 138.178101 38.277579 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(152.705661 43.179212)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"line2d_13\">\n",
" <defs>\n",
" <path id=\"m8a01fa0265\" d=\"M -3 3 \n",
"L 3 -3 \n",
"M -3 -3 \n",
"L 3 3 \n",
"\" style=\"stroke: #ff0000\"/>\n",
" </defs>\n",
" <g clip-path=\"url(#pd30e6f8bb8)\">\n",
" <use xlink:href=\"#m8a01fa0265\" x=\"77.028692\" y=\"73.313514\" style=\"fill: #ff0000; stroke: #ff0000\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"Line3DCollection_4\">\n",
" <path d=\"M 24.027361 80.414345 \n",
"L 24.411647 78.980971 \n",
"L 24.796899 77.569496 \n",
"L 25.183095 76.180005 \n",
"L 25.570215 74.812571 \n",
"L 25.958241 73.467261 \n",
"L 26.34715 72.144156 \n",
"L 26.736922 70.843316 \n",
"L 27.127538 69.564812 \n",
"L 27.518977 68.308701 \n",
"L 27.911219 67.075047 \n",
"L 28.304241 65.86391 \n",
"L 28.698023 64.675346 \n",
"L 29.092547 63.509407 \n",
"L 29.487788 62.366147 \n",
"L 29.883727 61.245615 \n",
"L 30.280345 60.147854 \n",
"L 30.677616 59.072914 \n",
"L 31.075523 58.020831 \n",
"L 31.474043 56.991649 \n",
"L 31.873154 55.985404 \n",
"L 32.272837 55.002131 \n",
"L 32.673069 54.041858 \n",
"L 33.073829 53.10462 \n",
"L 33.475097 52.190441 \n",
"L 33.87685 51.299349 \n",
"L 34.279065 50.431363 \n",
"L 34.681724 49.586504 \n",
"L 35.084804 48.764793 \n",
"L 35.488283 47.966241 \n",
"L 35.89214 47.19086 \n",
"L 36.296353 46.438665 \n",
"L 36.700902 45.709659 \n",
"L 37.105764 45.00385 \n",
"L 37.510918 44.321241 \n",
"L 37.916344 43.661831 \n",
"L 38.322017 43.025619 \n",
"L 38.72792 42.4126 \n",
"L 39.134028 41.822769 \n",
"L 39.540322 41.256115 \n",
"L 39.946779 40.71263 \n",
"L 40.353378 40.192297 \n",
"L 40.760098 39.6951 \n",
"L 41.166919 39.221022 \n",
"L 41.573818 38.770041 \n",
"L 41.980774 38.342134 \n",
"L 42.387767 37.937277 \n",
"L 42.794776 37.555441 \n",
"L 43.201779 37.196597 \n",
"L 43.608755 36.86071 \n",
"L 44.015684 36.547748 \n",
"L 44.422545 36.257674 \n",
"L 44.829317 35.990448 \n",
"L 45.23598 35.746028 \n",
"L 45.642512 35.524372 \n",
"L 46.048894 35.325434 \n",
"L 46.455107 35.149166 \n",
"L 46.861127 34.99552 \n",
"L 47.266935 34.864439 \n",
"L 47.672513 34.755873 \n",
"L 48.077838 34.669764 \n",
"L 48.482892 34.606053 \n",
"L 48.887655 34.564683 \n",
"L 49.292107 34.545587 \n",
"L 49.696229 34.548702 \n",
"L 50.1 34.573963 \n",
"L 50.503402 34.6213 \n",
"L 50.906415 34.690642 \n",
"L 51.30902 34.781918 \n",
"L 51.711199 34.895054 \n",
"L 52.112931 35.029973 \n",
"L 52.514199 35.186596 \n",
"L 52.914983 35.364845 \n",
"L 53.315267 35.564637 \n",
"L 53.715029 35.785888 \n",
"L 54.114254 36.028515 \n",
"L 54.512922 36.292429 \n",
"L 54.911015 36.577542 \n",
"L 55.308518 36.883763 \n",
"L 55.705409 37.211 \n",
"L 56.101674 37.559161 \n",
"L 56.497293 37.928148 \n",
"L 56.892251 38.317866 \n",
"L 57.286529 38.728216 \n",
"L 57.68011 39.159096 \n",
"L 58.07298 39.610408 \n",
"L 58.46512 40.082044 \n",
"L 58.856514 40.573906 \n",
"L 59.247145 41.085882 \n",
"L 59.636998 41.617866 \n",
"L 60.026059 42.169755 \n",
"L 60.414308 42.74143 \n",
"L 60.801731 43.332785 \n",
"L 61.188315 43.943707 \n",
"L 61.574041 44.574081 \n",
"L 61.958895 45.22379 \n",
"L 62.342864 45.892722 \n",
"L 62.725931 46.580755 \n",
"L 63.108082 47.287772 \n",
"L 63.489304 48.013655 \n",
"L 63.869581 48.758278 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 30.898473 92.271992 \n",
"L 31.28199 90.844121 \n",
"L 31.66635 89.437874 \n",
"L 32.051531 88.053334 \n",
"L 32.437514 86.690572 \n",
"L 32.824281 85.349657 \n",
"L 33.211808 84.030666 \n",
"L 33.600077 82.733662 \n",
"L 33.989068 81.458714 \n",
"L 34.37876 80.205879 \n",
"L 34.769133 78.975221 \n",
"L 35.160165 77.766799 \n",
"L 35.551837 76.58067 \n",
"L 35.944129 75.416884 \n",
"L 36.337018 74.275495 \n",
"L 36.730484 73.156555 \n",
"L 37.124508 72.060102 \n",
"L 37.519067 70.98619 \n",
"L 37.914142 69.934852 \n",
"L 38.30971 68.906136 \n",
"L 38.70575 67.900074 \n",
"L 39.102243 66.916703 \n",
"L 39.499169 65.956051 \n",
"L 39.896503 65.018153 \n",
"L 40.294227 64.103032 \n",
"L 40.692319 63.210717 \n",
"L 41.090758 62.34123 \n",
"L 41.489523 61.49459 \n",
"L 41.888592 60.670817 \n",
"L 42.287946 59.869923 \n",
"L 42.687562 59.091923 \n",
"L 43.08742 58.336829 \n",
"L 43.487498 57.604648 \n",
"L 43.887777 56.895385 \n",
"L 44.288234 56.209046 \n",
"L 44.688849 55.545629 \n",
"L 45.0896 54.905136 \n",
"L 45.490468 54.287561 \n",
"L 45.89143 53.6929 \n",
"L 46.292467 53.121142 \n",
"L 46.693557 52.57228 \n",
"L 47.094679 52.046298 \n",
"L 47.495813 51.543183 \n",
"L 47.896939 51.062914 \n",
"L 48.298035 50.605476 \n",
"L 48.69908 50.170843 \n",
"L 49.100056 49.758992 \n",
"L 49.50094 49.369896 \n",
"L 49.901713 49.003527 \n",
"L 50.302355 48.659851 \n",
"L 50.702845 48.338836 \n",
"L 51.103162 48.040448 \n",
"L 51.503288 47.764648 \n",
"L 51.903202 47.511393 \n",
"L 52.302883 47.280645 \n",
"L 52.702314 47.072358 \n",
"L 53.101473 46.886484 \n",
"L 53.500341 46.722976 \n",
"L 53.898898 46.581785 \n",
"L 54.297125 46.462854 \n",
"L 54.695002 46.36613 \n",
"L 55.092512 46.291556 \n",
"L 55.489633 46.239074 \n",
"L 55.886349 46.208621 \n",
"L 56.282638 46.200135 \n",
"L 56.678484 46.213552 \n",
"L 57.073866 46.248802 \n",
"L 57.468768 46.30582 \n",
"L 57.863169 46.384531 \n",
"L 58.257052 46.484866 \n",
"L 58.6504 46.60675 \n",
"L 59.043192 46.750104 \n",
"L 59.435413 46.914853 \n",
"L 59.827045 47.100914 \n",
"L 60.218069 47.308208 \n",
"L 60.608468 47.536649 \n",
"L 60.998225 47.786154 \n",
"L 61.387323 48.056633 \n",
"L 61.775745 48.348002 \n",
"L 62.163473 48.660166 \n",
"L 62.550493 48.993037 \n",
"L 62.936785 49.346518 \n",
"L 63.322335 49.720517 \n",
"L 63.707126 50.114935 \n",
"L 64.091141 50.529675 \n",
"L 64.474366 50.964638 \n",
"L 64.856783 51.41972 \n",
"L 65.238378 51.894823 \n",
"L 65.619135 52.389838 \n",
"L 65.999038 52.904662 \n",
"L 66.378075 53.439192 \n",
"L 66.756227 53.993312 \n",
"L 67.133481 54.566918 \n",
"L 67.509824 55.159899 \n",
"L 67.885239 55.772142 \n",
"L 68.259712 56.403532 \n",
"L 68.633231 57.05396 \n",
"L 69.00578 57.723305 \n",
"L 69.377346 58.411452 \n",
"L 69.747916 59.118285 \n",
"L 70.117475 59.843682 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 37.65677 101.898819 \n",
"L 38.040179 100.473794 \n",
"L 38.424312 99.070192 \n",
"L 38.809148 97.688097 \n",
"L 39.194667 96.32758 \n",
"L 39.580852 94.988709 \n",
"L 39.967679 93.671563 \n",
"L 40.355129 92.376203 \n",
"L 40.743182 91.102697 \n",
"L 41.131819 89.851104 \n",
"L 41.521019 88.621486 \n",
"L 41.91076 87.413903 \n",
"L 42.301023 86.228412 \n",
"L 42.691788 85.065062 \n",
"L 43.083033 83.923908 \n",
"L 43.474738 82.805001 \n",
"L 43.866884 81.708381 \n",
"L 44.259447 80.6341 \n",
"L 44.65241 79.582192 \n",
"L 45.04575 78.552704 \n",
"L 45.439446 77.54567 \n",
"L 45.833478 76.561126 \n",
"L 46.227827 75.599099 \n",
"L 46.62247 74.659627 \n",
"L 47.017387 73.742732 \n",
"L 47.412558 72.848442 \n",
"L 47.80796 71.976782 \n",
"L 48.203574 71.127768 \n",
"L 48.599379 70.301422 \n",
"L 48.995354 69.497756 \n",
"L 49.391479 68.716787 \n",
"L 49.787732 67.958526 \n",
"L 50.184094 67.222979 \n",
"L 50.580543 66.510154 \n",
"L 50.977059 65.820056 \n",
"L 51.373622 65.152684 \n",
"L 51.77021 64.50804 \n",
"L 52.166804 63.886119 \n",
"L 52.563383 63.286916 \n",
"L 52.959926 62.710426 \n",
"L 53.356413 62.156635 \n",
"L 53.752824 61.625533 \n",
"L 54.149138 61.117105 \n",
"L 54.545336 60.631333 \n",
"L 54.941397 60.168199 \n",
"L 55.337302 59.72768 \n",
"L 55.733029 59.309755 \n",
"L 56.12856 58.914396 \n",
"L 56.523875 58.541575 \n",
"L 56.918953 58.191262 \n",
"L 57.313775 57.863424 \n",
"L 57.708322 57.558026 \n",
"L 58.102573 57.275031 \n",
"L 58.49651 57.0144 \n",
"L 58.890114 56.776091 \n",
"L 59.283365 56.560061 \n",
"L 59.676244 56.366265 \n",
"L 60.068731 56.194654 \n",
"L 60.460809 56.04518 \n",
"L 60.852457 55.917789 \n",
"L 61.243659 55.812428 \n",
"L 61.634394 55.729041 \n",
"L 62.024644 55.667569 \n",
"L 62.414392 55.627954 \n",
"L 62.80362 55.610133 \n",
"L 63.192308 55.614042 \n",
"L 63.580438 55.639614 \n",
"L 63.967994 55.686784 \n",
"L 64.354957 55.75548 \n",
"L 64.74131 55.845632 \n",
"L 65.127036 55.957166 \n",
"L 65.512116 56.090006 \n",
"L 65.896534 56.244075 \n",
"L 66.280274 56.419296 \n",
"L 66.663318 56.615586 \n",
"L 67.045649 56.832865 \n",
"L 67.42725 57.071048 \n",
"L 67.808106 57.330048 \n",
"L 68.188201 57.609781 \n",
"L 68.567517 57.910154 \n",
"L 68.94604 58.23108 \n",
"L 69.323752 58.572464 \n",
"L 69.70064 58.934214 \n",
"L 70.076686 59.316234 \n",
"L 70.451876 59.718426 \n",
"L 70.826195 60.140694 \n",
"L 71.199627 60.582936 \n",
"L 71.572158 61.045053 \n",
"L 71.943773 61.526941 \n",
"L 72.314457 62.028494 \n",
"L 72.684198 62.549613 \n",
"L 73.052979 63.090184 \n",
"L 73.420788 63.650103 \n",
"L 73.78761 64.22926 \n",
"L 74.153432 64.827545 \n",
"L 74.518239 65.444844 \n",
"L 74.882021 66.081048 \n",
"L 75.244762 66.736038 \n",
"L 75.60645 67.409702 \n",
"L 75.967073 68.101924 \n",
"L 76.326616 68.812583 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 44.337948 109.355104 \n",
"L 44.721908 107.930294 \n",
"L 45.106477 106.526784 \n",
"L 45.491633 105.144655 \n",
"L 45.877357 103.78398 \n",
"L 46.26363 102.444827 \n",
"L 46.650429 101.127273 \n",
"L 47.037737 99.83138 \n",
"L 47.425531 98.557217 \n",
"L 47.813793 97.304843 \n",
"L 48.202503 96.074319 \n",
"L 48.591638 94.865706 \n",
"L 48.981179 93.67906 \n",
"L 49.371107 92.514431 \n",
"L 49.761399 91.371875 \n",
"L 50.152036 90.251442 \n",
"L 50.542998 89.153171 \n",
"L 50.934263 88.077116 \n",
"L 51.325813 87.023311 \n",
"L 51.717624 85.991802 \n",
"L 52.109677 84.982625 \n",
"L 52.501952 83.995815 \n",
"L 52.894429 83.031399 \n",
"L 53.287085 82.089416 \n",
"L 53.679902 81.169888 \n",
"L 54.072859 80.272844 \n",
"L 54.465933 79.398307 \n",
"L 54.859107 78.546295 \n",
"L 55.252358 77.71683 \n",
"L 55.645667 76.909926 \n",
"L 56.039013 76.125597 \n",
"L 56.432376 75.363856 \n",
"L 56.825734 74.624711 \n",
"L 57.219069 73.908168 \n",
"L 57.612359 73.214232 \n",
"L 58.005585 72.542905 \n",
"L 58.398726 71.894187 \n",
"L 58.791762 71.268074 \n",
"L 59.184674 70.664563 \n",
"L 59.57744 70.083646 \n",
"L 59.970041 69.525313 \n",
"L 60.362457 68.989552 \n",
"L 60.754668 68.476349 \n",
"L 61.146655 67.985688 \n",
"L 61.538397 67.517551 \n",
"L 61.929876 67.071915 \n",
"L 62.321071 66.648758 \n",
"L 62.711963 66.248053 \n",
"L 63.102533 65.869775 \n",
"L 63.492762 65.513892 \n",
"L 63.88263 65.180372 \n",
"L 64.272118 64.869182 \n",
"L 64.661207 64.580284 \n",
"L 65.049878 64.31364 \n",
"L 65.438113 64.069209 \n",
"L 65.825893 63.846948 \n",
"L 66.2132 63.646812 \n",
"L 66.600014 63.468754 \n",
"L 66.986317 63.312724 \n",
"L 67.372092 63.178673 \n",
"L 67.757319 63.066545 \n",
"L 68.141982 62.976286 \n",
"L 68.526061 62.907839 \n",
"L 68.909541 62.861143 \n",
"L 69.292402 62.836139 \n",
"L 69.674628 62.832761 \n",
"L 70.0562 62.850946 \n",
"L 70.437103 62.890626 \n",
"L 70.817317 62.951732 \n",
"L 71.196828 63.034194 \n",
"L 71.575619 63.137938 \n",
"L 71.953671 63.262891 \n",
"L 72.330968 63.408974 \n",
"L 72.707496 63.576112 \n",
"L 73.083237 63.764224 \n",
"L 73.458176 63.973228 \n",
"L 73.832295 64.20304 \n",
"L 74.20558 64.453577 \n",
"L 74.578017 64.724752 \n",
"L 74.949586 65.016475 \n",
"L 75.320277 65.328658 \n",
"L 75.69007 65.661209 \n",
"L 76.058954 66.014035 \n",
"L 76.426912 66.387041 \n",
"L 76.79393 66.780131 \n",
"L 77.159994 67.193209 \n",
"L 77.525088 67.626172 \n",
"L 77.8892 68.078925 \n",
"L 78.252315 68.551361 \n",
"L 78.61442 69.043379 \n",
"L 78.975501 69.554879 \n",
"L 79.335544 70.085746 \n",
"L 79.694537 70.635879 \n",
"L 80.052466 71.205169 \n",
"L 80.409319 71.793505 \n",
"L 80.765082 72.400775 \n",
"L 81.119744 73.02687 \n",
"L 81.473291 73.671674 \n",
"L 81.825712 74.335073 \n",
"L 82.176995 75.016953 \n",
"L 82.527128 75.717195 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 50.976794 114.678321 \n",
"L 51.361972 113.251101 \n",
"L 51.747644 111.845128 \n",
"L 52.133789 110.460487 \n",
"L 52.520388 109.097248 \n",
"L 52.907421 107.755481 \n",
"L 53.294867 106.435263 \n",
"L 53.682705 105.136657 \n",
"L 54.070916 103.859731 \n",
"L 54.45948 102.604544 \n",
"L 54.848377 101.371159 \n",
"L 55.237585 100.159636 \n",
"L 55.627084 98.970032 \n",
"L 56.016855 97.802396 \n",
"L 56.406876 96.656786 \n",
"L 56.797127 95.53325 \n",
"L 57.187588 94.43183 \n",
"L 57.578238 93.352578 \n",
"L 57.969057 92.295531 \n",
"L 58.360024 91.260732 \n",
"L 58.751118 90.248219 \n",
"L 59.14232 89.258027 \n",
"L 59.53361 88.290184 \n",
"L 59.924965 87.344729 \n",
"L 60.316367 86.421682 \n",
"L 60.707795 85.521075 \n",
"L 61.099227 84.64293 \n",
"L 61.490645 83.787267 \n",
"L 61.882027 82.954107 \n",
"L 62.273354 82.143463 \n",
"L 62.664605 81.355351 \n",
"L 63.05576 80.589785 \n",
"L 63.446798 79.84677 \n",
"L 63.837701 79.126316 \n",
"L 64.228447 78.428426 \n",
"L 64.619017 77.753103 \n",
"L 65.009391 77.100347 \n",
"L 65.399549 76.470156 \n",
"L 65.789471 75.862523 \n",
"L 66.179138 75.277445 \n",
"L 66.568529 74.71491 \n",
"L 66.957625 74.174906 \n",
"L 67.346407 73.657421 \n",
"L 67.734856 73.162437 \n",
"L 68.122951 72.689937 \n",
"L 68.510674 72.239899 \n",
"L 68.898006 71.812301 \n",
"L 69.284927 71.407117 \n",
"L 69.671418 71.02432 \n",
"L 70.057461 70.66388 \n",
"L 70.443037 70.325766 \n",
"L 70.828127 70.009943 \n",
"L 71.212712 69.716374 \n",
"L 71.596775 69.445023 \n",
"L 71.980296 69.195847 \n",
"L 72.363259 68.968804 \n",
"L 72.745644 68.76385 \n",
"L 73.127433 68.580937 \n",
"L 73.508609 68.420018 \n",
"L 73.889153 68.28104 \n",
"L 74.269049 68.16395 \n",
"L 74.648279 68.068694 \n",
"L 75.026825 67.995214 \n",
"L 75.404671 67.943451 \n",
"L 75.781799 67.913345 \n",
"L 76.158192 67.904831 \n",
"L 76.533834 67.917846 \n",
"L 76.908707 67.952322 \n",
"L 77.282796 68.00819 \n",
"L 77.656084 68.08538 \n",
"L 78.028555 68.18382 \n",
"L 78.400192 68.303435 \n",
"L 78.77098 68.444149 \n",
"L 79.140904 68.605884 \n",
"L 79.509947 68.788561 \n",
"L 79.878095 68.992098 \n",
"L 80.245331 69.216412 \n",
"L 80.61164 69.461419 \n",
"L 80.97701 69.727032 \n",
"L 81.341423 70.013162 \n",
"L 81.704866 70.319722 \n",
"L 82.067324 70.646619 \n",
"L 82.428783 70.993761 \n",
"L 82.789228 71.361052 \n",
"L 83.148646 71.748397 \n",
"L 83.507024 72.1557 \n",
"L 83.864347 72.582859 \n",
"L 84.220601 73.029779 \n",
"L 84.575775 73.496353 \n",
"L 84.929854 73.982479 \n",
"L 85.282827 74.488059 \n",
"L 85.63468 75.012978 \n",
"L 85.985401 75.557134 \n",
"L 86.334978 76.120419 \n",
"L 86.683397 76.702723 \n",
"L 87.030648 77.303932 \n",
"L 87.376719 77.92394 \n",
"L 87.721597 78.562629 \n",
"L 88.065273 79.219887 \n",
"L 88.407734 79.895599 \n",
"L 88.748969 80.589645 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 57.607816 117.883787 \n",
"L 57.99489 116.451502 \n",
"L 58.382345 115.040488 \n",
"L 58.770158 113.650827 \n",
"L 59.158311 112.282593 \n",
"L 59.546785 110.935854 \n",
"L 59.935556 109.610689 \n",
"L 60.324606 108.30716 \n",
"L 60.713913 107.025338 \n",
"L 61.103459 105.765279 \n",
"L 61.493223 104.527049 \n",
"L 61.883183 103.310709 \n",
"L 62.273319 102.116314 \n",
"L 62.663613 100.943916 \n",
"L 63.05404 99.793571 \n",
"L 63.444582 98.665329 \n",
"L 63.835221 97.559233 \n",
"L 64.225931 96.475334 \n",
"L 64.616697 95.413668 \n",
"L 65.007494 94.374283 \n",
"L 65.398304 93.357212 \n",
"L 65.789107 92.362494 \n",
"L 66.179882 91.390157 \n",
"L 66.570607 90.440237 \n",
"L 66.961265 89.51276 \n",
"L 67.351832 88.607753 \n",
"L 67.74229 87.725242 \n",
"L 68.132619 86.865245 \n",
"L 68.522797 86.027783 \n",
"L 68.912805 85.212872 \n",
"L 69.302623 84.420525 \n",
"L 69.692231 83.650758 \n",
"L 70.081608 82.903576 \n",
"L 70.470736 82.178988 \n",
"L 70.859593 81.477 \n",
"L 71.248161 80.797612 \n",
"L 71.636419 80.140827 \n",
"L 72.024348 79.50664 \n",
"L 72.411929 78.895047 \n",
"L 72.799141 78.306043 \n",
"L 73.185966 77.739618 \n",
"L 73.572383 77.195759 \n",
"L 73.958375 76.674454 \n",
"L 74.343921 76.175686 \n",
"L 74.729003 75.699436 \n",
"L 75.113602 75.245685 \n",
"L 75.497699 74.814408 \n",
"L 75.881274 74.405581 \n",
"L 76.264311 74.019177 \n",
"L 76.646789 73.655165 \n",
"L 77.028691 73.313514 \n",
"L 77.409998 72.994189 \n",
"L 77.790692 72.697155 \n",
"L 78.170756 72.422373 \n",
"L 78.55017 72.169802 \n",
"L 78.928918 71.9394 \n",
"L 79.306982 71.731121 \n",
"L 79.684344 71.544919 \n",
"L 80.060986 71.380745 \n",
"L 80.436892 71.238547 \n",
"L 80.812044 71.118272 \n",
"L 81.186425 71.019866 \n",
"L 81.560019 70.94327 \n",
"L 81.932808 70.888425 \n",
"L 82.304777 70.855271 \n",
"L 82.675908 70.843744 \n",
"L 83.046186 70.853778 \n",
"L 83.415594 70.885308 \n",
"L 83.784116 70.938263 \n",
"L 84.151737 71.012574 \n",
"L 84.518441 71.108167 \n",
"L 84.884212 71.224968 \n",
"L 85.249035 71.3629 \n",
"L 85.612896 71.521886 \n",
"L 85.975778 71.701845 \n",
"L 86.337668 71.902697 \n",
"L 86.698549 72.124357 \n",
"L 87.058409 72.36674 \n",
"L 87.417234 72.629761 \n",
"L 87.775007 72.91333 \n",
"L 88.131716 73.217358 \n",
"L 88.487346 73.541754 \n",
"L 88.841886 73.886423 \n",
"L 89.195319 74.251272 \n",
"L 89.547634 74.636204 \n",
"L 89.898818 75.041122 \n",
"L 90.248856 75.465925 \n",
"L 90.597738 75.910516 \n",
"L 90.945449 76.374789 \n",
"L 91.291978 76.858642 \n",
"L 91.637314 77.361975 \n",
"L 91.981443 77.884674 \n",
"L 92.324354 78.426636 \n",
"L 92.666036 78.987752 \n",
"L 93.006475 79.567913 \n",
"L 93.345663 80.167005 \n",
"L 93.683588 80.784919 \n",
"L 94.020237 81.421539 \n",
"L 94.355602 82.076751 \n",
"L 94.689672 82.750441 \n",
"L 95.022435 83.442489 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 64.265829 118.964692 \n",
"L 64.655497 117.524633 \n",
"L 65.045431 116.10594 \n",
"L 65.43561 114.708699 \n",
"L 65.826013 113.332983 \n",
"L 66.216622 111.978862 \n",
"L 66.607413 110.646414 \n",
"L 66.998367 109.335704 \n",
"L 67.389463 108.046801 \n",
"L 67.780681 106.779765 \n",
"L 68.172001 105.53466 \n",
"L 68.5634 104.311548 \n",
"L 68.954859 103.110486 \n",
"L 69.346359 101.931526 \n",
"L 69.737876 100.774724 \n",
"L 70.129391 99.640132 \n",
"L 70.520885 98.527791 \n",
"L 70.912334 97.437754 \n",
"L 71.303721 96.37006 \n",
"L 71.695023 95.324752 \n",
"L 72.086219 94.301869 \n",
"L 72.477291 93.301447 \n",
"L 72.868219 92.323513 \n",
"L 73.258979 91.368109 \n",
"L 73.649554 90.435256 \n",
"L 74.039922 89.524984 \n",
"L 74.430062 88.637318 \n",
"L 74.819956 87.772277 \n",
"L 75.209582 86.929882 \n",
"L 75.598921 86.110149 \n",
"L 75.987953 85.313093 \n",
"L 76.376658 84.538726 \n",
"L 76.765015 83.787058 \n",
"L 77.153006 83.058096 \n",
"L 77.540609 82.351845 \n",
"L 77.927807 81.668306 \n",
"L 78.314578 81.007482 \n",
"L 78.700904 80.369368 \n",
"L 79.086766 79.753962 \n",
"L 79.472143 79.161255 \n",
"L 79.857017 78.591239 \n",
"L 80.241368 78.043902 \n",
"L 80.625178 77.519231 \n",
"L 81.008428 77.017207 \n",
"L 81.391098 76.537814 \n",
"L 81.773171 76.081031 \n",
"L 82.154627 75.646833 \n",
"L 82.535449 75.235197 \n",
"L 82.915617 74.846093 \n",
"L 83.295114 74.479493 \n",
"L 83.673921 74.135363 \n",
"L 84.052021 73.81367 \n",
"L 84.429395 73.514377 \n",
"L 84.806027 73.237445 \n",
"L 85.181897 72.982833 \n",
"L 85.55699 72.750498 \n",
"L 85.931288 72.540395 \n",
"L 86.304773 72.352476 \n",
"L 86.677428 72.186691 \n",
"L 87.049237 72.04299 \n",
"L 87.420183 71.921318 \n",
"L 87.790249 71.821619 \n",
"L 88.159419 71.743836 \n",
"L 88.527677 71.68791 \n",
"L 88.895006 71.653777 \n",
"L 89.261391 71.641375 \n",
"L 89.626815 71.650637 \n",
"L 89.991264 71.681496 \n",
"L 90.354722 71.733882 \n",
"L 90.717173 71.807724 \n",
"L 91.078602 71.902949 \n",
"L 91.438995 72.019481 \n",
"L 91.798336 72.157242 \n",
"L 92.156611 72.316156 \n",
"L 92.513806 72.49614 \n",
"L 92.869906 72.697112 \n",
"L 93.224897 72.918988 \n",
"L 93.578765 73.161683 \n",
"L 93.931498 73.425109 \n",
"L 94.28308 73.709176 \n",
"L 94.633499 74.013795 \n",
"L 94.982741 74.338872 \n",
"L 95.330795 74.684315 \n",
"L 95.677645 75.050025 \n",
"L 96.02328 75.435908 \n",
"L 96.367689 75.841865 \n",
"L 96.710857 76.267794 \n",
"L 97.052773 76.713597 \n",
"L 97.393426 77.179167 \n",
"L 97.732803 77.664401 \n",
"L 98.070895 78.169198 \n",
"L 98.407688 78.693444 \n",
"L 98.743171 79.237034 \n",
"L 99.077336 79.799859 \n",
"L 99.410169 80.381807 \n",
"L 99.74166 80.982765 \n",
"L 100.0718 81.602622 \n",
"L 100.400578 82.241262 \n",
"L 100.727984 82.898569 \n",
"L 101.054009 83.574427 \n",
"L 101.378643 84.268717 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 70.986575 117.891576 \n",
"L 71.379563 116.440943 \n",
"L 71.772703 115.011851 \n",
"L 72.165969 113.604385 \n",
"L 72.559343 112.21862 \n",
"L 72.952804 110.854626 \n",
"L 73.34633 109.512484 \n",
"L 73.739901 108.192258 \n",
"L 74.133496 106.89402 \n",
"L 74.527094 105.617829 \n",
"L 74.920675 104.363752 \n",
"L 75.314216 103.13185 \n",
"L 75.707699 101.922182 \n",
"L 76.101102 100.734799 \n",
"L 76.494402 99.56976 \n",
"L 76.887581 98.427117 \n",
"L 77.280618 97.306911 \n",
"L 77.673491 96.209197 \n",
"L 78.066181 95.134011 \n",
"L 78.458665 94.081402 \n",
"L 78.850924 93.051406 \n",
"L 79.242936 92.044059 \n",
"L 79.634684 91.059392 \n",
"L 80.026143 90.097444 \n",
"L 80.417296 89.158237 \n",
"L 80.808121 88.241802 \n",
"L 81.198597 87.348165 \n",
"L 81.588706 86.477344 \n",
"L 81.978426 85.629361 \n",
"L 82.367738 84.804231 \n",
"L 82.756621 84.00197 \n",
"L 83.145055 83.222591 \n",
"L 83.533021 82.466102 \n",
"L 83.9205 81.732512 \n",
"L 84.30747 81.021824 \n",
"L 84.693914 80.334042 \n",
"L 85.079811 79.669166 \n",
"L 85.465141 79.027192 \n",
"L 85.849887 78.408117 \n",
"L 86.234027 77.811934 \n",
"L 86.617544 77.238633 \n",
"L 87.000418 76.688202 \n",
"L 87.382631 76.160627 \n",
"L 87.764164 75.655891 \n",
"L 88.144998 75.173975 \n",
"L 88.525114 74.714859 \n",
"L 88.904495 74.278518 \n",
"L 89.283122 73.864926 \n",
"L 89.660977 73.474056 \n",
"L 90.038043 73.105877 \n",
"L 90.4143 72.760355 \n",
"L 90.789732 72.437457 \n",
"L 91.164321 72.137144 \n",
"L 91.538049 71.859378 \n",
"L 91.9109 71.604117 \n",
"L 92.282857 71.371316 \n",
"L 92.653902 71.160929 \n",
"L 93.024018 70.972909 \n",
"L 93.393188 70.807206 \n",
"L 93.761397 70.663766 \n",
"L 94.128628 70.542534 \n",
"L 94.494865 70.443455 \n",
"L 94.860092 70.366469 \n",
"L 95.224293 70.311516 \n",
"L 95.587452 70.278533 \n",
"L 95.949554 70.267455 \n",
"L 96.310583 70.278215 \n",
"L 96.670525 70.310744 \n",
"L 97.029364 70.364972 \n",
"L 97.387086 70.440826 \n",
"L 97.743676 70.538232 \n",
"L 98.099118 70.657113 \n",
"L 98.4534 70.79739 \n",
"L 98.806508 70.958984 \n",
"L 99.158426 71.141813 \n",
"L 99.509143 71.345792 \n",
"L 99.858643 71.570838 \n",
"L 100.206913 71.816862 \n",
"L 100.553943 72.083777 \n",
"L 100.899715 72.371489 \n",
"L 101.244221 72.67991 \n",
"L 101.587444 73.008943 \n",
"L 101.929376 73.358495 \n",
"L 102.270001 73.728467 \n",
"L 102.609309 74.118761 \n",
"L 102.947288 74.529278 \n",
"L 103.283926 74.959914 \n",
"L 103.619212 75.41057 \n",
"L 103.953134 75.881137 \n",
"L 104.285682 76.371511 \n",
"L 104.616846 76.881589 \n",
"L 104.946613 77.411255 \n",
"L 105.274973 77.960402 \n",
"L 105.601918 78.528921 \n",
"L 105.927436 79.116697 \n",
"L 106.251517 79.723616 \n",
"L 106.574153 80.349567 \n",
"L 106.895333 80.994428 \n",
"L 107.215049 81.658085 \n",
"L 107.533291 82.34042 \n",
"L 107.850051 83.041311 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 77.807395 114.611168 \n",
"L 78.204467 113.147041 \n",
"L 78.601571 111.704708 \n",
"L 78.99868 110.284257 \n",
"L 79.395777 108.885763 \n",
"L 79.79284 107.509299 \n",
"L 80.189847 106.154947 \n",
"L 80.586776 104.822772 \n",
"L 80.983608 103.512846 \n",
"L 81.380319 102.225232 \n",
"L 81.776892 100.959994 \n",
"L 82.173301 99.717198 \n",
"L 82.569527 98.496902 \n",
"L 82.965551 97.299159 \n",
"L 83.361348 96.124028 \n",
"L 83.756899 94.971562 \n",
"L 84.152184 93.841803 \n",
"L 84.54718 92.734807 \n",
"L 84.941867 91.650612 \n",
"L 85.336224 90.589265 \n",
"L 85.730229 89.550803 \n",
"L 86.123864 88.535266 \n",
"L 86.517107 87.542681 \n",
"L 86.909936 86.573089 \n",
"L 87.302333 85.626514 \n",
"L 87.694275 84.702987 \n",
"L 88.085742 83.802532 \n",
"L 88.476716 82.925169 \n",
"L 88.867174 82.070921 \n",
"L 89.257097 81.239801 \n",
"L 89.646466 80.431827 \n",
"L 90.035259 79.647011 \n",
"L 90.423457 78.885362 \n",
"L 90.81104 78.146886 \n",
"L 91.197989 77.431591 \n",
"L 91.584285 76.739476 \n",
"L 91.969906 76.070543 \n",
"L 92.354836 75.424788 \n",
"L 92.739053 74.802207 \n",
"L 93.12254 74.202793 \n",
"L 93.505276 73.626536 \n",
"L 93.887243 73.073422 \n",
"L 94.268423 72.543437 \n",
"L 94.648797 72.036564 \n",
"L 95.028345 71.552785 \n",
"L 95.407051 71.092075 \n",
"L 95.784896 70.654413 \n",
"L 96.161862 70.23977 \n",
"L 96.53793 69.848118 \n",
"L 96.913084 69.479425 \n",
"L 97.287305 69.133658 \n",
"L 97.660577 68.810781 \n",
"L 98.032881 68.510755 \n",
"L 98.4042 68.233541 \n",
"L 98.774519 67.979094 \n",
"L 99.143819 67.747371 \n",
"L 99.512086 67.538324 \n",
"L 99.8793 67.351903 \n",
"L 100.245447 67.188058 \n",
"L 100.610511 67.046734 \n",
"L 100.974474 66.927875 \n",
"L 101.337322 66.831423 \n",
"L 101.699039 66.757318 \n",
"L 102.05961 66.705498 \n",
"L 102.419019 66.675899 \n",
"L 102.777251 66.668454 \n",
"L 103.134291 66.683094 \n",
"L 103.490125 66.71975 \n",
"L 103.844737 66.778349 \n",
"L 104.198115 66.858817 \n",
"L 104.550244 66.961078 \n",
"L 104.901108 67.085053 \n",
"L 105.250695 67.230662 \n",
"L 105.598993 67.397823 \n",
"L 105.945985 67.586454 \n",
"L 106.291661 67.796467 \n",
"L 106.636006 68.027777 \n",
"L 106.979008 68.280293 \n",
"L 107.320656 68.553926 \n",
"L 107.660934 68.848581 \n",
"L 107.999834 69.164167 \n",
"L 108.33734 69.500587 \n",
"L 108.673444 69.857742 \n",
"L 109.008131 70.235534 \n",
"L 109.341391 70.633862 \n",
"L 109.673214 71.052626 \n",
"L 110.003587 71.491718 \n",
"L 110.332501 71.951037 \n",
"L 110.659944 72.430474 \n",
"L 110.985907 72.92992 \n",
"L 111.31038 73.449271 \n",
"L 111.633352 73.988409 \n",
"L 111.954813 74.547225 \n",
"L 112.274756 75.125607 \n",
"L 112.593168 75.723438 \n",
"L 112.910042 76.340602 \n",
"L 113.22537 76.976984 \n",
"L 113.539141 77.632462 \n",
"L 113.851348 78.306918 \n",
"L 114.161983 79.000233 \n",
"L 114.471037 79.71228 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 84.76796 109.044539 \n",
"L 85.169925 107.563836 \n",
"L 85.571797 106.105268 \n",
"L 85.973549 104.668922 \n",
"L 86.375163 103.254877 \n",
"L 86.776618 101.863206 \n",
"L 87.177889 100.493993 \n",
"L 87.578956 99.147305 \n",
"L 87.979797 97.823215 \n",
"L 88.38039 96.521787 \n",
"L 88.780716 95.243089 \n",
"L 89.18075 93.987186 \n",
"L 89.580472 92.754136 \n",
"L 89.979862 91.543997 \n",
"L 90.378895 90.356826 \n",
"L 90.777552 89.192678 \n",
"L 91.175812 88.051597 \n",
"L 91.573652 86.933638 \n",
"L 91.971053 85.83884 \n",
"L 92.367991 84.767253 \n",
"L 92.764447 83.718914 \n",
"L 93.160399 82.693861 \n",
"L 93.555828 81.692125 \n",
"L 93.95071 80.713746 \n",
"L 94.345028 79.758748 \n",
"L 94.738758 78.827162 \n",
"L 95.13188 77.919015 \n",
"L 95.524375 77.034324 \n",
"L 95.916221 76.173114 \n",
"L 96.307399 75.335399 \n",
"L 96.697889 74.521194 \n",
"L 97.087669 73.730514 \n",
"L 97.47672 72.963365 \n",
"L 97.865024 72.219757 \n",
"L 98.252558 71.499693 \n",
"L 98.639306 70.803175 \n",
"L 99.025245 70.130205 \n",
"L 99.410358 69.480776 \n",
"L 99.794626 68.854885 \n",
"L 100.178028 68.252523 \n",
"L 100.560546 67.673682 \n",
"L 100.942162 67.118345 \n",
"L 101.322856 66.5865 \n",
"L 101.702611 66.078125 \n",
"L 102.081407 65.593205 \n",
"L 102.459226 65.131713 \n",
"L 102.836051 64.693625 \n",
"L 103.211864 64.278914 \n",
"L 103.586647 63.88755 \n",
"L 103.960382 63.519498 \n",
"L 104.333052 63.174725 \n",
"L 104.704639 62.853194 \n",
"L 105.075128 62.554866 \n",
"L 105.444499 62.279696 \n",
"L 105.812738 62.027643 \n",
"L 106.179828 61.798659 \n",
"L 106.545752 61.592696 \n",
"L 106.910493 61.409701 \n",
"L 107.274037 61.249625 \n",
"L 107.636366 61.112408 \n",
"L 107.997466 60.997994 \n",
"L 108.357321 60.906323 \n",
"L 108.715915 60.837334 \n",
"L 109.073235 60.790961 \n",
"L 109.429264 60.767139 \n",
"L 109.783989 60.765799 \n",
"L 110.137394 60.786871 \n",
"L 110.489465 60.830283 \n",
"L 110.840189 60.895957 \n",
"L 111.189552 60.983822 \n",
"L 111.53754 61.093796 \n",
"L 111.884138 61.225799 \n",
"L 112.229335 61.379749 \n",
"L 112.573118 61.555562 \n",
"L 112.915472 61.753151 \n",
"L 113.256386 61.972428 \n",
"L 113.595847 62.213304 \n",
"L 113.933843 62.475687 \n",
"L 114.270363 62.759485 \n",
"L 114.605394 63.0646 \n",
"L 114.938925 63.390938 \n",
"L 115.270943 63.738399 \n",
"L 115.601439 64.106883 \n",
"L 115.930401 64.496288 \n",
"L 116.257818 64.90651 \n",
"L 116.583681 65.337447 \n",
"L 116.907978 65.788987 \n",
"L 117.2307 66.261028 \n",
"L 117.551836 66.753457 \n",
"L 117.871377 67.266161 \n",
"L 118.189315 67.799036 \n",
"L 118.505639 68.351958 \n",
"L 118.82034 68.924816 \n",
"L 119.133411 69.517496 \n",
"L 119.444841 70.129877 \n",
"L 119.754623 70.761839 \n",
"L 120.06275 71.413266 \n",
"L 120.369211 72.084031 \n",
"L 120.674001 72.774015 \n",
"L 120.977112 73.483093 \n",
"L 121.278535 74.211138 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 91.9111 101.084466 \n",
"L 92.318825 99.583906 \n",
"L 92.726324 98.10591 \n",
"L 93.133573 96.650572 \n",
"L 93.540551 95.21797 \n",
"L 93.947236 93.808179 \n",
"L 94.353605 92.421286 \n",
"L 94.759635 91.057359 \n",
"L 95.165305 89.716475 \n",
"L 95.570593 88.398697 \n",
"L 95.975477 87.104096 \n",
"L 96.379933 85.832738 \n",
"L 96.783941 84.584684 \n",
"L 97.187479 83.359991 \n",
"L 97.590523 82.158719 \n",
"L 97.993054 80.980924 \n",
"L 98.395049 79.82665 \n",
"L 98.796485 78.695955 \n",
"L 99.197343 77.588879 \n",
"L 99.597599 76.50547 \n",
"L 99.997232 75.445768 \n",
"L 100.396222 74.409811 \n",
"L 100.794548 73.397633 \n",
"L 101.192187 72.409271 \n",
"L 101.58912 71.444753 \n",
"L 101.985325 70.504109 \n",
"L 102.38078 69.587362 \n",
"L 102.775465 68.694536 \n",
"L 103.16936 67.825654 \n",
"L 103.562445 66.980728 \n",
"L 103.954699 66.159774 \n",
"L 104.346101 65.362809 \n",
"L 104.736632 64.589836 \n",
"L 105.126272 63.840865 \n",
"L 105.515 63.115901 \n",
"L 105.902798 62.414944 \n",
"L 106.289645 61.737993 \n",
"L 106.675522 61.085045 \n",
"L 107.060411 60.456094 \n",
"L 107.444291 59.85113 \n",
"L 107.827144 59.270144 \n",
"L 108.208951 58.71312 \n",
"L 108.589693 58.180043 \n",
"L 108.969353 57.670893 \n",
"L 109.347911 57.185647 \n",
"L 109.72535 56.724284 \n",
"L 110.101651 56.286775 \n",
"L 110.476798 55.873093 \n",
"L 110.850771 55.483206 \n",
"L 111.223554 55.117078 \n",
"L 111.59513 54.774673 \n",
"L 111.965482 54.455954 \n",
"L 112.334592 54.160879 \n",
"L 112.702444 53.889403 \n",
"L 113.069022 53.64148 \n",
"L 113.434309 53.417063 \n",
"L 113.798289 53.216101 \n",
"L 114.160947 53.03854 \n",
"L 114.522266 52.884324 \n",
"L 114.88223 52.753397 \n",
"L 115.240826 52.645698 \n",
"L 115.598037 52.561163 \n",
"L 115.953848 52.499732 \n",
"L 116.308247 52.461334 \n",
"L 116.661216 52.445902 \n",
"L 117.012743 52.453365 \n",
"L 117.362813 52.483649 \n",
"L 117.711413 52.53668 \n",
"L 118.058528 52.612381 \n",
"L 118.404147 52.710671 \n",
"L 118.748254 52.83147 \n",
"L 119.090838 52.974694 \n",
"L 119.431885 53.140258 \n",
"L 119.771383 53.328074 \n",
"L 120.10932 53.538052 \n",
"L 120.445684 53.770104 \n",
"L 120.780463 54.024134 \n",
"L 121.113644 54.300049 \n",
"L 121.445219 54.597751 \n",
"L 121.775173 54.917141 \n",
"L 122.103498 55.258121 \n",
"L 122.430181 55.620588 \n",
"L 122.755214 56.004438 \n",
"L 123.078583 56.409567 \n",
"L 123.400281 56.835866 \n",
"L 123.720298 57.283228 \n",
"L 124.038623 57.751541 \n",
"L 124.355247 58.240697 \n",
"L 124.670162 58.750578 \n",
"L 124.983358 59.281072 \n",
"L 125.294828 59.832066 \n",
"L 125.604561 60.403435 \n",
"L 125.91255 60.995065 \n",
"L 126.218788 61.606836 \n",
"L 126.523265 62.238624 \n",
"L 126.825975 62.890305 \n",
"L 127.126911 63.56176 \n",
"L 127.426065 64.252858 \n",
"L 127.72343 64.963475 \n",
"L 128.019001 65.693483 \n",
"L 128.312769 66.44275 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 24.027361 80.414345 \n",
"L 24.720594 81.702703 \n",
"L 25.412368 82.968029 \n",
"L 26.102718 84.210411 \n",
"L 26.791683 85.429941 \n",
"L 27.479302 86.626711 \n",
"L 28.165609 87.800798 \n",
"L 28.850643 88.952292 \n",
"L 29.53444 90.081268 \n",
"L 30.217037 91.187811 \n",
"L 30.898473 92.271992 \n",
"L 31.57878 93.333883 \n",
"L 32.257996 94.373557 \n",
"L 32.93616 95.391082 \n",
"L 33.613303 96.386524 \n",
"L 34.289463 97.359944 \n",
"L 34.964678 98.311408 \n",
"L 35.63898 99.240971 \n",
"L 36.312407 100.148692 \n",
"L 36.984992 101.034624 \n",
"L 37.65677 101.898819 \n",
"L 38.327779 102.741327 \n",
"L 38.998054 103.562198 \n",
"L 39.667626 104.361473 \n",
"L 40.336535 105.139199 \n",
"L 41.004812 105.895413 \n",
"L 41.672492 106.630156 \n",
"L 42.339612 107.343465 \n",
"L 43.006204 108.035369 \n",
"L 43.672304 108.705906 \n",
"L 44.337948 109.355104 \n",
"L 45.003167 109.982987 \n",
"L 45.667998 110.589585 \n",
"L 46.332475 111.174918 \n",
"L 46.996632 111.739007 \n",
"L 47.660505 112.281871 \n",
"L 48.324126 112.803528 \n",
"L 48.987532 113.30399 \n",
"L 49.650755 113.78327 \n",
"L 50.313831 114.241379 \n",
"L 50.976794 114.678321 \n",
"L 51.639678 115.094104 \n",
"L 52.302519 115.488732 \n",
"L 52.96535 115.862204 \n",
"L 53.628206 116.21452 \n",
"L 54.291122 116.545677 \n",
"L 54.954132 116.855668 \n",
"L 55.617272 117.144485 \n",
"L 56.280576 117.412119 \n",
"L 56.944079 117.658558 \n",
"L 57.607816 117.883787 \n",
"L 58.271823 118.087788 \n",
"L 58.936133 118.270543 \n",
"L 59.600783 118.432032 \n",
"L 60.265808 118.572229 \n",
"L 60.931244 118.69111 \n",
"L 61.597126 118.788645 \n",
"L 62.263489 118.864804 \n",
"L 62.93037 118.919556 \n",
"L 63.597805 118.952865 \n",
"L 64.265829 118.964692 \n",
"L 64.934479 118.955 \n",
"L 65.603791 118.923743 \n",
"L 66.273803 118.87088 \n",
"L 66.944551 118.796362 \n",
"L 67.616072 118.70014 \n",
"L 68.288402 118.582163 \n",
"L 68.96158 118.442377 \n",
"L 69.635642 118.280723 \n",
"L 70.310628 118.097144 \n",
"L 70.986575 117.891576 \n",
"L 71.663519 117.663958 \n",
"L 72.3415 117.41422 \n",
"L 73.020559 117.142295 \n",
"L 73.700731 116.84811 \n",
"L 74.382059 116.531589 \n",
"L 75.064578 116.192656 \n",
"L 75.748331 115.831232 \n",
"L 76.43336 115.447233 \n",
"L 77.119699 115.040575 \n",
"L 77.807395 114.611168 \n",
"L 78.496483 114.158922 \n",
"L 79.18701 113.683744 \n",
"L 79.879012 113.185535 \n",
"L 80.572533 112.6642 \n",
"L 81.267618 112.119631 \n",
"L 81.964305 111.551729 \n",
"L 82.662639 110.96038 \n",
"L 83.362663 110.345476 \n",
"L 84.064422 109.706904 \n",
"L 84.76796 109.044539 \n",
"L 85.473318 108.358269 \n",
"L 86.180544 107.647967 \n",
"L 86.889684 106.913505 \n",
"L 87.600781 106.154755 \n",
"L 88.313881 105.371583 \n",
"L 89.029035 104.563848 \n",
"L 89.746284 103.731416 \n",
"L 90.465679 102.87414 \n",
"L 91.18727 101.991872 \n",
"L 91.9111 101.084466 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 27.911219 67.075047 \n",
"L 28.602823 68.368706 \n",
"L 29.293038 69.639101 \n",
"L 29.981898 70.886316 \n",
"L 30.66944 72.110447 \n",
"L 31.355706 73.311584 \n",
"L 32.040728 74.489807 \n",
"L 32.724545 75.645203 \n",
"L 33.407193 76.77785 \n",
"L 34.08871 77.887832 \n",
"L 34.769133 78.975221 \n",
"L 35.448495 80.04009 \n",
"L 36.126834 81.082511 \n",
"L 36.804189 82.102554 \n",
"L 37.48059 83.100282 \n",
"L 38.156075 84.07576 \n",
"L 38.830683 85.029052 \n",
"L 39.504445 85.960214 \n",
"L 40.177399 86.869306 \n",
"L 40.849578 87.756378 \n",
"L 41.521019 88.621486 \n",
"L 42.191756 89.464676 \n",
"L 42.861827 90.286003 \n",
"L 43.531262 91.085504 \n",
"L 44.200101 91.863228 \n",
"L 44.868375 92.619212 \n",
"L 45.536119 93.353495 \n",
"L 46.20337 94.066115 \n",
"L 46.870161 94.757104 \n",
"L 47.536527 95.426495 \n",
"L 48.202503 96.074319 \n",
"L 48.868122 96.700599 \n",
"L 49.533419 97.305364 \n",
"L 50.198431 97.888635 \n",
"L 50.863189 98.450432 \n",
"L 51.527731 98.990776 \n",
"L 52.192088 99.50968 \n",
"L 52.856298 100.00716 \n",
"L 53.520393 100.483227 \n",
"L 54.184408 100.937892 \n",
"L 54.848377 101.371159 \n",
"L 55.512336 101.783036 \n",
"L 56.176319 102.173524 \n",
"L 56.840362 102.542626 \n",
"L 57.504497 102.890336 \n",
"L 58.16876 103.216654 \n",
"L 58.833187 103.521572 \n",
"L 59.497811 103.805083 \n",
"L 60.162669 104.067174 \n",
"L 60.827794 104.307835 \n",
"L 61.493223 104.527049 \n",
"L 62.158991 104.724799 \n",
"L 62.825132 104.901065 \n",
"L 63.491682 105.055827 \n",
"L 64.158677 105.189058 \n",
"L 64.826153 105.300732 \n",
"L 65.494147 105.390821 \n",
"L 66.162691 105.459294 \n",
"L 66.831825 105.506115 \n",
"L 67.501582 105.53125 \n",
"L 68.172001 105.53466 \n",
"L 68.843117 105.516306 \n",
"L 69.514967 105.476142 \n",
"L 70.187589 105.414125 \n",
"L 70.86102 105.330206 \n",
"L 71.535296 105.224334 \n",
"L 72.210454 105.096457 \n",
"L 72.886533 104.946519 \n",
"L 73.563571 104.774464 \n",
"L 74.241605 104.580229 \n",
"L 74.920675 104.363752 \n",
"L 75.600817 104.124969 \n",
"L 76.282071 103.86381 \n",
"L 76.964478 103.580205 \n",
"L 77.648074 103.274081 \n",
"L 78.332901 102.945362 \n",
"L 79.018997 102.59397 \n",
"L 79.706404 102.219823 \n",
"L 80.395162 101.822835 \n",
"L 81.08531 101.402923 \n",
"L 81.776892 100.959994 \n",
"L 82.469945 100.493958 \n",
"L 83.164515 100.004718 \n",
"L 83.86064 99.492177 \n",
"L 84.558365 98.956235 \n",
"L 85.257733 98.396784 \n",
"L 85.958785 97.813723 \n",
"L 86.661565 97.206936 \n",
"L 87.366117 96.576313 \n",
"L 88.072485 95.92174 \n",
"L 88.780716 95.243089 \n",
"L 89.490851 94.540248 \n",
"L 90.202938 93.813087 \n",
"L 90.917023 93.061476 \n",
"L 91.63315 92.285284 \n",
"L 92.351367 91.484376 \n",
"L 93.071723 90.65861 \n",
"L 93.794263 89.807848 \n",
"L 94.519035 88.931942 \n",
"L 95.246092 88.030741 \n",
"L 95.975477 87.104096 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 31.873154 55.985404 \n",
"L 32.561899 57.281197 \n",
"L 33.249328 58.553572 \n",
"L 33.935475 59.802617 \n",
"L 34.620378 61.028425 \n",
"L 35.304077 62.231087 \n",
"L 35.986604 63.410683 \n",
"L 36.667998 64.567301 \n",
"L 37.348296 65.701018 \n",
"L 38.027534 66.811918 \n",
"L 38.70575 67.900074 \n",
"L 39.382976 68.965559 \n",
"L 40.059251 70.008443 \n",
"L 40.734611 71.028798 \n",
"L 41.409088 72.026688 \n",
"L 42.08272 73.002174 \n",
"L 42.755545 73.955324 \n",
"L 43.427593 74.886192 \n",
"L 44.098905 75.794838 \n",
"L 44.76951 76.681312 \n",
"L 45.439446 77.54567 \n",
"L 46.108748 78.387959 \n",
"L 46.777453 79.208232 \n",
"L 47.445592 80.006527 \n",
"L 48.113203 80.782893 \n",
"L 48.780318 81.537366 \n",
"L 49.446971 82.269985 \n",
"L 50.1132 82.980789 \n",
"L 50.779037 83.669809 \n",
"L 51.444518 84.337078 \n",
"L 52.109677 84.982625 \n",
"L 52.774547 85.606476 \n",
"L 53.439164 86.208657 \n",
"L 54.103563 86.789191 \n",
"L 54.767776 87.348097 \n",
"L 55.431841 87.885394 \n",
"L 56.095789 88.401098 \n",
"L 56.759657 88.895222 \n",
"L 57.423478 89.367779 \n",
"L 58.087287 89.818775 \n",
"L 58.751118 90.248219 \n",
"L 59.415006 90.656116 \n",
"L 60.078986 91.042467 \n",
"L 60.743093 91.407275 \n",
"L 61.407361 91.750535 \n",
"L 62.071824 92.072244 \n",
"L 62.736519 92.372395 \n",
"L 63.401479 92.65098 \n",
"L 64.06674 92.907987 \n",
"L 64.732337 93.143403 \n",
"L 65.398304 93.357212 \n",
"L 66.064678 93.549397 \n",
"L 66.731494 93.719938 \n",
"L 67.398788 93.868812 \n",
"L 68.066594 93.995993 \n",
"L 68.734949 94.101456 \n",
"L 69.40389 94.18517 \n",
"L 70.073451 94.247104 \n",
"L 70.743669 94.287223 \n",
"L 71.414579 94.305491 \n",
"L 72.086219 94.301869 \n",
"L 72.758626 94.276316 \n",
"L 73.431836 94.228788 \n",
"L 74.105887 94.159238 \n",
"L 74.780816 94.067619 \n",
"L 75.456659 93.953879 \n",
"L 76.133454 93.817965 \n",
"L 76.811241 93.65982 \n",
"L 77.490055 93.479387 \n",
"L 78.169937 93.276603 \n",
"L 78.850924 93.051406 \n",
"L 79.533054 92.803728 \n",
"L 80.216366 92.533503 \n",
"L 80.900903 92.240656 \n",
"L 81.586699 91.925116 \n",
"L 82.273799 91.586804 \n",
"L 82.962239 91.225642 \n",
"L 83.65206 90.841548 \n",
"L 84.343307 90.434435 \n",
"L 85.036015 90.004218 \n",
"L 85.730229 89.550803 \n",
"L 86.425988 89.0741 \n",
"L 87.123338 88.574012 \n",
"L 87.822316 88.050439 \n",
"L 88.522967 87.50328 \n",
"L 89.225336 86.932428 \n",
"L 89.929463 86.337779 \n",
"L 90.635393 85.719218 \n",
"L 91.34317 85.076633 \n",
"L 92.052839 84.409907 \n",
"L 92.764447 83.718914 \n",
"L 93.478035 83.003539 \n",
"L 94.193651 82.263651 \n",
"L 94.911343 81.499119 \n",
"L 95.631154 80.709811 \n",
"L 96.353132 79.895592 \n",
"L 97.077329 79.056317 \n",
"L 97.803786 78.191847 \n",
"L 98.532556 77.302034 \n",
"L 99.26369 76.386724 \n",
"L 99.997232 75.445768 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 35.89214 47.19086 \n",
"L 36.576824 48.485604 \n",
"L 37.26027 49.756862 \n",
"L 37.94251 51.004721 \n",
"L 38.623583 52.229273 \n",
"L 39.303528 53.430611 \n",
"L 39.982378 54.608813 \n",
"L 40.66017 55.763966 \n",
"L 41.33694 56.896148 \n",
"L 42.012725 58.005443 \n",
"L 42.687562 59.091923 \n",
"L 43.361483 60.155659 \n",
"L 44.034526 61.196724 \n",
"L 44.706728 62.215189 \n",
"L 45.37812 63.211115 \n",
"L 46.04874 64.184565 \n",
"L 46.718624 65.135607 \n",
"L 47.387804 66.064293 \n",
"L 48.056318 66.970685 \n",
"L 48.724197 67.854831 \n",
"L 49.391479 68.716787 \n",
"L 50.058197 69.5566 \n",
"L 50.724389 70.374322 \n",
"L 51.390084 71.169992 \n",
"L 52.055322 71.943657 \n",
"L 52.720134 72.695355 \n",
"L 53.384553 73.425123 \n",
"L 54.048618 74.133 \n",
"L 54.712359 74.819016 \n",
"L 55.375813 75.483206 \n",
"L 56.039013 76.125597 \n",
"L 56.701994 76.746215 \n",
"L 57.364789 77.345086 \n",
"L 58.027434 77.922232 \n",
"L 58.689962 78.477672 \n",
"L 59.352409 79.011426 \n",
"L 60.014806 79.523508 \n",
"L 60.677191 80.013931 \n",
"L 61.339596 80.482708 \n",
"L 62.002056 80.929845 \n",
"L 62.664605 81.355351 \n",
"L 63.327278 81.75923 \n",
"L 63.990109 82.141484 \n",
"L 64.653134 82.502112 \n",
"L 65.316386 82.841113 \n",
"L 65.9799 83.158482 \n",
"L 66.643711 83.454211 \n",
"L 67.307854 83.728293 \n",
"L 67.972363 83.980716 \n",
"L 68.637275 84.211465 \n",
"L 69.302623 84.420525 \n",
"L 69.968444 84.607879 \n",
"L 70.634772 84.773504 \n",
"L 71.301643 84.917379 \n",
"L 71.969092 85.03948 \n",
"L 72.637157 85.139776 \n",
"L 73.305872 85.218241 \n",
"L 73.975272 85.27484 \n",
"L 74.645395 85.30954 \n",
"L 75.316277 85.322304 \n",
"L 75.987953 85.313093 \n",
"L 76.660462 85.281864 \n",
"L 77.333839 85.228575 \n",
"L 78.008123 85.153178 \n",
"L 78.683349 85.055625 \n",
"L 79.359556 84.935864 \n",
"L 80.03678 84.793841 \n",
"L 80.715061 84.629501 \n",
"L 81.394435 84.442784 \n",
"L 82.074943 84.233628 \n",
"L 82.756621 84.00197 \n",
"L 83.439508 83.747743 \n",
"L 84.123643 83.470879 \n",
"L 84.809068 83.171305 \n",
"L 85.49582 82.848947 \n",
"L 86.183939 82.503728 \n",
"L 86.873466 82.135568 \n",
"L 87.56444 81.744385 \n",
"L 88.256906 81.330092 \n",
"L 88.950899 80.892604 \n",
"L 89.646466 80.431827 \n",
"L 90.343643 79.947669 \n",
"L 91.042478 79.440033 \n",
"L 91.743008 78.90882 \n",
"L 92.445278 78.353929 \n",
"L 93.149334 77.775251 \n",
"L 93.855214 77.172682 \n",
"L 94.562965 76.546106 \n",
"L 95.272631 75.895413 \n",
"L 95.984256 75.220484 \n",
"L 96.697889 74.521194 \n",
"L 97.413569 73.797426 \n",
"L 98.131346 73.049049 \n",
"L 98.851267 72.275933 \n",
"L 99.573376 71.477944 \n",
"L 100.297722 70.654947 \n",
"L 101.024354 69.806797 \n",
"L 101.753318 68.933356 \n",
"L 102.484663 68.034473 \n",
"L 103.218442 67.109996 \n",
"L 103.954699 66.159774 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 39.946779 40.71263 \n",
"L 40.626243 42.003175 \n",
"L 41.304549 43.270248 \n",
"L 41.98173 44.513936 \n",
"L 42.657823 45.734331 \n",
"L 43.332867 46.931524 \n",
"L 44.006893 48.105593 \n",
"L 44.679939 49.256625 \n",
"L 45.352041 50.384697 \n",
"L 46.023235 51.489891 \n",
"L 46.693557 52.57228 \n",
"L 47.363038 53.631935 \n",
"L 48.031717 54.668927 \n",
"L 48.699629 55.683325 \n",
"L 49.366806 56.675193 \n",
"L 50.033285 57.644592 \n",
"L 50.699101 58.591588 \n",
"L 51.364287 59.516234 \n",
"L 52.028879 60.418589 \n",
"L 52.692909 61.298706 \n",
"L 53.356413 62.156635 \n",
"L 54.019425 62.992425 \n",
"L 54.681981 63.806128 \n",
"L 55.344112 64.597781 \n",
"L 56.005856 65.367432 \n",
"L 56.667244 66.115117 \n",
"L 57.328309 66.840875 \n",
"L 57.989088 67.544742 \n",
"L 58.649613 68.226749 \n",
"L 59.309919 68.88693 \n",
"L 59.970041 69.525313 \n",
"L 60.63001 70.141921 \n",
"L 61.289861 70.736783 \n",
"L 61.94963 71.309918 \n",
"L 62.609349 71.861346 \n",
"L 63.269053 72.391087 \n",
"L 63.928774 72.899152 \n",
"L 64.588549 73.385558 \n",
"L 65.248411 73.850315 \n",
"L 65.908392 74.293429 \n",
"L 66.568529 74.71491 \n",
"L 67.228854 75.114759 \n",
"L 67.889403 75.492981 \n",
"L 68.55021 75.849573 \n",
"L 69.211308 76.184534 \n",
"L 69.872733 76.497859 \n",
"L 70.534519 76.789541 \n",
"L 71.1967 77.05957 \n",
"L 71.859312 77.307936 \n",
"L 72.522389 77.534624 \n",
"L 73.185966 77.739618 \n",
"L 73.850078 77.9229 \n",
"L 74.51476 78.08445 \n",
"L 75.180048 78.224244 \n",
"L 75.845977 78.342257 \n",
"L 76.512583 78.438462 \n",
"L 77.179902 78.51283 \n",
"L 77.847968 78.565326 \n",
"L 78.516819 78.595919 \n",
"L 79.18649 78.604569 \n",
"L 79.857017 78.591239 \n",
"L 80.528437 78.555887 \n",
"L 81.200788 78.498468 \n",
"L 81.874105 78.418936 \n",
"L 82.548427 78.317242 \n",
"L 83.22379 78.193335 \n",
"L 83.900231 78.047161 \n",
"L 84.57779 77.878664 \n",
"L 85.256502 77.687785 \n",
"L 85.936407 77.474463 \n",
"L 86.617544 77.238633 \n",
"L 87.29995 76.98023 \n",
"L 87.983665 76.699185 \n",
"L 88.668729 76.395425 \n",
"L 89.355179 76.068877 \n",
"L 90.043058 75.719463 \n",
"L 90.732403 75.347105 \n",
"L 91.423256 74.95172 \n",
"L 92.11566 74.533222 \n",
"L 92.809652 74.091525 \n",
"L 93.505276 73.626536 \n",
"L 94.202571 73.138163 \n",
"L 94.901582 72.626309 \n",
"L 95.602348 72.090878 \n",
"L 96.304914 71.531765 \n",
"L 97.009325 70.948864 \n",
"L 97.71562 70.342071 \n",
"L 98.423844 69.71127 \n",
"L 99.134044 69.056352 \n",
"L 99.846262 68.377198 \n",
"L 100.560546 67.673682 \n",
"L 101.276938 66.94569 \n",
"L 101.995486 66.193091 \n",
"L 102.716237 65.415753 \n",
"L 103.439236 64.613546 \n",
"L 104.16453 63.786333 \n",
"L 104.892171 62.933971 \n",
"L 105.622202 62.056321 \n",
"L 106.354675 61.153234 \n",
"L 107.08964 60.224559 \n",
"L 107.827144 59.270144 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 44.015684 36.547748 \n",
"L 44.688823 37.831014 \n",
"L 45.360887 39.090905 \n",
"L 46.031907 40.327506 \n",
"L 46.70192 41.540909 \n",
"L 47.370965 42.731204 \n",
"L 48.039072 43.898467 \n",
"L 48.706278 45.042785 \n",
"L 49.372619 46.164233 \n",
"L 50.038129 47.262893 \n",
"L 50.702845 48.338836 \n",
"L 51.366797 49.392133 \n",
"L 52.030023 50.422854 \n",
"L 52.692558 51.431067 \n",
"L 53.354433 52.416835 \n",
"L 54.015684 53.380218 \n",
"L 54.676347 54.321282 \n",
"L 55.336453 55.240079 \n",
"L 55.996038 56.136669 \n",
"L 56.655134 57.011099 \n",
"L 57.313775 57.863424 \n",
"L 57.971996 58.69369 \n",
"L 58.629833 59.501948 \n",
"L 59.287314 60.288235 \n",
"L 59.944478 61.052599 \n",
"L 60.601356 61.795075 \n",
"L 61.25798 62.5157 \n",
"L 61.914387 63.214512 \n",
"L 62.570607 63.89154 \n",
"L 63.226677 64.546818 \n",
"L 63.88263 65.180372 \n",
"L 64.538496 65.792229 \n",
"L 65.194312 66.382412 \n",
"L 65.850111 66.950944 \n",
"L 66.505926 67.497843 \n",
"L 67.161791 68.023127 \n",
"L 67.817738 68.52681 \n",
"L 68.473803 69.008907 \n",
"L 69.130019 69.469426 \n",
"L 69.786419 69.908377 \n",
"L 70.443037 70.325766 \n",
"L 71.099906 70.721596 \n",
"L 71.757062 71.09587 \n",
"L 72.414538 71.448588 \n",
"L 73.072367 71.779745 \n",
"L 73.730584 72.089338 \n",
"L 74.389224 72.37736 \n",
"L 75.04832 72.643801 \n",
"L 75.707907 72.88865 \n",
"L 76.368019 73.111893 \n",
"L 77.028691 73.313514 \n",
"L 77.689958 73.493494 \n",
"L 78.351854 73.651814 \n",
"L 79.014415 73.78845 \n",
"L 79.677676 73.903377 \n",
"L 80.341672 73.996568 \n",
"L 81.006439 74.067994 \n",
"L 81.672011 74.117621 \n",
"L 82.338425 74.145417 \n",
"L 83.005716 74.151344 \n",
"L 83.673921 74.135363 \n",
"L 84.343076 74.097434 \n",
"L 85.013217 74.037512 \n",
"L 85.684382 73.955551 \n",
"L 86.356606 73.851502 \n",
"L 87.029928 73.725316 \n",
"L 87.704382 73.576938 \n",
"L 88.38001 73.406312 \n",
"L 89.056846 73.213381 \n",
"L 89.73493 72.998083 \n",
"L 90.4143 72.760355 \n",
"L 91.094993 72.500132 \n",
"L 91.777049 72.217344 \n",
"L 92.460508 71.911921 \n",
"L 93.145407 71.58379 \n",
"L 93.831787 71.232873 \n",
"L 94.519687 70.859092 \n",
"L 95.209148 70.462367 \n",
"L 95.900212 70.04261 \n",
"L 96.592916 69.599738 \n",
"L 97.287305 69.133658 \n",
"L 97.983417 68.64428 \n",
"L 98.681297 68.131506 \n",
"L 99.380983 67.595241 \n",
"L 100.082521 67.035382 \n",
"L 100.785955 66.451825 \n",
"L 101.491323 65.844464 \n",
"L 102.198673 65.213187 \n",
"L 102.908047 64.557884 \n",
"L 103.619491 63.878438 \n",
"L 104.333052 63.174725 \n",
"L 105.04877 62.446631 \n",
"L 105.766693 61.694026 \n",
"L 106.48687 60.91678 \n",
"L 107.209344 60.114764 \n",
"L 107.934162 59.287844 \n",
"L 108.661377 58.435875 \n",
"L 109.39103 57.558723 \n",
"L 110.123174 56.656238 \n",
"L 110.857859 55.728271 \n",
"L 111.59513 54.774673 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 48.077838 34.669764 \n",
"L 48.74361 35.942783 \n",
"L 49.40839 37.192603 \n",
"L 50.07221 38.419308 \n",
"L 50.735105 39.622987 \n",
"L 51.397112 40.80373 \n",
"L 52.058263 41.961611 \n",
"L 52.718592 43.096716 \n",
"L 53.378136 44.20912 \n",
"L 54.036927 45.298901 \n",
"L 54.695002 46.36613 \n",
"L 55.352391 47.410878 \n",
"L 56.00913 48.433212 \n",
"L 56.665254 49.4332 \n",
"L 57.320792 50.410904 \n",
"L 57.975782 51.366382 \n",
"L 58.630257 52.299701 \n",
"L 59.284247 53.21091 \n",
"L 59.937791 54.100067 \n",
"L 60.590916 54.967223 \n",
"L 61.243659 55.812428 \n",
"L 61.896051 56.635727 \n",
"L 62.54813 57.437173 \n",
"L 63.199923 58.216801 \n",
"L 63.851467 58.974657 \n",
"L 64.502794 59.710777 \n",
"L 65.153935 60.425197 \n",
"L 65.804926 61.117954 \n",
"L 66.455797 61.789077 \n",
"L 67.106584 62.438598 \n",
"L 67.757319 63.066545 \n",
"L 68.408034 63.672942 \n",
"L 69.058762 64.257814 \n",
"L 69.709537 64.821181 \n",
"L 70.360392 65.363062 \n",
"L 71.01136 65.883475 \n",
"L 71.662473 66.382434 \n",
"L 72.313765 66.859952 \n",
"L 72.965271 67.316039 \n",
"L 73.617021 67.750703 \n",
"L 74.269049 68.16395 \n",
"L 74.92139 68.555784 \n",
"L 75.574076 68.926207 \n",
"L 76.227142 69.275219 \n",
"L 76.88062 69.602816 \n",
"L 77.534544 69.908994 \n",
"L 78.188948 70.193745 \n",
"L 78.843867 70.457062 \n",
"L 79.499332 70.698931 \n",
"L 80.15538 70.91934 \n",
"L 80.812044 71.118272 \n",
"L 81.469358 71.29571 \n",
"L 82.127357 71.451634 \n",
"L 82.786075 71.58602 \n",
"L 83.445548 71.698844 \n",
"L 84.10581 71.79008 \n",
"L 84.766896 71.859697 \n",
"L 85.428841 71.907664 \n",
"L 86.09168 71.933948 \n",
"L 86.755449 71.938512 \n",
"L 87.420183 71.921318 \n",
"L 88.085919 71.882324 \n",
"L 88.752692 71.821489 \n",
"L 89.42054 71.738766 \n",
"L 90.089498 71.634108 \n",
"L 90.759603 71.507464 \n",
"L 91.43089 71.358782 \n",
"L 92.1034 71.188007 \n",
"L 92.777167 70.995081 \n",
"L 93.452231 70.779944 \n",
"L 94.128628 70.542534 \n",
"L 94.806397 70.282786 \n",
"L 95.485575 70.000634 \n",
"L 96.166204 69.696005 \n",
"L 96.848319 69.368829 \n",
"L 97.531962 69.019029 \n",
"L 98.217171 68.646529 \n",
"L 98.903986 68.251248 \n",
"L 99.592449 67.833103 \n",
"L 100.282597 67.392009 \n",
"L 100.974474 66.927875 \n",
"L 101.668118 66.440612 \n",
"L 102.363574 65.930125 \n",
"L 103.06088 65.39632 \n",
"L 103.76008 64.839096 \n",
"L 104.461218 64.258348 \n",
"L 105.164334 63.653975 \n",
"L 105.869473 63.025864 \n",
"L 106.576678 62.373908 \n",
"L 107.285993 61.697993 \n",
"L 107.997466 60.997994 \n",
"L 108.711136 60.273802 \n",
"L 109.427052 59.525287 \n",
"L 110.145262 58.752321 \n",
"L 110.865807 57.954779 \n",
"L 111.588736 57.132526 \n",
"L 112.314099 56.285423 \n",
"L 113.04194 55.413335 \n",
"L 113.772308 54.516116 \n",
"L 114.505255 53.593619 \n",
"L 115.240826 52.645698 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 52.112931 35.029973 \n",
"L 52.770366 36.289923 \n",
"L 53.426892 37.526925 \n",
"L 54.082541 38.741061 \n",
"L 54.737348 39.932421 \n",
"L 55.391348 41.101089 \n",
"L 56.044573 42.24714 \n",
"L 56.697057 43.370658 \n",
"L 57.348833 44.471715 \n",
"L 57.999936 45.550389 \n",
"L 58.6504 46.60675 \n",
"L 59.300254 47.640865 \n",
"L 59.949534 48.652803 \n",
"L 60.598274 49.642628 \n",
"L 61.246504 50.610402 \n",
"L 61.894257 51.556182 \n",
"L 62.54157 52.480032 \n",
"L 63.18847 53.382001 \n",
"L 63.834994 54.262148 \n",
"L 64.481172 55.120519 \n",
"L 65.127036 55.957166 \n",
"L 65.77262 56.772132 \n",
"L 66.417958 57.565468 \n",
"L 67.063079 58.33721 \n",
"L 67.708019 59.087403 \n",
"L 68.352807 59.816081 \n",
"L 68.997476 60.523279 \n",
"L 69.64206 61.209035 \n",
"L 70.28659 61.873376 \n",
"L 70.931099 62.516335 \n",
"L 71.575619 63.137938 \n",
"L 72.220181 63.738209 \n",
"L 72.86482 64.317171 \n",
"L 73.509567 64.874846 \n",
"L 74.154454 65.411252 \n",
"L 74.799515 65.926407 \n",
"L 75.444781 66.420322 \n",
"L 76.090286 66.893012 \n",
"L 76.736062 67.344487 \n",
"L 77.38214 67.774755 \n",
"L 78.028555 68.18382 \n",
"L 78.675339 68.571688 \n",
"L 79.322524 68.938359 \n",
"L 79.970144 69.283835 \n",
"L 80.618232 69.608111 \n",
"L 81.26682 69.911183 \n",
"L 81.915943 70.193044 \n",
"L 82.565633 70.453685 \n",
"L 83.215924 70.693096 \n",
"L 83.866848 70.911261 \n",
"L 84.518441 71.108167 \n",
"L 85.170736 71.283795 \n",
"L 85.823765 71.438126 \n",
"L 86.477565 71.571137 \n",
"L 87.132168 71.682804 \n",
"L 87.78761 71.773101 \n",
"L 88.443925 71.841999 \n",
"L 89.101146 71.889467 \n",
"L 89.75931 71.915472 \n",
"L 90.41845 71.919978 \n",
"L 91.078602 71.902949 \n",
"L 91.739802 71.864343 \n",
"L 92.402085 71.804119 \n",
"L 93.065487 71.722232 \n",
"L 93.730044 71.618636 \n",
"L 94.395792 71.49328 \n",
"L 95.062767 71.346114 \n",
"L 95.731006 71.177084 \n",
"L 96.400545 70.986133 \n",
"L 97.071423 70.773203 \n",
"L 97.743676 70.538232 \n",
"L 98.41734 70.281157 \n",
"L 99.092456 70.001913 \n",
"L 99.769061 69.700429 \n",
"L 100.447192 69.376636 \n",
"L 101.126891 69.03046 \n",
"L 101.808193 68.661825 \n",
"L 102.491139 68.270652 \n",
"L 103.175771 67.856859 \n",
"L 103.862124 67.420364 \n",
"L 104.550244 66.961078 \n",
"L 105.240165 66.478914 \n",
"L 105.931935 65.973778 \n",
"L 106.625588 65.445577 \n",
"L 107.321171 64.894214 \n",
"L 108.018725 64.319587 \n",
"L 108.71829 63.721595 \n",
"L 109.41991 63.100129 \n",
"L 110.123629 62.455083 \n",
"L 110.82949 61.786346 \n",
"L 111.53754 61.093796 \n",
"L 112.247818 60.377325 \n",
"L 112.960371 59.636807 \n",
"L 113.675247 58.872118 \n",
"L 114.392487 58.083133 \n",
"L 115.112141 57.269722 \n",
"L 115.834255 56.431747 \n",
"L 116.558875 55.569078 \n",
"L 117.286048 54.681571 \n",
"L 118.015827 53.769083 \n",
"L 118.748254 52.83147 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 56.101674 37.559161 \n",
"L 56.749878 38.803396 \n",
"L 57.397257 40.025006 \n",
"L 58.043842 41.22407 \n",
"L 58.689667 42.400674 \n",
"L 59.334767 43.554903 \n",
"L 59.97917 44.686829 \n",
"L 60.622912 45.796533 \n",
"L 61.266024 46.884086 \n",
"L 61.90854 47.949565 \n",
"L 62.550493 48.993037 \n",
"L 63.191912 50.014568 \n",
"L 63.832832 51.014224 \n",
"L 64.473286 51.99207 \n",
"L 65.113302 52.948164 \n",
"L 65.752914 53.882564 \n",
"L 66.392157 54.795331 \n",
"L 67.031057 55.686513 \n",
"L 67.669652 56.556168 \n",
"L 68.307968 57.40434 \n",
"L 68.94604 58.23108 \n",
"L 69.583899 59.036432 \n",
"L 70.221579 59.820443 \n",
"L 70.859107 60.58315 \n",
"L 71.496519 61.324595 \n",
"L 72.133844 62.044813 \n",
"L 72.771113 62.743839 \n",
"L 73.40836 63.421707 \n",
"L 74.045614 64.078447 \n",
"L 74.68291 64.714089 \n",
"L 75.320277 65.328658 \n",
"L 75.957747 65.922179 \n",
"L 76.595351 66.494674 \n",
"L 77.233124 67.046164 \n",
"L 77.871093 67.576667 \n",
"L 78.509294 68.0862 \n",
"L 79.147757 68.574775 \n",
"L 79.786514 69.042407 \n",
"L 80.425596 69.489104 \n",
"L 81.065037 69.914873 \n",
"L 81.704866 70.319722 \n",
"L 82.345118 70.703654 \n",
"L 82.985824 71.066671 \n",
"L 83.627017 71.408772 \n",
"L 84.268729 71.729954 \n",
"L 84.910991 72.030213 \n",
"L 85.553838 72.309543 \n",
"L 86.197301 72.567934 \n",
"L 86.841413 72.805376 \n",
"L 87.486207 73.021855 \n",
"L 88.131716 73.217358 \n",
"L 88.777974 73.391866 \n",
"L 89.425013 73.545361 \n",
"L 90.072866 73.67782 \n",
"L 90.721569 73.789221 \n",
"L 91.371153 73.879537 \n",
"L 92.021655 73.94874 \n",
"L 92.673105 73.996801 \n",
"L 93.32554 74.023687 \n",
"L 93.978993 74.029364 \n",
"L 94.633499 74.013795 \n",
"L 95.289093 73.976941 \n",
"L 95.94581 73.918762 \n",
"L 96.603684 73.839212 \n",
"L 97.262753 73.738249 \n",
"L 97.923049 73.615822 \n",
"L 98.584609 73.471883 \n",
"L 99.247471 73.306377 \n",
"L 99.911668 73.119252 \n",
"L 100.57724 72.91045 \n",
"L 101.244221 72.67991 \n",
"L 101.912648 72.427572 \n",
"L 102.582559 72.153371 \n",
"L 103.253992 71.857239 \n",
"L 103.926983 71.539109 \n",
"L 104.601573 71.198909 \n",
"L 105.277798 70.836564 \n",
"L 105.955697 70.451998 \n",
"L 106.635311 70.045131 \n",
"L 107.316675 69.615883 \n",
"L 107.999834 69.164167 \n",
"L 108.684822 68.689899 \n",
"L 109.371685 68.192988 \n",
"L 110.060459 67.673343 \n",
"L 110.751186 67.130868 \n",
"L 111.44391 66.565465 \n",
"L 112.13867 65.977037 \n",
"L 112.835508 65.365476 \n",
"L 113.534467 64.73068 \n",
"L 114.235591 64.07254 \n",
"L 114.938925 63.390938 \n",
"L 115.644507 62.685769 \n",
"L 116.352385 61.95691 \n",
"L 117.062605 61.20424 \n",
"L 117.775208 60.427638 \n",
"L 118.490242 59.626978 \n",
"L 119.207754 58.802125 \n",
"L 119.927787 57.952952 \n",
"L 120.65039 57.07932 \n",
"L 121.375613 56.18109 \n",
"L 122.103498 55.258121 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 60.026059 42.169755 \n",
"L 60.664223 43.395833 \n",
"L 61.301646 44.599672 \n",
"L 61.938355 45.78135 \n",
"L 62.574385 46.94095 \n",
"L 63.209769 48.078555 \n",
"L 63.844535 49.194235 \n",
"L 64.478717 50.288068 \n",
"L 65.112346 51.360123 \n",
"L 65.745455 52.410476 \n",
"L 66.378075 53.439192 \n",
"L 67.010235 54.446334 \n",
"L 67.641968 55.431967 \n",
"L 68.273308 56.396154 \n",
"L 68.90428 57.338951 \n",
"L 69.534919 58.260415 \n",
"L 70.165258 59.160606 \n",
"L 70.795322 60.039569 \n",
"L 71.425148 60.897362 \n",
"L 72.054763 61.734027 \n",
"L 72.684198 62.549613 \n",
"L 73.313486 63.344164 \n",
"L 73.942658 64.117726 \n",
"L 74.571742 64.870334 \n",
"L 75.200772 65.602029 \n",
"L 75.829776 66.312846 \n",
"L 76.458785 67.002817 \n",
"L 77.087833 67.671978 \n",
"L 77.716946 68.320356 \n",
"L 78.346159 68.947981 \n",
"L 78.975501 69.554879 \n",
"L 79.605002 70.14107 \n",
"L 80.234694 70.70658 \n",
"L 80.864609 71.251428 \n",
"L 81.494776 71.775631 \n",
"L 82.125227 72.279206 \n",
"L 82.755993 72.762164 \n",
"L 83.387105 73.224521 \n",
"L 84.018594 73.666284 \n",
"L 84.650491 74.087462 \n",
"L 85.282827 74.488059 \n",
"L 85.915635 74.868081 \n",
"L 86.548945 75.227528 \n",
"L 87.182789 75.566402 \n",
"L 87.817198 75.884697 \n",
"L 88.452205 76.182412 \n",
"L 89.087841 76.459539 \n",
"L 89.724138 76.716071 \n",
"L 90.361127 76.951995 \n",
"L 90.998842 77.167302 \n",
"L 91.637314 77.361975 \n",
"L 92.276576 77.535997 \n",
"L 92.916661 77.689351 \n",
"L 93.5576 77.822017 \n",
"L 94.199428 77.933971 \n",
"L 94.842176 78.025187 \n",
"L 95.485879 78.09564 \n",
"L 96.130568 78.1453 \n",
"L 96.776279 78.174135 \n",
"L 97.423043 78.182113 \n",
"L 98.070895 78.169198 \n",
"L 98.71987 78.135354 \n",
"L 99.37 78.080538 \n",
"L 100.021322 78.004711 \n",
"L 100.673869 77.907827 \n",
"L 101.327677 77.78984 \n",
"L 101.982778 77.650702 \n",
"L 102.639211 77.490362 \n",
"L 103.297008 77.308768 \n",
"L 103.956208 77.105863 \n",
"L 104.616846 76.881589 \n",
"L 105.278956 76.635888 \n",
"L 105.942577 76.368696 \n",
"L 106.607745 76.07995 \n",
"L 107.274496 75.769583 \n",
"L 107.94287 75.437524 \n",
"L 108.612901 75.083703 \n",
"L 109.284629 74.708045 \n",
"L 109.958094 74.310473 \n",
"L 110.63333 73.89091 \n",
"L 111.31038 73.449271 \n",
"L 111.98928 72.985475 \n",
"L 112.670072 72.499433 \n",
"L 113.352793 71.991058 \n",
"L 114.037484 71.460258 \n",
"L 114.724189 70.906936 \n",
"L 115.412943 70.330999 \n",
"L 116.103791 69.732343 \n",
"L 116.796774 69.110869 \n",
"L 117.491934 68.466471 \n",
"L 118.189315 67.799036 \n",
"L 118.888957 67.10846 \n",
"L 119.590904 66.394627 \n",
"L 120.295202 65.657418 \n",
"L 121.001892 64.896716 \n",
"L 121.71102 64.112399 \n",
"L 122.422633 63.304338 \n",
"L 123.136773 62.472409 \n",
"L 123.853487 61.616477 \n",
"L 124.572824 60.736407 \n",
"L 125.294828 59.832066 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" <path d=\"M 63.869581 48.758278 \n",
"L 64.496983 49.963977 \n",
"L 65.123724 51.147884 \n",
"L 65.749831 52.310072 \n",
"L 66.375337 53.450623 \n",
"L 67.000275 54.569616 \n",
"L 67.624671 55.66712 \n",
"L 68.248558 56.74321 \n",
"L 68.871968 57.797953 \n",
"L 69.494929 58.831423 \n",
"L 70.117475 59.843682 \n",
"L 70.739632 60.834792 \n",
"L 71.361432 61.804815 \n",
"L 71.982908 62.753813 \n",
"L 72.604086 63.68184 \n",
"L 73.224997 64.588951 \n",
"L 73.845674 65.475204 \n",
"L 74.466143 66.340643 \n",
"L 75.086438 67.185323 \n",
"L 75.706585 68.009288 \n",
"L 76.326616 68.812583 \n",
"L 76.946561 69.595251 \n",
"L 77.566452 70.357336 \n",
"L 78.186314 71.098873 \n",
"L 78.806181 71.819902 \n",
"L 79.426081 72.520455 \n",
"L 80.046044 73.200567 \n",
"L 80.6661 73.86027 \n",
"L 81.286279 74.499589 \n",
"L 81.906612 75.118556 \n",
"L 82.527128 75.717195 \n",
"L 83.147856 76.295527 \n",
"L 83.768827 76.853575 \n",
"L 84.390072 77.391359 \n",
"L 85.01162 77.908894 \n",
"L 85.633501 78.406198 \n",
"L 86.255746 78.883284 \n",
"L 86.878385 79.340163 \n",
"L 87.501449 79.776845 \n",
"L 88.124966 80.193338 \n",
"L 88.748969 80.589645 \n",
"L 89.373487 80.965774 \n",
"L 89.998551 81.321724 \n",
"L 90.624193 81.657495 \n",
"L 91.250442 81.973087 \n",
"L 91.877329 82.268495 \n",
"L 92.504887 82.543712 \n",
"L 93.133145 82.798731 \n",
"L 93.762134 83.033541 \n",
"L 94.391888 83.248132 \n",
"L 95.022435 83.442489 \n",
"L 95.653809 83.616596 \n",
"L 96.286041 83.770435 \n",
"L 96.919163 83.903988 \n",
"L 97.553207 84.017231 \n",
"L 98.188205 84.110141 \n",
"L 98.824191 84.182692 \n",
"L 99.461193 84.234855 \n",
"L 100.099248 84.266603 \n",
"L 100.738387 84.277902 \n",
"L 101.378643 84.268717 \n",
"L 102.020049 84.239014 \n",
"L 102.662639 84.188752 \n",
"L 103.306447 84.117894 \n",
"L 103.951506 84.026395 \n",
"L 104.59785 83.914211 \n",
"L 105.245513 83.781296 \n",
"L 105.89453 83.627602 \n",
"L 106.544935 83.453075 \n",
"L 107.196763 83.257663 \n",
"L 107.850051 83.041311 \n",
"L 108.50483 82.803962 \n",
"L 109.161139 82.545555 \n",
"L 109.819015 82.266028 \n",
"L 110.47849 81.965318 \n",
"L 111.139605 81.643355 \n",
"L 111.802393 81.300073 \n",
"L 112.466892 80.935399 \n",
"L 113.133142 80.54926 \n",
"L 113.801176 80.141581 \n",
"L 114.471037 79.71228 \n",
"L 115.142758 79.261279 \n",
"L 115.816381 78.788494 \n",
"L 116.491943 78.293838 \n",
"L 117.169484 77.777225 \n",
"L 117.849045 77.23856 \n",
"L 118.530663 76.677755 \n",
"L 119.21438 76.094709 \n",
"L 119.900237 75.489327 \n",
"L 120.588274 74.861506 \n",
"L 121.278535 74.211138 \n",
"L 121.971058 73.538124 \n",
"L 122.665888 72.842351 \n",
"L 123.363068 72.123704 \n",
"L 124.062638 71.382071 \n",
"L 124.764644 70.617335 \n",
"L 125.469132 69.829371 \n",
"L 126.176142 69.01806 \n",
"L 126.88572 68.183273 \n",
"L 127.597915 67.324879 \n",
"L 128.312769 66.44275 \n",
"\" clip-path=\"url(#pd30e6f8bb8)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"pd30e6f8bb8\">\n",
" <rect x=\"7.242206\" y=\"7.2\" width=\"135.9\" height=\"135.9\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 252x180 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"x, y = torch.meshgrid(\n",
" torch.linspace(-1.0, 1.0, 101), torch.linspace(-1.0, 1.0, 101))\n",
"z = x**2 - y**2\n",
"\n",
"ax = d2l.plt.figure().add_subplot(111, projection='3d')\n",
"ax.plot_wireframe(x, y, z, **{'rstride': 10, 'cstride': 10})\n",
"ax.plot([0], [0], [0], 'rx')\n",
"ticks = [-1, 0, 1]\n",
"d2l.plt.xticks(ticks)\n",
"d2l.plt.yticks(ticks)\n",
"ax.set_zticks(ticks)\n",
"d2l.plt.xlabel('x')\n",
"d2l.plt.ylabel('y');"
]
},
{
"cell_type": "markdown",
"id": "5878b89a",
"metadata": {
"origin_pos": 19
},
"source": [
"我们假设函数的输入是$k$维向量,其输出是标量,因此其Hessian矩阵(也称黑塞矩阵)将有$k$个特征值(参考[特征分解的在线附录](https://d2l.ai/chapter_appendix-mathematics-for-deep-learning/eigendecomposition.html))。函数的解可能是局部最小值、局部最大值或函数梯度为零位置处的鞍点:\n",
"\n",
"* 当函数在零梯度位置处的Hessian矩阵的特征值全部为正值时,我们有该函数的局部最小值;\n",
"* 当函数在零梯度位置处的Hessian矩阵的特征值全部为负值时,我们有该函数的局部最大值;\n",
"* 当函数在零梯度位置处的Hessian矩阵的特征值为负值和正值时,我们有该函数的一个鞍点。\n",
"\n",
"对于高维度问题,至少*部分*特征值为负的可能性相当高。这使得鞍点比局部最小值更有可能出现。我们将在下一节介绍凸性时讨论这种情况的一些例外。简而言之,凸函数是Hessian函数的特征值永远不为负值的函数。不幸的是,大多数深度学习问题并不属于这一类。尽管如此,它还是研究优化算法的一个很好的工具。\n",
"\n",
"### 梯度消失\n",
"\n",
"可能遇到的最隐蔽问题是梯度消失。回想一下我们在 :numref:`subsec_activation_functions`中常用的激活函数及其衍生函数。例如,假设我们想最小化函数$f(x) = \\tanh(x)$,然后我们恰好从$x = 4$开始。正如我们所看到的那样,$f$的梯度接近零。更具体地说,$f'(x) = 1 - \\tanh^2(x)$,因此是$f'(4) = 0.0013$。因此,在我们取得进展之前,优化将会停滞很长一段时间。事实证明,这是在引入ReLU激活函数之前训练深度学习模型相当棘手的原因之一。\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "5aa88d1a",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:07:09.362938Z",
"iopub.status.busy": "2023-08-18T07:07:09.362153Z",
"iopub.status.idle": "2023-08-18T07:07:09.566124Z",
"shell.execute_reply": "2023-08-18T07:07:09.565284Z"
},
"origin_pos": 20,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"263.314464pt\" height=\"180.65625pt\" viewBox=\"0 0 263.314464 180.65625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T07:07:09.526643</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 180.65625 \n",
"L 263.314464 180.65625 \n",
"L 263.314464 0 \n",
"L 0 0 \n",
"L 0 180.65625 \n",
"z\n",
"\" style=\"fill: none\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 52.160938 143.1 \n",
"L 247.460938 143.1 \n",
"L 247.460938 7.2 \n",
"L 52.160938 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 61.03821 143.1 \n",
"L 61.03821 7.2 \n",
"\" clip-path=\"url(#p171b45cbce)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_2\">\n",
" <defs>\n",
" <path id=\"mc46b4706a0\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#mc46b4706a0\" x=\"61.03821\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(53.667116 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
"L 4684 2272 \n",
"L 4684 1741 \n",
"L 678 1741 \n",
"L 678 2272 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 111.838056 143.1 \n",
"L 111.838056 7.2 \n",
"\" clip-path=\"url(#p171b45cbce)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use xlink:href=\"#mc46b4706a0\" x=\"111.838056\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(108.656806 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 162.637901 143.1 \n",
"L 162.637901 7.2 \n",
"\" clip-path=\"url(#p171b45cbce)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#mc46b4706a0\" x=\"162.637901\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(159.456651 157.698438)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 213.437747 143.1 \n",
"L 213.437747 7.2 \n",
"\" clip-path=\"url(#p171b45cbce)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use xlink:href=\"#mc46b4706a0\" x=\"213.437747\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 4 -->\n",
" <g transform=\"translate(210.256497 157.698438)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
"L 825 1625 \n",
"L 2419 1625 \n",
"L 2419 4116 \n",
"z\n",
"M 2253 4666 \n",
"L 3047 4666 \n",
"L 3047 1625 \n",
"L 3713 1625 \n",
"L 3713 1100 \n",
"L 3047 1100 \n",
"L 3047 0 \n",
"L 2419 0 \n",
"L 2419 1100 \n",
"L 313 1100 \n",
"L 313 1709 \n",
"L 2253 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- x -->\n",
" <g transform=\"translate(146.851563 171.376563)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 52.160938 139.185648 \n",
"L 247.460938 139.185648 \n",
"\" clip-path=\"url(#p171b45cbce)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_10\">\n",
" <defs>\n",
" <path id=\"m49bbb10a47\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m49bbb10a47\" x=\"52.160938\" y=\"139.185648\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- 1.0 -->\n",
" <g transform=\"translate(20.878125 142.984866)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
"L 1344 794 \n",
"L 1344 0 \n",
"L 684 0 \n",
"L 684 794 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"179.199219\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 52.160938 107.732097 \n",
"L 247.460938 107.732097 \n",
"\" clip-path=\"url(#p171b45cbce)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_12\">\n",
" <g>\n",
" <use xlink:href=\"#m49bbb10a47\" x=\"52.160938\" y=\"107.732097\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- 0.5 -->\n",
" <g transform=\"translate(20.878125 111.531316)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
"L 3169 4666 \n",
"L 3169 4134 \n",
"L 1269 4134 \n",
"L 1269 2991 \n",
"Q 1406 3038 1543 3061 \n",
"Q 1681 3084 1819 3084 \n",
"Q 2600 3084 3056 2656 \n",
"Q 3513 2228 3513 1497 \n",
"Q 3513 744 3044 326 \n",
"Q 2575 -91 1722 -91 \n",
"Q 1428 -91 1123 -41 \n",
"Q 819 9 494 109 \n",
"L 494 744 \n",
"Q 775 591 1075 516 \n",
"Q 1375 441 1709 441 \n",
"Q 2250 441 2565 725 \n",
"Q 2881 1009 2881 1497 \n",
"Q 2881 1984 2565 2268 \n",
"Q 2250 2553 1709 2553 \n",
"Q 1456 2553 1204 2497 \n",
"Q 953 2441 691 2322 \n",
"L 691 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"179.199219\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_13\">\n",
" <path d=\"M 52.160938 76.278547 \n",
"L 247.460938 76.278547 \n",
"\" clip-path=\"url(#p171b45cbce)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use xlink:href=\"#m49bbb10a47\" x=\"52.160938\" y=\"76.278547\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0.0 -->\n",
" <g transform=\"translate(29.257812 80.077766)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_15\">\n",
" <path d=\"M 52.160938 44.824996 \n",
"L 247.460938 44.824996 \n",
"\" clip-path=\"url(#p171b45cbce)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use xlink:href=\"#m49bbb10a47\" x=\"52.160938\" y=\"44.824996\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 0.5 -->\n",
" <g transform=\"translate(29.257812 48.624215)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_17\">\n",
" <path d=\"M 52.160938 13.371446 \n",
"L 247.460938 13.371446 \n",
"\" clip-path=\"url(#p171b45cbce)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_18\">\n",
" <g>\n",
" <use xlink:href=\"#m49bbb10a47\" x=\"52.160938\" y=\"13.371446\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 1.0 -->\n",
" <g transform=\"translate(29.257812 17.170665)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- f(x) -->\n",
" <g transform=\"translate(14.798438 83.771094)rotate(-90)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
"L 2375 4384 \n",
"L 1825 4384 \n",
"Q 1516 4384 1395 4259 \n",
"Q 1275 4134 1275 3809 \n",
"L 1275 3500 \n",
"L 2222 3500 \n",
"L 2222 3053 \n",
"L 1275 3053 \n",
"L 1275 0 \n",
"L 697 0 \n",
"L 697 3053 \n",
"L 147 3053 \n",
"L 147 3500 \n",
"L 697 3500 \n",
"L 697 3744 \n",
"Q 697 4328 969 4595 \n",
"Q 1241 4863 1831 4863 \n",
"L 2375 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
"Q 1566 4138 1362 3434 \n",
"Q 1159 2731 1159 2009 \n",
"Q 1159 1288 1364 580 \n",
"Q 1569 -128 1984 -844 \n",
"L 1484 -844 \n",
"Q 1016 -109 783 600 \n",
"Q 550 1309 550 2009 \n",
"Q 550 2706 781 3412 \n",
"Q 1013 4119 1484 4856 \n",
"L 1984 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
"L 1013 4856 \n",
"Q 1481 4119 1714 3412 \n",
"Q 1947 2706 1947 2009 \n",
"Q 1947 1309 1714 600 \n",
"Q 1481 -109 1013 -844 \n",
"L 513 -844 \n",
"Q 928 -128 1133 580 \n",
"Q 1338 1288 1338 2009 \n",
"Q 1338 2731 1133 3434 \n",
"Q 928 4138 513 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-66\"/>\n",
" <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
" <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
" <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_19\">\n",
" <path d=\"M 61.03821 136.922727 \n",
"L 64.848201 136.150124 \n",
"L 68.404187 135.200328 \n",
"L 71.452178 134.162473 \n",
"L 74.24617 132.987284 \n",
"L 76.786159 131.696638 \n",
"L 79.326155 130.157528 \n",
"L 81.612146 128.527855 \n",
"L 83.89814 126.63562 \n",
"L 86.184134 124.450416 \n",
"L 88.470128 121.942646 \n",
"L 90.75612 119.08532 \n",
"L 93.042113 115.856244 \n",
"L 95.328105 112.240649 \n",
"L 97.868098 107.764823 \n",
"L 100.408091 102.818989 \n",
"L 103.202082 96.8792 \n",
"L 106.504072 89.298209 \n",
"L 111.076058 78.165194 \n",
"L 117.934037 61.464192 \n",
"L 121.236027 54.009954 \n",
"L 124.030018 48.206655 \n",
"L 126.570011 43.399179 \n",
"L 129.110004 39.067775 \n",
"L 131.395997 35.582087 \n",
"L 133.68199 32.478997 \n",
"L 135.967982 29.74094 \n",
"L 138.253975 27.343864 \n",
"L 140.539968 25.259678 \n",
"L 142.825962 23.458335 \n",
"L 145.365952 21.751683 \n",
"L 147.905945 20.316459 \n",
"L 150.699937 19.006375 \n",
"L 153.493929 17.933935 \n",
"L 156.54192 16.988413 \n",
"L 160.097909 16.124422 \n",
"L 164.161895 15.382569 \n",
"L 168.733883 14.781372 \n",
"L 174.321867 14.283124 \n",
"L 181.179846 13.904336 \n",
"L 190.577814 13.626258 \n",
"L 204.801769 13.454701 \n",
"L 233.24968 13.380314 \n",
"L 238.583665 13.377273 \n",
"L 238.583665 13.377273 \n",
"\" clip-path=\"url(#p171b45cbce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 52.160938 143.1 \n",
"L 52.160938 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 247.460938 143.1 \n",
"L 247.460938 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 52.160937 143.1 \n",
"L 247.460938 143.1 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 52.160937 7.2 \n",
"L 247.460938 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_7\">\n",
" <path d=\"M 209.972699 64.685354 \n",
"Q 211.637864 40.025927 213.227703 16.481993 \n",
"\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
" <path d=\"M 210.962755 20.338159 \n",
"L 213.227703 16.481993 \n",
"L 214.953666 20.607651 \n",
"\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
" </g>\n",
" <g id=\"text_12\">\n",
" <!-- vanishing gradient -->\n",
" <g transform=\"translate(162.637901 76.278547)scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-76\" d=\"M 191 3500 \n",
"L 800 3500 \n",
"L 1894 563 \n",
"L 2988 3500 \n",
"L 3597 3500 \n",
"L 2284 0 \n",
"L 1503 0 \n",
"L 191 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
"Q 1497 1759 1228 1600 \n",
"Q 959 1441 959 1056 \n",
"Q 959 750 1161 570 \n",
"Q 1363 391 1709 391 \n",
"Q 2188 391 2477 730 \n",
"Q 2766 1069 2766 1631 \n",
"L 2766 1759 \n",
"L 2194 1759 \n",
"z\n",
"M 3341 1997 \n",
"L 3341 0 \n",
"L 2766 0 \n",
"L 2766 531 \n",
"Q 2569 213 2275 61 \n",
"Q 1981 -91 1556 -91 \n",
"Q 1019 -91 701 211 \n",
"Q 384 513 384 1019 \n",
"Q 384 1609 779 1909 \n",
"Q 1175 2209 1959 2209 \n",
"L 2766 2209 \n",
"L 2766 2266 \n",
"Q 2766 2663 2505 2880 \n",
"Q 2244 3097 1772 3097 \n",
"Q 1472 3097 1187 3025 \n",
"Q 903 2953 641 2809 \n",
"L 641 3341 \n",
"Q 956 3463 1253 3523 \n",
"Q 1550 3584 1831 3584 \n",
"Q 2591 3584 2966 3190 \n",
"Q 3341 2797 3341 1997 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
"L 1178 3500 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 3500 \n",
"z\n",
"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 4134 \n",
"L 603 4134 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
"L 2834 2853 \n",
"Q 2591 2978 2328 3040 \n",
"Q 2066 3103 1784 3103 \n",
"Q 1356 3103 1142 2972 \n",
"Q 928 2841 928 2578 \n",
"Q 928 2378 1081 2264 \n",
"Q 1234 2150 1697 2047 \n",
"L 1894 2003 \n",
"Q 2506 1872 2764 1633 \n",
"Q 3022 1394 3022 966 \n",
"Q 3022 478 2636 193 \n",
"Q 2250 -91 1575 -91 \n",
"Q 1294 -91 989 -36 \n",
"Q 684 19 347 128 \n",
"L 347 722 \n",
"Q 666 556 975 473 \n",
"Q 1284 391 1588 391 \n",
"Q 1994 391 2212 530 \n",
"Q 2431 669 2431 922 \n",
"Q 2431 1156 2273 1281 \n",
"Q 2116 1406 1581 1522 \n",
"L 1381 1569 \n",
"Q 847 1681 609 1914 \n",
"Q 372 2147 372 2553 \n",
"Q 372 3047 722 3315 \n",
"Q 1072 3584 1716 3584 \n",
"Q 2034 3584 2315 3537 \n",
"Q 2597 3491 2834 3397 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 4863 \n",
"L 1159 4863 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-67\" d=\"M 2906 1791 \n",
"Q 2906 2416 2648 2759 \n",
"Q 2391 3103 1925 3103 \n",
"Q 1463 3103 1205 2759 \n",
"Q 947 2416 947 1791 \n",
"Q 947 1169 1205 825 \n",
"Q 1463 481 1925 481 \n",
"Q 2391 481 2648 825 \n",
"Q 2906 1169 2906 1791 \n",
"z\n",
"M 3481 434 \n",
"Q 3481 -459 3084 -895 \n",
"Q 2688 -1331 1869 -1331 \n",
"Q 1566 -1331 1297 -1286 \n",
"Q 1028 -1241 775 -1147 \n",
"L 775 -588 \n",
"Q 1028 -725 1275 -790 \n",
"Q 1522 -856 1778 -856 \n",
"Q 2344 -856 2625 -561 \n",
"Q 2906 -266 2906 331 \n",
"L 2906 616 \n",
"Q 2728 306 2450 153 \n",
"Q 2172 0 1784 0 \n",
"Q 1141 0 747 490 \n",
"Q 353 981 353 1791 \n",
"Q 353 2603 747 3093 \n",
"Q 1141 3584 1784 3584 \n",
"Q 2172 3584 2450 3431 \n",
"Q 2728 3278 2906 2969 \n",
"L 2906 3500 \n",
"L 3481 3500 \n",
"L 3481 434 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
"Q 2534 3019 2420 3045 \n",
"Q 2306 3072 2169 3072 \n",
"Q 1681 3072 1420 2755 \n",
"Q 1159 2438 1159 1844 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1341 3275 1631 3429 \n",
"Q 1922 3584 2338 3584 \n",
"Q 2397 3584 2469 3576 \n",
"Q 2541 3569 2628 3553 \n",
"L 2631 2963 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-64\" d=\"M 2906 2969 \n",
"L 2906 4863 \n",
"L 3481 4863 \n",
"L 3481 0 \n",
"L 2906 0 \n",
"L 2906 525 \n",
"Q 2725 213 2448 61 \n",
"Q 2172 -91 1784 -91 \n",
"Q 1150 -91 751 415 \n",
"Q 353 922 353 1747 \n",
"Q 353 2572 751 3078 \n",
"Q 1150 3584 1784 3584 \n",
"Q 2172 3584 2448 3432 \n",
"Q 2725 3281 2906 2969 \n",
"z\n",
"M 947 1747 \n",
"Q 947 1113 1208 752 \n",
"Q 1469 391 1925 391 \n",
"Q 2381 391 2643 752 \n",
"Q 2906 1113 2906 1747 \n",
"Q 2906 2381 2643 2742 \n",
"Q 2381 3103 1925 3103 \n",
"Q 1469 3103 1208 2742 \n",
"Q 947 2381 947 1747 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
"L 3597 1613 \n",
"L 953 1613 \n",
"Q 991 1019 1311 708 \n",
"Q 1631 397 2203 397 \n",
"Q 2534 397 2845 478 \n",
"Q 3156 559 3463 722 \n",
"L 3463 178 \n",
"Q 3153 47 2828 -22 \n",
"Q 2503 -91 2169 -91 \n",
"Q 1331 -91 842 396 \n",
"Q 353 884 353 1716 \n",
"Q 353 2575 817 3079 \n",
"Q 1281 3584 2069 3584 \n",
"Q 2775 3584 3186 3129 \n",
"Q 3597 2675 3597 1894 \n",
"z\n",
"M 3022 2063 \n",
"Q 3016 2534 2758 2815 \n",
"Q 2500 3097 2075 3097 \n",
"Q 1594 3097 1305 2825 \n",
"Q 1016 2553 972 2059 \n",
"L 3022 2063 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
"L 1172 3500 \n",
"L 2356 3500 \n",
"L 2356 3053 \n",
"L 1172 3053 \n",
"L 1172 1153 \n",
"Q 1172 725 1289 603 \n",
"Q 1406 481 1766 481 \n",
"L 2356 481 \n",
"L 2356 0 \n",
"L 1766 0 \n",
"Q 1100 0 847 248 \n",
"Q 594 497 594 1153 \n",
"L 594 3053 \n",
"L 172 3053 \n",
"L 172 3500 \n",
"L 594 3500 \n",
"L 594 4494 \n",
"L 1172 4494 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-76\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"59.179688\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"120.458984\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"183.837891\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"211.621094\"/>\n",
" <use xlink:href=\"#DejaVuSans-68\" x=\"263.720703\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"327.099609\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"354.882812\"/>\n",
" <use xlink:href=\"#DejaVuSans-67\" x=\"418.261719\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"481.738281\"/>\n",
" <use xlink:href=\"#DejaVuSans-67\" x=\"513.525391\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"577.001953\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"618.115234\"/>\n",
" <use xlink:href=\"#DejaVuSans-64\" x=\"679.394531\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"742.871094\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"770.654297\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"832.177734\"/>\n",
" <use xlink:href=\"#DejaVuSans-74\" x=\"895.556641\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p171b45cbce\">\n",
" <rect x=\"52.160938\" y=\"7.2\" width=\"195.3\" height=\"135.9\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 252x180 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"x = torch.arange(-2.0, 5.0, 0.01)\n",
"d2l.plot(x, [torch.tanh(x)], 'x', 'f(x)')\n",
"annotate('vanishing gradient', (4, 1), (2, 0.0))"
]
},
{
"cell_type": "markdown",
"id": "7cc9c3a2",
"metadata": {
"origin_pos": 22
},
"source": [
"正如我们所看到的那样,深度学习的优化充满挑战。幸运的是,有一系列强大的算法表现良好,即使对于初学者也很容易使用。此外,没有必要找到最优解。局部最优解或其近似解仍然非常有用。\n",
"\n",
"## 小结\n",
"\n",
"* 最小化训练误差并*不能*保证我们找到最佳的参数集来最小化泛化误差。\n",
"* 优化问题可能有许多局部最小值。\n",
"* 一个问题可能有很多的鞍点,因为问题通常不是凸的。\n",
"* 梯度消失可能会导致优化停滞,重参数化通常会有所帮助。对参数进行良好的初始化也可能是有益的。\n",
"\n",
"## 练习\n",
"\n",
"1. 考虑一个简单的MLP,它有一个隐藏层,比如,隐藏层中维度为$d$和一个输出。证明对于任何局部最小值,至少有$d!$个等效方案。\n",
"1. 假设我们有一个对称随机矩阵$\\mathbf{M}$,其中条目$M_{ij} = M_{ji}$各自从某种概率分布$p_{ij}$中抽取。此外,假设$p_{ij}(x) = p_{ij}(-x)$,即分布是对称的(详情请参见 :cite:`Wigner.1958`)。\n",
" 1. 证明特征值的分布也是对称的。也就是说,对于任何特征向量$\\mathbf{v}$,关联的特征值$\\lambda$满足$P(\\lambda > 0) = P(\\lambda < 0)$的概率为$P(\\lambda > 0) = P(\\lambda < 0)$。\n",
" 1. 为什么以上*没有*暗示$P(\\lambda > 0) = 0.5$\n",
"1. 你能想到深度学习优化还涉及哪些其他挑战?\n",
"1. 假设你想在(真实的)鞍上平衡一个(真实的)球。\n",
" 1. 为什么这很难?\n",
" 1. 能利用这种效应来优化算法吗?\n"
]
},
{
"cell_type": "markdown",
"id": "ea89bc0e",
"metadata": {
"origin_pos": 24,
"tab": [
"pytorch"
]
},
"source": [
"[Discussions](https://discuss.d2l.ai/t/3841)\n"
]
}
],
"metadata": {
"language_info": {
"name": "python"
},
"required_libs": []
},
"nbformat": 4,
"nbformat_minor": 5
}