1566 lines
59 KiB
Plaintext
1566 lines
59 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c6b5f483",
|
|
"metadata": {
|
|
"origin_pos": 0
|
|
},
|
|
"source": [
|
|
"# Adadelta\n",
|
|
":label:`sec_adadelta`\n",
|
|
"\n",
|
|
"\n",
|
|
"Adadelta是AdaGrad的另一种变体（ :numref:`sec_adagrad`），\n",
|
|
"主要区别在于前者减少了学习率适应坐标的数量。\n",
|
|
"此外，广义上Adadelta被称为没有学习率，因为它使用变化量本身作为未来变化的校准。\n",
|
|
"Adadelta算法是在 :cite:`Zeiler.2012`中提出的。\n",
|
|
"\n",
|
|
"## Adadelta算法\n",
|
|
"\n",
|
|
"简而言之，Adadelta使用两个状态变量，$\\mathbf{s}_t$用于存储梯度二阶导数的泄露平均值，$\\Delta\\mathbf{x}_t$用于存储模型本身中参数变化二阶导数的泄露平均值。请注意，为了与其他出版物和实现的兼容性，我们使用作者的原始符号和命名（没有其它真正理由让大家使用不同的希腊变量来表示在动量法、AdaGrad、RMSProp和Adadelta中用于相同用途的参数）。\n",
|
|
"\n",
|
|
"以下是Adadelta的技术细节。鉴于参数du jour是$\\rho$，我们获得了与 :numref:`sec_rmsprop`类似的以下泄漏更新：\n",
|
|
"\n",
|
|
"$$\\begin{aligned}\n",
|
|
" \\mathbf{s}_t & = \\rho \\mathbf{s}_{t-1} + (1 - \\rho) \\mathbf{g}_t^2.\n",
|
|
"\\end{aligned}$$\n",
|
|
"\n",
|
|
"与 :numref:`sec_rmsprop`的区别在于，我们使用重新缩放的梯度$\\mathbf{g}_t'$执行更新，即\n",
|
|
"\n",
|
|
"$$\\begin{aligned}\n",
|
|
" \\mathbf{x}_t & = \\mathbf{x}_{t-1} - \\mathbf{g}_t'. \\\\\n",
|
|
"\\end{aligned}$$\n",
|
|
"\n",
|
|
"那么，调整后的梯度$\\mathbf{g}_t'$是什么？我们可以按如下方式计算它：\n",
|
|
"\n",
|
|
"$$\\begin{aligned}\n",
|
|
" \\mathbf{g}_t' & = \\frac{\\sqrt{\\Delta\\mathbf{x}_{t-1} + \\epsilon}}{\\sqrt{{\\mathbf{s}_t + \\epsilon}}} \\odot \\mathbf{g}_t, \\\\\n",
|
|
"\\end{aligned}$$\n",
|
|
"\n",
|
|
"其中$\\Delta \\mathbf{x}_{t-1}$是重新缩放梯度的平方$\\mathbf{g}_t'$的泄漏平均值。我们将$\\Delta \\mathbf{x}_{0}$初始化为$0$，然后在每个步骤中使用$\\mathbf{g}_t'$更新它，即\n",
|
|
"\n",
|
|
"$$\\begin{aligned}\n",
|
|
" \\Delta \\mathbf{x}_t & = \\rho \\Delta\\mathbf{x}_{t-1} + (1 - \\rho) {\\mathbf{g}_t'}^2,\n",
|
|
"\\end{aligned}$$\n",
|
|
"\n",
|
|
"和$\\epsilon$（例如$10^{-5}$这样的小值）是为了保持数字稳定性而加入的。\n",
|
|
"\n",
|
|
"## 代码实现\n",
|
|
"\n",
|
|
"Adadelta需要为每个变量维护两个状态变量，即$\\mathbf{s}_t$和$\\Delta\\mathbf{x}_t$。这将产生以下实现。\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "b249f128",
|
|
"metadata": {
|
|
"execution": {
|
|
"iopub.execute_input": "2023-08-18T07:05:09.332337Z",
|
|
"iopub.status.busy": "2023-08-18T07:05:09.331524Z",
|
|
"iopub.status.idle": "2023-08-18T07:05:11.424308Z",
|
|
"shell.execute_reply": "2023-08-18T07:05:11.423239Z"
|
|
},
|
|
"origin_pos": 2,
|
|
"tab": [
|
|
"pytorch"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%matplotlib inline\n",
|
|
"import torch\n",
|
|
"from d2l import torch as d2l\n",
|
|
"\n",
|
|
"\n",
|
|
"def init_adadelta_states(feature_dim):\n",
|
|
" s_w, s_b = torch.zeros((feature_dim, 1)), torch.zeros(1)\n",
|
|
" delta_w, delta_b = torch.zeros((feature_dim, 1)), torch.zeros(1)\n",
|
|
" return ((s_w, delta_w), (s_b, delta_b))\n",
|
|
"\n",
|
|
"def adadelta(params, states, hyperparams):\n",
|
|
" rho, eps = hyperparams['rho'], 1e-5\n",
|
|
" for p, (s, delta) in zip(params, states):\n",
|
|
" with torch.no_grad():\n",
|
|
" # In-placeupdatesvia[:]\n",
|
|
" s[:] = rho * s + (1 - rho) * torch.square(p.grad)\n",
|
|
" g = (torch.sqrt(delta + eps) / torch.sqrt(s + eps)) * p.grad\n",
|
|
" p[:] -= g\n",
|
|
" delta[:] = rho * delta + (1 - rho) * g * g\n",
|
|
" p.grad.data.zero_()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4411aabf",
|
|
"metadata": {
|
|
"origin_pos": 5
|
|
},
|
|
"source": [
|
|
"对于每次参数更新，选择$\\rho = 0.9$相当于10个半衰期。由此我们得到：\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"id": "4f8025df",
|
|
"metadata": {
|
|
"execution": {
|
|
"iopub.execute_input": "2023-08-18T07:05:11.429258Z",
|
|
"iopub.status.busy": "2023-08-18T07:05:11.428414Z",
|
|
"iopub.status.idle": "2023-08-18T07:05:14.081998Z",
|
|
"shell.execute_reply": "2023-08-18T07:05:14.081152Z"
|
|
},
|
|
"origin_pos": 6,
|
|
"tab": [
|
|
"pytorch"
|
|
]
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"loss: 0.243, 0.014 sec/epoch\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/svg+xml": [
|
|
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
|
|
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
|
|
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
|
|
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"266.957813pt\" height=\"184.455469pt\" viewBox=\"0 0 266.957813 184.455469\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
|
|
" <metadata>\n",
|
|
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
|
|
" <cc:Work>\n",
|
|
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
|
|
" <dc:date>2023-08-18T07:05:14.047429</dc:date>\n",
|
|
" <dc:format>image/svg+xml</dc:format>\n",
|
|
" <dc:creator>\n",
|
|
" <cc:Agent>\n",
|
|
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
|
|
" </cc:Agent>\n",
|
|
" </dc:creator>\n",
|
|
" </cc:Work>\n",
|
|
" </rdf:RDF>\n",
|
|
" </metadata>\n",
|
|
" <defs>\n",
|
|
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
|
|
" </defs>\n",
|
|
" <g id=\"figure_1\">\n",
|
|
" <g id=\"patch_1\">\n",
|
|
" <path d=\"M -0 184.455469 \n",
|
|
"L 266.957813 184.455469 \n",
|
|
"L 266.957813 0 \n",
|
|
"L -0 0 \n",
|
|
"L -0 184.455469 \n",
|
|
"z\n",
|
|
"\" style=\"fill: none\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"axes_1\">\n",
|
|
" <g id=\"patch_2\">\n",
|
|
" <path d=\"M 56.50625 146.899219 \n",
|
|
"L 251.80625 146.899219 \n",
|
|
"L 251.80625 10.999219 \n",
|
|
"L 56.50625 10.999219 \n",
|
|
"z\n",
|
|
"\" style=\"fill: #ffffff\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"matplotlib.axis_1\">\n",
|
|
" <g id=\"xtick_1\">\n",
|
|
" <g id=\"line2d_1\">\n",
|
|
" <path d=\"M 56.50625 146.899219 \n",
|
|
"L 56.50625 10.999219 \n",
|
|
"\" clip-path=\"url(#p3fc86941e4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_2\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"mc11186c2e7\" d=\"M 0 0 \n",
|
|
"L 0 3.5 \n",
|
|
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </defs>\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#mc11186c2e7\" x=\"56.50625\" y=\"146.899219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_1\">\n",
|
|
" <!-- 0.0 -->\n",
|
|
" <g transform=\"translate(48.554688 161.497656)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
|
|
"Q 1547 4250 1301 3770 \n",
|
|
"Q 1056 3291 1056 2328 \n",
|
|
"Q 1056 1369 1301 889 \n",
|
|
"Q 1547 409 2034 409 \n",
|
|
"Q 2525 409 2770 889 \n",
|
|
"Q 3016 1369 3016 2328 \n",
|
|
"Q 3016 3291 2770 3770 \n",
|
|
"Q 2525 4250 2034 4250 \n",
|
|
"z\n",
|
|
"M 2034 4750 \n",
|
|
"Q 2819 4750 3233 4129 \n",
|
|
"Q 3647 3509 3647 2328 \n",
|
|
"Q 3647 1150 3233 529 \n",
|
|
"Q 2819 -91 2034 -91 \n",
|
|
"Q 1250 -91 836 529 \n",
|
|
"Q 422 1150 422 2328 \n",
|
|
"Q 422 3509 836 4129 \n",
|
|
"Q 1250 4750 2034 4750 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
|
|
"L 1344 794 \n",
|
|
"L 1344 0 \n",
|
|
"L 684 0 \n",
|
|
"L 684 794 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"xtick_2\">\n",
|
|
" <g id=\"line2d_3\">\n",
|
|
" <path d=\"M 105.33125 146.899219 \n",
|
|
"L 105.33125 10.999219 \n",
|
|
"\" clip-path=\"url(#p3fc86941e4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_4\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#mc11186c2e7\" x=\"105.33125\" y=\"146.899219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_2\">\n",
|
|
" <!-- 0.5 -->\n",
|
|
" <g transform=\"translate(97.379688 161.497656)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
|
|
"L 3169 4666 \n",
|
|
"L 3169 4134 \n",
|
|
"L 1269 4134 \n",
|
|
"L 1269 2991 \n",
|
|
"Q 1406 3038 1543 3061 \n",
|
|
"Q 1681 3084 1819 3084 \n",
|
|
"Q 2600 3084 3056 2656 \n",
|
|
"Q 3513 2228 3513 1497 \n",
|
|
"Q 3513 744 3044 326 \n",
|
|
"Q 2575 -91 1722 -91 \n",
|
|
"Q 1428 -91 1123 -41 \n",
|
|
"Q 819 9 494 109 \n",
|
|
"L 494 744 \n",
|
|
"Q 775 591 1075 516 \n",
|
|
"Q 1375 441 1709 441 \n",
|
|
"Q 2250 441 2565 725 \n",
|
|
"Q 2881 1009 2881 1497 \n",
|
|
"Q 2881 1984 2565 2268 \n",
|
|
"Q 2250 2553 1709 2553 \n",
|
|
"Q 1456 2553 1204 2497 \n",
|
|
"Q 953 2441 691 2322 \n",
|
|
"L 691 4666 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"xtick_3\">\n",
|
|
" <g id=\"line2d_5\">\n",
|
|
" <path d=\"M 154.15625 146.899219 \n",
|
|
"L 154.15625 10.999219 \n",
|
|
"\" clip-path=\"url(#p3fc86941e4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_6\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#mc11186c2e7\" x=\"154.15625\" y=\"146.899219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_3\">\n",
|
|
" <!-- 1.0 -->\n",
|
|
" <g transform=\"translate(146.204688 161.497656)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
|
|
"L 1825 531 \n",
|
|
"L 1825 4091 \n",
|
|
"L 703 3866 \n",
|
|
"L 703 4441 \n",
|
|
"L 1819 4666 \n",
|
|
"L 2450 4666 \n",
|
|
"L 2450 531 \n",
|
|
"L 3481 531 \n",
|
|
"L 3481 0 \n",
|
|
"L 794 0 \n",
|
|
"L 794 531 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"xtick_4\">\n",
|
|
" <g id=\"line2d_7\">\n",
|
|
" <path d=\"M 202.98125 146.899219 \n",
|
|
"L 202.98125 10.999219 \n",
|
|
"\" clip-path=\"url(#p3fc86941e4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_8\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#mc11186c2e7\" x=\"202.98125\" y=\"146.899219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_4\">\n",
|
|
" <!-- 1.5 -->\n",
|
|
" <g transform=\"translate(195.029688 161.497656)scale(0.1 -0.1)\">\n",
|
|
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"xtick_5\">\n",
|
|
" <g id=\"line2d_9\">\n",
|
|
" <path d=\"M 251.80625 146.899219 \n",
|
|
"L 251.80625 10.999219 \n",
|
|
"\" clip-path=\"url(#p3fc86941e4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_10\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#mc11186c2e7\" x=\"251.80625\" y=\"146.899219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_5\">\n",
|
|
" <!-- 2.0 -->\n",
|
|
" <g transform=\"translate(243.854688 161.497656)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
|
|
"L 3431 531 \n",
|
|
"L 3431 0 \n",
|
|
"L 469 0 \n",
|
|
"L 469 531 \n",
|
|
"Q 828 903 1448 1529 \n",
|
|
"Q 2069 2156 2228 2338 \n",
|
|
"Q 2531 2678 2651 2914 \n",
|
|
"Q 2772 3150 2772 3378 \n",
|
|
"Q 2772 3750 2511 3984 \n",
|
|
"Q 2250 4219 1831 4219 \n",
|
|
"Q 1534 4219 1204 4116 \n",
|
|
"Q 875 4013 500 3803 \n",
|
|
"L 500 4441 \n",
|
|
"Q 881 4594 1212 4672 \n",
|
|
"Q 1544 4750 1819 4750 \n",
|
|
"Q 2544 4750 2975 4387 \n",
|
|
"Q 3406 4025 3406 3419 \n",
|
|
"Q 3406 3131 3298 2873 \n",
|
|
"Q 3191 2616 2906 2266 \n",
|
|
"Q 2828 2175 2409 1742 \n",
|
|
"Q 1991 1309 1228 531 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_6\">\n",
|
|
" <!-- epoch -->\n",
|
|
" <g transform=\"translate(138.928125 175.175781)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
|
|
"L 3597 1613 \n",
|
|
"L 953 1613 \n",
|
|
"Q 991 1019 1311 708 \n",
|
|
"Q 1631 397 2203 397 \n",
|
|
"Q 2534 397 2845 478 \n",
|
|
"Q 3156 559 3463 722 \n",
|
|
"L 3463 178 \n",
|
|
"Q 3153 47 2828 -22 \n",
|
|
"Q 2503 -91 2169 -91 \n",
|
|
"Q 1331 -91 842 396 \n",
|
|
"Q 353 884 353 1716 \n",
|
|
"Q 353 2575 817 3079 \n",
|
|
"Q 1281 3584 2069 3584 \n",
|
|
"Q 2775 3584 3186 3129 \n",
|
|
"Q 3597 2675 3597 1894 \n",
|
|
"z\n",
|
|
"M 3022 2063 \n",
|
|
"Q 3016 2534 2758 2815 \n",
|
|
"Q 2500 3097 2075 3097 \n",
|
|
"Q 1594 3097 1305 2825 \n",
|
|
"Q 1016 2553 972 2059 \n",
|
|
"L 3022 2063 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
|
|
"L 1159 -1331 \n",
|
|
"L 581 -1331 \n",
|
|
"L 581 3500 \n",
|
|
"L 1159 3500 \n",
|
|
"L 1159 2969 \n",
|
|
"Q 1341 3281 1617 3432 \n",
|
|
"Q 1894 3584 2278 3584 \n",
|
|
"Q 2916 3584 3314 3078 \n",
|
|
"Q 3713 2572 3713 1747 \n",
|
|
"Q 3713 922 3314 415 \n",
|
|
"Q 2916 -91 2278 -91 \n",
|
|
"Q 1894 -91 1617 61 \n",
|
|
"Q 1341 213 1159 525 \n",
|
|
"z\n",
|
|
"M 3116 1747 \n",
|
|
"Q 3116 2381 2855 2742 \n",
|
|
"Q 2594 3103 2138 3103 \n",
|
|
"Q 1681 3103 1420 2742 \n",
|
|
"Q 1159 2381 1159 1747 \n",
|
|
"Q 1159 1113 1420 752 \n",
|
|
"Q 1681 391 2138 391 \n",
|
|
"Q 2594 391 2855 752 \n",
|
|
"Q 3116 1113 3116 1747 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
|
|
"Q 1497 3097 1228 2736 \n",
|
|
"Q 959 2375 959 1747 \n",
|
|
"Q 959 1119 1226 758 \n",
|
|
"Q 1494 397 1959 397 \n",
|
|
"Q 2419 397 2687 759 \n",
|
|
"Q 2956 1122 2956 1747 \n",
|
|
"Q 2956 2369 2687 2733 \n",
|
|
"Q 2419 3097 1959 3097 \n",
|
|
"z\n",
|
|
"M 1959 3584 \n",
|
|
"Q 2709 3584 3137 3096 \n",
|
|
"Q 3566 2609 3566 1747 \n",
|
|
"Q 3566 888 3137 398 \n",
|
|
"Q 2709 -91 1959 -91 \n",
|
|
"Q 1206 -91 779 398 \n",
|
|
"Q 353 888 353 1747 \n",
|
|
"Q 353 2609 779 3096 \n",
|
|
"Q 1206 3584 1959 3584 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
|
|
"L 3122 2828 \n",
|
|
"Q 2878 2963 2633 3030 \n",
|
|
"Q 2388 3097 2138 3097 \n",
|
|
"Q 1578 3097 1268 2742 \n",
|
|
"Q 959 2388 959 1747 \n",
|
|
"Q 959 1106 1268 751 \n",
|
|
"Q 1578 397 2138 397 \n",
|
|
"Q 2388 397 2633 464 \n",
|
|
"Q 2878 531 3122 666 \n",
|
|
"L 3122 134 \n",
|
|
"Q 2881 22 2623 -34 \n",
|
|
"Q 2366 -91 2075 -91 \n",
|
|
"Q 1284 -91 818 406 \n",
|
|
"Q 353 903 353 1747 \n",
|
|
"Q 353 2603 823 3093 \n",
|
|
"Q 1294 3584 2113 3584 \n",
|
|
"Q 2378 3584 2631 3529 \n",
|
|
"Q 2884 3475 3122 3366 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
|
|
"L 3513 0 \n",
|
|
"L 2938 0 \n",
|
|
"L 2938 2094 \n",
|
|
"Q 2938 2591 2744 2837 \n",
|
|
"Q 2550 3084 2163 3084 \n",
|
|
"Q 1697 3084 1428 2787 \n",
|
|
"Q 1159 2491 1159 1978 \n",
|
|
"L 1159 0 \n",
|
|
"L 581 0 \n",
|
|
"L 581 4863 \n",
|
|
"L 1159 4863 \n",
|
|
"L 1159 2956 \n",
|
|
"Q 1366 3272 1645 3428 \n",
|
|
"Q 1925 3584 2291 3584 \n",
|
|
"Q 2894 3584 3203 3211 \n",
|
|
"Q 3513 2838 3513 2113 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-65\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"matplotlib.axis_2\">\n",
|
|
" <g id=\"ytick_1\">\n",
|
|
" <g id=\"line2d_11\">\n",
|
|
" <path d=\"M 56.50625 141.672296 \n",
|
|
"L 251.80625 141.672296 \n",
|
|
"\" clip-path=\"url(#p3fc86941e4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_12\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"m48330adb50\" d=\"M 0 0 \n",
|
|
"L -3.5 0 \n",
|
|
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </defs>\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#m48330adb50\" x=\"56.50625\" y=\"141.672296\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_7\">\n",
|
|
" <!-- 0.225 -->\n",
|
|
" <g transform=\"translate(20.878125 145.471514)scale(0.1 -0.1)\">\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-32\" x=\"95.410156\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-32\" x=\"159.033203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-35\" x=\"222.65625\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"ytick_2\">\n",
|
|
" <g id=\"line2d_13\">\n",
|
|
" <path d=\"M 56.50625 115.53768 \n",
|
|
"L 251.80625 115.53768 \n",
|
|
"\" clip-path=\"url(#p3fc86941e4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_14\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#m48330adb50\" x=\"56.50625\" y=\"115.53768\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_8\">\n",
|
|
" <!-- 0.250 -->\n",
|
|
" <g transform=\"translate(20.878125 119.336899)scale(0.1 -0.1)\">\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-32\" x=\"95.410156\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-35\" x=\"159.033203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\" x=\"222.65625\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"ytick_3\">\n",
|
|
" <g id=\"line2d_15\">\n",
|
|
" <path d=\"M 56.50625 89.403065 \n",
|
|
"L 251.80625 89.403065 \n",
|
|
"\" clip-path=\"url(#p3fc86941e4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_16\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#m48330adb50\" x=\"56.50625\" y=\"89.403065\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_9\">\n",
|
|
" <!-- 0.275 -->\n",
|
|
" <g transform=\"translate(20.878125 93.202284)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-37\" d=\"M 525 4666 \n",
|
|
"L 3525 4666 \n",
|
|
"L 3525 4397 \n",
|
|
"L 1831 0 \n",
|
|
"L 1172 0 \n",
|
|
"L 2766 4134 \n",
|
|
"L 525 4134 \n",
|
|
"L 525 4666 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-32\" x=\"95.410156\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-37\" x=\"159.033203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-35\" x=\"222.65625\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"ytick_4\">\n",
|
|
" <g id=\"line2d_17\">\n",
|
|
" <path d=\"M 56.50625 63.26845 \n",
|
|
"L 251.80625 63.26845 \n",
|
|
"\" clip-path=\"url(#p3fc86941e4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_18\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#m48330adb50\" x=\"56.50625\" y=\"63.26845\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_10\">\n",
|
|
" <!-- 0.300 -->\n",
|
|
" <g transform=\"translate(20.878125 67.067668)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
|
|
"Q 3050 2419 3304 2112 \n",
|
|
"Q 3559 1806 3559 1356 \n",
|
|
"Q 3559 666 3084 287 \n",
|
|
"Q 2609 -91 1734 -91 \n",
|
|
"Q 1441 -91 1130 -33 \n",
|
|
"Q 819 25 488 141 \n",
|
|
"L 488 750 \n",
|
|
"Q 750 597 1062 519 \n",
|
|
"Q 1375 441 1716 441 \n",
|
|
"Q 2309 441 2620 675 \n",
|
|
"Q 2931 909 2931 1356 \n",
|
|
"Q 2931 1769 2642 2001 \n",
|
|
"Q 2353 2234 1838 2234 \n",
|
|
"L 1294 2234 \n",
|
|
"L 1294 2753 \n",
|
|
"L 1863 2753 \n",
|
|
"Q 2328 2753 2575 2939 \n",
|
|
"Q 2822 3125 2822 3475 \n",
|
|
"Q 2822 3834 2567 4026 \n",
|
|
"Q 2313 4219 1838 4219 \n",
|
|
"Q 1578 4219 1281 4162 \n",
|
|
"Q 984 4106 628 3988 \n",
|
|
"L 628 4550 \n",
|
|
"Q 988 4650 1302 4700 \n",
|
|
"Q 1616 4750 1894 4750 \n",
|
|
"Q 2613 4750 3031 4423 \n",
|
|
"Q 3450 4097 3450 3541 \n",
|
|
"Q 3450 3153 3228 2886 \n",
|
|
"Q 3006 2619 2597 2516 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-33\" x=\"95.410156\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\" x=\"159.033203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\" x=\"222.65625\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"ytick_5\">\n",
|
|
" <g id=\"line2d_19\">\n",
|
|
" <path d=\"M 56.50625 37.133834 \n",
|
|
"L 251.80625 37.133834 \n",
|
|
"\" clip-path=\"url(#p3fc86941e4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_20\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#m48330adb50\" x=\"56.50625\" y=\"37.133834\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_11\">\n",
|
|
" <!-- 0.325 -->\n",
|
|
" <g transform=\"translate(20.878125 40.933053)scale(0.1 -0.1)\">\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-33\" x=\"95.410156\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-32\" x=\"159.033203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-35\" x=\"222.65625\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"ytick_6\">\n",
|
|
" <g id=\"line2d_21\">\n",
|
|
" <path d=\"M 56.50625 10.999219 \n",
|
|
"L 251.80625 10.999219 \n",
|
|
"\" clip-path=\"url(#p3fc86941e4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_22\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#m48330adb50\" x=\"56.50625\" y=\"10.999219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_12\">\n",
|
|
" <!-- 0.350 -->\n",
|
|
" <g transform=\"translate(20.878125 14.798437)scale(0.1 -0.1)\">\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-33\" x=\"95.410156\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-35\" x=\"159.033203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\" x=\"222.65625\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_13\">\n",
|
|
" <!-- loss -->\n",
|
|
" <g transform=\"translate(14.798438 88.607031)rotate(-90)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
|
|
"L 1178 4863 \n",
|
|
"L 1178 0 \n",
|
|
"L 603 0 \n",
|
|
"L 603 4863 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
|
|
"L 2834 2853 \n",
|
|
"Q 2591 2978 2328 3040 \n",
|
|
"Q 2066 3103 1784 3103 \n",
|
|
"Q 1356 3103 1142 2972 \n",
|
|
"Q 928 2841 928 2578 \n",
|
|
"Q 928 2378 1081 2264 \n",
|
|
"Q 1234 2150 1697 2047 \n",
|
|
"L 1894 2003 \n",
|
|
"Q 2506 1872 2764 1633 \n",
|
|
"Q 3022 1394 3022 966 \n",
|
|
"Q 3022 478 2636 193 \n",
|
|
"Q 2250 -91 1575 -91 \n",
|
|
"Q 1294 -91 989 -36 \n",
|
|
"Q 684 19 347 128 \n",
|
|
"L 347 722 \n",
|
|
"Q 666 556 975 473 \n",
|
|
"Q 1284 391 1588 391 \n",
|
|
"Q 1994 391 2212 530 \n",
|
|
"Q 2431 669 2431 922 \n",
|
|
"Q 2431 1156 2273 1281 \n",
|
|
"Q 2116 1406 1581 1522 \n",
|
|
"L 1381 1569 \n",
|
|
"Q 847 1681 609 1914 \n",
|
|
"Q 372 2147 372 2553 \n",
|
|
"Q 372 3047 722 3315 \n",
|
|
"Q 1072 3584 1716 3584 \n",
|
|
"Q 2034 3584 2315 3537 \n",
|
|
"Q 2597 3491 2834 3397 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-6c\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-6f\" x=\"27.783203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-73\" x=\"88.964844\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-73\" x=\"141.064453\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_23\">\n",
|
|
" <path d=\"M 77.336976 -1 \n",
|
|
"L 82.54625 23.179505 \n",
|
|
"L 95.56625 60.579845 \n",
|
|
"L 108.58625 89.887007 \n",
|
|
"L 121.60625 103.062288 \n",
|
|
"L 134.62625 114.75221 \n",
|
|
"L 147.64625 121.826247 \n",
|
|
"L 160.66625 120.998062 \n",
|
|
"L 173.68625 122.775371 \n",
|
|
"L 186.70625 123.587037 \n",
|
|
"L 199.72625 120.594352 \n",
|
|
"L 212.74625 122.703567 \n",
|
|
"L 225.76625 120.829798 \n",
|
|
"L 238.78625 121.914748 \n",
|
|
"L 251.80625 123.145342 \n",
|
|
"\" clip-path=\"url(#p3fc86941e4)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"patch_3\">\n",
|
|
" <path d=\"M 56.50625 146.899219 \n",
|
|
"L 56.50625 10.999219 \n",
|
|
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"patch_4\">\n",
|
|
" <path d=\"M 251.80625 146.899219 \n",
|
|
"L 251.80625 10.999219 \n",
|
|
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"patch_5\">\n",
|
|
" <path d=\"M 56.50625 146.899219 \n",
|
|
"L 251.80625 146.899219 \n",
|
|
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"patch_6\">\n",
|
|
" <path d=\"M 56.50625 10.999219 \n",
|
|
"L 251.80625 10.999219 \n",
|
|
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <defs>\n",
|
|
" <clipPath id=\"p3fc86941e4\">\n",
|
|
" <rect x=\"56.50625\" y=\"10.999219\" width=\"195.3\" height=\"135.9\"/>\n",
|
|
" </clipPath>\n",
|
|
" </defs>\n",
|
|
"</svg>\n"
|
|
],
|
|
"text/plain": [
|
|
"<Figure size 252x180 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"data_iter, feature_dim = d2l.get_data_ch11(batch_size=10)\n",
|
|
"d2l.train_ch11(adadelta, init_adadelta_states(feature_dim),\n",
|
|
" {'rho': 0.9}, data_iter, feature_dim);"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a8cd0fd5",
|
|
"metadata": {
|
|
"origin_pos": 7
|
|
},
|
|
"source": [
|
|
"为了简洁实现，我们只需使用高级API中的Adadelta算法。\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"id": "25f0fd32",
|
|
"metadata": {
|
|
"execution": {
|
|
"iopub.execute_input": "2023-08-18T07:05:14.085468Z",
|
|
"iopub.status.busy": "2023-08-18T07:05:14.085189Z",
|
|
"iopub.status.idle": "2023-08-18T07:05:19.137299Z",
|
|
"shell.execute_reply": "2023-08-18T07:05:19.136478Z"
|
|
},
|
|
"origin_pos": 9,
|
|
"tab": [
|
|
"pytorch"
|
|
]
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"loss: 0.243, 0.013 sec/epoch\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/svg+xml": [
|
|
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
|
|
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
|
|
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
|
|
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"262.1875pt\" height=\"184.455469pt\" viewBox=\"0 0 262.1875 184.455469\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
|
|
" <metadata>\n",
|
|
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
|
|
" <cc:Work>\n",
|
|
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
|
|
" <dc:date>2023-08-18T07:05:19.103343</dc:date>\n",
|
|
" <dc:format>image/svg+xml</dc:format>\n",
|
|
" <dc:creator>\n",
|
|
" <cc:Agent>\n",
|
|
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
|
|
" </cc:Agent>\n",
|
|
" </dc:creator>\n",
|
|
" </cc:Work>\n",
|
|
" </rdf:RDF>\n",
|
|
" </metadata>\n",
|
|
" <defs>\n",
|
|
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
|
|
" </defs>\n",
|
|
" <g id=\"figure_1\">\n",
|
|
" <g id=\"patch_1\">\n",
|
|
" <path d=\"M -0 184.455469 \n",
|
|
"L 262.1875 184.455469 \n",
|
|
"L 262.1875 0 \n",
|
|
"L -0 0 \n",
|
|
"L -0 184.455469 \n",
|
|
"z\n",
|
|
"\" style=\"fill: none\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"axes_1\">\n",
|
|
" <g id=\"patch_2\">\n",
|
|
" <path d=\"M 56.50625 146.899219 \n",
|
|
"L 251.80625 146.899219 \n",
|
|
"L 251.80625 10.999219 \n",
|
|
"L 56.50625 10.999219 \n",
|
|
"z\n",
|
|
"\" style=\"fill: #ffffff\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"matplotlib.axis_1\">\n",
|
|
" <g id=\"xtick_1\">\n",
|
|
" <g id=\"line2d_1\">\n",
|
|
" <path d=\"M 56.50625 146.899219 \n",
|
|
"L 56.50625 10.999219 \n",
|
|
"\" clip-path=\"url(#p0138888679)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_2\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"mdbdc969c06\" d=\"M 0 0 \n",
|
|
"L 0 3.5 \n",
|
|
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </defs>\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#mdbdc969c06\" x=\"56.50625\" y=\"146.899219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_1\">\n",
|
|
" <!-- 0 -->\n",
|
|
" <g transform=\"translate(53.325 161.497656)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
|
|
"Q 1547 4250 1301 3770 \n",
|
|
"Q 1056 3291 1056 2328 \n",
|
|
"Q 1056 1369 1301 889 \n",
|
|
"Q 1547 409 2034 409 \n",
|
|
"Q 2525 409 2770 889 \n",
|
|
"Q 3016 1369 3016 2328 \n",
|
|
"Q 3016 3291 2770 3770 \n",
|
|
"Q 2525 4250 2034 4250 \n",
|
|
"z\n",
|
|
"M 2034 4750 \n",
|
|
"Q 2819 4750 3233 4129 \n",
|
|
"Q 3647 3509 3647 2328 \n",
|
|
"Q 3647 1150 3233 529 \n",
|
|
"Q 2819 -91 2034 -91 \n",
|
|
"Q 1250 -91 836 529 \n",
|
|
"Q 422 1150 422 2328 \n",
|
|
"Q 422 3509 836 4129 \n",
|
|
"Q 1250 4750 2034 4750 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"xtick_2\">\n",
|
|
" <g id=\"line2d_3\">\n",
|
|
" <path d=\"M 105.33125 146.899219 \n",
|
|
"L 105.33125 10.999219 \n",
|
|
"\" clip-path=\"url(#p0138888679)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_4\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#mdbdc969c06\" x=\"105.33125\" y=\"146.899219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_2\">\n",
|
|
" <!-- 1 -->\n",
|
|
" <g transform=\"translate(102.15 161.497656)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
|
|
"L 1825 531 \n",
|
|
"L 1825 4091 \n",
|
|
"L 703 3866 \n",
|
|
"L 703 4441 \n",
|
|
"L 1819 4666 \n",
|
|
"L 2450 4666 \n",
|
|
"L 2450 531 \n",
|
|
"L 3481 531 \n",
|
|
"L 3481 0 \n",
|
|
"L 794 0 \n",
|
|
"L 794 531 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"xtick_3\">\n",
|
|
" <g id=\"line2d_5\">\n",
|
|
" <path d=\"M 154.15625 146.899219 \n",
|
|
"L 154.15625 10.999219 \n",
|
|
"\" clip-path=\"url(#p0138888679)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_6\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#mdbdc969c06\" x=\"154.15625\" y=\"146.899219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_3\">\n",
|
|
" <!-- 2 -->\n",
|
|
" <g transform=\"translate(150.975 161.497656)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
|
|
"L 3431 531 \n",
|
|
"L 3431 0 \n",
|
|
"L 469 0 \n",
|
|
"L 469 531 \n",
|
|
"Q 828 903 1448 1529 \n",
|
|
"Q 2069 2156 2228 2338 \n",
|
|
"Q 2531 2678 2651 2914 \n",
|
|
"Q 2772 3150 2772 3378 \n",
|
|
"Q 2772 3750 2511 3984 \n",
|
|
"Q 2250 4219 1831 4219 \n",
|
|
"Q 1534 4219 1204 4116 \n",
|
|
"Q 875 4013 500 3803 \n",
|
|
"L 500 4441 \n",
|
|
"Q 881 4594 1212 4672 \n",
|
|
"Q 1544 4750 1819 4750 \n",
|
|
"Q 2544 4750 2975 4387 \n",
|
|
"Q 3406 4025 3406 3419 \n",
|
|
"Q 3406 3131 3298 2873 \n",
|
|
"Q 3191 2616 2906 2266 \n",
|
|
"Q 2828 2175 2409 1742 \n",
|
|
"Q 1991 1309 1228 531 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"xtick_4\">\n",
|
|
" <g id=\"line2d_7\">\n",
|
|
" <path d=\"M 202.98125 146.899219 \n",
|
|
"L 202.98125 10.999219 \n",
|
|
"\" clip-path=\"url(#p0138888679)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_8\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#mdbdc969c06\" x=\"202.98125\" y=\"146.899219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_4\">\n",
|
|
" <!-- 3 -->\n",
|
|
" <g transform=\"translate(199.8 161.497656)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
|
|
"Q 3050 2419 3304 2112 \n",
|
|
"Q 3559 1806 3559 1356 \n",
|
|
"Q 3559 666 3084 287 \n",
|
|
"Q 2609 -91 1734 -91 \n",
|
|
"Q 1441 -91 1130 -33 \n",
|
|
"Q 819 25 488 141 \n",
|
|
"L 488 750 \n",
|
|
"Q 750 597 1062 519 \n",
|
|
"Q 1375 441 1716 441 \n",
|
|
"Q 2309 441 2620 675 \n",
|
|
"Q 2931 909 2931 1356 \n",
|
|
"Q 2931 1769 2642 2001 \n",
|
|
"Q 2353 2234 1838 2234 \n",
|
|
"L 1294 2234 \n",
|
|
"L 1294 2753 \n",
|
|
"L 1863 2753 \n",
|
|
"Q 2328 2753 2575 2939 \n",
|
|
"Q 2822 3125 2822 3475 \n",
|
|
"Q 2822 3834 2567 4026 \n",
|
|
"Q 2313 4219 1838 4219 \n",
|
|
"Q 1578 4219 1281 4162 \n",
|
|
"Q 984 4106 628 3988 \n",
|
|
"L 628 4550 \n",
|
|
"Q 988 4650 1302 4700 \n",
|
|
"Q 1616 4750 1894 4750 \n",
|
|
"Q 2613 4750 3031 4423 \n",
|
|
"Q 3450 4097 3450 3541 \n",
|
|
"Q 3450 3153 3228 2886 \n",
|
|
"Q 3006 2619 2597 2516 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-33\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"xtick_5\">\n",
|
|
" <g id=\"line2d_9\">\n",
|
|
" <path d=\"M 251.80625 146.899219 \n",
|
|
"L 251.80625 10.999219 \n",
|
|
"\" clip-path=\"url(#p0138888679)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_10\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#mdbdc969c06\" x=\"251.80625\" y=\"146.899219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_5\">\n",
|
|
" <!-- 4 -->\n",
|
|
" <g transform=\"translate(248.625 161.497656)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
|
|
"L 825 1625 \n",
|
|
"L 2419 1625 \n",
|
|
"L 2419 4116 \n",
|
|
"z\n",
|
|
"M 2253 4666 \n",
|
|
"L 3047 4666 \n",
|
|
"L 3047 1625 \n",
|
|
"L 3713 1625 \n",
|
|
"L 3713 1100 \n",
|
|
"L 3047 1100 \n",
|
|
"L 3047 0 \n",
|
|
"L 2419 0 \n",
|
|
"L 2419 1100 \n",
|
|
"L 313 1100 \n",
|
|
"L 313 1709 \n",
|
|
"L 2253 4666 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_6\">\n",
|
|
" <!-- epoch -->\n",
|
|
" <g transform=\"translate(138.928125 175.175781)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
|
|
"L 3597 1613 \n",
|
|
"L 953 1613 \n",
|
|
"Q 991 1019 1311 708 \n",
|
|
"Q 1631 397 2203 397 \n",
|
|
"Q 2534 397 2845 478 \n",
|
|
"Q 3156 559 3463 722 \n",
|
|
"L 3463 178 \n",
|
|
"Q 3153 47 2828 -22 \n",
|
|
"Q 2503 -91 2169 -91 \n",
|
|
"Q 1331 -91 842 396 \n",
|
|
"Q 353 884 353 1716 \n",
|
|
"Q 353 2575 817 3079 \n",
|
|
"Q 1281 3584 2069 3584 \n",
|
|
"Q 2775 3584 3186 3129 \n",
|
|
"Q 3597 2675 3597 1894 \n",
|
|
"z\n",
|
|
"M 3022 2063 \n",
|
|
"Q 3016 2534 2758 2815 \n",
|
|
"Q 2500 3097 2075 3097 \n",
|
|
"Q 1594 3097 1305 2825 \n",
|
|
"Q 1016 2553 972 2059 \n",
|
|
"L 3022 2063 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
|
|
"L 1159 -1331 \n",
|
|
"L 581 -1331 \n",
|
|
"L 581 3500 \n",
|
|
"L 1159 3500 \n",
|
|
"L 1159 2969 \n",
|
|
"Q 1341 3281 1617 3432 \n",
|
|
"Q 1894 3584 2278 3584 \n",
|
|
"Q 2916 3584 3314 3078 \n",
|
|
"Q 3713 2572 3713 1747 \n",
|
|
"Q 3713 922 3314 415 \n",
|
|
"Q 2916 -91 2278 -91 \n",
|
|
"Q 1894 -91 1617 61 \n",
|
|
"Q 1341 213 1159 525 \n",
|
|
"z\n",
|
|
"M 3116 1747 \n",
|
|
"Q 3116 2381 2855 2742 \n",
|
|
"Q 2594 3103 2138 3103 \n",
|
|
"Q 1681 3103 1420 2742 \n",
|
|
"Q 1159 2381 1159 1747 \n",
|
|
"Q 1159 1113 1420 752 \n",
|
|
"Q 1681 391 2138 391 \n",
|
|
"Q 2594 391 2855 752 \n",
|
|
"Q 3116 1113 3116 1747 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
|
|
"Q 1497 3097 1228 2736 \n",
|
|
"Q 959 2375 959 1747 \n",
|
|
"Q 959 1119 1226 758 \n",
|
|
"Q 1494 397 1959 397 \n",
|
|
"Q 2419 397 2687 759 \n",
|
|
"Q 2956 1122 2956 1747 \n",
|
|
"Q 2956 2369 2687 2733 \n",
|
|
"Q 2419 3097 1959 3097 \n",
|
|
"z\n",
|
|
"M 1959 3584 \n",
|
|
"Q 2709 3584 3137 3096 \n",
|
|
"Q 3566 2609 3566 1747 \n",
|
|
"Q 3566 888 3137 398 \n",
|
|
"Q 2709 -91 1959 -91 \n",
|
|
"Q 1206 -91 779 398 \n",
|
|
"Q 353 888 353 1747 \n",
|
|
"Q 353 2609 779 3096 \n",
|
|
"Q 1206 3584 1959 3584 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
|
|
"L 3122 2828 \n",
|
|
"Q 2878 2963 2633 3030 \n",
|
|
"Q 2388 3097 2138 3097 \n",
|
|
"Q 1578 3097 1268 2742 \n",
|
|
"Q 959 2388 959 1747 \n",
|
|
"Q 959 1106 1268 751 \n",
|
|
"Q 1578 397 2138 397 \n",
|
|
"Q 2388 397 2633 464 \n",
|
|
"Q 2878 531 3122 666 \n",
|
|
"L 3122 134 \n",
|
|
"Q 2881 22 2623 -34 \n",
|
|
"Q 2366 -91 2075 -91 \n",
|
|
"Q 1284 -91 818 406 \n",
|
|
"Q 353 903 353 1747 \n",
|
|
"Q 353 2603 823 3093 \n",
|
|
"Q 1294 3584 2113 3584 \n",
|
|
"Q 2378 3584 2631 3529 \n",
|
|
"Q 2884 3475 3122 3366 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
|
|
"L 3513 0 \n",
|
|
"L 2938 0 \n",
|
|
"L 2938 2094 \n",
|
|
"Q 2938 2591 2744 2837 \n",
|
|
"Q 2550 3084 2163 3084 \n",
|
|
"Q 1697 3084 1428 2787 \n",
|
|
"Q 1159 2491 1159 1978 \n",
|
|
"L 1159 0 \n",
|
|
"L 581 0 \n",
|
|
"L 581 4863 \n",
|
|
"L 1159 4863 \n",
|
|
"L 1159 2956 \n",
|
|
"Q 1366 3272 1645 3428 \n",
|
|
"Q 1925 3584 2291 3584 \n",
|
|
"Q 2894 3584 3203 3211 \n",
|
|
"Q 3513 2838 3513 2113 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-65\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"matplotlib.axis_2\">\n",
|
|
" <g id=\"ytick_1\">\n",
|
|
" <g id=\"line2d_11\">\n",
|
|
" <path d=\"M 56.50625 141.672296 \n",
|
|
"L 251.80625 141.672296 \n",
|
|
"\" clip-path=\"url(#p0138888679)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_12\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"ma92c651954\" d=\"M 0 0 \n",
|
|
"L -3.5 0 \n",
|
|
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </defs>\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#ma92c651954\" x=\"56.50625\" y=\"141.672296\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_7\">\n",
|
|
" <!-- 0.225 -->\n",
|
|
" <g transform=\"translate(20.878125 145.471514)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
|
|
"L 1344 794 \n",
|
|
"L 1344 0 \n",
|
|
"L 684 0 \n",
|
|
"L 684 794 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
|
|
"L 3169 4666 \n",
|
|
"L 3169 4134 \n",
|
|
"L 1269 4134 \n",
|
|
"L 1269 2991 \n",
|
|
"Q 1406 3038 1543 3061 \n",
|
|
"Q 1681 3084 1819 3084 \n",
|
|
"Q 2600 3084 3056 2656 \n",
|
|
"Q 3513 2228 3513 1497 \n",
|
|
"Q 3513 744 3044 326 \n",
|
|
"Q 2575 -91 1722 -91 \n",
|
|
"Q 1428 -91 1123 -41 \n",
|
|
"Q 819 9 494 109 \n",
|
|
"L 494 744 \n",
|
|
"Q 775 591 1075 516 \n",
|
|
"Q 1375 441 1709 441 \n",
|
|
"Q 2250 441 2565 725 \n",
|
|
"Q 2881 1009 2881 1497 \n",
|
|
"Q 2881 1984 2565 2268 \n",
|
|
"Q 2250 2553 1709 2553 \n",
|
|
"Q 1456 2553 1204 2497 \n",
|
|
"Q 953 2441 691 2322 \n",
|
|
"L 691 4666 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-32\" x=\"95.410156\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-32\" x=\"159.033203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-35\" x=\"222.65625\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"ytick_2\">\n",
|
|
" <g id=\"line2d_13\">\n",
|
|
" <path d=\"M 56.50625 115.53768 \n",
|
|
"L 251.80625 115.53768 \n",
|
|
"\" clip-path=\"url(#p0138888679)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_14\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#ma92c651954\" x=\"56.50625\" y=\"115.53768\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_8\">\n",
|
|
" <!-- 0.250 -->\n",
|
|
" <g transform=\"translate(20.878125 119.336899)scale(0.1 -0.1)\">\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-32\" x=\"95.410156\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-35\" x=\"159.033203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\" x=\"222.65625\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"ytick_3\">\n",
|
|
" <g id=\"line2d_15\">\n",
|
|
" <path d=\"M 56.50625 89.403065 \n",
|
|
"L 251.80625 89.403065 \n",
|
|
"\" clip-path=\"url(#p0138888679)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_16\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#ma92c651954\" x=\"56.50625\" y=\"89.403065\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_9\">\n",
|
|
" <!-- 0.275 -->\n",
|
|
" <g transform=\"translate(20.878125 93.202284)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-37\" d=\"M 525 4666 \n",
|
|
"L 3525 4666 \n",
|
|
"L 3525 4397 \n",
|
|
"L 1831 0 \n",
|
|
"L 1172 0 \n",
|
|
"L 2766 4134 \n",
|
|
"L 525 4134 \n",
|
|
"L 525 4666 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-32\" x=\"95.410156\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-37\" x=\"159.033203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-35\" x=\"222.65625\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"ytick_4\">\n",
|
|
" <g id=\"line2d_17\">\n",
|
|
" <path d=\"M 56.50625 63.26845 \n",
|
|
"L 251.80625 63.26845 \n",
|
|
"\" clip-path=\"url(#p0138888679)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_18\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#ma92c651954\" x=\"56.50625\" y=\"63.26845\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_10\">\n",
|
|
" <!-- 0.300 -->\n",
|
|
" <g transform=\"translate(20.878125 67.067668)scale(0.1 -0.1)\">\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-33\" x=\"95.410156\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\" x=\"159.033203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\" x=\"222.65625\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"ytick_5\">\n",
|
|
" <g id=\"line2d_19\">\n",
|
|
" <path d=\"M 56.50625 37.133834 \n",
|
|
"L 251.80625 37.133834 \n",
|
|
"\" clip-path=\"url(#p0138888679)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_20\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#ma92c651954\" x=\"56.50625\" y=\"37.133834\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_11\">\n",
|
|
" <!-- 0.325 -->\n",
|
|
" <g transform=\"translate(20.878125 40.933053)scale(0.1 -0.1)\">\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-33\" x=\"95.410156\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-32\" x=\"159.033203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-35\" x=\"222.65625\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"ytick_6\">\n",
|
|
" <g id=\"line2d_21\">\n",
|
|
" <path d=\"M 56.50625 10.999219 \n",
|
|
"L 251.80625 10.999219 \n",
|
|
"\" clip-path=\"url(#p0138888679)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_22\">\n",
|
|
" <g>\n",
|
|
" <use xlink:href=\"#ma92c651954\" x=\"56.50625\" y=\"10.999219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_12\">\n",
|
|
" <!-- 0.350 -->\n",
|
|
" <g transform=\"translate(20.878125 14.798437)scale(0.1 -0.1)\">\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-33\" x=\"95.410156\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-35\" x=\"159.033203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-30\" x=\"222.65625\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"text_13\">\n",
|
|
" <!-- loss -->\n",
|
|
" <g transform=\"translate(14.798438 88.607031)rotate(-90)scale(0.1 -0.1)\">\n",
|
|
" <defs>\n",
|
|
" <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
|
|
"L 1178 4863 \n",
|
|
"L 1178 0 \n",
|
|
"L 603 0 \n",
|
|
"L 603 4863 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
|
|
"L 2834 2853 \n",
|
|
"Q 2591 2978 2328 3040 \n",
|
|
"Q 2066 3103 1784 3103 \n",
|
|
"Q 1356 3103 1142 2972 \n",
|
|
"Q 928 2841 928 2578 \n",
|
|
"Q 928 2378 1081 2264 \n",
|
|
"Q 1234 2150 1697 2047 \n",
|
|
"L 1894 2003 \n",
|
|
"Q 2506 1872 2764 1633 \n",
|
|
"Q 3022 1394 3022 966 \n",
|
|
"Q 3022 478 2636 193 \n",
|
|
"Q 2250 -91 1575 -91 \n",
|
|
"Q 1294 -91 989 -36 \n",
|
|
"Q 684 19 347 128 \n",
|
|
"L 347 722 \n",
|
|
"Q 666 556 975 473 \n",
|
|
"Q 1284 391 1588 391 \n",
|
|
"Q 1994 391 2212 530 \n",
|
|
"Q 2431 669 2431 922 \n",
|
|
"Q 2431 1156 2273 1281 \n",
|
|
"Q 2116 1406 1581 1522 \n",
|
|
"L 1381 1569 \n",
|
|
"Q 847 1681 609 1914 \n",
|
|
"Q 372 2147 372 2553 \n",
|
|
"Q 372 3047 722 3315 \n",
|
|
"Q 1072 3584 1716 3584 \n",
|
|
"Q 2034 3584 2315 3537 \n",
|
|
"Q 2597 3491 2834 3397 \n",
|
|
"z\n",
|
|
"\" transform=\"scale(0.015625)\"/>\n",
|
|
" </defs>\n",
|
|
" <use xlink:href=\"#DejaVuSans-6c\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-6f\" x=\"27.783203\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-73\" x=\"88.964844\"/>\n",
|
|
" <use xlink:href=\"#DejaVuSans-73\" x=\"141.064453\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <g id=\"line2d_23\">\n",
|
|
" <path d=\"M 89.913143 -1 \n",
|
|
"L 95.56625 18.697397 \n",
|
|
"L 102.07625 36.468935 \n",
|
|
"L 108.58625 57.397616 \n",
|
|
"L 115.09625 67.480601 \n",
|
|
"L 121.60625 78.949194 \n",
|
|
"L 128.11625 89.651494 \n",
|
|
"L 134.62625 96.687236 \n",
|
|
"L 141.13625 101.218879 \n",
|
|
"L 147.64625 107.021718 \n",
|
|
"L 154.15625 110.322871 \n",
|
|
"L 160.66625 113.953173 \n",
|
|
"L 167.17625 116.00154 \n",
|
|
"L 173.68625 117.270062 \n",
|
|
"L 180.19625 118.598213 \n",
|
|
"L 186.70625 120.013712 \n",
|
|
"L 193.21625 120.971153 \n",
|
|
"L 199.72625 120.994942 \n",
|
|
"L 206.23625 121.464986 \n",
|
|
"L 212.74625 121.61574 \n",
|
|
"L 219.25625 121.594855 \n",
|
|
"L 225.76625 122.039876 \n",
|
|
"L 232.27625 122.300064 \n",
|
|
"L 238.78625 122.881812 \n",
|
|
"L 245.29625 123.178551 \n",
|
|
"L 251.80625 123.0222 \n",
|
|
"\" clip-path=\"url(#p0138888679)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"patch_3\">\n",
|
|
" <path d=\"M 56.50625 146.899219 \n",
|
|
"L 56.50625 10.999219 \n",
|
|
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"patch_4\">\n",
|
|
" <path d=\"M 251.80625 146.899219 \n",
|
|
"L 251.80625 10.999219 \n",
|
|
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"patch_5\">\n",
|
|
" <path d=\"M 56.50625 146.899219 \n",
|
|
"L 251.80625 146.899219 \n",
|
|
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" <g id=\"patch_6\">\n",
|
|
" <path d=\"M 56.50625 10.999219 \n",
|
|
"L 251.80625 10.999219 \n",
|
|
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" </g>\n",
|
|
" <defs>\n",
|
|
" <clipPath id=\"p0138888679\">\n",
|
|
" <rect x=\"56.50625\" y=\"10.999219\" width=\"195.3\" height=\"135.9\"/>\n",
|
|
" </clipPath>\n",
|
|
" </defs>\n",
|
|
"</svg>\n"
|
|
],
|
|
"text/plain": [
|
|
"<Figure size 252x180 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"trainer = torch.optim.Adadelta\n",
|
|
"d2l.train_concise_ch11(trainer, {'rho': 0.9}, data_iter)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ff4cd9a5",
|
|
"metadata": {
|
|
"origin_pos": 12
|
|
},
|
|
"source": [
|
|
"## 小结\n",
|
|
"\n",
|
|
"* Adadelta没有学习率参数。相反，它使用参数本身的变化率来调整学习率。\n",
|
|
"* Adadelta需要两个状态变量来存储梯度的二阶导数和参数的变化。\n",
|
|
"* Adadelta使用泄漏的平均值来保持对适当统计数据的运行估计。\n",
|
|
"\n",
|
|
"## 练习\n",
|
|
"\n",
|
|
"1. 调整$\\rho$的值，会发生什么？\n",
|
|
"1. 展示如何在不使用$\\mathbf{g}_t'$的情况下实现算法。为什么这是个好主意？\n",
|
|
"1. Adadelta真的是学习率为0吗？能找到Adadelta无法解决的优化问题吗？\n",
|
|
"1. 将Adadelta的收敛行为与AdaGrad和RMSProp进行比较。\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "074f9479",
|
|
"metadata": {
|
|
"origin_pos": 14,
|
|
"tab": [
|
|
"pytorch"
|
|
]
|
|
},
|
|
"source": [
|
|
"[Discussions](https://discuss.d2l.ai/t/5772)\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"language_info": {
|
|
"name": "python"
|
|
},
|
|
"required_libs": []
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
} |