1224 lines
45 KiB
Plaintext
1224 lines
45 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "7f91a115",
|
||
"metadata": {
|
||
"origin_pos": 0
|
||
},
|
||
"source": [
|
||
"# 微积分\n",
|
||
":label:`sec_calculus`\n",
|
||
"\n",
|
||
"在2500年前，古希腊人把一个多边形分成三角形，并把它们的面积相加，才找到计算多边形面积的方法。\n",
|
||
"为了求出曲线形状（比如圆）的面积，古希腊人在这样的形状上刻内接多边形。\n",
|
||
"如 :numref:`fig_circle_area`所示，内接多边形的等长边越多，就越接近圆。\n",
|
||
"这个过程也被称为*逼近法*（method of exhaustion）。\n",
|
||
"\n",
|
||
"\n",
|
||
":label:`fig_circle_area`\n",
|
||
"\n",
|
||
"事实上，逼近法就是*积分*（integral calculus）的起源。\n",
|
||
"2000多年后，微积分的另一支，*微分*（differential calculus）被发明出来。\n",
|
||
"在微分学最重要的应用是优化问题，即考虑如何把事情做到最好。\n",
|
||
"正如在 :numref:`subsec_norms_and_objectives`中讨论的那样，\n",
|
||
"这种问题在深度学习中是无处不在的。\n",
|
||
"\n",
|
||
"在深度学习中，我们“训练”模型，不断更新它们，使它们在看到越来越多的数据时变得越来越好。\n",
|
||
"通常情况下，变得更好意味着最小化一个*损失函数*（loss function），\n",
|
||
"即一个衡量“模型有多糟糕”这个问题的分数。\n",
|
||
"最终，我们真正关心的是生成一个模型，它能够在从未见过的数据上表现良好。\n",
|
||
"但“训练”模型只能将模型与我们实际能看到的数据相拟合。\n",
|
||
"因此，我们可以将拟合模型的任务分解为两个关键问题：\n",
|
||
"\n",
|
||
"* *优化*（optimization）：用模型拟合观测数据的过程；\n",
|
||
"* *泛化*（generalization）：数学原理和实践者的智慧，能够指导我们生成出有效性超出用于训练的数据集本身的模型。\n",
|
||
"\n",
|
||
"为了帮助读者在后面的章节中更好地理解优化问题和方法，\n",
|
||
"本节提供了一个非常简短的入门教程，帮助读者快速掌握深度学习中常用的微分知识。\n",
|
||
"\n",
|
||
"## 导数和微分\n",
|
||
"\n",
|
||
"我们首先讨论导数的计算，这是几乎所有深度学习优化算法的关键步骤。\n",
|
||
"在深度学习中，我们通常选择对于模型参数可微的损失函数。\n",
|
||
"简而言之，对于每个参数，\n",
|
||
"如果我们把这个参数*增加*或*减少*一个无穷小的量，可以知道损失会以多快的速度增加或减少，\n",
|
||
"\n",
|
||
"假设我们有一个函数$f: \\mathbb{R} \\rightarrow \\mathbb{R}$，其输入和输出都是标量。\n",
|
||
"(**如果$f$的*导数*存在，这个极限被定义为**)\n",
|
||
"\n",
|
||
"(**$$f'(x) = \\lim_{h \\rightarrow 0} \\frac{f(x+h) - f(x)}{h}.$$**)\n",
|
||
":eqlabel:`eq_derivative`\n",
|
||
"\n",
|
||
"如果$f'(a)$存在，则称$f$在$a$处是*可微*（differentiable）的。\n",
|
||
"如果$f$在一个区间内的每个数上都是可微的，则此函数在此区间中是可微的。\n",
|
||
"我们可以将 :eqref:`eq_derivative`中的导数$f'(x)$解释为$f(x)$相对于$x$的*瞬时*（instantaneous）变化率。\n",
|
||
"所谓的瞬时变化率是基于$x$中的变化$h$，且$h$接近$0$。\n",
|
||
"\n",
|
||
"为了更好地解释导数，让我们做一个实验。\n",
|
||
"(**定义$u=f(x)=3x^2-4x$**)如下：\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "02d617cb",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:01:25.065994Z",
|
||
"iopub.status.busy": "2023-08-18T07:01:25.065245Z",
|
||
"iopub.status.idle": "2023-08-18T07:01:27.381378Z",
|
||
"shell.execute_reply": "2023-08-18T07:01:27.380233Z"
|
||
},
|
||
"origin_pos": 2,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"%matplotlib inline\n",
|
||
"import numpy as np\n",
|
||
"from matplotlib_inline import backend_inline\n",
|
||
"from d2l import torch as d2l\n",
|
||
"\n",
|
||
"\n",
|
||
"def f(x):\n",
|
||
" return 3 * x ** 2 - 4 * x"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "60a0915b",
|
||
"metadata": {
|
||
"origin_pos": 5
|
||
},
|
||
"source": [
|
||
"[**通过令$x=1$并让$h$接近$0$，**] :eqref:`eq_derivative`中(**$\\frac{f(x+h)-f(x)}{h}$的数值结果接近$2$**)。\n",
|
||
"虽然这个实验不是一个数学证明，但稍后会看到，当$x=1$时，导数$u'$是$2$。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "39cf9942",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:01:27.387542Z",
|
||
"iopub.status.busy": "2023-08-18T07:01:27.386582Z",
|
||
"iopub.status.idle": "2023-08-18T07:01:27.394057Z",
|
||
"shell.execute_reply": "2023-08-18T07:01:27.393090Z"
|
||
},
|
||
"origin_pos": 6,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"h=0.10000, numerical limit=2.30000\n",
|
||
"h=0.01000, numerical limit=2.03000\n",
|
||
"h=0.00100, numerical limit=2.00300\n",
|
||
"h=0.00010, numerical limit=2.00030\n",
|
||
"h=0.00001, numerical limit=2.00003\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"def numerical_lim(f, x, h):\n",
|
||
" return (f(x + h) - f(x)) / h\n",
|
||
"\n",
|
||
"h = 0.1\n",
|
||
"for i in range(5):\n",
|
||
" print(f'h={h:.5f}, numerical limit={numerical_lim(f, 1, h):.5f}')\n",
|
||
" h *= 0.1"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ea011f86",
|
||
"metadata": {
|
||
"origin_pos": 7
|
||
},
|
||
"source": [
|
||
"让我们熟悉一下导数的几个等价符号。\n",
|
||
"给定$y=f(x)$，其中$x$和$y$分别是函数$f$的自变量和因变量。以下表达式是等价的：\n",
|
||
"\n",
|
||
"$$f'(x) = y' = \\frac{dy}{dx} = \\frac{df}{dx} = \\frac{d}{dx} f(x) = Df(x) = D_x f(x),$$\n",
|
||
"\n",
|
||
"其中符号$\\frac{d}{dx}$和$D$是*微分运算符*，表示*微分*操作。\n",
|
||
"我们可以使用以下规则来对常见函数求微分：\n",
|
||
"\n",
|
||
"* $DC = 0$（$C$是一个常数）\n",
|
||
"* $Dx^n = nx^{n-1}$（*幂律*（power rule），$n$是任意实数）\n",
|
||
"* $De^x = e^x$\n",
|
||
"* $D\\ln(x) = 1/x$\n",
|
||
"\n",
|
||
"为了微分一个由一些常见函数组成的函数，下面的一些法则方便使用。\n",
|
||
"假设函数$f$和$g$都是可微的，$C$是一个常数，则：\n",
|
||
"\n",
|
||
"*常数相乘法则*\n",
|
||
"$$\\frac{d}{dx} [Cf(x)] = C \\frac{d}{dx} f(x),$$\n",
|
||
"\n",
|
||
"*加法法则*\n",
|
||
"\n",
|
||
"$$\\frac{d}{dx} [f(x) + g(x)] = \\frac{d}{dx} f(x) + \\frac{d}{dx} g(x),$$\n",
|
||
"\n",
|
||
"*乘法法则*\n",
|
||
"\n",
|
||
"$$\\frac{d}{dx} [f(x)g(x)] = f(x) \\frac{d}{dx} [g(x)] + g(x) \\frac{d}{dx} [f(x)],$$\n",
|
||
"\n",
|
||
"*除法法则*\n",
|
||
"\n",
|
||
"$$\\frac{d}{dx} \\left[\\frac{f(x)}{g(x)}\\right] = \\frac{g(x) \\frac{d}{dx} [f(x)] - f(x) \\frac{d}{dx} [g(x)]}{[g(x)]^2}.$$\n",
|
||
"\n",
|
||
"现在我们可以应用上述几个法则来计算$u'=f'(x)=3\\frac{d}{dx}x^2-4\\frac{d}{dx}x=6x-4$。\n",
|
||
"令$x=1$，我们有$u'=2$：在这个实验中，数值结果接近$2$，\n",
|
||
"这一点得到了在本节前面的实验的支持。\n",
|
||
"当$x=1$时，此导数也是曲线$u=f(x)$切线的斜率。\n",
|
||
"\n",
|
||
"[**为了对导数的这种解释进行可视化，我们将使用`matplotlib`**]，\n",
|
||
"这是一个Python中流行的绘图库。\n",
|
||
"要配置`matplotlib`生成图形的属性，我们需要(**定义几个函数**)。\n",
|
||
"在下面，`use_svg_display`函数指定`matplotlib`软件包输出svg图表以获得更清晰的图像。\n",
|
||
"\n",
|
||
"注意，注释`#@save`是一个特殊的标记，会将对应的函数、类或语句保存在`d2l`包中。\n",
|
||
"因此，以后无须重新定义就可以直接调用它们（例如，`d2l.use_svg_display()`）。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "a0efe8c9",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:01:27.399278Z",
|
||
"iopub.status.busy": "2023-08-18T07:01:27.398487Z",
|
||
"iopub.status.idle": "2023-08-18T07:01:27.403514Z",
|
||
"shell.execute_reply": "2023-08-18T07:01:27.402414Z"
|
||
},
|
||
"origin_pos": 8,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def use_svg_display(): #@save\n",
|
||
" \"\"\"使用svg格式在Jupyter中显示绘图\"\"\"\n",
|
||
" backend_inline.set_matplotlib_formats('svg')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8b1650c6",
|
||
"metadata": {
|
||
"origin_pos": 9
|
||
},
|
||
"source": [
|
||
"我们定义`set_figsize`函数来设置图表大小。\n",
|
||
"注意，这里可以直接使用`d2l.plt`，因为导入语句\n",
|
||
"`from matplotlib import pyplot as plt`已标记为保存到`d2l`包中。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "acef7e22",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:01:27.408462Z",
|
||
"iopub.status.busy": "2023-08-18T07:01:27.407659Z",
|
||
"iopub.status.idle": "2023-08-18T07:01:27.414090Z",
|
||
"shell.execute_reply": "2023-08-18T07:01:27.412718Z"
|
||
},
|
||
"origin_pos": 10,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def set_figsize(figsize=(3.5, 2.5)): #@save\n",
|
||
" \"\"\"设置matplotlib的图表大小\"\"\"\n",
|
||
" use_svg_display()\n",
|
||
" d2l.plt.rcParams['figure.figsize'] = figsize"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "71a62720",
|
||
"metadata": {
|
||
"origin_pos": 11
|
||
},
|
||
"source": [
|
||
"下面的`set_axes`函数用于设置由`matplotlib`生成图表的轴的属性。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"id": "0ad890f8",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:01:27.419140Z",
|
||
"iopub.status.busy": "2023-08-18T07:01:27.418455Z",
|
||
"iopub.status.idle": "2023-08-18T07:01:27.426061Z",
|
||
"shell.execute_reply": "2023-08-18T07:01:27.424739Z"
|
||
},
|
||
"origin_pos": 12,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"#@save\n",
|
||
"def set_axes(axes, xlabel, ylabel, xlim, ylim, xscale, yscale, legend):\n",
|
||
" \"\"\"设置matplotlib的轴\"\"\"\n",
|
||
" axes.set_xlabel(xlabel)\n",
|
||
" axes.set_ylabel(ylabel)\n",
|
||
" axes.set_xscale(xscale)\n",
|
||
" axes.set_yscale(yscale)\n",
|
||
" axes.set_xlim(xlim)\n",
|
||
" axes.set_ylim(ylim)\n",
|
||
" if legend:\n",
|
||
" axes.legend(legend)\n",
|
||
" axes.grid()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "30e5a1f9",
|
||
"metadata": {
|
||
"origin_pos": 13
|
||
},
|
||
"source": [
|
||
"通过这三个用于图形配置的函数，定义一个`plot`函数来简洁地绘制多条曲线，\n",
|
||
"因为我们需要在整个书中可视化许多曲线。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "00c43fac",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:01:27.431229Z",
|
||
"iopub.status.busy": "2023-08-18T07:01:27.430462Z",
|
||
"iopub.status.idle": "2023-08-18T07:01:27.441418Z",
|
||
"shell.execute_reply": "2023-08-18T07:01:27.440390Z"
|
||
},
|
||
"origin_pos": 14,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"#@save\n",
|
||
"def plot(X, Y=None, xlabel=None, ylabel=None, legend=None, xlim=None,\n",
|
||
" ylim=None, xscale='linear', yscale='linear',\n",
|
||
" fmts=('-', 'm--', 'g-.', 'r:'), figsize=(3.5, 2.5), axes=None):\n",
|
||
" \"\"\"绘制数据点\"\"\"\n",
|
||
" if legend is None:\n",
|
||
" legend = []\n",
|
||
"\n",
|
||
" set_figsize(figsize)\n",
|
||
" axes = axes if axes else d2l.plt.gca()\n",
|
||
"\n",
|
||
" # 如果X有一个轴，输出True\n",
|
||
" def has_one_axis(X):\n",
|
||
" return (hasattr(X, \"ndim\") and X.ndim == 1 or isinstance(X, list)\n",
|
||
" and not hasattr(X[0], \"__len__\"))\n",
|
||
"\n",
|
||
" if has_one_axis(X):\n",
|
||
" X = [X]\n",
|
||
" if Y is None:\n",
|
||
" X, Y = [[]] * len(X), X\n",
|
||
" elif has_one_axis(Y):\n",
|
||
" Y = [Y]\n",
|
||
" if len(X) != len(Y):\n",
|
||
" X = X * len(Y)\n",
|
||
" axes.cla()\n",
|
||
" for x, y, fmt in zip(X, Y, fmts):\n",
|
||
" if len(x):\n",
|
||
" axes.plot(x, y, fmt)\n",
|
||
" else:\n",
|
||
" axes.plot(y, fmt)\n",
|
||
" set_axes(axes, xlabel, ylabel, xlim, ylim, xscale, yscale, legend)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "d9dbfa1f",
|
||
"metadata": {
|
||
"origin_pos": 15
|
||
},
|
||
"source": [
|
||
"现在我们可以[**绘制函数$u=f(x)$及其在$x=1$处的切线$y=2x-3$**]，\n",
|
||
"其中系数$2$是切线的斜率。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"id": "f09a2c12",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:01:27.445931Z",
|
||
"iopub.status.busy": "2023-08-18T07:01:27.445122Z",
|
||
"iopub.status.idle": "2023-08-18T07:01:27.699931Z",
|
||
"shell.execute_reply": "2023-08-18T07:01:27.698662Z"
|
||
},
|
||
"origin_pos": 16,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"image/svg+xml": [
|
||
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
|
||
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
|
||
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
|
||
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"243.529359pt\" height=\"180.65625pt\" viewBox=\"0 0 243.529359 180.65625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
|
||
" <metadata>\n",
|
||
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
|
||
" <cc:Work>\n",
|
||
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
|
||
" <dc:date>2023-08-18T07:01:27.649475</dc:date>\n",
|
||
" <dc:format>image/svg+xml</dc:format>\n",
|
||
" <dc:creator>\n",
|
||
" <cc:Agent>\n",
|
||
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
|
||
" </cc:Agent>\n",
|
||
" </dc:creator>\n",
|
||
" </cc:Work>\n",
|
||
" </rdf:RDF>\n",
|
||
" </metadata>\n",
|
||
" <defs>\n",
|
||
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
|
||
" </defs>\n",
|
||
" <g id=\"figure_1\">\n",
|
||
" <g id=\"patch_1\">\n",
|
||
" <path d=\"M 0 180.65625 \n",
|
||
"L 243.529359 180.65625 \n",
|
||
"L 243.529359 0 \n",
|
||
"L 0 0 \n",
|
||
"L 0 180.65625 \n",
|
||
"z\n",
|
||
"\" style=\"fill: none\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"axes_1\">\n",
|
||
" <g id=\"patch_2\">\n",
|
||
" <path d=\"M 40.603125 143.1 \n",
|
||
"L 235.903125 143.1 \n",
|
||
"L 235.903125 7.2 \n",
|
||
"L 40.603125 7.2 \n",
|
||
"z\n",
|
||
"\" style=\"fill: #ffffff\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"matplotlib.axis_1\">\n",
|
||
" <g id=\"xtick_1\">\n",
|
||
" <g id=\"line2d_1\">\n",
|
||
" <path d=\"M 49.480398 143.1 \n",
|
||
"L 49.480398 7.2 \n",
|
||
"\" clip-path=\"url(#pb34a262701)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_2\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"meab27cbf96\" d=\"M 0 0 \n",
|
||
"L 0 3.5 \n",
|
||
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </defs>\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#meab27cbf96\" x=\"49.480398\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_1\">\n",
|
||
" <!-- 0 -->\n",
|
||
" <g transform=\"translate(46.299148 157.698438)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
|
||
"Q 1547 4250 1301 3770 \n",
|
||
"Q 1056 3291 1056 2328 \n",
|
||
"Q 1056 1369 1301 889 \n",
|
||
"Q 1547 409 2034 409 \n",
|
||
"Q 2525 409 2770 889 \n",
|
||
"Q 3016 1369 3016 2328 \n",
|
||
"Q 3016 3291 2770 3770 \n",
|
||
"Q 2525 4250 2034 4250 \n",
|
||
"z\n",
|
||
"M 2034 4750 \n",
|
||
"Q 2819 4750 3233 4129 \n",
|
||
"Q 3647 3509 3647 2328 \n",
|
||
"Q 3647 1150 3233 529 \n",
|
||
"Q 2819 -91 2034 -91 \n",
|
||
"Q 1250 -91 836 529 \n",
|
||
"Q 422 1150 422 2328 \n",
|
||
"Q 422 3509 836 4129 \n",
|
||
"Q 1250 4750 2034 4750 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"xtick_2\">\n",
|
||
" <g id=\"line2d_3\">\n",
|
||
" <path d=\"M 110.702968 143.1 \n",
|
||
"L 110.702968 7.2 \n",
|
||
"\" clip-path=\"url(#pb34a262701)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_4\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#meab27cbf96\" x=\"110.702968\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_2\">\n",
|
||
" <!-- 1 -->\n",
|
||
" <g transform=\"translate(107.521718 157.698438)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
|
||
"L 1825 531 \n",
|
||
"L 1825 4091 \n",
|
||
"L 703 3866 \n",
|
||
"L 703 4441 \n",
|
||
"L 1819 4666 \n",
|
||
"L 2450 4666 \n",
|
||
"L 2450 531 \n",
|
||
"L 3481 531 \n",
|
||
"L 3481 0 \n",
|
||
"L 794 0 \n",
|
||
"L 794 531 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"xtick_3\">\n",
|
||
" <g id=\"line2d_5\">\n",
|
||
" <path d=\"M 171.925539 143.1 \n",
|
||
"L 171.925539 7.2 \n",
|
||
"\" clip-path=\"url(#pb34a262701)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_6\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#meab27cbf96\" x=\"171.925539\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_3\">\n",
|
||
" <!-- 2 -->\n",
|
||
" <g transform=\"translate(168.744289 157.698438)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
|
||
"L 3431 531 \n",
|
||
"L 3431 0 \n",
|
||
"L 469 0 \n",
|
||
"L 469 531 \n",
|
||
"Q 828 903 1448 1529 \n",
|
||
"Q 2069 2156 2228 2338 \n",
|
||
"Q 2531 2678 2651 2914 \n",
|
||
"Q 2772 3150 2772 3378 \n",
|
||
"Q 2772 3750 2511 3984 \n",
|
||
"Q 2250 4219 1831 4219 \n",
|
||
"Q 1534 4219 1204 4116 \n",
|
||
"Q 875 4013 500 3803 \n",
|
||
"L 500 4441 \n",
|
||
"Q 881 4594 1212 4672 \n",
|
||
"Q 1544 4750 1819 4750 \n",
|
||
"Q 2544 4750 2975 4387 \n",
|
||
"Q 3406 4025 3406 3419 \n",
|
||
"Q 3406 3131 3298 2873 \n",
|
||
"Q 3191 2616 2906 2266 \n",
|
||
"Q 2828 2175 2409 1742 \n",
|
||
"Q 1991 1309 1228 531 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"xtick_4\">\n",
|
||
" <g id=\"line2d_7\">\n",
|
||
" <path d=\"M 233.148109 143.1 \n",
|
||
"L 233.148109 7.2 \n",
|
||
"\" clip-path=\"url(#pb34a262701)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_8\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#meab27cbf96\" x=\"233.148109\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_4\">\n",
|
||
" <!-- 3 -->\n",
|
||
" <g transform=\"translate(229.966859 157.698438)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
|
||
"Q 3050 2419 3304 2112 \n",
|
||
"Q 3559 1806 3559 1356 \n",
|
||
"Q 3559 666 3084 287 \n",
|
||
"Q 2609 -91 1734 -91 \n",
|
||
"Q 1441 -91 1130 -33 \n",
|
||
"Q 819 25 488 141 \n",
|
||
"L 488 750 \n",
|
||
"Q 750 597 1062 519 \n",
|
||
"Q 1375 441 1716 441 \n",
|
||
"Q 2309 441 2620 675 \n",
|
||
"Q 2931 909 2931 1356 \n",
|
||
"Q 2931 1769 2642 2001 \n",
|
||
"Q 2353 2234 1838 2234 \n",
|
||
"L 1294 2234 \n",
|
||
"L 1294 2753 \n",
|
||
"L 1863 2753 \n",
|
||
"Q 2328 2753 2575 2939 \n",
|
||
"Q 2822 3125 2822 3475 \n",
|
||
"Q 2822 3834 2567 4026 \n",
|
||
"Q 2313 4219 1838 4219 \n",
|
||
"Q 1578 4219 1281 4162 \n",
|
||
"Q 984 4106 628 3988 \n",
|
||
"L 628 4550 \n",
|
||
"Q 988 4650 1302 4700 \n",
|
||
"Q 1616 4750 1894 4750 \n",
|
||
"Q 2613 4750 3031 4423 \n",
|
||
"Q 3450 4097 3450 3541 \n",
|
||
"Q 3450 3153 3228 2886 \n",
|
||
"Q 3006 2619 2597 2516 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-33\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_5\">\n",
|
||
" <!-- x -->\n",
|
||
" <g transform=\"translate(135.29375 171.376563)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
|
||
"L 2247 1797 \n",
|
||
"L 3578 0 \n",
|
||
"L 2900 0 \n",
|
||
"L 1881 1375 \n",
|
||
"L 863 0 \n",
|
||
"L 184 0 \n",
|
||
"L 1544 1831 \n",
|
||
"L 300 3500 \n",
|
||
"L 978 3500 \n",
|
||
"L 1906 2253 \n",
|
||
"L 2834 3500 \n",
|
||
"L 3513 3500 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"matplotlib.axis_2\">\n",
|
||
" <g id=\"ytick_1\">\n",
|
||
" <g id=\"line2d_9\">\n",
|
||
" <path d=\"M 40.603125 114.635514 \n",
|
||
"L 235.903125 114.635514 \n",
|
||
"\" clip-path=\"url(#pb34a262701)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_10\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"m629c214856\" d=\"M 0 0 \n",
|
||
"L -3.5 0 \n",
|
||
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </defs>\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m629c214856\" x=\"40.603125\" y=\"114.635514\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_6\">\n",
|
||
" <!-- 0 -->\n",
|
||
" <g transform=\"translate(27.240625 118.434732)scale(0.1 -0.1)\">\n",
|
||
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"ytick_2\">\n",
|
||
" <g id=\"line2d_11\">\n",
|
||
" <path d=\"M 40.603125 77.490157 \n",
|
||
"L 235.903125 77.490157 \n",
|
||
"\" clip-path=\"url(#pb34a262701)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_12\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m629c214856\" x=\"40.603125\" y=\"77.490157\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_7\">\n",
|
||
" <!-- 5 -->\n",
|
||
" <g transform=\"translate(27.240625 81.289376)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
|
||
"L 3169 4666 \n",
|
||
"L 3169 4134 \n",
|
||
"L 1269 4134 \n",
|
||
"L 1269 2991 \n",
|
||
"Q 1406 3038 1543 3061 \n",
|
||
"Q 1681 3084 1819 3084 \n",
|
||
"Q 2600 3084 3056 2656 \n",
|
||
"Q 3513 2228 3513 1497 \n",
|
||
"Q 3513 744 3044 326 \n",
|
||
"Q 2575 -91 1722 -91 \n",
|
||
"Q 1428 -91 1123 -41 \n",
|
||
"Q 819 9 494 109 \n",
|
||
"L 494 744 \n",
|
||
"Q 775 591 1075 516 \n",
|
||
"Q 1375 441 1709 441 \n",
|
||
"Q 2250 441 2565 725 \n",
|
||
"Q 2881 1009 2881 1497 \n",
|
||
"Q 2881 1984 2565 2268 \n",
|
||
"Q 2250 2553 1709 2553 \n",
|
||
"Q 1456 2553 1204 2497 \n",
|
||
"Q 953 2441 691 2322 \n",
|
||
"L 691 4666 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-35\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"ytick_3\">\n",
|
||
" <g id=\"line2d_13\">\n",
|
||
" <path d=\"M 40.603125 40.344801 \n",
|
||
"L 235.903125 40.344801 \n",
|
||
"\" clip-path=\"url(#pb34a262701)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_14\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m629c214856\" x=\"40.603125\" y=\"40.344801\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_8\">\n",
|
||
" <!-- 10 -->\n",
|
||
" <g transform=\"translate(20.878125 44.14402)scale(0.1 -0.1)\">\n",
|
||
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_9\">\n",
|
||
" <!-- f(x) -->\n",
|
||
" <g transform=\"translate(14.798437 83.771094)rotate(-90)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
|
||
"L 2375 4384 \n",
|
||
"L 1825 4384 \n",
|
||
"Q 1516 4384 1395 4259 \n",
|
||
"Q 1275 4134 1275 3809 \n",
|
||
"L 1275 3500 \n",
|
||
"L 2222 3500 \n",
|
||
"L 2222 3053 \n",
|
||
"L 1275 3053 \n",
|
||
"L 1275 0 \n",
|
||
"L 697 0 \n",
|
||
"L 697 3053 \n",
|
||
"L 147 3053 \n",
|
||
"L 147 3500 \n",
|
||
"L 697 3500 \n",
|
||
"L 697 3744 \n",
|
||
"Q 697 4328 969 4595 \n",
|
||
"Q 1241 4863 1831 4863 \n",
|
||
"L 2375 4863 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
|
||
"Q 1566 4138 1362 3434 \n",
|
||
"Q 1159 2731 1159 2009 \n",
|
||
"Q 1159 1288 1364 580 \n",
|
||
"Q 1569 -128 1984 -844 \n",
|
||
"L 1484 -844 \n",
|
||
"Q 1016 -109 783 600 \n",
|
||
"Q 550 1309 550 2009 \n",
|
||
"Q 550 2706 781 3412 \n",
|
||
"Q 1013 4119 1484 4856 \n",
|
||
"L 1984 4856 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
|
||
"L 1013 4856 \n",
|
||
"Q 1481 4119 1714 3412 \n",
|
||
"Q 1947 2706 1947 2009 \n",
|
||
"Q 1947 1309 1714 600 \n",
|
||
"Q 1481 -109 1013 -844 \n",
|
||
"L 513 -844 \n",
|
||
"Q 928 -128 1133 580 \n",
|
||
"Q 1338 1288 1338 2009 \n",
|
||
"Q 1338 2731 1133 3434 \n",
|
||
"Q 928 4138 513 4856 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-66\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_15\">\n",
|
||
" <path d=\"M 49.480398 114.635514 \n",
|
||
"L 55.602655 117.38427 \n",
|
||
"L 61.724912 119.687282 \n",
|
||
"L 67.847169 121.54455 \n",
|
||
"L 73.969426 122.956073 \n",
|
||
"L 80.091683 123.921853 \n",
|
||
"L 86.21394 124.441888 \n",
|
||
"L 92.336197 124.516178 \n",
|
||
"L 98.458454 124.144725 \n",
|
||
"L 104.580711 123.327527 \n",
|
||
"L 110.702968 122.064585 \n",
|
||
"L 116.825225 120.355898 \n",
|
||
"L 122.947482 118.201468 \n",
|
||
"L 129.069739 115.601293 \n",
|
||
"L 135.191996 112.555374 \n",
|
||
"L 141.314254 109.06371 \n",
|
||
"L 147.436511 105.126302 \n",
|
||
"L 153.558768 100.74315 \n",
|
||
"L 159.681025 95.914254 \n",
|
||
"L 165.803282 90.639614 \n",
|
||
"L 171.925539 84.919229 \n",
|
||
"L 178.047796 78.7531 \n",
|
||
"L 184.170053 72.141226 \n",
|
||
"L 190.29231 65.083608 \n",
|
||
"L 196.414567 57.580247 \n",
|
||
"L 202.536824 49.63114 \n",
|
||
"L 208.659081 41.23629 \n",
|
||
"L 214.781338 32.395695 \n",
|
||
"L 220.903595 23.109356 \n",
|
||
"L 227.025852 13.377273 \n",
|
||
"\" clip-path=\"url(#pb34a262701)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_16\">\n",
|
||
" <path d=\"M 49.480398 136.922727 \n",
|
||
"L 55.602655 135.436913 \n",
|
||
"L 61.724912 133.951099 \n",
|
||
"L 67.847169 132.465285 \n",
|
||
"L 73.969426 130.97947 \n",
|
||
"L 80.091683 129.493656 \n",
|
||
"L 86.21394 128.007842 \n",
|
||
"L 92.336197 126.522028 \n",
|
||
"L 98.458454 125.036213 \n",
|
||
"L 104.580711 123.550399 \n",
|
||
"L 110.702968 122.064585 \n",
|
||
"L 116.825225 120.578771 \n",
|
||
"L 122.947482 119.092956 \n",
|
||
"L 129.069739 117.607142 \n",
|
||
"L 135.191996 116.121328 \n",
|
||
"L 141.314254 114.635514 \n",
|
||
"L 147.436511 113.149699 \n",
|
||
"L 153.558768 111.663885 \n",
|
||
"L 159.681025 110.178071 \n",
|
||
"L 165.803282 108.692257 \n",
|
||
"L 171.925539 107.206442 \n",
|
||
"L 178.047796 105.720628 \n",
|
||
"L 184.170053 104.234814 \n",
|
||
"L 190.29231 102.749 \n",
|
||
"L 196.414567 101.263185 \n",
|
||
"L 202.536824 99.777371 \n",
|
||
"L 208.659081 98.291557 \n",
|
||
"L 214.781338 96.805743 \n",
|
||
"L 220.903595 95.319928 \n",
|
||
"L 227.025852 93.834114 \n",
|
||
"\" clip-path=\"url(#pb34a262701)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"patch_3\">\n",
|
||
" <path d=\"M 40.603125 143.1 \n",
|
||
"L 40.603125 7.2 \n",
|
||
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"patch_4\">\n",
|
||
" <path d=\"M 235.903125 143.1 \n",
|
||
"L 235.903125 7.2 \n",
|
||
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"patch_5\">\n",
|
||
" <path d=\"M 40.603125 143.1 \n",
|
||
"L 235.903125 143.1 \n",
|
||
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"patch_6\">\n",
|
||
" <path d=\"M 40.603125 7.2 \n",
|
||
"L 235.903125 7.2 \n",
|
||
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"legend_1\">\n",
|
||
" <g id=\"patch_7\">\n",
|
||
" <path d=\"M 47.603125 44.55625 \n",
|
||
"L 172.153125 44.55625 \n",
|
||
"Q 174.153125 44.55625 174.153125 42.55625 \n",
|
||
"L 174.153125 14.2 \n",
|
||
"Q 174.153125 12.2 172.153125 12.2 \n",
|
||
"L 47.603125 12.2 \n",
|
||
"Q 45.603125 12.2 45.603125 14.2 \n",
|
||
"L 45.603125 42.55625 \n",
|
||
"Q 45.603125 44.55625 47.603125 44.55625 \n",
|
||
"z\n",
|
||
"\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_17\">\n",
|
||
" <path d=\"M 49.603125 20.298437 \n",
|
||
"L 59.603125 20.298437 \n",
|
||
"L 69.603125 20.298437 \n",
|
||
"\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_10\">\n",
|
||
" <!-- f(x) -->\n",
|
||
" <g transform=\"translate(77.603125 23.798437)scale(0.1 -0.1)\">\n",
|
||
" <use xlink:href=\"#DejaVuSans-66\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_18\">\n",
|
||
" <path d=\"M 49.603125 34.976562 \n",
|
||
"L 59.603125 34.976562 \n",
|
||
"L 69.603125 34.976562 \n",
|
||
"\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_11\">\n",
|
||
" <!-- Tangent line (x=1) -->\n",
|
||
" <g transform=\"translate(77.603125 38.476562)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-54\" d=\"M -19 4666 \n",
|
||
"L 3928 4666 \n",
|
||
"L 3928 4134 \n",
|
||
"L 2272 4134 \n",
|
||
"L 2272 0 \n",
|
||
"L 1638 0 \n",
|
||
"L 1638 4134 \n",
|
||
"L -19 4134 \n",
|
||
"L -19 4666 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
|
||
"Q 1497 1759 1228 1600 \n",
|
||
"Q 959 1441 959 1056 \n",
|
||
"Q 959 750 1161 570 \n",
|
||
"Q 1363 391 1709 391 \n",
|
||
"Q 2188 391 2477 730 \n",
|
||
"Q 2766 1069 2766 1631 \n",
|
||
"L 2766 1759 \n",
|
||
"L 2194 1759 \n",
|
||
"z\n",
|
||
"M 3341 1997 \n",
|
||
"L 3341 0 \n",
|
||
"L 2766 0 \n",
|
||
"L 2766 531 \n",
|
||
"Q 2569 213 2275 61 \n",
|
||
"Q 1981 -91 1556 -91 \n",
|
||
"Q 1019 -91 701 211 \n",
|
||
"Q 384 513 384 1019 \n",
|
||
"Q 384 1609 779 1909 \n",
|
||
"Q 1175 2209 1959 2209 \n",
|
||
"L 2766 2209 \n",
|
||
"L 2766 2266 \n",
|
||
"Q 2766 2663 2505 2880 \n",
|
||
"Q 2244 3097 1772 3097 \n",
|
||
"Q 1472 3097 1187 3025 \n",
|
||
"Q 903 2953 641 2809 \n",
|
||
"L 641 3341 \n",
|
||
"Q 956 3463 1253 3523 \n",
|
||
"Q 1550 3584 1831 3584 \n",
|
||
"Q 2591 3584 2966 3190 \n",
|
||
"Q 3341 2797 3341 1997 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
|
||
"L 3513 0 \n",
|
||
"L 2938 0 \n",
|
||
"L 2938 2094 \n",
|
||
"Q 2938 2591 2744 2837 \n",
|
||
"Q 2550 3084 2163 3084 \n",
|
||
"Q 1697 3084 1428 2787 \n",
|
||
"Q 1159 2491 1159 1978 \n",
|
||
"L 1159 0 \n",
|
||
"L 581 0 \n",
|
||
"L 581 3500 \n",
|
||
"L 1159 3500 \n",
|
||
"L 1159 2956 \n",
|
||
"Q 1366 3272 1645 3428 \n",
|
||
"Q 1925 3584 2291 3584 \n",
|
||
"Q 2894 3584 3203 3211 \n",
|
||
"Q 3513 2838 3513 2113 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-67\" d=\"M 2906 1791 \n",
|
||
"Q 2906 2416 2648 2759 \n",
|
||
"Q 2391 3103 1925 3103 \n",
|
||
"Q 1463 3103 1205 2759 \n",
|
||
"Q 947 2416 947 1791 \n",
|
||
"Q 947 1169 1205 825 \n",
|
||
"Q 1463 481 1925 481 \n",
|
||
"Q 2391 481 2648 825 \n",
|
||
"Q 2906 1169 2906 1791 \n",
|
||
"z\n",
|
||
"M 3481 434 \n",
|
||
"Q 3481 -459 3084 -895 \n",
|
||
"Q 2688 -1331 1869 -1331 \n",
|
||
"Q 1566 -1331 1297 -1286 \n",
|
||
"Q 1028 -1241 775 -1147 \n",
|
||
"L 775 -588 \n",
|
||
"Q 1028 -725 1275 -790 \n",
|
||
"Q 1522 -856 1778 -856 \n",
|
||
"Q 2344 -856 2625 -561 \n",
|
||
"Q 2906 -266 2906 331 \n",
|
||
"L 2906 616 \n",
|
||
"Q 2728 306 2450 153 \n",
|
||
"Q 2172 0 1784 0 \n",
|
||
"Q 1141 0 747 490 \n",
|
||
"Q 353 981 353 1791 \n",
|
||
"Q 353 2603 747 3093 \n",
|
||
"Q 1141 3584 1784 3584 \n",
|
||
"Q 2172 3584 2450 3431 \n",
|
||
"Q 2728 3278 2906 2969 \n",
|
||
"L 2906 3500 \n",
|
||
"L 3481 3500 \n",
|
||
"L 3481 434 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
|
||
"L 3597 1613 \n",
|
||
"L 953 1613 \n",
|
||
"Q 991 1019 1311 708 \n",
|
||
"Q 1631 397 2203 397 \n",
|
||
"Q 2534 397 2845 478 \n",
|
||
"Q 3156 559 3463 722 \n",
|
||
"L 3463 178 \n",
|
||
"Q 3153 47 2828 -22 \n",
|
||
"Q 2503 -91 2169 -91 \n",
|
||
"Q 1331 -91 842 396 \n",
|
||
"Q 353 884 353 1716 \n",
|
||
"Q 353 2575 817 3079 \n",
|
||
"Q 1281 3584 2069 3584 \n",
|
||
"Q 2775 3584 3186 3129 \n",
|
||
"Q 3597 2675 3597 1894 \n",
|
||
"z\n",
|
||
"M 3022 2063 \n",
|
||
"Q 3016 2534 2758 2815 \n",
|
||
"Q 2500 3097 2075 3097 \n",
|
||
"Q 1594 3097 1305 2825 \n",
|
||
"Q 1016 2553 972 2059 \n",
|
||
"L 3022 2063 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
|
||
"L 1172 3500 \n",
|
||
"L 2356 3500 \n",
|
||
"L 2356 3053 \n",
|
||
"L 1172 3053 \n",
|
||
"L 1172 1153 \n",
|
||
"Q 1172 725 1289 603 \n",
|
||
"Q 1406 481 1766 481 \n",
|
||
"L 2356 481 \n",
|
||
"L 2356 0 \n",
|
||
"L 1766 0 \n",
|
||
"Q 1100 0 847 248 \n",
|
||
"Q 594 497 594 1153 \n",
|
||
"L 594 3053 \n",
|
||
"L 172 3053 \n",
|
||
"L 172 3500 \n",
|
||
"L 594 3500 \n",
|
||
"L 594 4494 \n",
|
||
"L 1172 4494 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
|
||
"L 1178 4863 \n",
|
||
"L 1178 0 \n",
|
||
"L 603 0 \n",
|
||
"L 603 4863 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
|
||
"L 1178 3500 \n",
|
||
"L 1178 0 \n",
|
||
"L 603 0 \n",
|
||
"L 603 3500 \n",
|
||
"z\n",
|
||
"M 603 4863 \n",
|
||
"L 1178 4863 \n",
|
||
"L 1178 4134 \n",
|
||
"L 603 4134 \n",
|
||
"L 603 4863 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-3d\" d=\"M 678 2906 \n",
|
||
"L 4684 2906 \n",
|
||
"L 4684 2381 \n",
|
||
"L 678 2381 \n",
|
||
"L 678 2906 \n",
|
||
"z\n",
|
||
"M 678 1631 \n",
|
||
"L 4684 1631 \n",
|
||
"L 4684 1100 \n",
|
||
"L 678 1100 \n",
|
||
"L 678 1631 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-54\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-61\" x=\"44.583984\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-6e\" x=\"105.863281\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-67\" x=\"169.242188\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-65\" x=\"232.71875\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-6e\" x=\"294.242188\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-74\" x=\"357.621094\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-20\" x=\"396.830078\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-6c\" x=\"428.617188\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-69\" x=\"456.400391\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-6e\" x=\"484.183594\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-65\" x=\"547.5625\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-20\" x=\"609.085938\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-28\" x=\"640.873047\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-78\" x=\"679.886719\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-3d\" x=\"739.066406\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-31\" x=\"822.855469\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-29\" x=\"886.478516\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <defs>\n",
|
||
" <clipPath id=\"pb34a262701\">\n",
|
||
" <rect x=\"40.603125\" y=\"7.2\" width=\"195.3\" height=\"135.9\"/>\n",
|
||
" </clipPath>\n",
|
||
" </defs>\n",
|
||
"</svg>\n"
|
||
],
|
||
"text/plain": [
|
||
"<Figure size 252x180 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"needs_background": "light"
|
||
},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"x = np.arange(0, 3, 0.1)\n",
|
||
"plot(x, [f(x), 2 * x - 3], 'x', 'f(x)', legend=['f(x)', 'Tangent line (x=1)'])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c292a783",
|
||
"metadata": {
|
||
"origin_pos": 17
|
||
},
|
||
"source": [
|
||
"## 偏导数\n",
|
||
"\n",
|
||
"到目前为止，我们只讨论了仅含一个变量的函数的微分。\n",
|
||
"在深度学习中，函数通常依赖于许多变量。\n",
|
||
"因此，我们需要将微分的思想推广到*多元函数*（multivariate function）上。\n",
|
||
"\n",
|
||
"设$y = f(x_1, x_2, \\ldots, x_n)$是一个具有$n$个变量的函数。\n",
|
||
"$y$关于第$i$个参数$x_i$的*偏导数*（partial derivative）为：\n",
|
||
"\n",
|
||
"$$ \\frac{\\partial y}{\\partial x_i} = \\lim_{h \\rightarrow 0} \\frac{f(x_1, \\ldots, x_{i-1}, x_i+h, x_{i+1}, \\ldots, x_n) - f(x_1, \\ldots, x_i, \\ldots, x_n)}{h}.$$\n",
|
||
"\n",
|
||
"为了计算$\\frac{\\partial y}{\\partial x_i}$，\n",
|
||
"我们可以简单地将$x_1, \\ldots, x_{i-1}, x_{i+1}, \\ldots, x_n$看作常数，\n",
|
||
"并计算$y$关于$x_i$的导数。\n",
|
||
"对于偏导数的表示，以下是等价的：\n",
|
||
"\n",
|
||
"$$\\frac{\\partial y}{\\partial x_i} = \\frac{\\partial f}{\\partial x_i} = f_{x_i} = f_i = D_i f = D_{x_i} f.$$\n",
|
||
"\n",
|
||
"## 梯度\n",
|
||
":label:`subsec_calculus-grad`\n",
|
||
"\n",
|
||
"我们可以连结一个多元函数对其所有变量的偏导数，以得到该函数的*梯度*（gradient）向量。\n",
|
||
"具体而言，设函数$f:\\mathbb{R}^n\\rightarrow\\mathbb{R}$的输入是\n",
|
||
"一个$n$维向量$\\mathbf{x}=[x_1,x_2,\\ldots,x_n]^\\top$，并且输出是一个标量。\n",
|
||
"函数$f(\\mathbf{x})$相对于$\\mathbf{x}$的梯度是一个包含$n$个偏导数的向量:\n",
|
||
"\n",
|
||
"$$\\nabla_{\\mathbf{x}} f(\\mathbf{x}) = \\bigg[\\frac{\\partial f(\\mathbf{x})}{\\partial x_1}, \\frac{\\partial f(\\mathbf{x})}{\\partial x_2}, \\ldots, \\frac{\\partial f(\\mathbf{x})}{\\partial x_n}\\bigg]^\\top,$$\n",
|
||
"\n",
|
||
"其中$\\nabla_{\\mathbf{x}} f(\\mathbf{x})$通常在没有歧义时被$\\nabla f(\\mathbf{x})$取代。\n",
|
||
"\n",
|
||
"假设$\\mathbf{x}$为$n$维向量，在微分多元函数时经常使用以下规则:\n",
|
||
"\n",
|
||
"* 对于所有$\\mathbf{A} \\in \\mathbb{R}^{m \\times n}$，都有$\\nabla_{\\mathbf{x}} \\mathbf{A} \\mathbf{x} = \\mathbf{A}^\\top$\n",
|
||
"* 对于所有$\\mathbf{A} \\in \\mathbb{R}^{n \\times m}$，都有$\\nabla_{\\mathbf{x}} \\mathbf{x}^\\top \\mathbf{A} = \\mathbf{A}$\n",
|
||
"* 对于所有$\\mathbf{A} \\in \\mathbb{R}^{n \\times n}$，都有$\\nabla_{\\mathbf{x}} \\mathbf{x}^\\top \\mathbf{A} \\mathbf{x} = (\\mathbf{A} + \\mathbf{A}^\\top)\\mathbf{x}$\n",
|
||
"* $\\nabla_{\\mathbf{x}} \\|\\mathbf{x} \\|^2 = \\nabla_{\\mathbf{x}} \\mathbf{x}^\\top \\mathbf{x} = 2\\mathbf{x}$\n",
|
||
"\n",
|
||
"同样，对于任何矩阵$\\mathbf{X}$，都有$\\nabla_{\\mathbf{X}} \\|\\mathbf{X} \\|_F^2 = 2\\mathbf{X}$。\n",
|
||
"正如我们之后将看到的，梯度对于设计深度学习中的优化算法有很大用处。\n",
|
||
"\n",
|
||
"## 链式法则\n",
|
||
"\n",
|
||
"然而，上面方法可能很难找到梯度。\n",
|
||
"这是因为在深度学习中，多元函数通常是*复合*（composite）的，\n",
|
||
"所以难以应用上述任何规则来微分这些函数。\n",
|
||
"幸运的是，链式法则可以被用来微分复合函数。\n",
|
||
"\n",
|
||
"让我们先考虑单变量函数。假设函数$y=f(u)$和$u=g(x)$都是可微的，根据链式法则：\n",
|
||
"\n",
|
||
"$$\\frac{dy}{dx} = \\frac{dy}{du} \\frac{du}{dx}.$$\n",
|
||
"\n",
|
||
"现在考虑一个更一般的场景，即函数具有任意数量的变量的情况。\n",
|
||
"假设可微分函数$y$有变量$u_1, u_2, \\ldots, u_m$，其中每个可微分函数$u_i$都有变量$x_1, x_2, \\ldots, x_n$。\n",
|
||
"注意，$y$是$x_1, x_2， \\ldots, x_n$的函数。\n",
|
||
"对于任意$i = 1, 2, \\ldots, n$，链式法则给出：\n",
|
||
"\n",
|
||
"$$\\frac{\\partial y}{\\partial x_i} = \\frac{\\partial y}{\\partial u_1} \\frac{\\partial u_1}{\\partial x_i} + \\frac{\\partial y}{\\partial u_2} \\frac{\\partial u_2}{\\partial x_i} + \\cdots + \\frac{\\partial y}{\\partial u_m} \\frac{\\partial u_m}{\\partial x_i}$$\n",
|
||
"\n",
|
||
"## 小结\n",
|
||
"\n",
|
||
"* 微分和积分是微积分的两个分支，前者可以应用于深度学习中的优化问题。\n",
|
||
"* 导数可以被解释为函数相对于其变量的瞬时变化率，它也是函数曲线的切线的斜率。\n",
|
||
"* 梯度是一个向量，其分量是多变量函数相对于其所有变量的偏导数。\n",
|
||
"* 链式法则可以用来微分复合函数。\n",
|
||
"\n",
|
||
"## 练习\n",
|
||
"\n",
|
||
"1. 绘制函数$y = f(x) = x^3 - \\frac{1}{x}$和其在$x = 1$处切线的图像。\n",
|
||
"1. 求函数$f(\\mathbf{x}) = 3x_1^2 + 5e^{x_2}$的梯度。\n",
|
||
"1. 函数$f(\\mathbf{x}) = \\|\\mathbf{x}\\|_2$的梯度是什么？\n",
|
||
"1. 尝试写出函数$u = f(x, y, z)$，其中$x = x(a, b)$，$y = y(a, b)$，$z = z(a, b)$的链式法则。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "fed29a76",
|
||
"metadata": {
|
||
"origin_pos": 19,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"source": [
|
||
"[Discussions](https://discuss.d2l.ai/t/1756)\n"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"language_info": {
|
||
"name": "python"
|
||
},
|
||
"required_libs": []
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
} |