1738 lines
72 KiB
Plaintext
1738 lines
72 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "d128f467",
|
||
"metadata": {
|
||
"origin_pos": 0
|
||
},
|
||
"source": [
|
||
"# 线性回归\n",
|
||
":label:`sec_linear_regression`\n",
|
||
"\n",
|
||
"*回归*（regression）是能为一个或多个自变量与因变量之间关系建模的一类方法。\n",
|
||
"在自然科学和社会科学领域，回归经常用来表示输入和输出之间的关系。\n",
|
||
"\n",
|
||
"在机器学习领域中的大多数任务通常都与*预测*（prediction）有关。\n",
|
||
"当我们想预测一个数值时，就会涉及到回归问题。\n",
|
||
"常见的例子包括：预测价格（房屋、股票等）、预测住院时间（针对住院病人等）、\n",
|
||
"预测需求（零售销量等）。\n",
|
||
"但不是所有的*预测*都是回归问题。\n",
|
||
"在后面的章节中，我们将介绍分类问题。分类问题的目标是预测数据属于一组类别中的哪一个。\n",
|
||
"\n",
|
||
"## 线性回归的基本元素\n",
|
||
"\n",
|
||
"*线性回归*（linear regression）可以追溯到19世纪初，\n",
|
||
"它在回归的各种标准工具中最简单而且最流行。\n",
|
||
"线性回归基于几个简单的假设：\n",
|
||
"首先，假设自变量$\\mathbf{x}$和因变量$y$之间的关系是线性的，\n",
|
||
"即$y$可以表示为$\\mathbf{x}$中元素的加权和，这里通常允许包含观测值的一些噪声；\n",
|
||
"其次，我们假设任何噪声都比较正常，如噪声遵循正态分布。\n",
|
||
"\n",
|
||
"为了解释*线性回归*，我们举一个实际的例子：\n",
|
||
"我们希望根据房屋的面积（平方英尺）和房龄（年）来估算房屋价格（美元）。\n",
|
||
"为了开发一个能预测房价的模型，我们需要收集一个真实的数据集。\n",
|
||
"这个数据集包括了房屋的销售价格、面积和房龄。\n",
|
||
"在机器学习的术语中，该数据集称为*训练数据集*（training data set）\n",
|
||
"或*训练集*（training set）。\n",
|
||
"每行数据（比如一次房屋交易相对应的数据）称为*样本*（sample），\n",
|
||
"也可以称为*数据点*（data point）或*数据样本*（data instance）。\n",
|
||
"我们把试图预测的目标（比如预测房屋价格）称为*标签*（label）或*目标*（target）。\n",
|
||
"预测所依据的自变量（面积和房龄）称为*特征*（feature）或*协变量*（covariate）。\n",
|
||
"\n",
|
||
"通常，我们使用$n$来表示数据集中的样本数。\n",
|
||
"对索引为$i$的样本，其输入表示为$\\mathbf{x}^{(i)} = [x_1^{(i)}, x_2^{(i)}]^\\top$，\n",
|
||
"其对应的标签是$y^{(i)}$。\n",
|
||
"\n",
|
||
"### 线性模型\n",
|
||
":label:`subsec_linear_model`\n",
|
||
"\n",
|
||
"线性假设是指目标（房屋价格）可以表示为特征（面积和房龄）的加权和，如下面的式子：\n",
|
||
"\n",
|
||
"$$\\mathrm{price} = w_{\\mathrm{area}} \\cdot \\mathrm{area} + w_{\\mathrm{age}} \\cdot \\mathrm{age} + b.$$\n",
|
||
":eqlabel:`eq_price-area`\n",
|
||
"\n",
|
||
" :eqref:`eq_price-area`中的$w_{\\mathrm{area}}$和$w_{\\mathrm{age}}$\n",
|
||
"称为*权重*（weight），权重决定了每个特征对我们预测值的影响。\n",
|
||
"$b$称为*偏置*（bias）、*偏移量*（offset）或*截距*（intercept）。\n",
|
||
"偏置是指当所有特征都取值为0时，预测值应该为多少。\n",
|
||
"即使现实中不会有任何房子的面积是0或房龄正好是0年，我们仍然需要偏置项。\n",
|
||
"如果没有偏置项，我们模型的表达能力将受到限制。\n",
|
||
"严格来说， :eqref:`eq_price-area`是输入特征的一个\n",
|
||
"*仿射变换*（affine transformation）。\n",
|
||
"仿射变换的特点是通过加权和对特征进行*线性变换*（linear transformation），\n",
|
||
"并通过偏置项来进行*平移*（translation）。\n",
|
||
"\n",
|
||
"给定一个数据集，我们的目标是寻找模型的权重$\\mathbf{w}$和偏置$b$，\n",
|
||
"使得根据模型做出的预测大体符合数据里的真实价格。\n",
|
||
"输出的预测值由输入特征通过*线性模型*的仿射变换决定，仿射变换由所选权重和偏置确定。\n",
|
||
"\n",
|
||
"而在机器学习领域，我们通常使用的是高维数据集，建模时采用线性代数表示法会比较方便。\n",
|
||
"当我们的输入包含$d$个特征时，我们将预测结果$\\hat{y}$\n",
|
||
"（通常使用“尖角”符号表示$y$的估计值）表示为：\n",
|
||
"\n",
|
||
"$$\\hat{y} = w_1 x_1 + ... + w_d x_d + b.$$\n",
|
||
"\n",
|
||
"将所有特征放到向量$\\mathbf{x} \\in \\mathbb{R}^d$中，\n",
|
||
"并将所有权重放到向量$\\mathbf{w} \\in \\mathbb{R}^d$中，\n",
|
||
"我们可以用点积形式来简洁地表达模型：\n",
|
||
"\n",
|
||
"$$\\hat{y} = \\mathbf{w}^\\top \\mathbf{x} + b.$$\n",
|
||
":eqlabel:`eq_linreg-y`\n",
|
||
"\n",
|
||
"在 :eqref:`eq_linreg-y`中，\n",
|
||
"向量$\\mathbf{x}$对应于单个数据样本的特征。\n",
|
||
"用符号表示的矩阵$\\mathbf{X} \\in \\mathbb{R}^{n \\times d}$\n",
|
||
"可以很方便地引用我们整个数据集的$n$个样本。\n",
|
||
"其中，$\\mathbf{X}$的每一行是一个样本，每一列是一种特征。\n",
|
||
"\n",
|
||
"对于特征集合$\\mathbf{X}$，预测值$\\hat{\\mathbf{y}} \\in \\mathbb{R}^n$\n",
|
||
"可以通过矩阵-向量乘法表示为：\n",
|
||
"\n",
|
||
"$${\\hat{\\mathbf{y}}} = \\mathbf{X} \\mathbf{w} + b$$\n",
|
||
"\n",
|
||
"这个过程中的求和将使用广播机制\n",
|
||
"（广播机制在 :numref:`subsec_broadcasting`中有详细介绍）。\n",
|
||
"给定训练数据特征$\\mathbf{X}$和对应的已知标签$\\mathbf{y}$，\n",
|
||
"线性回归的目标是找到一组权重向量$\\mathbf{w}$和偏置$b$：\n",
|
||
"当给定从$\\mathbf{X}$的同分布中取样的新样本特征时，\n",
|
||
"这组权重向量和偏置能够使得新样本预测标签的误差尽可能小。\n",
|
||
"\n",
|
||
"虽然我们相信给定$\\mathbf{x}$预测$y$的最佳模型会是线性的，\n",
|
||
"但我们很难找到一个有$n$个样本的真实数据集，其中对于所有的$1 \\leq i \\leq n$，$y^{(i)}$完全等于$\\mathbf{w}^\\top \\mathbf{x}^{(i)}+b$。\n",
|
||
"无论我们使用什么手段来观察特征$\\mathbf{X}$和标签$\\mathbf{y}$，\n",
|
||
"都可能会出现少量的观测误差。\n",
|
||
"因此，即使确信特征与标签的潜在关系是线性的，\n",
|
||
"我们也会加入一个噪声项来考虑观测误差带来的影响。\n",
|
||
"\n",
|
||
"在开始寻找最好的*模型参数*（model parameters）$\\mathbf{w}$和$b$之前，\n",
|
||
"我们还需要两个东西：\n",
|
||
"（1）一种模型质量的度量方式；\n",
|
||
"（2）一种能够更新模型以提高模型预测质量的方法。\n",
|
||
"\n",
|
||
"### 损失函数\n",
|
||
"\n",
|
||
"在我们开始考虑如何用模型*拟合*（fit）数据之前，我们需要确定一个拟合程度的度量。\n",
|
||
"*损失函数*（loss function）能够量化目标的*实际*值与*预测*值之间的差距。\n",
|
||
"通常我们会选择非负数作为损失，且数值越小表示损失越小，完美预测时的损失为0。\n",
|
||
"回归问题中最常用的损失函数是平方误差函数。\n",
|
||
"当样本$i$的预测值为$\\hat{y}^{(i)}$，其相应的真实标签为$y^{(i)}$时，\n",
|
||
"平方误差可以定义为以下公式：\n",
|
||
"\n",
|
||
"$$l^{(i)}(\\mathbf{w}, b) = \\frac{1}{2} \\left(\\hat{y}^{(i)} - y^{(i)}\\right)^2.$$\n",
|
||
":eqlabel:`eq_mse`\n",
|
||
"\n",
|
||
"常数$\\frac{1}{2}$不会带来本质的差别，但这样在形式上稍微简单一些\n",
|
||
"（因为当我们对损失函数求导后常数系数为1）。\n",
|
||
"由于训练数据集并不受我们控制，所以经验误差只是关于模型参数的函数。\n",
|
||
"为了进一步说明，来看下面的例子。\n",
|
||
"我们为一维情况下的回归问题绘制图像，如 :numref:`fig_fit_linreg`所示。\n",
|
||
"\n",
|
||
"\n",
|
||
":label:`fig_fit_linreg`\n",
|
||
"\n",
|
||
"由于平方误差函数中的二次方项，\n",
|
||
"估计值$\\hat{y}^{(i)}$和观测值$y^{(i)}$之间较大的差异将导致更大的损失。\n",
|
||
"为了度量模型在整个数据集上的质量，我们需计算在训练集$n$个样本上的损失均值（也等价于求和）。\n",
|
||
"\n",
|
||
"$$L(\\mathbf{w}, b) =\\frac{1}{n}\\sum_{i=1}^n l^{(i)}(\\mathbf{w}, b) =\\frac{1}{n} \\sum_{i=1}^n \\frac{1}{2}\\left(\\mathbf{w}^\\top \\mathbf{x}^{(i)} + b - y^{(i)}\\right)^2.$$\n",
|
||
"\n",
|
||
"在训练模型时，我们希望寻找一组参数（$\\mathbf{w}^*, b^*$），\n",
|
||
"这组参数能最小化在所有训练样本上的总损失。如下式：\n",
|
||
"\n",
|
||
"$$\\mathbf{w}^*, b^* = \\operatorname*{argmin}_{\\mathbf{w}, b}\\ L(\\mathbf{w}, b).$$\n",
|
||
"\n",
|
||
"### 解析解\n",
|
||
"\n",
|
||
"线性回归刚好是一个很简单的优化问题。\n",
|
||
"与我们将在本书中所讲到的其他大部分模型不同，线性回归的解可以用一个公式简单地表达出来，\n",
|
||
"这类解叫作解析解（analytical solution）。\n",
|
||
"首先，我们将偏置$b$合并到参数$\\mathbf{w}$中，合并方法是在包含所有参数的矩阵中附加一列。\n",
|
||
"我们的预测问题是最小化$\\|\\mathbf{y} - \\mathbf{X}\\mathbf{w}\\|^2$。\n",
|
||
"这在损失平面上只有一个临界点，这个临界点对应于整个区域的损失极小点。\n",
|
||
"将损失关于$\\mathbf{w}$的导数设为0，得到解析解：\n",
|
||
"\n",
|
||
"$$\\mathbf{w}^* = (\\mathbf X^\\top \\mathbf X)^{-1}\\mathbf X^\\top \\mathbf{y}.$$\n",
|
||
"\n",
|
||
"像线性回归这样的简单问题存在解析解，但并不是所有的问题都存在解析解。\n",
|
||
"解析解可以进行很好的数学分析，但解析解对问题的限制很严格，导致它无法广泛应用在深度学习里。\n",
|
||
"\n",
|
||
"### 随机梯度下降\n",
|
||
"\n",
|
||
"即使在我们无法得到解析解的情况下，我们仍然可以有效地训练模型。\n",
|
||
"在许多任务上，那些难以优化的模型效果要更好。\n",
|
||
"因此，弄清楚如何训练这些难以优化的模型是非常重要的。\n",
|
||
"\n",
|
||
"本书中我们用到一种名为*梯度下降*（gradient descent）的方法，\n",
|
||
"这种方法几乎可以优化所有深度学习模型。\n",
|
||
"它通过不断地在损失函数递减的方向上更新参数来降低误差。\n",
|
||
"\n",
|
||
"梯度下降最简单的用法是计算损失函数（数据集中所有样本的损失均值）\n",
|
||
"关于模型参数的导数（在这里也可以称为梯度）。\n",
|
||
"但实际中的执行可能会非常慢：因为在每一次更新参数之前，我们必须遍历整个数据集。\n",
|
||
"因此，我们通常会在每次需要计算更新的时候随机抽取一小批样本，\n",
|
||
"这种变体叫做*小批量随机梯度下降*（minibatch stochastic gradient descent）。\n",
|
||
"\n",
|
||
"在每次迭代中，我们首先随机抽样一个小批量$\\mathcal{B}$，\n",
|
||
"它是由固定数量的训练样本组成的。\n",
|
||
"然后，我们计算小批量的平均损失关于模型参数的导数（也可以称为梯度）。\n",
|
||
"最后，我们将梯度乘以一个预先确定的正数$\\eta$，并从当前参数的值中减掉。\n",
|
||
"\n",
|
||
"我们用下面的数学公式来表示这一更新过程（$\\partial$表示偏导数）：\n",
|
||
"\n",
|
||
"$$(\\mathbf{w},b) \\leftarrow (\\mathbf{w},b) - \\frac{\\eta}{|\\mathcal{B}|} \\sum_{i \\in \\mathcal{B}} \\partial_{(\\mathbf{w},b)} l^{(i)}(\\mathbf{w},b).$$\n",
|
||
"\n",
|
||
"总结一下，算法的步骤如下：\n",
|
||
"（1）初始化模型参数的值，如随机初始化；\n",
|
||
"（2）从数据集中随机抽取小批量样本且在负梯度的方向上更新参数，并不断迭代这一步骤。\n",
|
||
"对于平方损失和仿射变换，我们可以明确地写成如下形式:\n",
|
||
"\n",
|
||
"$$\\begin{aligned} \\mathbf{w} &\\leftarrow \\mathbf{w} - \\frac{\\eta}{|\\mathcal{B}|} \\sum_{i \\in \\mathcal{B}} \\partial_{\\mathbf{w}} l^{(i)}(\\mathbf{w}, b) = \\mathbf{w} - \\frac{\\eta}{|\\mathcal{B}|} \\sum_{i \\in \\mathcal{B}} \\mathbf{x}^{(i)} \\left(\\mathbf{w}^\\top \\mathbf{x}^{(i)} + b - y^{(i)}\\right),\\\\ b &\\leftarrow b - \\frac{\\eta}{|\\mathcal{B}|} \\sum_{i \\in \\mathcal{B}} \\partial_b l^{(i)}(\\mathbf{w}, b) = b - \\frac{\\eta}{|\\mathcal{B}|} \\sum_{i \\in \\mathcal{B}} \\left(\\mathbf{w}^\\top \\mathbf{x}^{(i)} + b - y^{(i)}\\right). \\end{aligned}$$\n",
|
||
":eqlabel:`eq_linreg_batch_update`\n",
|
||
"\n",
|
||
"公式 :eqref:`eq_linreg_batch_update`中的$\\mathbf{w}$和$\\mathbf{x}$都是向量。\n",
|
||
"在这里，更优雅的向量表示法比系数表示法（如$w_1, w_2, \\ldots, w_d$）更具可读性。\n",
|
||
"$|\\mathcal{B}|$表示每个小批量中的样本数，这也称为*批量大小*（batch size）。\n",
|
||
"$\\eta$表示*学习率*（learning rate）。\n",
|
||
"批量大小和学习率的值通常是手动预先指定，而不是通过模型训练得到的。\n",
|
||
"这些可以调整但不在训练过程中更新的参数称为*超参数*（hyperparameter）。\n",
|
||
"*调参*（hyperparameter tuning）是选择超参数的过程。\n",
|
||
"超参数通常是我们根据训练迭代结果来调整的，\n",
|
||
"而训练迭代结果是在独立的*验证数据集*（validation dataset）上评估得到的。\n",
|
||
"\n",
|
||
"在训练了预先确定的若干迭代次数后（或者直到满足某些其他停止条件后），\n",
|
||
"我们记录下模型参数的估计值，表示为$\\hat{\\mathbf{w}}, \\hat{b}$。\n",
|
||
"但是，即使我们的函数确实是线性的且无噪声，这些估计值也不会使损失函数真正地达到最小值。\n",
|
||
"因为算法会使得损失向最小值缓慢收敛，但却不能在有限的步数内非常精确地达到最小值。\n",
|
||
"\n",
|
||
"线性回归恰好是一个在整个域中只有一个最小值的学习问题。\n",
|
||
"但是对像深度神经网络这样复杂的模型来说，损失平面上通常包含多个最小值。\n",
|
||
"深度学习实践者很少会去花费大力气寻找这样一组参数，使得在*训练集*上的损失达到最小。\n",
|
||
"事实上，更难做到的是找到一组参数，这组参数能够在我们从未见过的数据上实现较低的损失，\n",
|
||
"这一挑战被称为*泛化*（generalization）。\n",
|
||
"\n",
|
||
"### 用模型进行预测\n",
|
||
"\n",
|
||
"给定“已学习”的线性回归模型$\\hat{\\mathbf{w}}^\\top \\mathbf{x} + \\hat{b}$，\n",
|
||
"现在我们可以通过房屋面积$x_1$和房龄$x_2$来估计一个（未包含在训练数据中的）新房屋价格。\n",
|
||
"给定特征估计目标的过程通常称为*预测*（prediction）或*推断*（inference）。\n",
|
||
"\n",
|
||
"本书将尝试坚持使用*预测*这个词。\n",
|
||
"虽然*推断*这个词已经成为深度学习的标准术语，但其实*推断*这个词有些用词不当。\n",
|
||
"在统计学中，*推断*更多地表示基于数据集估计参数。\n",
|
||
"当深度学习从业者与统计学家交谈时，术语的误用经常导致一些误解。\n",
|
||
"\n",
|
||
"## 矢量化加速\n",
|
||
"\n",
|
||
"在训练我们的模型时，我们经常希望能够同时处理整个小批量的样本。\n",
|
||
"为了实现这一点，需要(**我们对计算进行矢量化，\n",
|
||
"从而利用线性代数库，而不是在Python中编写开销高昂的for循环**)。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "00f34f5f",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:05:53.225628Z",
|
||
"iopub.status.busy": "2023-08-18T07:05:53.224888Z",
|
||
"iopub.status.idle": "2023-08-18T07:05:55.670980Z",
|
||
"shell.execute_reply": "2023-08-18T07:05:55.670021Z"
|
||
},
|
||
"origin_pos": 2,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"%matplotlib inline\n",
|
||
"import math\n",
|
||
"import time\n",
|
||
"import numpy as np\n",
|
||
"import torch\n",
|
||
"from d2l import torch as d2l"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "16134deb",
|
||
"metadata": {
|
||
"origin_pos": 5
|
||
},
|
||
"source": [
|
||
"为了说明矢量化为什么如此重要，我们考虑(**对向量相加的两种方法**)。\n",
|
||
"我们实例化两个全为1的10000维向量。\n",
|
||
"在一种方法中，我们将使用Python的for循环遍历向量；\n",
|
||
"在另一种方法中，我们将依赖对`+`的调用。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "6dab2aed",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:05:55.677138Z",
|
||
"iopub.status.busy": "2023-08-18T07:05:55.676615Z",
|
||
"iopub.status.idle": "2023-08-18T07:05:55.683088Z",
|
||
"shell.execute_reply": "2023-08-18T07:05:55.682108Z"
|
||
},
|
||
"origin_pos": 6,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"n = 10000\n",
|
||
"a = torch.ones([n])\n",
|
||
"b = torch.ones([n])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "4d858f1b",
|
||
"metadata": {
|
||
"origin_pos": 7
|
||
},
|
||
"source": [
|
||
"由于在本书中我们将频繁地进行运行时间的基准测试，所以[**我们定义一个计时器**]：\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "debf1949",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:05:55.687766Z",
|
||
"iopub.status.busy": "2023-08-18T07:05:55.687275Z",
|
||
"iopub.status.idle": "2023-08-18T07:05:55.694676Z",
|
||
"shell.execute_reply": "2023-08-18T07:05:55.693692Z"
|
||
},
|
||
"origin_pos": 8,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"class Timer: #@save\n",
|
||
" \"\"\"记录多次运行时间\"\"\"\n",
|
||
" def __init__(self):\n",
|
||
" self.times = []\n",
|
||
" self.start()\n",
|
||
"\n",
|
||
" def start(self):\n",
|
||
" \"\"\"启动计时器\"\"\"\n",
|
||
" self.tik = time.time()\n",
|
||
"\n",
|
||
" def stop(self):\n",
|
||
" \"\"\"停止计时器并将时间记录在列表中\"\"\"\n",
|
||
" self.times.append(time.time() - self.tik)\n",
|
||
" return self.times[-1]\n",
|
||
"\n",
|
||
" def avg(self):\n",
|
||
" \"\"\"返回平均时间\"\"\"\n",
|
||
" return sum(self.times) / len(self.times)\n",
|
||
"\n",
|
||
" def sum(self):\n",
|
||
" \"\"\"返回时间总和\"\"\"\n",
|
||
" return sum(self.times)\n",
|
||
"\n",
|
||
" def cumsum(self):\n",
|
||
" \"\"\"返回累计时间\"\"\"\n",
|
||
" return np.array(self.times).cumsum().tolist()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "68e4d328",
|
||
"metadata": {
|
||
"origin_pos": 9
|
||
},
|
||
"source": [
|
||
"现在我们可以对工作负载进行基准测试。\n",
|
||
"\n",
|
||
"首先，[**我们使用for循环，每次执行一位的加法**]。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "53998dfd",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:05:55.699851Z",
|
||
"iopub.status.busy": "2023-08-18T07:05:55.699125Z",
|
||
"iopub.status.idle": "2023-08-18T07:05:55.877034Z",
|
||
"shell.execute_reply": "2023-08-18T07:05:55.875673Z"
|
||
},
|
||
"origin_pos": 10,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"'0.16749 sec'"
|
||
]
|
||
},
|
||
"execution_count": 4,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"c = torch.zeros(n)\n",
|
||
"timer = Timer()\n",
|
||
"for i in range(n):\n",
|
||
" c[i] = a[i] + b[i]\n",
|
||
"f'{timer.stop():.5f} sec'"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c48c5704",
|
||
"metadata": {
|
||
"origin_pos": 13
|
||
},
|
||
"source": [
|
||
"(**或者，我们使用重载的`+`运算符来计算按元素的和**)。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"id": "7a6c052c",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:05:55.882550Z",
|
||
"iopub.status.busy": "2023-08-18T07:05:55.881702Z",
|
||
"iopub.status.idle": "2023-08-18T07:05:55.890012Z",
|
||
"shell.execute_reply": "2023-08-18T07:05:55.888724Z"
|
||
},
|
||
"origin_pos": 14,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"'0.00042 sec'"
|
||
]
|
||
},
|
||
"execution_count": 5,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"timer.start()\n",
|
||
"d = a + b\n",
|
||
"f'{timer.stop():.5f} sec'"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5e343af9",
|
||
"metadata": {
|
||
"origin_pos": 15
|
||
},
|
||
"source": [
|
||
"结果很明显，第二种方法比第一种方法快得多。\n",
|
||
"矢量化代码通常会带来数量级的加速。\n",
|
||
"另外，我们将更多的数学运算放到库中，而无须自己编写那么多的计算，从而减少了出错的可能性。\n",
|
||
"\n",
|
||
"## 正态分布与平方损失\n",
|
||
":label:`subsec_normal_distribution_and_squared_loss`\n",
|
||
"\n",
|
||
"接下来，我们通过对噪声分布的假设来解读平方损失目标函数。\n",
|
||
"\n",
|
||
"正态分布和线性回归之间的关系很密切。\n",
|
||
"正态分布（normal distribution），也称为*高斯分布*（Gaussian distribution），\n",
|
||
"最早由德国数学家高斯（Gauss）应用于天文学研究。\n",
|
||
"简单的说，若随机变量$x$具有均值$\\mu$和方差$\\sigma^2$（标准差$\\sigma$），其正态分布概率密度函数如下：\n",
|
||
"\n",
|
||
"$$p(x) = \\frac{1}{\\sqrt{2 \\pi \\sigma^2}} \\exp\\left(-\\frac{1}{2 \\sigma^2} (x - \\mu)^2\\right).$$\n",
|
||
"\n",
|
||
"下面[**我们定义一个Python函数来计算正态分布**]。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "fa7e3595",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:05:55.895314Z",
|
||
"iopub.status.busy": "2023-08-18T07:05:55.894485Z",
|
||
"iopub.status.idle": "2023-08-18T07:05:55.900942Z",
|
||
"shell.execute_reply": "2023-08-18T07:05:55.899745Z"
|
||
},
|
||
"origin_pos": 16,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def normal(x, mu, sigma):\n",
|
||
" p = 1 / math.sqrt(2 * math.pi * sigma**2)\n",
|
||
" return p * np.exp(-0.5 / sigma**2 * (x - mu)**2)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ea5ec143",
|
||
"metadata": {
|
||
"origin_pos": 17
|
||
},
|
||
"source": [
|
||
"我们现在(**可视化正态分布**)。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"id": "7d3e1e42",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T07:05:55.906310Z",
|
||
"iopub.status.busy": "2023-08-18T07:05:55.905361Z",
|
||
"iopub.status.idle": "2023-08-18T07:05:56.341164Z",
|
||
"shell.execute_reply": "2023-08-18T07:05:56.339902Z"
|
||
},
|
||
"origin_pos": 19,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"image/svg+xml": [
|
||
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
|
||
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
|
||
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
|
||
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"302.08125pt\" height=\"180.65625pt\" viewBox=\"0 0 302.08125 180.65625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
|
||
" <metadata>\n",
|
||
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
|
||
" <cc:Work>\n",
|
||
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
|
||
" <dc:date>2023-08-18T07:05:56.251185</dc:date>\n",
|
||
" <dc:format>image/svg+xml</dc:format>\n",
|
||
" <dc:creator>\n",
|
||
" <cc:Agent>\n",
|
||
" <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
|
||
" </cc:Agent>\n",
|
||
" </dc:creator>\n",
|
||
" </cc:Work>\n",
|
||
" </rdf:RDF>\n",
|
||
" </metadata>\n",
|
||
" <defs>\n",
|
||
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
|
||
" </defs>\n",
|
||
" <g id=\"figure_1\">\n",
|
||
" <g id=\"patch_1\">\n",
|
||
" <path d=\"M 0 180.65625 \n",
|
||
"L 302.08125 180.65625 \n",
|
||
"L 302.08125 0 \n",
|
||
"L 0 0 \n",
|
||
"L 0 180.65625 \n",
|
||
"z\n",
|
||
"\" style=\"fill: none\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"axes_1\">\n",
|
||
" <g id=\"patch_2\">\n",
|
||
" <path d=\"M 43.78125 143.1 \n",
|
||
"L 294.88125 143.1 \n",
|
||
"L 294.88125 7.2 \n",
|
||
"L 43.78125 7.2 \n",
|
||
"z\n",
|
||
"\" style=\"fill: #ffffff\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"matplotlib.axis_1\">\n",
|
||
" <g id=\"xtick_1\">\n",
|
||
" <g id=\"line2d_1\">\n",
|
||
" <path d=\"M 71.511736 143.1 \n",
|
||
"L 71.511736 7.2 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_2\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"m5a952af5d8\" d=\"M 0 0 \n",
|
||
"L 0 3.5 \n",
|
||
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </defs>\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m5a952af5d8\" x=\"71.511736\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_1\">\n",
|
||
" <!-- −6 -->\n",
|
||
" <g transform=\"translate(64.140642 157.698438)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
|
||
"L 4684 2272 \n",
|
||
"L 4684 1741 \n",
|
||
"L 678 1741 \n",
|
||
"L 678 2272 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
|
||
"Q 1688 2584 1439 2293 \n",
|
||
"Q 1191 2003 1191 1497 \n",
|
||
"Q 1191 994 1439 701 \n",
|
||
"Q 1688 409 2113 409 \n",
|
||
"Q 2538 409 2786 701 \n",
|
||
"Q 3034 994 3034 1497 \n",
|
||
"Q 3034 2003 2786 2293 \n",
|
||
"Q 2538 2584 2113 2584 \n",
|
||
"z\n",
|
||
"M 3366 4563 \n",
|
||
"L 3366 3988 \n",
|
||
"Q 3128 4100 2886 4159 \n",
|
||
"Q 2644 4219 2406 4219 \n",
|
||
"Q 1781 4219 1451 3797 \n",
|
||
"Q 1122 3375 1075 2522 \n",
|
||
"Q 1259 2794 1537 2939 \n",
|
||
"Q 1816 3084 2150 3084 \n",
|
||
"Q 2853 3084 3261 2657 \n",
|
||
"Q 3669 2231 3669 1497 \n",
|
||
"Q 3669 778 3244 343 \n",
|
||
"Q 2819 -91 2113 -91 \n",
|
||
"Q 1303 -91 875 529 \n",
|
||
"Q 447 1150 447 2328 \n",
|
||
"Q 447 3434 972 4092 \n",
|
||
"Q 1497 4750 2381 4750 \n",
|
||
"Q 2619 4750 2861 4703 \n",
|
||
"Q 3103 4656 3366 4563 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-36\" x=\"83.789062\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"xtick_2\">\n",
|
||
" <g id=\"line2d_3\">\n",
|
||
" <path d=\"M 104.145435 143.1 \n",
|
||
"L 104.145435 7.2 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_4\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m5a952af5d8\" x=\"104.145435\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_2\">\n",
|
||
" <!-- −4 -->\n",
|
||
" <g transform=\"translate(96.774342 157.698438)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
|
||
"L 825 1625 \n",
|
||
"L 2419 1625 \n",
|
||
"L 2419 4116 \n",
|
||
"z\n",
|
||
"M 2253 4666 \n",
|
||
"L 3047 4666 \n",
|
||
"L 3047 1625 \n",
|
||
"L 3713 1625 \n",
|
||
"L 3713 1100 \n",
|
||
"L 3047 1100 \n",
|
||
"L 3047 0 \n",
|
||
"L 2419 0 \n",
|
||
"L 2419 1100 \n",
|
||
"L 313 1100 \n",
|
||
"L 313 1709 \n",
|
||
"L 2253 4666 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-34\" x=\"83.789062\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"xtick_3\">\n",
|
||
" <g id=\"line2d_5\">\n",
|
||
" <path d=\"M 136.779135 143.1 \n",
|
||
"L 136.779135 7.2 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_6\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m5a952af5d8\" x=\"136.779135\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_3\">\n",
|
||
" <!-- −2 -->\n",
|
||
" <g transform=\"translate(129.408041 157.698438)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
|
||
"L 3431 531 \n",
|
||
"L 3431 0 \n",
|
||
"L 469 0 \n",
|
||
"L 469 531 \n",
|
||
"Q 828 903 1448 1529 \n",
|
||
"Q 2069 2156 2228 2338 \n",
|
||
"Q 2531 2678 2651 2914 \n",
|
||
"Q 2772 3150 2772 3378 \n",
|
||
"Q 2772 3750 2511 3984 \n",
|
||
"Q 2250 4219 1831 4219 \n",
|
||
"Q 1534 4219 1204 4116 \n",
|
||
"Q 875 4013 500 3803 \n",
|
||
"L 500 4441 \n",
|
||
"Q 881 4594 1212 4672 \n",
|
||
"Q 1544 4750 1819 4750 \n",
|
||
"Q 2544 4750 2975 4387 \n",
|
||
"Q 3406 4025 3406 3419 \n",
|
||
"Q 3406 3131 3298 2873 \n",
|
||
"Q 3191 2616 2906 2266 \n",
|
||
"Q 2828 2175 2409 1742 \n",
|
||
"Q 1991 1309 1228 531 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"xtick_4\">\n",
|
||
" <g id=\"line2d_7\">\n",
|
||
" <path d=\"M 169.412834 143.1 \n",
|
||
"L 169.412834 7.2 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_8\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m5a952af5d8\" x=\"169.412834\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_4\">\n",
|
||
" <!-- 0 -->\n",
|
||
" <g transform=\"translate(166.231584 157.698438)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
|
||
"Q 1547 4250 1301 3770 \n",
|
||
"Q 1056 3291 1056 2328 \n",
|
||
"Q 1056 1369 1301 889 \n",
|
||
"Q 1547 409 2034 409 \n",
|
||
"Q 2525 409 2770 889 \n",
|
||
"Q 3016 1369 3016 2328 \n",
|
||
"Q 3016 3291 2770 3770 \n",
|
||
"Q 2525 4250 2034 4250 \n",
|
||
"z\n",
|
||
"M 2034 4750 \n",
|
||
"Q 2819 4750 3233 4129 \n",
|
||
"Q 3647 3509 3647 2328 \n",
|
||
"Q 3647 1150 3233 529 \n",
|
||
"Q 2819 -91 2034 -91 \n",
|
||
"Q 1250 -91 836 529 \n",
|
||
"Q 422 1150 422 2328 \n",
|
||
"Q 422 3509 836 4129 \n",
|
||
"Q 1250 4750 2034 4750 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"xtick_5\">\n",
|
||
" <g id=\"line2d_9\">\n",
|
||
" <path d=\"M 202.046534 143.1 \n",
|
||
"L 202.046534 7.2 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_10\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m5a952af5d8\" x=\"202.046534\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_5\">\n",
|
||
" <!-- 2 -->\n",
|
||
" <g transform=\"translate(198.865284 157.698438)scale(0.1 -0.1)\">\n",
|
||
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"xtick_6\">\n",
|
||
" <g id=\"line2d_11\">\n",
|
||
" <path d=\"M 234.680233 143.1 \n",
|
||
"L 234.680233 7.2 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_12\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m5a952af5d8\" x=\"234.680233\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_6\">\n",
|
||
" <!-- 4 -->\n",
|
||
" <g transform=\"translate(231.498983 157.698438)scale(0.1 -0.1)\">\n",
|
||
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"xtick_7\">\n",
|
||
" <g id=\"line2d_13\">\n",
|
||
" <path d=\"M 267.313932 143.1 \n",
|
||
"L 267.313932 7.2 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_14\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m5a952af5d8\" x=\"267.313932\" y=\"143.1\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_7\">\n",
|
||
" <!-- 6 -->\n",
|
||
" <g transform=\"translate(264.132682 157.698438)scale(0.1 -0.1)\">\n",
|
||
" <use xlink:href=\"#DejaVuSans-36\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_8\">\n",
|
||
" <!-- x -->\n",
|
||
" <g transform=\"translate(166.371875 171.376563)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
|
||
"L 2247 1797 \n",
|
||
"L 3578 0 \n",
|
||
"L 2900 0 \n",
|
||
"L 1881 1375 \n",
|
||
"L 863 0 \n",
|
||
"L 184 0 \n",
|
||
"L 1544 1831 \n",
|
||
"L 300 3500 \n",
|
||
"L 978 3500 \n",
|
||
"L 1906 2253 \n",
|
||
"L 2834 3500 \n",
|
||
"L 3513 3500 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"matplotlib.axis_2\">\n",
|
||
" <g id=\"ytick_1\">\n",
|
||
" <g id=\"line2d_15\">\n",
|
||
" <path d=\"M 43.78125 136.922727 \n",
|
||
"L 294.88125 136.922727 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_16\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"m2bd57c3439\" d=\"M 0 0 \n",
|
||
"L -3.5 0 \n",
|
||
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </defs>\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m2bd57c3439\" x=\"43.78125\" y=\"136.922727\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_9\">\n",
|
||
" <!-- 0.0 -->\n",
|
||
" <g transform=\"translate(20.878125 140.721946)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
|
||
"L 1344 794 \n",
|
||
"L 1344 0 \n",
|
||
"L 684 0 \n",
|
||
"L 684 794 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"ytick_2\">\n",
|
||
" <g id=\"line2d_17\">\n",
|
||
" <path d=\"M 43.78125 105.954474 \n",
|
||
"L 294.88125 105.954474 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_18\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m2bd57c3439\" x=\"43.78125\" y=\"105.954474\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_10\">\n",
|
||
" <!-- 0.1 -->\n",
|
||
" <g transform=\"translate(20.878125 109.753693)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
|
||
"L 1825 531 \n",
|
||
"L 1825 4091 \n",
|
||
"L 703 3866 \n",
|
||
"L 703 4441 \n",
|
||
"L 1819 4666 \n",
|
||
"L 2450 4666 \n",
|
||
"L 2450 531 \n",
|
||
"L 3481 531 \n",
|
||
"L 3481 0 \n",
|
||
"L 794 0 \n",
|
||
"L 794 531 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-31\" x=\"95.410156\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"ytick_3\">\n",
|
||
" <g id=\"line2d_19\">\n",
|
||
" <path d=\"M 43.78125 74.986221 \n",
|
||
"L 294.88125 74.986221 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_20\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m2bd57c3439\" x=\"43.78125\" y=\"74.986221\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_11\">\n",
|
||
" <!-- 0.2 -->\n",
|
||
" <g transform=\"translate(20.878125 78.78544)scale(0.1 -0.1)\">\n",
|
||
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-32\" x=\"95.410156\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"ytick_4\">\n",
|
||
" <g id=\"line2d_21\">\n",
|
||
" <path d=\"M 43.78125 44.017968 \n",
|
||
"L 294.88125 44.017968 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_22\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m2bd57c3439\" x=\"43.78125\" y=\"44.017968\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_12\">\n",
|
||
" <!-- 0.3 -->\n",
|
||
" <g transform=\"translate(20.878125 47.817187)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
|
||
"Q 3050 2419 3304 2112 \n",
|
||
"Q 3559 1806 3559 1356 \n",
|
||
"Q 3559 666 3084 287 \n",
|
||
"Q 2609 -91 1734 -91 \n",
|
||
"Q 1441 -91 1130 -33 \n",
|
||
"Q 819 25 488 141 \n",
|
||
"L 488 750 \n",
|
||
"Q 750 597 1062 519 \n",
|
||
"Q 1375 441 1716 441 \n",
|
||
"Q 2309 441 2620 675 \n",
|
||
"Q 2931 909 2931 1356 \n",
|
||
"Q 2931 1769 2642 2001 \n",
|
||
"Q 2353 2234 1838 2234 \n",
|
||
"L 1294 2234 \n",
|
||
"L 1294 2753 \n",
|
||
"L 1863 2753 \n",
|
||
"Q 2328 2753 2575 2939 \n",
|
||
"Q 2822 3125 2822 3475 \n",
|
||
"Q 2822 3834 2567 4026 \n",
|
||
"Q 2313 4219 1838 4219 \n",
|
||
"Q 1578 4219 1281 4162 \n",
|
||
"Q 984 4106 628 3988 \n",
|
||
"L 628 4550 \n",
|
||
"Q 988 4650 1302 4700 \n",
|
||
"Q 1616 4750 1894 4750 \n",
|
||
"Q 2613 4750 3031 4423 \n",
|
||
"Q 3450 4097 3450 3541 \n",
|
||
"Q 3450 3153 3228 2886 \n",
|
||
"Q 3006 2619 2597 2516 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-33\" x=\"95.410156\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"ytick_5\">\n",
|
||
" <g id=\"line2d_23\">\n",
|
||
" <path d=\"M 43.78125 13.049715 \n",
|
||
"L 294.88125 13.049715 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_24\">\n",
|
||
" <g>\n",
|
||
" <use xlink:href=\"#m2bd57c3439\" x=\"43.78125\" y=\"13.049715\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_13\">\n",
|
||
" <!-- 0.4 -->\n",
|
||
" <g transform=\"translate(20.878125 16.848934)scale(0.1 -0.1)\">\n",
|
||
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-34\" x=\"95.410156\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_14\">\n",
|
||
" <!-- p(x) -->\n",
|
||
" <g transform=\"translate(14.798438 85.185156)rotate(-90)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
|
||
"L 1159 -1331 \n",
|
||
"L 581 -1331 \n",
|
||
"L 581 3500 \n",
|
||
"L 1159 3500 \n",
|
||
"L 1159 2969 \n",
|
||
"Q 1341 3281 1617 3432 \n",
|
||
"Q 1894 3584 2278 3584 \n",
|
||
"Q 2916 3584 3314 3078 \n",
|
||
"Q 3713 2572 3713 1747 \n",
|
||
"Q 3713 922 3314 415 \n",
|
||
"Q 2916 -91 2278 -91 \n",
|
||
"Q 1894 -91 1617 61 \n",
|
||
"Q 1341 213 1159 525 \n",
|
||
"z\n",
|
||
"M 3116 1747 \n",
|
||
"Q 3116 2381 2855 2742 \n",
|
||
"Q 2594 3103 2138 3103 \n",
|
||
"Q 1681 3103 1420 2742 \n",
|
||
"Q 1159 2381 1159 1747 \n",
|
||
"Q 1159 1113 1420 752 \n",
|
||
"Q 1681 391 2138 391 \n",
|
||
"Q 2594 391 2855 752 \n",
|
||
"Q 3116 1113 3116 1747 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
|
||
"Q 1566 4138 1362 3434 \n",
|
||
"Q 1159 2731 1159 2009 \n",
|
||
"Q 1159 1288 1364 580 \n",
|
||
"Q 1569 -128 1984 -844 \n",
|
||
"L 1484 -844 \n",
|
||
"Q 1016 -109 783 600 \n",
|
||
"Q 550 1309 550 2009 \n",
|
||
"Q 550 2706 781 3412 \n",
|
||
"Q 1013 4119 1484 4856 \n",
|
||
"L 1984 4856 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
|
||
"L 1013 4856 \n",
|
||
"Q 1481 4119 1714 3412 \n",
|
||
"Q 1947 2706 1947 2009 \n",
|
||
"Q 1947 1309 1714 600 \n",
|
||
"Q 1481 -109 1013 -844 \n",
|
||
"L 513 -844 \n",
|
||
"Q 928 -128 1133 580 \n",
|
||
"Q 1338 1288 1338 2009 \n",
|
||
"Q 1338 2731 1133 3434 \n",
|
||
"Q 928 4138 513 4856 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-70\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-28\" x=\"63.476562\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-78\" x=\"102.490234\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-29\" x=\"161.669922\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_25\">\n",
|
||
" <path d=\"M 55.194886 136.922727 \n",
|
||
"L 108.224648 136.813535 \n",
|
||
"L 113.609208 136.566288 \n",
|
||
"L 117.198915 136.184417 \n",
|
||
"L 119.97278 135.668952 \n",
|
||
"L 122.257139 135.025186 \n",
|
||
"L 124.215161 134.257847 \n",
|
||
"L 126.010014 133.330368 \n",
|
||
"L 127.804868 132.138334 \n",
|
||
"L 129.436552 130.779445 \n",
|
||
"L 131.068237 129.113085 \n",
|
||
"L 132.699922 127.093512 \n",
|
||
"L 134.331607 124.67477 \n",
|
||
"L 135.963292 121.812692 \n",
|
||
"L 137.594977 118.467288 \n",
|
||
"L 139.226662 114.605495 \n",
|
||
"L 141.021516 109.733818 \n",
|
||
"L 142.816369 104.197078 \n",
|
||
"L 144.774391 97.412516 \n",
|
||
"L 146.895582 89.247632 \n",
|
||
"L 149.506278 78.224523 \n",
|
||
"L 153.259153 61.239305 \n",
|
||
"L 157.501534 42.275258 \n",
|
||
"L 159.785893 33.113056 \n",
|
||
"L 161.580746 26.820515 \n",
|
||
"L 163.049263 22.424523 \n",
|
||
"L 164.354611 19.173268 \n",
|
||
"L 165.49679 16.884633 \n",
|
||
"L 166.475801 15.362585 \n",
|
||
"L 167.454812 14.263605 \n",
|
||
"L 168.270655 13.679589 \n",
|
||
"L 169.086497 13.401979 \n",
|
||
"L 169.739171 13.401979 \n",
|
||
"L 170.391845 13.599455 \n",
|
||
"L 171.207688 14.122466 \n",
|
||
"L 172.02353 14.948577 \n",
|
||
"L 173.002541 16.331186 \n",
|
||
"L 173.981552 18.12656 \n",
|
||
"L 175.123732 20.717347 \n",
|
||
"L 176.42908 24.286979 \n",
|
||
"L 177.897596 29.000684 \n",
|
||
"L 179.69245 35.615359 \n",
|
||
"L 181.81364 44.367165 \n",
|
||
"L 184.913841 58.245076 \n",
|
||
"L 190.46157 83.161144 \n",
|
||
"L 192.909098 93.115047 \n",
|
||
"L 195.030288 100.899673 \n",
|
||
"L 196.98831 107.299489 \n",
|
||
"L 198.783164 112.473243 \n",
|
||
"L 200.578017 116.986089 \n",
|
||
"L 202.209702 120.534567 \n",
|
||
"L 203.841387 123.5855 \n",
|
||
"L 205.473072 126.176453 \n",
|
||
"L 207.104757 128.35023 \n",
|
||
"L 208.736442 130.152334 \n",
|
||
"L 210.368127 131.628806 \n",
|
||
"L 211.999812 132.824481 \n",
|
||
"L 213.794665 133.865803 \n",
|
||
"L 215.752687 134.73293 \n",
|
||
"L 217.873878 135.421686 \n",
|
||
"L 220.321405 135.97206 \n",
|
||
"L 223.258438 136.389278 \n",
|
||
"L 227.011314 136.67952 \n",
|
||
"L 232.395874 136.850878 \n",
|
||
"L 243.001826 136.917995 \n",
|
||
"L 283.467614 136.922727 \n",
|
||
"L 283.467614 136.922727 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_26\">\n",
|
||
" <path d=\"M 55.194886 136.7876 \n",
|
||
"L 65.800839 136.522951 \n",
|
||
"L 73.143421 136.126434 \n",
|
||
"L 79.017487 135.590287 \n",
|
||
"L 83.912542 134.926494 \n",
|
||
"L 88.318091 134.105229 \n",
|
||
"L 92.234135 133.153577 \n",
|
||
"L 95.823842 132.06316 \n",
|
||
"L 99.250381 130.798749 \n",
|
||
"L 102.51375 129.367709 \n",
|
||
"L 105.77712 127.695008 \n",
|
||
"L 109.04049 125.764111 \n",
|
||
"L 112.30386 123.563438 \n",
|
||
"L 115.56723 121.087897 \n",
|
||
"L 118.993769 118.196043 \n",
|
||
"L 122.583476 114.860886 \n",
|
||
"L 126.49952 110.903144 \n",
|
||
"L 131.068237 105.948686 \n",
|
||
"L 138.247651 97.770807 \n",
|
||
"L 144.611223 90.644946 \n",
|
||
"L 148.527267 86.589675 \n",
|
||
"L 151.790637 83.530672 \n",
|
||
"L 154.564501 81.224436 \n",
|
||
"L 157.175197 79.344214 \n",
|
||
"L 159.459556 77.957408 \n",
|
||
"L 161.743915 76.832365 \n",
|
||
"L 163.865105 76.036198 \n",
|
||
"L 165.823127 75.522597 \n",
|
||
"L 167.781149 75.227168 \n",
|
||
"L 169.739171 75.153089 \n",
|
||
"L 171.697193 75.301158 \n",
|
||
"L 173.655215 75.66978 \n",
|
||
"L 175.613237 76.254995 \n",
|
||
"L 177.734428 77.126088 \n",
|
||
"L 179.855618 78.233161 \n",
|
||
"L 182.139977 79.673625 \n",
|
||
"L 184.587504 81.48006 \n",
|
||
"L 187.361369 83.820867 \n",
|
||
"L 190.46157 86.751111 \n",
|
||
"L 194.051277 90.469337 \n",
|
||
"L 198.783164 95.721751 \n",
|
||
"L 211.673475 110.215093 \n",
|
||
"L 215.752687 114.383393 \n",
|
||
"L 219.505563 117.90542 \n",
|
||
"L 222.932101 120.82526 \n",
|
||
"L 226.195471 123.328273 \n",
|
||
"L 229.458841 125.556324 \n",
|
||
"L 232.722211 127.513772 \n",
|
||
"L 235.985581 129.211619 \n",
|
||
"L 239.248951 130.665971 \n",
|
||
"L 242.675489 131.95258 \n",
|
||
"L 246.265196 133.063568 \n",
|
||
"L 250.18124 134.034481 \n",
|
||
"L 254.423621 134.846714 \n",
|
||
"L 259.155508 135.514669 \n",
|
||
"L 264.540068 136.040363 \n",
|
||
"L 270.903639 136.432354 \n",
|
||
"L 279.225233 136.707949 \n",
|
||
"L 283.467614 136.785216 \n",
|
||
"L 283.467614 136.785216 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_27\">\n",
|
||
" <path d=\"M 55.194886 136.922727 \n",
|
||
"L 157.175197 136.813535 \n",
|
||
"L 162.559757 136.566288 \n",
|
||
"L 166.149464 136.184417 \n",
|
||
"L 168.923329 135.668952 \n",
|
||
"L 171.207688 135.025186 \n",
|
||
"L 173.16571 134.257847 \n",
|
||
"L 174.960563 133.330368 \n",
|
||
"L 176.755417 132.138334 \n",
|
||
"L 178.387102 130.779445 \n",
|
||
"L 180.018787 129.113085 \n",
|
||
"L 181.650472 127.093512 \n",
|
||
"L 183.282156 124.67477 \n",
|
||
"L 184.913841 121.812692 \n",
|
||
"L 186.545526 118.467288 \n",
|
||
"L 188.177211 114.605495 \n",
|
||
"L 189.972065 109.733818 \n",
|
||
"L 191.766918 104.197078 \n",
|
||
"L 193.72494 97.412516 \n",
|
||
"L 195.846131 89.247632 \n",
|
||
"L 198.456827 78.224523 \n",
|
||
"L 202.209702 61.239305 \n",
|
||
"L 206.452083 42.275258 \n",
|
||
"L 208.736442 33.113056 \n",
|
||
"L 210.531295 26.820515 \n",
|
||
"L 211.999812 22.424523 \n",
|
||
"L 213.30516 19.173268 \n",
|
||
"L 214.447339 16.884633 \n",
|
||
"L 215.42635 15.362585 \n",
|
||
"L 216.405361 14.263605 \n",
|
||
"L 217.221204 13.679589 \n",
|
||
"L 218.037046 13.401979 \n",
|
||
"L 218.68972 13.401979 \n",
|
||
"L 219.342394 13.599455 \n",
|
||
"L 220.158237 14.122466 \n",
|
||
"L 220.974079 14.948577 \n",
|
||
"L 221.95309 16.331186 \n",
|
||
"L 222.932101 18.12656 \n",
|
||
"L 224.074281 20.717347 \n",
|
||
"L 225.379629 24.286979 \n",
|
||
"L 226.848145 29.000684 \n",
|
||
"L 228.642999 35.615359 \n",
|
||
"L 230.764189 44.367165 \n",
|
||
"L 233.864391 58.245076 \n",
|
||
"L 239.412119 83.161144 \n",
|
||
"L 241.859647 93.115047 \n",
|
||
"L 243.980837 100.899673 \n",
|
||
"L 245.938859 107.299489 \n",
|
||
"L 247.733713 112.473243 \n",
|
||
"L 249.528566 116.986089 \n",
|
||
"L 251.160251 120.534567 \n",
|
||
"L 252.791936 123.5855 \n",
|
||
"L 254.423621 126.176453 \n",
|
||
"L 256.055306 128.35023 \n",
|
||
"L 257.686991 130.152334 \n",
|
||
"L 259.318676 131.628806 \n",
|
||
"L 260.950361 132.824481 \n",
|
||
"L 262.745215 133.865803 \n",
|
||
"L 264.703236 134.73293 \n",
|
||
"L 266.824427 135.421686 \n",
|
||
"L 269.271954 135.97206 \n",
|
||
"L 272.208987 136.389278 \n",
|
||
"L 275.961863 136.67952 \n",
|
||
"L 281.346423 136.850878 \n",
|
||
"L 283.467614 136.879593 \n",
|
||
"L 283.467614 136.879593 \n",
|
||
"\" clip-path=\"url(#p0a634f4fb7)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"patch_3\">\n",
|
||
" <path d=\"M 43.78125 143.1 \n",
|
||
"L 43.78125 7.2 \n",
|
||
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"patch_4\">\n",
|
||
" <path d=\"M 294.88125 143.1 \n",
|
||
"L 294.88125 7.2 \n",
|
||
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"patch_5\">\n",
|
||
" <path d=\"M 43.78125 143.1 \n",
|
||
"L 294.88125 143.1 \n",
|
||
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"patch_6\">\n",
|
||
" <path d=\"M 43.78125 7.2 \n",
|
||
"L 294.88125 7.2 \n",
|
||
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"legend_1\">\n",
|
||
" <g id=\"patch_7\">\n",
|
||
" <path d=\"M 50.78125 59.234375 \n",
|
||
"L 152.05625 59.234375 \n",
|
||
"Q 154.05625 59.234375 154.05625 57.234375 \n",
|
||
"L 154.05625 14.2 \n",
|
||
"Q 154.05625 12.2 152.05625 12.2 \n",
|
||
"L 50.78125 12.2 \n",
|
||
"Q 48.78125 12.2 48.78125 14.2 \n",
|
||
"L 48.78125 57.234375 \n",
|
||
"Q 48.78125 59.234375 50.78125 59.234375 \n",
|
||
"z\n",
|
||
"\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_28\">\n",
|
||
" <path d=\"M 52.78125 20.298438 \n",
|
||
"L 62.78125 20.298438 \n",
|
||
"L 72.78125 20.298438 \n",
|
||
"\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_15\">\n",
|
||
" <!-- mean 0, std 1 -->\n",
|
||
" <g transform=\"translate(80.78125 23.798438)scale(0.1 -0.1)\">\n",
|
||
" <defs>\n",
|
||
" <path id=\"DejaVuSans-6d\" d=\"M 3328 2828 \n",
|
||
"Q 3544 3216 3844 3400 \n",
|
||
"Q 4144 3584 4550 3584 \n",
|
||
"Q 5097 3584 5394 3201 \n",
|
||
"Q 5691 2819 5691 2113 \n",
|
||
"L 5691 0 \n",
|
||
"L 5113 0 \n",
|
||
"L 5113 2094 \n",
|
||
"Q 5113 2597 4934 2840 \n",
|
||
"Q 4756 3084 4391 3084 \n",
|
||
"Q 3944 3084 3684 2787 \n",
|
||
"Q 3425 2491 3425 1978 \n",
|
||
"L 3425 0 \n",
|
||
"L 2847 0 \n",
|
||
"L 2847 2094 \n",
|
||
"Q 2847 2600 2669 2842 \n",
|
||
"Q 2491 3084 2119 3084 \n",
|
||
"Q 1678 3084 1418 2786 \n",
|
||
"Q 1159 2488 1159 1978 \n",
|
||
"L 1159 0 \n",
|
||
"L 581 0 \n",
|
||
"L 581 3500 \n",
|
||
"L 1159 3500 \n",
|
||
"L 1159 2956 \n",
|
||
"Q 1356 3278 1631 3431 \n",
|
||
"Q 1906 3584 2284 3584 \n",
|
||
"Q 2666 3584 2933 3390 \n",
|
||
"Q 3200 3197 3328 2828 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
|
||
"L 3597 1613 \n",
|
||
"L 953 1613 \n",
|
||
"Q 991 1019 1311 708 \n",
|
||
"Q 1631 397 2203 397 \n",
|
||
"Q 2534 397 2845 478 \n",
|
||
"Q 3156 559 3463 722 \n",
|
||
"L 3463 178 \n",
|
||
"Q 3153 47 2828 -22 \n",
|
||
"Q 2503 -91 2169 -91 \n",
|
||
"Q 1331 -91 842 396 \n",
|
||
"Q 353 884 353 1716 \n",
|
||
"Q 353 2575 817 3079 \n",
|
||
"Q 1281 3584 2069 3584 \n",
|
||
"Q 2775 3584 3186 3129 \n",
|
||
"Q 3597 2675 3597 1894 \n",
|
||
"z\n",
|
||
"M 3022 2063 \n",
|
||
"Q 3016 2534 2758 2815 \n",
|
||
"Q 2500 3097 2075 3097 \n",
|
||
"Q 1594 3097 1305 2825 \n",
|
||
"Q 1016 2553 972 2059 \n",
|
||
"L 3022 2063 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
|
||
"Q 1497 1759 1228 1600 \n",
|
||
"Q 959 1441 959 1056 \n",
|
||
"Q 959 750 1161 570 \n",
|
||
"Q 1363 391 1709 391 \n",
|
||
"Q 2188 391 2477 730 \n",
|
||
"Q 2766 1069 2766 1631 \n",
|
||
"L 2766 1759 \n",
|
||
"L 2194 1759 \n",
|
||
"z\n",
|
||
"M 3341 1997 \n",
|
||
"L 3341 0 \n",
|
||
"L 2766 0 \n",
|
||
"L 2766 531 \n",
|
||
"Q 2569 213 2275 61 \n",
|
||
"Q 1981 -91 1556 -91 \n",
|
||
"Q 1019 -91 701 211 \n",
|
||
"Q 384 513 384 1019 \n",
|
||
"Q 384 1609 779 1909 \n",
|
||
"Q 1175 2209 1959 2209 \n",
|
||
"L 2766 2209 \n",
|
||
"L 2766 2266 \n",
|
||
"Q 2766 2663 2505 2880 \n",
|
||
"Q 2244 3097 1772 3097 \n",
|
||
"Q 1472 3097 1187 3025 \n",
|
||
"Q 903 2953 641 2809 \n",
|
||
"L 641 3341 \n",
|
||
"Q 956 3463 1253 3523 \n",
|
||
"Q 1550 3584 1831 3584 \n",
|
||
"Q 2591 3584 2966 3190 \n",
|
||
"Q 3341 2797 3341 1997 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
|
||
"L 3513 0 \n",
|
||
"L 2938 0 \n",
|
||
"L 2938 2094 \n",
|
||
"Q 2938 2591 2744 2837 \n",
|
||
"Q 2550 3084 2163 3084 \n",
|
||
"Q 1697 3084 1428 2787 \n",
|
||
"Q 1159 2491 1159 1978 \n",
|
||
"L 1159 0 \n",
|
||
"L 581 0 \n",
|
||
"L 581 3500 \n",
|
||
"L 1159 3500 \n",
|
||
"L 1159 2956 \n",
|
||
"Q 1366 3272 1645 3428 \n",
|
||
"Q 1925 3584 2291 3584 \n",
|
||
"Q 2894 3584 3203 3211 \n",
|
||
"Q 3513 2838 3513 2113 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-2c\" d=\"M 750 794 \n",
|
||
"L 1409 794 \n",
|
||
"L 1409 256 \n",
|
||
"L 897 -744 \n",
|
||
"L 494 -744 \n",
|
||
"L 750 256 \n",
|
||
"L 750 794 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
|
||
"L 2834 2853 \n",
|
||
"Q 2591 2978 2328 3040 \n",
|
||
"Q 2066 3103 1784 3103 \n",
|
||
"Q 1356 3103 1142 2972 \n",
|
||
"Q 928 2841 928 2578 \n",
|
||
"Q 928 2378 1081 2264 \n",
|
||
"Q 1234 2150 1697 2047 \n",
|
||
"L 1894 2003 \n",
|
||
"Q 2506 1872 2764 1633 \n",
|
||
"Q 3022 1394 3022 966 \n",
|
||
"Q 3022 478 2636 193 \n",
|
||
"Q 2250 -91 1575 -91 \n",
|
||
"Q 1294 -91 989 -36 \n",
|
||
"Q 684 19 347 128 \n",
|
||
"L 347 722 \n",
|
||
"Q 666 556 975 473 \n",
|
||
"Q 1284 391 1588 391 \n",
|
||
"Q 1994 391 2212 530 \n",
|
||
"Q 2431 669 2431 922 \n",
|
||
"Q 2431 1156 2273 1281 \n",
|
||
"Q 2116 1406 1581 1522 \n",
|
||
"L 1381 1569 \n",
|
||
"Q 847 1681 609 1914 \n",
|
||
"Q 372 2147 372 2553 \n",
|
||
"Q 372 3047 722 3315 \n",
|
||
"Q 1072 3584 1716 3584 \n",
|
||
"Q 2034 3584 2315 3537 \n",
|
||
"Q 2597 3491 2834 3397 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
|
||
"L 1172 3500 \n",
|
||
"L 2356 3500 \n",
|
||
"L 2356 3053 \n",
|
||
"L 1172 3053 \n",
|
||
"L 1172 1153 \n",
|
||
"Q 1172 725 1289 603 \n",
|
||
"Q 1406 481 1766 481 \n",
|
||
"L 2356 481 \n",
|
||
"L 2356 0 \n",
|
||
"L 1766 0 \n",
|
||
"Q 1100 0 847 248 \n",
|
||
"Q 594 497 594 1153 \n",
|
||
"L 594 3053 \n",
|
||
"L 172 3053 \n",
|
||
"L 172 3500 \n",
|
||
"L 594 3500 \n",
|
||
"L 594 4494 \n",
|
||
"L 1172 4494 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" <path id=\"DejaVuSans-64\" d=\"M 2906 2969 \n",
|
||
"L 2906 4863 \n",
|
||
"L 3481 4863 \n",
|
||
"L 3481 0 \n",
|
||
"L 2906 0 \n",
|
||
"L 2906 525 \n",
|
||
"Q 2725 213 2448 61 \n",
|
||
"Q 2172 -91 1784 -91 \n",
|
||
"Q 1150 -91 751 415 \n",
|
||
"Q 353 922 353 1747 \n",
|
||
"Q 353 2572 751 3078 \n",
|
||
"Q 1150 3584 1784 3584 \n",
|
||
"Q 2172 3584 2448 3432 \n",
|
||
"Q 2725 3281 2906 2969 \n",
|
||
"z\n",
|
||
"M 947 1747 \n",
|
||
"Q 947 1113 1208 752 \n",
|
||
"Q 1469 391 1925 391 \n",
|
||
"Q 2381 391 2643 752 \n",
|
||
"Q 2906 1113 2906 1747 \n",
|
||
"Q 2906 2381 2643 2742 \n",
|
||
"Q 2381 3103 1925 3103 \n",
|
||
"Q 1469 3103 1208 2742 \n",
|
||
"Q 947 2381 947 1747 \n",
|
||
"z\n",
|
||
"\" transform=\"scale(0.015625)\"/>\n",
|
||
" </defs>\n",
|
||
" <use xlink:href=\"#DejaVuSans-6d\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-65\" x=\"97.412109\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-61\" x=\"158.935547\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-6e\" x=\"220.214844\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-20\" x=\"283.59375\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-30\" x=\"315.380859\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-2c\" x=\"379.003906\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-20\" x=\"410.791016\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-73\" x=\"442.578125\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-74\" x=\"494.677734\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-64\" x=\"533.886719\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-20\" x=\"597.363281\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-31\" x=\"629.150391\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_29\">\n",
|
||
" <path d=\"M 52.78125 34.976562 \n",
|
||
"L 62.78125 34.976562 \n",
|
||
"L 72.78125 34.976562 \n",
|
||
"\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_16\">\n",
|
||
" <!-- mean 0, std 2 -->\n",
|
||
" <g transform=\"translate(80.78125 38.476562)scale(0.1 -0.1)\">\n",
|
||
" <use xlink:href=\"#DejaVuSans-6d\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-65\" x=\"97.412109\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-61\" x=\"158.935547\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-6e\" x=\"220.214844\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-20\" x=\"283.59375\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-30\" x=\"315.380859\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-2c\" x=\"379.003906\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-20\" x=\"410.791016\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-73\" x=\"442.578125\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-74\" x=\"494.677734\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-64\" x=\"533.886719\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-20\" x=\"597.363281\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-32\" x=\"629.150391\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <g id=\"line2d_30\">\n",
|
||
" <path d=\"M 52.78125 49.654688 \n",
|
||
"L 62.78125 49.654688 \n",
|
||
"L 72.78125 49.654688 \n",
|
||
"\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
|
||
" </g>\n",
|
||
" <g id=\"text_17\">\n",
|
||
" <!-- mean 3, std 1 -->\n",
|
||
" <g transform=\"translate(80.78125 53.154688)scale(0.1 -0.1)\">\n",
|
||
" <use xlink:href=\"#DejaVuSans-6d\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-65\" x=\"97.412109\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-61\" x=\"158.935547\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-6e\" x=\"220.214844\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-20\" x=\"283.59375\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-33\" x=\"315.380859\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-2c\" x=\"379.003906\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-20\" x=\"410.791016\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-73\" x=\"442.578125\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-74\" x=\"494.677734\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-64\" x=\"533.886719\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-20\" x=\"597.363281\"/>\n",
|
||
" <use xlink:href=\"#DejaVuSans-31\" x=\"629.150391\"/>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" </g>\n",
|
||
" <defs>\n",
|
||
" <clipPath id=\"p0a634f4fb7\">\n",
|
||
" <rect x=\"43.78125\" y=\"7.2\" width=\"251.1\" height=\"135.9\"/>\n",
|
||
" </clipPath>\n",
|
||
" </defs>\n",
|
||
"</svg>\n"
|
||
],
|
||
"text/plain": [
|
||
"<Figure size 324x180 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"needs_background": "light"
|
||
},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"# 再次使用numpy进行可视化\n",
|
||
"x = np.arange(-7, 7, 0.01)\n",
|
||
"\n",
|
||
"# 均值和标准差对\n",
|
||
"params = [(0, 1), (0, 2), (3, 1)]\n",
|
||
"d2l.plot(x, [normal(x, mu, sigma) for mu, sigma in params], xlabel='x',\n",
|
||
" ylabel='p(x)', figsize=(4.5, 2.5),\n",
|
||
" legend=[f'mean {mu}, std {sigma}' for mu, sigma in params])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "7509f1f1",
|
||
"metadata": {
|
||
"origin_pos": 20
|
||
},
|
||
"source": [
|
||
"就像我们所看到的，改变均值会产生沿$x$轴的偏移，增加方差将会分散分布、降低其峰值。\n",
|
||
"\n",
|
||
"均方误差损失函数（简称均方损失）可以用于线性回归的一个原因是：\n",
|
||
"我们假设了观测中包含噪声，其中噪声服从正态分布。\n",
|
||
"噪声正态分布如下式:\n",
|
||
"\n",
|
||
"$$y = \\mathbf{w}^\\top \\mathbf{x} + b + \\epsilon,$$\n",
|
||
"\n",
|
||
"其中，$\\epsilon \\sim \\mathcal{N}(0, \\sigma^2)$。\n",
|
||
"\n",
|
||
"因此，我们现在可以写出通过给定的$\\mathbf{x}$观测到特定$y$的*似然*（likelihood）：\n",
|
||
"\n",
|
||
"$$P(y \\mid \\mathbf{x}) = \\frac{1}{\\sqrt{2 \\pi \\sigma^2}} \\exp\\left(-\\frac{1}{2 \\sigma^2} (y - \\mathbf{w}^\\top \\mathbf{x} - b)^2\\right).$$\n",
|
||
"\n",
|
||
"现在，根据极大似然估计法，参数$\\mathbf{w}$和$b$的最优值是使整个数据集的*似然*最大的值：\n",
|
||
"\n",
|
||
"$$P(\\mathbf y \\mid \\mathbf X) = \\prod_{i=1}^{n} p(y^{(i)}|\\mathbf{x}^{(i)}).$$\n",
|
||
"\n",
|
||
"根据极大似然估计法选择的估计量称为*极大似然估计量*。\n",
|
||
"虽然使许多指数函数的乘积最大化看起来很困难，\n",
|
||
"但是我们可以在不改变目标的前提下，通过最大化似然对数来简化。\n",
|
||
"由于历史原因，优化通常是说最小化而不是最大化。\n",
|
||
"我们可以改为*最小化负对数似然*$-\\log P(\\mathbf y \\mid \\mathbf X)$。\n",
|
||
"由此可以得到的数学公式是：\n",
|
||
"\n",
|
||
"$$-\\log P(\\mathbf y \\mid \\mathbf X) = \\sum_{i=1}^n \\frac{1}{2} \\log(2 \\pi \\sigma^2) + \\frac{1}{2 \\sigma^2} \\left(y^{(i)} - \\mathbf{w}^\\top \\mathbf{x}^{(i)} - b\\right)^2.$$\n",
|
||
"\n",
|
||
"现在我们只需要假设$\\sigma$是某个固定常数就可以忽略第一项，\n",
|
||
"因为第一项不依赖于$\\mathbf{w}$和$b$。\n",
|
||
"现在第二项除了常数$\\frac{1}{\\sigma^2}$外，其余部分和前面介绍的均方误差是一样的。\n",
|
||
"幸运的是，上面式子的解并不依赖于$\\sigma$。\n",
|
||
"因此，在高斯噪声的假设下，最小化均方误差等价于对线性模型的极大似然估计。\n",
|
||
"\n",
|
||
"## 从线性回归到深度网络\n",
|
||
"\n",
|
||
"到目前为止，我们只谈论了线性模型。\n",
|
||
"尽管神经网络涵盖了更多更为丰富的模型，我们依然可以用描述神经网络的方式来描述线性模型，\n",
|
||
"从而把线性模型看作一个神经网络。\n",
|
||
"首先，我们用“层”符号来重写这个模型。\n",
|
||
"\n",
|
||
"### 神经网络图\n",
|
||
"\n",
|
||
"深度学习从业者喜欢绘制图表来可视化模型中正在发生的事情。\n",
|
||
"在 :numref:`fig_single_neuron`中，我们将线性回归模型描述为一个神经网络。\n",
|
||
"需要注意的是，该图只显示连接模式，即只显示每个输入如何连接到输出，隐去了权重和偏置的值。\n",
|
||
"\n",
|
||
"\n",
|
||
":label:`fig_single_neuron`\n",
|
||
"\n",
|
||
"在 :numref:`fig_single_neuron`所示的神经网络中，输入为$x_1, \\ldots, x_d$，\n",
|
||
"因此输入层中的*输入数*（或称为*特征维度*，feature dimensionality）为$d$。\n",
|
||
"网络的输出为$o_1$，因此输出层中的*输出数*是1。\n",
|
||
"需要注意的是，输入值都是已经给定的，并且只有一个*计算*神经元。\n",
|
||
"由于模型重点在发生计算的地方，所以通常我们在计算层数时不考虑输入层。\n",
|
||
"也就是说， :numref:`fig_single_neuron`中神经网络的*层数*为1。\n",
|
||
"我们可以将线性回归模型视为仅由单个人工神经元组成的神经网络，或称为单层神经网络。\n",
|
||
"\n",
|
||
"对于线性回归，每个输入都与每个输出（在本例中只有一个输出）相连，\n",
|
||
"我们将这种变换（ :numref:`fig_single_neuron`中的输出层）\n",
|
||
"称为*全连接层*（fully-connected layer）或称为*稠密层*（dense layer）。\n",
|
||
"下一章将详细讨论由这些层组成的网络。\n",
|
||
"\n",
|
||
"### 生物学\n",
|
||
"\n",
|
||
"线性回归发明的时间（1795年）早于计算神经科学，所以将线性回归描述为神经网络似乎不合适。\n",
|
||
"当控制学家、神经生物学家沃伦·麦库洛奇和沃尔特·皮茨开始开发人工神经元模型时，\n",
|
||
"他们为什么将线性模型作为一个起点呢？\n",
|
||
"我们来看一张图片 :numref:`fig_Neuron`：\n",
|
||
"这是一张由*树突*（dendrites，输入终端）、\n",
|
||
"*细胞核*（nucleus，CPU）组成的生物神经元图片。\n",
|
||
"*轴突*（axon，输出线）和*轴突端子*（axon terminal，输出端子）\n",
|
||
"通过*突触*（synapse）与其他神经元连接。\n",
|
||
"\n",
|
||
"\n",
|
||
":label:`fig_Neuron`\n",
|
||
"\n",
|
||
"树突中接收到来自其他神经元（或视网膜等环境传感器）的信息$x_i$。\n",
|
||
"该信息通过*突触权重*$w_i$来加权，以确定输入的影响（即，通过$x_i w_i$相乘来激活或抑制）。\n",
|
||
"来自多个源的加权输入以加权和$y = \\sum_i x_i w_i + b$的形式汇聚在细胞核中，\n",
|
||
"然后将这些信息发送到轴突$y$中进一步处理，通常会通过$\\sigma(y)$进行一些非线性处理。\n",
|
||
"之后，它要么到达目的地（例如肌肉），要么通过树突进入另一个神经元。\n",
|
||
"\n",
|
||
"当然，许多这样的单元可以通过正确连接和正确的学习算法拼凑在一起，\n",
|
||
"从而产生的行为会比单独一个神经元所产生的行为更有趣、更复杂，\n",
|
||
"这种想法归功于我们对真实生物神经系统的研究。\n",
|
||
"\n",
|
||
"当今大多数深度学习的研究几乎没有直接从神经科学中获得灵感。\n",
|
||
"我们援引斯图尔特·罗素和彼得·诺维格在他们的经典人工智能教科书\n",
|
||
"*Artificial Intelligence:A Modern Approach* :cite:`Russell.Norvig.2016`\n",
|
||
"中所说的：虽然飞机可能受到鸟类的启发，但几个世纪以来，鸟类学并不是航空创新的主要驱动力。\n",
|
||
"同样地，如今在深度学习中的灵感同样或更多地来自数学、统计学和计算机科学。\n",
|
||
"\n",
|
||
"## 小结\n",
|
||
"\n",
|
||
"* 机器学习模型中的关键要素是训练数据、损失函数、优化算法，还有模型本身。\n",
|
||
"* 矢量化使数学表达上更简洁，同时运行的更快。\n",
|
||
"* 最小化目标函数和执行极大似然估计等价。\n",
|
||
"* 线性回归模型也是一个简单的神经网络。\n",
|
||
"\n",
|
||
"## 练习\n",
|
||
"\n",
|
||
"1. 假设我们有一些数据$x_1, \\ldots, x_n \\in \\mathbb{R}$。我们的目标是找到一个常数$b$，使得最小化$\\sum_i (x_i - b)^2$。\n",
|
||
" 1. 找到最优值$b$的解析解。\n",
|
||
" 1. 这个问题及其解与正态分布有什么关系?\n",
|
||
"1. 推导出使用平方误差的线性回归优化问题的解析解。为了简化问题，可以忽略偏置$b$（我们可以通过向$\\mathbf X$添加所有值为1的一列来做到这一点）。\n",
|
||
" 1. 用矩阵和向量表示法写出优化问题（将所有数据视为单个矩阵，将所有目标值视为单个向量）。\n",
|
||
" 1. 计算损失对$w$的梯度。\n",
|
||
" 1. 通过将梯度设为0、求解矩阵方程来找到解析解。\n",
|
||
" 1. 什么时候可能比使用随机梯度下降更好？这种方法何时会失效？\n",
|
||
"1. 假定控制附加噪声$\\epsilon$的噪声模型是指数分布。也就是说，$p(\\epsilon) = \\frac{1}{2} \\exp(-|\\epsilon|)$\n",
|
||
" 1. 写出模型$-\\log P(\\mathbf y \\mid \\mathbf X)$下数据的负对数似然。\n",
|
||
" 1. 请试着写出解析解。\n",
|
||
" 1. 提出一种随机梯度下降算法来解决这个问题。哪里可能出错？（提示：当我们不断更新参数时，在驻点附近会发生什么情况）请尝试解决这个问题。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3c59afc9",
|
||
"metadata": {
|
||
"origin_pos": 22,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"source": [
|
||
"[Discussions](https://discuss.d2l.ai/t/1775)\n"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"language_info": {
|
||
"name": "python"
|
||
},
|
||
"required_libs": []
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
} |