768 lines
22 KiB
Plaintext
768 lines
22 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "618fd23a",
|
||
"metadata": {
|
||
"origin_pos": 0
|
||
},
|
||
"source": [
|
||
"# GPU\n",
|
||
":label:`sec_use_gpu`\n",
|
||
"\n",
|
||
"在 :numref:`tab_intro_decade`中，\n",
|
||
"我们回顾了过去20年计算能力的快速增长。\n",
|
||
"简而言之，自2000年以来，GPU性能每十年增长1000倍。\n",
|
||
"\n",
|
||
"本节，我们将讨论如何利用这种计算性能进行研究。\n",
|
||
"首先是如何使用单个GPU，然后是如何使用多个GPU和多个服务器（具有多个GPU）。\n",
|
||
"\n",
|
||
"我们先看看如何使用单个NVIDIA GPU进行计算。\n",
|
||
"首先，确保至少安装了一个NVIDIA GPU。\n",
|
||
"然后，下载[NVIDIA驱动和CUDA](https://developer.nvidia.com/cuda-downloads)\n",
|
||
"并按照提示设置适当的路径。\n",
|
||
"当这些准备工作完成，就可以使用`nvidia-smi`命令来(**查看显卡信息。**)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "369d9baa",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T06:58:06.499888Z",
|
||
"iopub.status.busy": "2023-08-18T06:58:06.499324Z",
|
||
"iopub.status.idle": "2023-08-18T06:58:06.859541Z",
|
||
"shell.execute_reply": "2023-08-18T06:58:06.858210Z"
|
||
},
|
||
"origin_pos": 1,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Fri Aug 18 06:58:06 2023 \r\n",
|
||
"+-----------------------------------------------------------------------------+\r\n",
|
||
"| NVIDIA-SMI 470.161.03 Driver Version: 470.161.03 CUDA Version: 11.7 |\r\n",
|
||
"|-------------------------------+----------------------+----------------------+\r\n",
|
||
"| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\r\n",
|
||
"| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\r\n",
|
||
"| | | MIG M. |\r\n",
|
||
"|===============================+======================+======================|\r\n",
|
||
"| 0 Tesla V100-SXM2... Off | 00000000:00:1B.0 Off | 0 |\r\n",
|
||
"| N/A 41C P0 42W / 300W | 0MiB / 16160MiB | 0% Default |\r\n",
|
||
"| | | N/A |\r\n",
|
||
"+-------------------------------+----------------------+----------------------+\r\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"| 1 Tesla V100-SXM2... Off | 00000000:00:1C.0 Off | 0 |\r\n",
|
||
"| N/A 44C P0 113W / 300W | 1456MiB / 16160MiB | 53% Default |\r\n",
|
||
"| | | N/A |\r\n",
|
||
"+-------------------------------+----------------------+----------------------+\r\n",
|
||
"| 2 Tesla V100-SXM2... Off | 00000000:00:1D.0 Off | 0 |\r\n",
|
||
"| N/A 43C P0 120W / 300W | 1358MiB / 16160MiB | 55% Default |\r\n",
|
||
"| | | N/A |\r\n",
|
||
"+-------------------------------+----------------------+----------------------+\r\n",
|
||
"| 3 Tesla V100-SXM2... Off | 00000000:00:1E.0 Off | 0 |\r\n",
|
||
"| N/A 42C P0 47W / 300W | 0MiB / 16160MiB | 0% Default |\r\n",
|
||
"| | | N/A |\r\n",
|
||
"+-------------------------------+----------------------+----------------------+\r\n",
|
||
" \r\n",
|
||
"+-----------------------------------------------------------------------------+\r\n",
|
||
"| Processes: |\r\n",
|
||
"| GPU GI CI PID Type Process name GPU Memory |\r\n",
|
||
"| ID ID Usage |\r\n",
|
||
"|=============================================================================|\r\n",
|
||
"+-----------------------------------------------------------------------------+\r\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"!nvidia-smi"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "23e1982b",
|
||
"metadata": {
|
||
"origin_pos": 3,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"source": [
|
||
"在PyTorch中，每个数组都有一个设备（device），\n",
|
||
"我们通常将其称为环境（context）。\n",
|
||
"默认情况下，所有变量和相关的计算都分配给CPU。\n",
|
||
"有时环境可能是GPU。\n",
|
||
"当我们跨多个服务器部署作业时，事情会变得更加棘手。\n",
|
||
"通过智能地将数组分配给环境，\n",
|
||
"我们可以最大限度地减少在设备之间传输数据的时间。\n",
|
||
"例如，当在带有GPU的服务器上训练神经网络时，\n",
|
||
"我们通常希望模型的参数在GPU上。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "aeacf63c",
|
||
"metadata": {
|
||
"origin_pos": 5
|
||
},
|
||
"source": [
|
||
"要运行此部分中的程序，至少需要两个GPU。\n",
|
||
"注意，对大多数桌面计算机来说，这可能是奢侈的，但在云中很容易获得。\n",
|
||
"例如可以使用AWS EC2的多GPU实例。\n",
|
||
"本书的其他章节大都不需要多个GPU，\n",
|
||
"而本节只是为了展示数据如何在不同的设备之间传递。\n",
|
||
"\n",
|
||
"## [**计算设备**]\n",
|
||
"\n",
|
||
"我们可以指定用于存储和计算的设备，如CPU和GPU。\n",
|
||
"默认情况下，张量是在内存中创建的，然后使用CPU计算它。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "872e46f0",
|
||
"metadata": {
|
||
"origin_pos": 7,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"source": [
|
||
"在PyTorch中，CPU和GPU可以用`torch.device('cpu')`\n",
|
||
"和`torch.device('cuda')`表示。\n",
|
||
"应该注意的是，`cpu`设备意味着所有物理CPU和内存，\n",
|
||
"这意味着PyTorch的计算将尝试使用所有CPU核心。\n",
|
||
"然而，`gpu`设备只代表一个卡和相应的显存。\n",
|
||
"如果有多个GPU，我们使用`torch.device(f'cuda:{i}')`\n",
|
||
"来表示第$i$块GPU（$i$从0开始）。\n",
|
||
"另外，`cuda:0`和`cuda`是等价的。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "9f69ad46",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T06:58:06.865430Z",
|
||
"iopub.status.busy": "2023-08-18T06:58:06.864979Z",
|
||
"iopub.status.idle": "2023-08-18T06:58:07.970615Z",
|
||
"shell.execute_reply": "2023-08-18T06:58:07.969801Z"
|
||
},
|
||
"origin_pos": 10,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"(device(type='cpu'), device(type='cuda'), device(type='cuda', index=1))"
|
||
]
|
||
},
|
||
"execution_count": 2,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"import torch\n",
|
||
"from torch import nn\n",
|
||
"\n",
|
||
"torch.device('cpu'), torch.device('cuda'), torch.device('cuda:1')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "248784cc",
|
||
"metadata": {
|
||
"origin_pos": 13
|
||
},
|
||
"source": [
|
||
"我们可以(**查询可用gpu的数量。**)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "c29151b0",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T06:58:07.974568Z",
|
||
"iopub.status.busy": "2023-08-18T06:58:07.973917Z",
|
||
"iopub.status.idle": "2023-08-18T06:58:07.979097Z",
|
||
"shell.execute_reply": "2023-08-18T06:58:07.978337Z"
|
||
},
|
||
"origin_pos": 15,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"2"
|
||
]
|
||
},
|
||
"execution_count": 3,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"torch.cuda.device_count()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "6e1bc4a6",
|
||
"metadata": {
|
||
"origin_pos": 18
|
||
},
|
||
"source": [
|
||
"现在我们定义了两个方便的函数，\n",
|
||
"[**这两个函数允许我们在不存在所需所有GPU的情况下运行代码。**]\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "cda0ab76",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T06:58:07.983261Z",
|
||
"iopub.status.busy": "2023-08-18T06:58:07.982604Z",
|
||
"iopub.status.idle": "2023-08-18T06:58:07.990309Z",
|
||
"shell.execute_reply": "2023-08-18T06:58:07.989541Z"
|
||
},
|
||
"origin_pos": 20,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"(device(type='cuda', index=0),\n",
|
||
" device(type='cpu'),\n",
|
||
" [device(type='cuda', index=0), device(type='cuda', index=1)])"
|
||
]
|
||
},
|
||
"execution_count": 4,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"def try_gpu(i=0): #@save\n",
|
||
" \"\"\"如果存在，则返回gpu(i)，否则返回cpu()\"\"\"\n",
|
||
" if torch.cuda.device_count() >= i + 1:\n",
|
||
" return torch.device(f'cuda:{i}')\n",
|
||
" return torch.device('cpu')\n",
|
||
"\n",
|
||
"def try_all_gpus(): #@save\n",
|
||
" \"\"\"返回所有可用的GPU，如果没有GPU，则返回[cpu(),]\"\"\"\n",
|
||
" devices = [torch.device(f'cuda:{i}')\n",
|
||
" for i in range(torch.cuda.device_count())]\n",
|
||
" return devices if devices else [torch.device('cpu')]\n",
|
||
"\n",
|
||
"try_gpu(), try_gpu(10), try_all_gpus()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "034b0d3b",
|
||
"metadata": {
|
||
"origin_pos": 23
|
||
},
|
||
"source": [
|
||
"## 张量与GPU\n",
|
||
"\n",
|
||
"我们可以[**查询张量所在的设备。**]\n",
|
||
"默认情况下，张量是在CPU上创建的。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"id": "f6ab0f26",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T06:58:07.994741Z",
|
||
"iopub.status.busy": "2023-08-18T06:58:07.994126Z",
|
||
"iopub.status.idle": "2023-08-18T06:58:07.999439Z",
|
||
"shell.execute_reply": "2023-08-18T06:58:07.998673Z"
|
||
},
|
||
"origin_pos": 25,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"device(type='cpu')"
|
||
]
|
||
},
|
||
"execution_count": 5,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"x = torch.tensor([1, 2, 3])\n",
|
||
"x.device"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f39b0efa",
|
||
"metadata": {
|
||
"origin_pos": 28
|
||
},
|
||
"source": [
|
||
"需要注意的是，无论何时我们要对多个项进行操作，\n",
|
||
"它们都必须在同一个设备上。\n",
|
||
"例如，如果我们对两个张量求和，\n",
|
||
"我们需要确保两个张量都位于同一个设备上，\n",
|
||
"否则框架将不知道在哪里存储结果，甚至不知道在哪里执行计算。\n",
|
||
"\n",
|
||
"### [**存储在GPU上**]\n",
|
||
"\n",
|
||
"有几种方法可以在GPU上存储张量。\n",
|
||
"例如，我们可以在创建张量时指定存储设备。接\n",
|
||
"下来，我们在第一个`gpu`上创建张量变量`X`。\n",
|
||
"在GPU上创建的张量只消耗这个GPU的显存。\n",
|
||
"我们可以使用`nvidia-smi`命令查看显存使用情况。\n",
|
||
"一般来说，我们需要确保不创建超过GPU显存限制的数据。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "a67dbf2f",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T06:58:08.004162Z",
|
||
"iopub.status.busy": "2023-08-18T06:58:08.003541Z",
|
||
"iopub.status.idle": "2023-08-18T06:58:09.277879Z",
|
||
"shell.execute_reply": "2023-08-18T06:58:09.277008Z"
|
||
},
|
||
"origin_pos": 30,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"tensor([[1., 1., 1.],\n",
|
||
" [1., 1., 1.]], device='cuda:0')"
|
||
]
|
||
},
|
||
"execution_count": 6,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"X = torch.ones(2, 3, device=try_gpu())\n",
|
||
"X"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "dd17f6d7",
|
||
"metadata": {
|
||
"origin_pos": 33
|
||
},
|
||
"source": [
|
||
"假设我们至少有两个GPU，下面的代码将在(**第二个GPU上创建一个随机张量。**)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"id": "7c0d4a84",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T06:58:09.282814Z",
|
||
"iopub.status.busy": "2023-08-18T06:58:09.282230Z",
|
||
"iopub.status.idle": "2023-08-18T06:58:10.279046Z",
|
||
"shell.execute_reply": "2023-08-18T06:58:10.278227Z"
|
||
},
|
||
"origin_pos": 35,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"tensor([[0.4860, 0.1285, 0.0440],\n",
|
||
" [0.9743, 0.4159, 0.9979]], device='cuda:1')"
|
||
]
|
||
},
|
||
"execution_count": 7,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"Y = torch.rand(2, 3, device=try_gpu(1))\n",
|
||
"Y"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "71646fa2",
|
||
"metadata": {
|
||
"origin_pos": 38
|
||
},
|
||
"source": [
|
||
"### 复制\n",
|
||
"\n",
|
||
"如果我们[**要计算`X + Y`，我们需要决定在哪里执行这个操作**]。\n",
|
||
"例如，如 :numref:`fig_copyto`所示，\n",
|
||
"我们可以将`X`传输到第二个GPU并在那里执行操作。\n",
|
||
"*不要*简单地`X`加上`Y`，因为这会导致异常，\n",
|
||
"运行时引擎不知道该怎么做：它在同一设备上找不到数据会导致失败。\n",
|
||
"由于`Y`位于第二个GPU上，所以我们需要将`X`移到那里，\n",
|
||
"然后才能执行相加运算。\n",
|
||
"\n",
|
||
"\n",
|
||
":label:`fig_copyto`\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"id": "9e700cd2",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T06:58:10.284097Z",
|
||
"iopub.status.busy": "2023-08-18T06:58:10.283529Z",
|
||
"iopub.status.idle": "2023-08-18T06:58:10.290795Z",
|
||
"shell.execute_reply": "2023-08-18T06:58:10.290007Z"
|
||
},
|
||
"origin_pos": 40,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"tensor([[1., 1., 1.],\n",
|
||
" [1., 1., 1.]], device='cuda:0')\n",
|
||
"tensor([[1., 1., 1.],\n",
|
||
" [1., 1., 1.]], device='cuda:1')\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"Z = X.cuda(1)\n",
|
||
"print(X)\n",
|
||
"print(Z)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f57eab12",
|
||
"metadata": {
|
||
"origin_pos": 42
|
||
},
|
||
"source": [
|
||
"[**现在数据在同一个GPU上（`Z`和`Y`都在），我们可以将它们相加。**]\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"id": "b2f04f35",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T06:58:10.295377Z",
|
||
"iopub.status.busy": "2023-08-18T06:58:10.294845Z",
|
||
"iopub.status.idle": "2023-08-18T06:58:10.301122Z",
|
||
"shell.execute_reply": "2023-08-18T06:58:10.300297Z"
|
||
},
|
||
"origin_pos": 43,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"tensor([[1.4860, 1.1285, 1.0440],\n",
|
||
" [1.9743, 1.4159, 1.9979]], device='cuda:1')"
|
||
]
|
||
},
|
||
"execution_count": 9,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"Y + Z"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "9acbe573",
|
||
"metadata": {
|
||
"origin_pos": 45,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"source": [
|
||
"假设变量`Z`已经存在于第二个GPU上。\n",
|
||
"如果我们还是调用`Z.cuda(1)`会发生什么？\n",
|
||
"它将返回`Z`，而不会复制并分配新内存。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 10,
|
||
"id": "d6b95aa1",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T06:58:10.305143Z",
|
||
"iopub.status.busy": "2023-08-18T06:58:10.304592Z",
|
||
"iopub.status.idle": "2023-08-18T06:58:10.309707Z",
|
||
"shell.execute_reply": "2023-08-18T06:58:10.308894Z"
|
||
},
|
||
"origin_pos": 48,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"True"
|
||
]
|
||
},
|
||
"execution_count": 10,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"Z.cuda(1) is Z"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "35568455",
|
||
"metadata": {
|
||
"origin_pos": 50
|
||
},
|
||
"source": [
|
||
"### 旁注\n",
|
||
"\n",
|
||
"人们使用GPU来进行机器学习，因为单个GPU相对运行速度快。\n",
|
||
"但是在设备（CPU、GPU和其他机器）之间传输数据比计算慢得多。\n",
|
||
"这也使得并行化变得更加困难，因为我们必须等待数据被发送（或者接收），\n",
|
||
"然后才能继续进行更多的操作。\n",
|
||
"这就是为什么拷贝操作要格外小心。\n",
|
||
"根据经验，多个小操作比一个大操作糟糕得多。\n",
|
||
"此外，一次执行几个操作比代码中散布的许多单个操作要好得多。\n",
|
||
"如果一个设备必须等待另一个设备才能执行其他操作，\n",
|
||
"那么这样的操作可能会阻塞。\n",
|
||
"这有点像排队订购咖啡，而不像通过电话预先订购：\n",
|
||
"当客人到店的时候，咖啡已经准备好了。\n",
|
||
"\n",
|
||
"最后，当我们打印张量或将张量转换为NumPy格式时，\n",
|
||
"如果数据不在内存中，框架会首先将其复制到内存中，\n",
|
||
"这会导致额外的传输开销。\n",
|
||
"更糟糕的是，它现在受制于全局解释器锁，使得一切都得等待Python完成。\n",
|
||
"\n",
|
||
"## [**神经网络与GPU**]\n",
|
||
"\n",
|
||
"类似地，神经网络模型可以指定设备。\n",
|
||
"下面的代码将模型参数放在GPU上。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 11,
|
||
"id": "587af904",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T06:58:10.313163Z",
|
||
"iopub.status.busy": "2023-08-18T06:58:10.312623Z",
|
||
"iopub.status.idle": "2023-08-18T06:58:10.336351Z",
|
||
"shell.execute_reply": "2023-08-18T06:58:10.335568Z"
|
||
},
|
||
"origin_pos": 52,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"net = nn.Sequential(nn.Linear(3, 1))\n",
|
||
"net = net.to(device=try_gpu())"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "a834a04c",
|
||
"metadata": {
|
||
"origin_pos": 55
|
||
},
|
||
"source": [
|
||
"在接下来的几章中，\n",
|
||
"我们将看到更多关于如何在GPU上运行模型的例子，\n",
|
||
"因为它们将变得更加计算密集。\n",
|
||
"\n",
|
||
"当输入为GPU上的张量时，模型将在同一GPU上计算结果。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 12,
|
||
"id": "955f7f67",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T06:58:10.340989Z",
|
||
"iopub.status.busy": "2023-08-18T06:58:10.340312Z",
|
||
"iopub.status.idle": "2023-08-18T06:58:10.930969Z",
|
||
"shell.execute_reply": "2023-08-18T06:58:10.930143Z"
|
||
},
|
||
"origin_pos": 56,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"tensor([[-0.4275],\n",
|
||
" [-0.4275]], device='cuda:0', grad_fn=<AddmmBackward0>)"
|
||
]
|
||
},
|
||
"execution_count": 12,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"net(X)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "fb9f9aef",
|
||
"metadata": {
|
||
"origin_pos": 57
|
||
},
|
||
"source": [
|
||
"让我们(**确认模型参数存储在同一个GPU上。**)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 13,
|
||
"id": "bd727993",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2023-08-18T06:58:10.935087Z",
|
||
"iopub.status.busy": "2023-08-18T06:58:10.934497Z",
|
||
"iopub.status.idle": "2023-08-18T06:58:10.939740Z",
|
||
"shell.execute_reply": "2023-08-18T06:58:10.938974Z"
|
||
},
|
||
"origin_pos": 59,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"device(type='cuda', index=0)"
|
||
]
|
||
},
|
||
"execution_count": 13,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"net[0].weight.data.device"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "cf1bf3b2",
|
||
"metadata": {
|
||
"origin_pos": 62
|
||
},
|
||
"source": [
|
||
"总之，只要所有的数据和参数都在同一个设备上，\n",
|
||
"我们就可以有效地学习模型。\n",
|
||
"在下面的章节中，我们将看到几个这样的例子。\n",
|
||
"\n",
|
||
"## 小结\n",
|
||
"\n",
|
||
"* 我们可以指定用于存储和计算的设备，例如CPU或GPU。默认情况下，数据在主内存中创建，然后使用CPU进行计算。\n",
|
||
"* 深度学习框架要求计算的所有输入数据都在同一设备上，无论是CPU还是GPU。\n",
|
||
"* 不经意地移动数据可能会显著降低性能。一个典型的错误如下：计算GPU上每个小批量的损失，并在命令行中将其报告给用户（或将其记录在NumPy `ndarray`中）时，将触发全局解释器锁，从而使所有GPU阻塞。最好是为GPU内部的日志分配内存，并且只移动较大的日志。\n",
|
||
"\n",
|
||
"## 练习\n",
|
||
"\n",
|
||
"1. 尝试一个计算量更大的任务，比如大矩阵的乘法，看看CPU和GPU之间的速度差异。再试一个计算量很小的任务呢？\n",
|
||
"1. 我们应该如何在GPU上读写模型参数？\n",
|
||
"1. 测量计算1000个$100 \\times 100$矩阵的矩阵乘法所需的时间，并记录输出矩阵的Frobenius范数，一次记录一个结果，而不是在GPU上保存日志并仅传输最终结果。\n",
|
||
"1. 测量同时在两个GPU上执行两个矩阵乘法与在一个GPU上按顺序执行两个矩阵乘法所需的时间。提示：应该看到近乎线性的缩放。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0460f3be",
|
||
"metadata": {
|
||
"origin_pos": 64,
|
||
"tab": [
|
||
"pytorch"
|
||
]
|
||
},
|
||
"source": [
|
||
"[Discussions](https://discuss.d2l.ai/t/1841)\n"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"language_info": {
|
||
"name": "python"
|
||
},
|
||
"required_libs": []
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
} |