{
"cells": [
{
"cell_type": "markdown",
"id": "2dc00e55",
"metadata": {
"origin_pos": 0
},
"source": [
"# 目标检测和边界框\n",
":label:`sec_bbox`\n",
"\n",
"前面的章节（例如 :numref:`sec_alexnet`— :numref:`sec_googlenet`）介绍了各种图像分类模型。\n",
"在图像分类任务中，我们假设图像中只有一个主要物体对象，我们只关注如何识别其类别。\n",
"然而，很多时候图像里有多个我们感兴趣的目标，我们不仅想知道它们的类别，还想得到它们在图像中的具体位置。\n",
"在计算机视觉里，我们将这类任务称为*目标检测*（object detection）或*目标识别*（object recognition）。\n",
"\n",
"目标检测在多个领域中被广泛使用。\n",
"例如，在无人驾驶里，我们需要通过识别拍摄到的视频图像里的车辆、行人、道路和障碍物的位置来规划行进线路。\n",
"机器人也常通过该任务来检测感兴趣的目标。安防领域则需要检测异常目标，如歹徒或者炸弹。\n",
"\n",
"接下来的几节将介绍几种用于目标检测的深度学习方法。\n",
"我们将首先介绍目标的*位置*。\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "587086b2",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:03:18.634362Z",
"iopub.status.busy": "2023-08-18T07:03:18.633524Z",
"iopub.status.idle": "2023-08-18T07:03:21.225101Z",
"shell.execute_reply": "2023-08-18T07:03:21.223830Z"
},
"origin_pos": 2,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import torch\n",
"from d2l import torch as d2l"
]
},
{
"cell_type": "markdown",
"id": "8745073e",
"metadata": {
"origin_pos": 5
},
"source": [
"下面加载本节将使用的示例图像。可以看到图像左边是一只狗，右边是一只猫。\n",
"它们是这张图像里的两个主要目标。\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "ca6c12ce",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:03:21.231076Z",
"iopub.status.busy": "2023-08-18T07:03:21.230144Z",
"iopub.status.idle": "2023-08-18T07:03:21.553660Z",
"shell.execute_reply": "2023-08-18T07:03:21.552412Z"
},
"origin_pos": 7,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"d2l.set_figsize()\n",
"img = d2l.plt.imread('../img/catdog.jpg')\n",
"d2l.plt.imshow(img);"
]
},
{
"cell_type": "markdown",
"id": "1b37e085",
"metadata": {
"origin_pos": 8
},
"source": [
"## 边界框\n",
"\n",
"在目标检测中，我们通常使用*边界框*（bounding box）来描述对象的空间位置。\n",
"边界框是矩形的，由矩形左上角的以及右下角的$x$和$y$坐标决定。\n",
"另一种常用的边界框表示方法是边界框中心的$(x, y)$轴坐标以及框的宽度和高度。\n",
"\n",
"在这里，我们[**定义在这两种表示法之间进行转换的函数**]：`box_corner_to_center`从两角表示法转换为中心宽度表示法，而`box_center_to_corner`反之亦然。\n",
"输入参数`boxes`可以是长度为4的张量，也可以是形状为（$n$，4）的二维张量，其中$n$是边界框的数量。\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "396275f2",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:03:21.559910Z",
"iopub.status.busy": "2023-08-18T07:03:21.559503Z",
"iopub.status.idle": "2023-08-18T07:03:21.570997Z",
"shell.execute_reply": "2023-08-18T07:03:21.569943Z"
},
"origin_pos": 9,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"#@save\n",
"def box_corner_to_center(boxes):\n",
" \"\"\"从（左上，右下）转换到（中间，宽度，高度）\"\"\"\n",
" x1, y1, x2, y2 = boxes[:, 0], boxes[:, 1], boxes[:, 2], boxes[:, 3]\n",
" cx = (x1 + x2) / 2\n",
" cy = (y1 + y2) / 2\n",
" w = x2 - x1\n",
" h = y2 - y1\n",
" boxes = torch.stack((cx, cy, w, h), axis=-1)\n",
" return boxes\n",
"\n",
"#@save\n",
"def box_center_to_corner(boxes):\n",
" \"\"\"从（中间，宽度，高度）转换到（左上，右下）\"\"\"\n",
" cx, cy, w, h = boxes[:, 0], boxes[:, 1], boxes[:, 2], boxes[:, 3]\n",
" x1 = cx - 0.5 * w\n",
" y1 = cy - 0.5 * h\n",
" x2 = cx + 0.5 * w\n",
" y2 = cy + 0.5 * h\n",
" boxes = torch.stack((x1, y1, x2, y2), axis=-1)\n",
" return boxes"
]
},
{
"cell_type": "markdown",
"id": "de7237f0",
"metadata": {
"origin_pos": 10
},
"source": [
"我们将根据坐标信息[**定义图像中狗和猫的边界框**]。\n",
"图像中坐标的原点是图像的左上角，向右的方向为$x$轴的正方向，向下的方向为$y$轴的正方向。\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "7847c0b0",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:03:21.575847Z",
"iopub.status.busy": "2023-08-18T07:03:21.575128Z",
"iopub.status.idle": "2023-08-18T07:03:21.580290Z",
"shell.execute_reply": "2023-08-18T07:03:21.579258Z"
},
"origin_pos": 11,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"# bbox是边界框的英文缩写\n",
"dog_bbox, cat_bbox = [60.0, 45.0, 378.0, 516.0], [400.0, 112.0, 655.0, 493.0]"
]
},
{
"cell_type": "markdown",
"id": "bd3a0db4",
"metadata": {
"origin_pos": 12
},
"source": [
"我们可以通过转换两次来验证边界框转换函数的正确性。\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "9d2b1c9f",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:03:21.584966Z",
"iopub.status.busy": "2023-08-18T07:03:21.584258Z",
"iopub.status.idle": "2023-08-18T07:03:21.612392Z",
"shell.execute_reply": "2023-08-18T07:03:21.611303Z"
},
"origin_pos": 13,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"text/plain": [
"tensor([[True, True, True, True],\n",
" [True, True, True, True]])"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"boxes = torch.tensor((dog_bbox, cat_bbox))\n",
"box_center_to_corner(box_corner_to_center(boxes)) == boxes"
]
},
{
"cell_type": "markdown",
"id": "d6416af2",
"metadata": {
"origin_pos": 14
},
"source": [
"我们可以[**将边界框在图中画出**]，以检查其是否准确。\n",
"画之前，我们定义一个辅助函数`bbox_to_rect`。\n",
"它将边界框表示成`matplotlib`的边界框格式。\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "8608613e",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:03:21.617749Z",
"iopub.status.busy": "2023-08-18T07:03:21.616867Z",
"iopub.status.idle": "2023-08-18T07:03:21.623750Z",
"shell.execute_reply": "2023-08-18T07:03:21.622704Z"
},
"origin_pos": 15,
"tab": [
"pytorch"
]
},
"outputs": [],
"source": [
"#@save\n",
"def bbox_to_rect(bbox, color):\n",
" # 将边界框(左上x,左上y,右下x,右下y)格式转换成matplotlib格式：\n",
" # ((左上x,左上y),宽,高)\n",
" return d2l.plt.Rectangle(\n",
" xy=(bbox[0], bbox[1]), width=bbox[2]-bbox[0], height=bbox[3]-bbox[1],\n",
" fill=False, edgecolor=color, linewidth=2)"
]
},
{
"cell_type": "markdown",
"id": "ff1f3379",
"metadata": {
"origin_pos": 16
},
"source": [
"在图像上添加边界框之后，我们可以看到两个物体的主要轮廓基本上在两个框内。\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "c14af7d0",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T07:03:21.628443Z",
"iopub.status.busy": "2023-08-18T07:03:21.627843Z",
"iopub.status.idle": "2023-08-18T07:03:21.938549Z",
"shell.execute_reply": "2023-08-18T07:03:21.937424Z"
},
"origin_pos": 17,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"fig = d2l.plt.imshow(img)\n",
"fig.axes.add_patch(bbox_to_rect(dog_bbox, 'blue'))\n",
"fig.axes.add_patch(bbox_to_rect(cat_bbox, 'red'));"
]
},
{
"cell_type": "markdown",
"id": "1730012f",
"metadata": {
"origin_pos": 18
},
"source": [
"## 小结\n",
"\n",
"* 目标检测不仅可以识别图像中所有感兴趣的物体，还能识别它们的位置，该位置通常由矩形边界框表示。\n",
"* 我们可以在两种常用的边界框表示（中间，宽度，高度）和（左上，右下）坐标之间进行转换。\n",
"\n",
"## 练习\n",
"\n",
"1. 找到另一张图像，然后尝试标记包含该对象的边界框。比较标注边界框和标注类别哪个需要更长的时间？\n",
"1. 为什么`box_corner_to_center`和`box_center_to_corner`的输入参数的最内层维度总是4？\n"
]
},
{
"cell_type": "markdown",
"id": "c9770a7c",
"metadata": {
"origin_pos": 20,
"tab": [
"pytorch"
]
},
"source": [
"[Discussions](https://discuss.d2l.ai/t/2944)\n"
]
}
],
"metadata": {
"language_info": {
"name": "python"
},
"required_libs": []
},
"nbformat": 4,
"nbformat_minor": 5
}