卷积神经网络基础概念理解（二）

电玩女神 2023-09-24 13:50 47阅读 0赞

### 基于 PyTorch 的手写数字识别 ###

*  一 背景说明
 *   *  1.数据集
     *  2.框架 Pytorch
 *  二 准备工作
 *   *  1.数据集处理
     *  2.网络
 *  三 按步实现
 *   *  1.下载数据集
     *  2.标准化参数
     *  3.数据集查看
     *  4.网络结构设计
     *  5.完整训练程序
     *  6.结果打印
     *  7.手写图片验证
     *   *  1.进行多次训练
         *  2.测试

# 一 背景说明 #

后学末进，多为参考前人经验所得，文中如有谬误，请大家不吝促教，及时留言指正，以免不当之处误人子弟

## 1.数据集 ##

利用 MNIST 数据集实现一个手写数字识别的卷积神经网络实现，堪称人工智能界的"Hello World"
    MNIST 包含的是尺寸为 28 x 28 的数字灰度图，以及数字标签，分为一个训练集和一个测试集

[MINST 数据地址][MINST]

<table> 
 <thead> 
  <tr> 
   <th>文件</th> 
   <th>说明（样本与标签一一对应）</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>train-images-idx3-ubyte.gz</td> 
   <td>样本数：60000</td> 
  </tr> 
  <tr> 
   <td>train-labels-idx1-ubyte.gz</td> 
   <td>标签数：60000</td> 
  </tr> 
  <tr> 
   <td>t10k-images-idx3-ubyte.gz</td> 
   <td>样本数：10000</td> 
  </tr> 
  <tr> 
   <td>t10k-labels-idx1-ubyte.gz</td> 
   <td>标签数：10000</td> 
  </tr> 
 </tbody> 
</table>

## 2.框架 Pytorch ##

PyTorch是一个开源的Python机器学习库
    已兼容 Windows(CUDA,CPU)、MacOS(CPU)、Linux(CUDA,ROCm,CPU)
    其他深度学习框架还有诸如：PaddlePaddle、Tensorflow、 Caffe等

[Pytorch 文档地址][Pytorch]

# 二 准备工作 #

## 1.数据集处理 ##

数据标准化：对输入数据做归一化处理，使输入值尽可能平滑分区，缩小值域区间
    所以标准化又称为：特征缩放
    为什么要进行数据标准化呢？我们试想，红苹果、绿苹果，于我们人类而言，都是苹果，但是于机器而言，
    输入图就是一组矩阵值，在卷积计算后，可能突出红色特征或者绿色特征，然后我们给它的参考值是苹果，
    机器就可能反复修正网络权重去兼容红、绿特征，如果不影响收敛还好，如果有影响网络训练再久也是没
    结果，收敛曲线反复横跳
    除此外，亮度、不同度量单位的特征数据导致数据分布高度离散，都不利于学习收敛
    包括 R G B 通道值，可能差异很大，不加以处理会导致无法使训练输出值理想化
    怎么归一化？
    举个例子：  一组正整数集合，归一化到 (0,1] 的区间，怎么处理？

<table> 
 <thead> 
  <tr> 
   <th>算法</th> 
   <th>公式</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>离差标准化</td> 
   <td>x = (x - min)/(max - min)</td> 
  </tr> 
  <tr> 
   <td>z-score标准化</td> 
   <td>x = (x - μ)/σ 其中 μ 平均数 σ 标准差</td> 
  </tr> 
  <tr> 
   <td>反正切函数标准化</td> 
   <td></td> 
  </tr> 
 </tbody> 
</table>

## 2.网络 ##

前面我们说了很多关于卷积的概念，但是到底该如何基于理论，去做一个实现呢？
    首先，我们应该思考一下，卷积能提取特征，但这些特征值是怎么使用的呢；我们说了池化、全连接、激
    活，但是其实还不全面，如何根据卷积结果，和预期结果，来调整入参，重复计算，直至得到我们要的值，才是完整的学习方案

# 三 按步实现 #

环境版本

![在这里插入图片描述][1a369ace77db48319ddb0e2db075e8e0.png]

程序已添加详细注释，关于 Python 语法或者 Torch 函数，不理解的可以分别查看网络资料与

上述 Torch 官网查询函数名称

## 1.下载数据集 ##

可以去网站下载并放到程序目录，或者我们通过程序自动首次下载
    代码如下

import torch
    import torchvision
    
    ## 如果数据不存在 则下载数据到当前目录下的 data 下
    train_set=torchvision.datasets.MNIST('./data/', train=True, download=True,
               transform=torchvision.transforms.Compose([
                   torchvision.transforms.ToTensor()
               ]))
    
    test_set=torchvision.datasets.MNIST('./data/', train=False, download=True,
               transform=torchvision.transforms.Compose([
                   torchvision.transforms.ToTensor()
               ]))

![在这里插入图片描述][f98b75b2a7ec47e0b142dc2d6cc1cd39.png]

## 2.标准化参数 ##

使用 z-score 标准化算法，Pytorch 能直接计算出参数值

import torch
    import torchvision
    from torch.utils.data import DataLoader
    
    train_set = torchvision.datasets.MNIST('./data/', train=True, download=True,
                                   transform=torchvision.transforms.Compose([
                                       torchvision.transforms.ToTensor()
                                   ]))
    
    loader=torch.utils.data.DataLoader(
        train_set,batch_size=len(train_set),num_workers=1
        )
    
    data=next(iter(loader))
    
    print(data[0].mean(),",",data[0].std())

![在这里插入图片描述][2612dc3043d44021861d799009972451.png]

对于当前数据集，这里取了全量数据进行计算；实际处理时数据是分批操作的

## 3.数据集查看 ##

import torch
    import torchvision
    import cv2
    import numpy as np
    
    ## 超参数（不随迭代更新）
    batch_size_train=64
    
    ## 加载数据
    train_loader=torch.utils.data.DataLoader(
        torchvision.datasets.MNIST('./data/', train=True, download=True,
                                   transform=torchvision.transforms.Compose([
                                       torchvision.transforms.ToTensor(),
                                       torchvision.transforms.Normalize(
                                           (0.1307,), (0.3081,))
                                   ])),
        batch_size=batch_size_train, shuffle=True)
    
    examples = enumerate(train_loader)
    batch_idx, (example_data, example_val) = next(examples)
    
    
    ## 打印批次、数字值数据结构、灰度图像素数据结构
    print("batch_idx:",batch_idx)
    print("val_shape",example_val.shape)
    print("data shape",example_data.shape)
    
    ## 将数字值拆分合并一下，按 8 * 8 打印出来 
    val_temp=torch.zeros(8,8)
    for x in range(8):
        for y in range(8):
            val_temp[x][y] = example_val[ x*8 + y ]
        
    print("val_8_8",val_temp)
    
    ## 遍历像素数据矩阵 64 * 1* 28 * 28 将每次矩阵转为 np 在拼接
    for x in range(8):
        start = x * 8
        end = (x + 1) * 8
        np_temp = np.squeeze(torch.sigmoid(example_data[start]).cpu().data.numpy())
        for y in range(start + 1,end):
            temp=np.squeeze(torch.sigmoid(example_data[y]).cpu().data.numpy())
            np_temp = np.hstack((np_temp,temp))
        if x == 0:
            np_result = np_temp
        else:
            np_result = np.vstack((np_result,np_temp))
    
    cv2.imshow('Result Image',np_result)
    cv2.waitKey(0)

执行上述程序，我们将第一批次 64 个数据转为图片，查看效果

![在这里插入图片描述][5bad35c60aba4cbbba6d62097478a759.png]

## 4.网络结构设计 ##

![在这里插入图片描述][d67747dcced64159835c7d6a734cad7d.png]

继承 torch 的 module 定义网络

## 定义一个网络类,并继承 torch.nn.Module
    class Net(nn.Module):
    
        ## 显示定义构造方法 __init__ 时,除额外参数,第一个参必须为对象自身关键字 self
        def __init__(self):
    
            ## 调用父类初始化方法
            super(Net, self).__init__()
            ## 定义卷积核 输入通道数 输出通道数 卷积核尺寸 nn.Conv2d(in_cahnnels=1, out_channels=10, kernel_size=5) 默认步长为 1 不填充
            self.conv1 = nn.Conv2d(1, 10, 5)
            self.conv2 = nn.Conv2d(10, 20, 5)
            ## 定义通道值随机清零处理方法 在多参数、小样本的场景下,网络容易过拟合,现象为训练集里表现很好,但在测试集并不友好,原因如下
            ## 1.多参数、小样本情况下,由于不同特征间的复杂依赖关系,使得我们的网络难以厘出细粒度的特征区别,比如1000个特征,有600个一致,100不同,300无参考价值
            ##   实际起到作用的100特征在所有特整里面所占比例太低,虽然在训练集里面有一个良好表现,但换到测试集完全可能走向另一种可能
            ## 2.Dropout2d 实际就是随机丢弃一些特征,以干扰当前训练,抑制不同特征的依赖关系,提升最终结果可靠性
            self.conv2_drop = nn.Dropout2d()
            ## 全连接层 二维图像处理时全连接输入、输出一般重置为二维张量,不同于卷积计算时是四维张量
            ## 第一层全连接 320 来源下面计算说明
            self.fc1 = nn.Linear(320, 50)
            self.fc2 = nn.Linear(50, 10)
    
        ## 前向传播
        def forward(self, x):
            ## 卷积 self.conv1(x) 
            ## 最大池化 Fun.max_pool2d(self.conv1(x), kernel_size=2) 池化核大小为 2 **stride default kernel_size** 下面计算输出大小过程步长应取 2 
            ## 此方法可在上面提到的 Pytorch 官方查到 API 说明,里面有更多、更详细的参数说明 
            x = Fun.relu(Fun.max_pool2d(self.conv1(x), 2))
            ## 卷积 self.conv2(x) 
            ## 让某个神经元的激活值以一定的概率停止工作 self.conv2_drop 具体实现会做比字面上更多的处理和优化
            ## 最大池化 Fun.max_pool2d
            x = Fun.relu(Fun.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
            ## 将 x 转为二维张量 -1 表示其他维度,此时 tensor 会根据自身总量和输出列自动计算行数
            ## 第一次卷积输出：28 x 28 填充 0 步长 1 => (28 + 2 * 0 - 5)/1 + 1 = 24 => 24 * 10 = 240
            ## 第一次池化输出：24 x 24 填充 0 步长 2 => (24 + 2 * 0 - 2)/2 + 1 = 12
            ## 第二次卷积输出：12 x 12 填充 0 步长 1 => (12 + 2 * 0 - 5)/1 + 1 = 8
            ## 第二次池化输出：8 x 8 填充 0 步长 1 => (8 + 2 * 0 - 2)/2 + 1 = 4
            ## 最终输出20通道：4 x 4 共 20 * 4 * 4 = 320 个点
            x = x.view(-1, 320)
            ## 全连接并激活
            x = Fun.relu(self.fc1(x))
            ## 随机丢弃
            x = Fun.dropout(x, training=self.training)
            ## 再次全连接
            x = self.fc2(x)
            ## 计算损失 log_softmax 为 log(softmax(x)) 的变形,官方说法是：单独执行 softmax 和 log 这两个操作速度较慢且数值不稳定
            ## softmax 是概率分布计算 比如我们的模型预测结果 A:0.256 B:0.687 C:0.057 但是实际我们输入是B 所以结果应为 A:0.000 B:1.000 C:0.000
            ## crossEntropy 是交叉熵 用来做似然估计,即损失 计算如下：-(0.0 * ln(0.256) + 1.0 * ln(0.687 + 0.0 * ln(0.025)) => [0,+ ∞)
            ## log_softmax 将 crossEntropy 的操作提前了,直接对概率分布求对数, 将 (0,1] 映射到了 (- ∞,0]
            return Fun.log_softmax(x,dim = 1)

## 5.完整训练程序 ##

所有语句均添加了详细注释说明
    1.train方法，可单独调用一次，查看随机卷积核情况下损失情况
    2.训练 3 个循环，并每次结束时通过测试集验证精度
    3.测试方法，可以自己手写一些数字图片进行验证，最好尺寸直接控制在 28 x 28 否则情况可能不太好

import cv2
    import numpy as np
    import torch
    import torch.nn as nn
    import torch.nn.functional as Fun
    import torch.optim as optim
    import torchvision
    from torch.utils.data import DataLoader
    
    ## 超参数定义
    ## 训练和验证时每批次处理的数据量
    batch_size_train = 64
    batch_size_test = 1000
    ## 参数打印间隔
    log_interval = 10
    ## SGD 定义优化器参数 学习率 冲量
    ## p′ = p − lr∗dp (p - 学习率 乘以自己的一阶导数)
    ## v′ = −dx∗lr+v∗momemtum 相对于旧的把 −dx∗lr 作为梯度更新量的方式，引入v∗momemtum后 则如果上次更新方向与本次一致，产生加速效果，反之不一致就是减速效果
    learning_rate = 0.01
    momentum = 0.5
    ## 损失值记录
    train_losses = []
    train_counter = []
    test_losses = []
    # test_counter = [i * len(train_loader.dataset) for i in range(n_epochs + 1)]
    ## 平均数 和 标准差 前面已经计算获得
    μ=0.1307
    σ=0.3081
    ## 模型参数保存
    module_w='./weight/model.pth'
    optimizer_w='./weight/optimizer.pth'
    
    ## 设置随机种子
    torch.manual_seed(1)
    
    ## ----------------------------------------------------------------------------------------------------------------------
    ## 加载数据
    train_loader=DataLoader(
        torchvision.datasets.MNIST('./data/', train=True, download=True,
                                   transform=torchvision.transforms.Compose([
                                       torchvision.transforms.ToTensor(),
                                       torchvision.transforms.Normalize((μ,), (σ,))
                                   ])),
        ## shuffle = true 对输入数据进行打乱处理，先打乱，再batch
        batch_size=batch_size_train, shuffle=True)
    
    test_loader=DataLoader(
        torchvision.datasets.MNIST('./data/', train=False, download=True,
                                   transform=torchvision.transforms.Compose([
                                       torchvision.transforms.ToTensor(),
                                       torchvision.transforms.Normalize((μ,), (σ,))
                                   ])),
        batch_size=batch_size_test, shuffle=True)
    
    ## ----------------------------------------------------------------------------------------------------------------------
    ## 定义一个网络类,并继承 torch.nn.Module
    class Net(nn.Module):
    
        ## 显示定义构造方法 __init__ 时,除额外参数,第一个参必须为对象自身关键字 self
        def __init__(self):
    
            ## 调用父类初始化方法
            super(Net, self).__init__()
            ## 定义卷积核 输入通道数 输出通道数 卷积核尺寸 nn.Conv2d(in_cahnnels=1, out_channels=10, kernel_size=5) 默认步长为 1 不填充
            self.conv1 = nn.Conv2d(1, 10, 5)
            self.conv2 = nn.Conv2d(10, 20, 5)
            ## 定义通道值随机清零处理方法 在多参数、小样本的场景下,网络容易过拟合,现象为训练集里表现很好,但在测试集并不友好,原因如下
            ## 1.多参数、小样本情况下,由于不同特征间的复杂依赖关系,使得我们的网络难以厘出细粒度的特征区别,比如1000个特征,有600个一致,100不同,300无参考价值
            ##   实际起到作用的100特征在所有特整里面所占比例太低,虽然在训练集里面有一个良好表现,但换到测试集完全可能走向另一种可能
            ## 2.Dropout2d 实际就是随机丢弃一些特征,以干扰当前训练,抑制不同特征的依赖关系,提升最终结果可靠性
            self.conv2_drop = nn.Dropout2d()
            ## 全连接层 二维图像处理时全连接输入、输出一般重置为二维张量,不同于卷积计算时是四维张量
            ## 第一层全连接 320 来源下面计算说明
            self.fc1 = nn.Linear(320, 50)
            self.fc2 = nn.Linear(50, 10)
    
        ## 前向传播
        def forward(self, x):
            ## 卷积 self.conv1(x) 
            ## 最大池化 Fun.max_pool2d(self.conv1(x), kernel_size=2) 池化核大小为 2 **stride default kernel_size** 下面计算输出大小过程步长应取 2 
            ## 此方法可在上面提到的 Pytorch 官方查到 API 说明,里面有更多、更详细的参数说明 
            x = Fun.relu(Fun.max_pool2d(self.conv1(x), 2))
            ## 卷积 self.conv2(x) 
            ## 让某个神经元的激活值以一定的概率停止工作 self.conv2_drop 具体实现会做比字面上更多的处理和优化
            ## 最大池化 Fun.max_pool2d
            x = Fun.relu(Fun.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
            ## 将 x 转为二维张量 -1 表示其他维度,此时 tensor 会根据自身总量和输出列自动计算行数
            ## 第一次卷积输出：28 x 28 填充 0 步长 1 => (28 + 2 * 0 - 5)/1 + 1 = 24 => 24 * 10 = 240
            ## 第一次池化输出：24 x 24 填充 0 步长 2 => (24 + 2 * 0 - 2)/2 + 1 = 12
            ## 第二次卷积输出：12 x 12 填充 0 步长 1 => (12 + 2 * 0 - 5)/1 + 1 = 8
            ## 第二次池化输出：8 x 8 填充 0 步长 1 => (8 + 2 * 0 - 2)/2 + 1 = 4
            ## 最终输出20通道：4 x 4 共 20 * 4 * 4 = 320 个点
            x = x.view(-1, 320)
            ## 全连接并激活
            x = Fun.relu(self.fc1(x))
            ## 随机丢弃
            x = Fun.dropout(x, training=self.training)
            ## 再次全连接
            x = self.fc2(x)
            ## 计算损失 log_softmax 为 log(softmax(x)) 的变形,官方说法是：单独执行 softmax 和 log 这两个操作速度较慢且数值不稳定
            ## softmax 是概率分布计算 比如我们的模型预测结果 A:0.256 B:0.687 C:0.057 但是实际我们输入是B 所以结果应为 A:0.000 B:1.000 C:0.000
            ## crossEntropy 是交叉熵 用来做似然估计,即损失 计算如下：-(0.0 * ln(0.256) + 1.0 * ln(0.687 + 0.0 * ln(0.025)) => [0,+ ∞)
            ## log_softmax 将 crossEntropy 的操作提前了,直接对概率分布求对数, 将 (0,1] 映射到了 (- ∞,0]
            return Fun.log_softmax(x,dim = 1)
    
    ## ----------------------------------------------------------------------------------------------------------------------
    ## 定义训练方法 epoch 迭代次数
    def train(epoch):
      ## 加载已有模型参数
      network
      ## train 表示启用：批次标准化和随机丢弃 
      ## Batch Normalization 保障 bn 层能够用到每一批次数据的平均值和标准差
      ## Dropout 随机取一部分网络连接来训练更新参数
      network.train()
      ## enumerate 取出索引、值 即批次 batch_idx 和数据 (data, target)
      for batch_idx, (data, target) in enumerate(train_loader):
        ## 将模型的参数梯度初始化为 0 避免不同批次的梯度被累计使用
        optimizer.zero_grad()
        ## 传入数据
        output = network(data)
        ## 交叉熵函数 值越小则预测结果越接近真实结果
        loss = Fun.nll_loss(output, target)
        ## 反向传播计算梯度
        loss.backward()
        ## 更新所有参数 
        optimizer.step()
        ## 每 10 批打印一次，一批处理 64 张，所以每次打印实际处理了 640 张
        if batch_idx % log_interval == 0:
          print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
            epoch, batch_idx * len(data), len(train_loader.dataset),
            100. * batch_idx / len(train_loader), loss.item()))
          train_losses.append(loss.item())
          train_counter.append((batch_idx*64) + ((epoch-1)*len(train_loader.dataset)))
          torch.save(network.state_dict(), module_w)
          torch.save(optimizer.state_dict(), optimizer_w)
              
    ## ----------------------------------------------------------------------------------------------------------------------
    ## 定义验证方法
    def check():
        ## eval 表示不启用：标准化和随机丢弃
        ## Batch Normalization 保障BN层能够用全部训练数据的均值和标准差
        ## Dropout 利用所有网络连接（不进行随机舍弃神经元）
        ## 训练完成后 在模型测试时执行 eval 避免因为有新的输入导致权值更新
        network.eval()
        ## 
        test_loss = 0
        correct = 0
        ## tensor 有一个参数 requires_grad 默认为 false 设置为 true 以后则反向传播时会自动求导
        ## with torch.no_grad 语法使得当前模块计算所得的新的 tensor 的 requires_grad 都自动设置为False 可以在验证阶段节省GPU或CPU内存
        with torch.no_grad():
            ## 遍历测试数据
            for data, target in test_loader:
                ## 传入数据
                output = network(data)
                ## reduction='sum'  对N个样本损失求和返回
                ## reduction='none' 对N个样本损失求直接返回
                ## reduction='mean' 对N个样本损失求平均返回（已弃用）
                test_loss += Fun.nll_loss(output, target, reduction='sum').item()
                ## 取最大值 max(input, dim, keepdim=False)
                ## input 输入 tensor dim 维度 0 按列 1 按行 keepdim true 不压缩 false 压缩（输出结果比输入低一个维度）
                ## max 返回两个 tensor 前者为值、后者为索引 此处我们取索引即恰好为分析所得结果
                pred = output.data.max(1, keepdim=True)[1]
                ## 逐元素比较 返回一个 boolean 类型张量 再求和 => 输出即识别结果和真实结果一致的数量
                correct += pred.eq(target.data.view_as(pred)).sum()
        ## 平均损失 = 损失和/数据量
        test_loss /= len(test_loader.dataset)
        test_losses.append(test_loss)
        print('\nTest set: Avg. loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
            test_loss, correct, len(test_loader.dataset),
            100. * correct / len(test_loader.dataset)))
    
    ## ----------------------------------------------------------------------------------------------------------------------
    ## 功能测试
    def test():
        ## 加载图片
        original_img=cv2.imdecode(np.fromfile('88.png',dtype=np.uint8), 1)
        ## 转换
        new_img = cv2.cvtColor(cv2.resize(original_img,(28,28)),cv2.COLOR_RGB2GRAY)
        ## 下面两行注释取消后，可以查看转化出的灰度图
        ## cv2.imshow('Result Image',new_img)
        ## cv2.waitKey(0)
        ## 定义一个四维张量
        data = torch.randn(1,1,28,28)
        ## 填充灰度化的图片数据
        data[0][0] = torch.from_numpy(new_img)
        ## print(data.shape)
        ## 标准化
        μ=data.mean()
        σ=data.std()
        inputdata=data.sub(μ).div_(σ)
        ## 加载权重
        network.load_state_dict(torch.load(module_w))
        optimizer.load_state_dict(torch.load(optimizer_w))
        network.eval()
        with torch.no_grad():
          output = network(inputdata)
        ## 取出最大值索引(对于数字来说，无需二次映射，直接就是输出结果了)
        pred = output.data.max(1, keepdim=True)[1]
        ## print(output.shape)
        ## print(output)
        print(pred)
    
    ## ----------------------------------------------------------------------------------------------------------------------
    ## 初始化网络
    network = Net()
    ## 定义优化器 保存当前的参数状态并且基于计算梯度更新参数
    optimizer = optim.SGD(network.parameters(), lr=learning_rate, momentum=momentum)
    
    ## 加载已有模型数据
    # network.load_state_dict(torch.load(module_w))
    # optimizer.load_state_dict(torch.load(optimizer_w))
    
    ## 1.执行一次训练
    #  train(1)
    
    ## 2.训练
    # for epoch in range(1, 1 + 3):
    #     train(epoch)
    #     check()
    
    ## 3.测试
    ## test()

## 6.结果打印 ##

训练随机输入训练 1 次的效果

训练 3 个循环的效果

Train Epoch: 1 [0/60000 (0%)]   Loss: 2.371851
    Train Epoch: 1 [640/60000 (1%)] Loss: 2.366941
    Train Epoch: 1 [1280/60000 (2%)]        Loss: 2.253085
    Train Epoch: 1 [1920/60000 (3%)]        Loss: 2.258119
    Train Epoch: 1 [2560/60000 (4%)]        Loss: 2.279200
    Train Epoch: 1 [3200/60000 (5%)]        Loss: 2.284118
    Train Epoch: 1 [3840/60000 (6%)]        Loss: 2.224427
    Train Epoch: 1 [4480/60000 (7%)]        Loss: 2.182901
    Train Epoch: 1 [5120/60000 (9%)]        Loss: 2.092957
    Train Epoch: 1 [5760/60000 (10%)]       Loss: 2.014003
    Train Epoch: 1 [6400/60000 (11%)]       Loss: 2.066881
    Train Epoch: 1 [7040/60000 (12%)]       Loss: 1.937319
    Train Epoch: 1 [7680/60000 (13%)]       Loss: 1.780319
    Train Epoch: 1 [8320/60000 (14%)]       Loss: 1.727874
    Train Epoch: 1 [8960/60000 (15%)]       Loss: 1.722512
    Train Epoch: 1 [9600/60000 (16%)]       Loss: 1.627026
    Train Epoch: 1 [10240/60000 (17%)]      Loss: 1.649031
    Train Epoch: 1 [10880/60000 (18%)]      Loss: 1.317975
    Train Epoch: 1 [11520/60000 (19%)]      Loss: 1.215969
    Train Epoch: 1 [12160/60000 (20%)]      Loss: 1.311500
    Train Epoch: 1 [12800/60000 (21%)]      Loss: 1.205322
    Train Epoch: 1 [13440/60000 (22%)]      Loss: 1.249098
    Train Epoch: 1 [14080/60000 (23%)]      Loss: 1.102184
    Train Epoch: 1 [14720/60000 (25%)]      Loss: 1.101858
    Train Epoch: 1 [15360/60000 (26%)]      Loss: 1.056550
    Train Epoch: 1 [16000/60000 (27%)]      Loss: 0.966378
    Train Epoch: 1 [16640/60000 (28%)]      Loss: 0.898759
    Train Epoch: 1 [17280/60000 (29%)]      Loss: 1.141907
    Train Epoch: 1 [17920/60000 (30%)]      Loss: 0.744676
    Train Epoch: 1 [18560/60000 (31%)]      Loss: 0.865845
    Train Epoch: 1 [19200/60000 (32%)]      Loss: 0.743363
    Train Epoch: 1 [19840/60000 (33%)]      Loss: 0.896839
    Train Epoch: 1 [20480/60000 (34%)]      Loss: 0.797301
    Train Epoch: 1 [21120/60000 (35%)]      Loss: 0.760073
    Train Epoch: 1 [21760/60000 (36%)]      Loss: 0.796578
    Train Epoch: 1 [22400/60000 (37%)]      Loss: 0.632670
    Train Epoch: 1 [23040/60000 (38%)]      Loss: 0.671789
    Train Epoch: 1 [23680/60000 (39%)]      Loss: 0.852199
    Train Epoch: 1 [24320/60000 (41%)]      Loss: 0.684957
    Train Epoch: 1 [24960/60000 (42%)]      Loss: 0.658827
    Train Epoch: 1 [25600/60000 (43%)]      Loss: 0.599369
    Train Epoch: 1 [26240/60000 (44%)]      Loss: 0.925203
    Train Epoch: 1 [26880/60000 (45%)]      Loss: 0.569247
    Train Epoch: 1 [27520/60000 (46%)]      Loss: 0.595495
    Train Epoch: 1 [28160/60000 (47%)]      Loss: 0.787057
    Train Epoch: 1 [28800/60000 (48%)]      Loss: 0.709622
    Train Epoch: 1 [29440/60000 (49%)]      Loss: 0.771295
    Train Epoch: 1 [30080/60000 (50%)]      Loss: 0.459200
    Train Epoch: 1 [30720/60000 (51%)]      Loss: 0.515915
    Train Epoch: 1 [31360/60000 (52%)]      Loss: 0.510923
    Train Epoch: 1 [32000/60000 (53%)]      Loss: 0.789831
    Train Epoch: 1 [32640/60000 (54%)]      Loss: 0.668127
    Train Epoch: 1 [33280/60000 (55%)]      Loss: 0.596397
    Train Epoch: 1 [33920/60000 (57%)]      Loss: 0.545272
    Train Epoch: 1 [34560/60000 (58%)]      Loss: 0.791866
    Train Epoch: 1 [35200/60000 (59%)]      Loss: 0.623321
    Train Epoch: 1 [35840/60000 (60%)]      Loss: 0.501244
    Train Epoch: 1 [36480/60000 (61%)]      Loss: 0.753823
    Train Epoch: 1 [37120/60000 (62%)]      Loss: 0.610182
    Train Epoch: 1 [37760/60000 (63%)]      Loss: 0.472686
    Train Epoch: 1 [38400/60000 (64%)]      Loss: 0.742567
    Train Epoch: 1 [39040/60000 (65%)]      Loss: 0.577439
    Train Epoch: 1 [39680/60000 (66%)]      Loss: 0.659224
    Train Epoch: 1 [40320/60000 (67%)]      Loss: 0.758697
    Train Epoch: 1 [40960/60000 (68%)]      Loss: 0.801238
    Train Epoch: 1 [41600/60000 (69%)]      Loss: 0.456015
    Train Epoch: 1 [42240/60000 (70%)]      Loss: 0.584893
    Train Epoch: 1 [42880/60000 (71%)]      Loss: 0.763128
    Train Epoch: 1 [43520/60000 (72%)]      Loss: 0.408243
    Train Epoch: 1 [44160/60000 (74%)]      Loss: 0.462214
    Train Epoch: 1 [44800/60000 (75%)]      Loss: 0.511222
    Train Epoch: 1 [45440/60000 (76%)]      Loss: 0.520635
    Train Epoch: 1 [46080/60000 (77%)]      Loss: 0.404983
    Train Epoch: 1 [46720/60000 (78%)]      Loss: 0.551706
    Train Epoch: 1 [47360/60000 (79%)]      Loss: 0.539516
    Train Epoch: 1 [48000/60000 (80%)]      Loss: 0.649675
    Train Epoch: 1 [48640/60000 (81%)]      Loss: 0.387086
    Train Epoch: 1 [49280/60000 (82%)]      Loss: 0.531161
    Train Epoch: 1 [49920/60000 (83%)]      Loss: 0.468416
    Train Epoch: 1 [50560/60000 (84%)]      Loss: 0.313190
    Train Epoch: 1 [51200/60000 (85%)]      Loss: 0.473787
    Train Epoch: 1 [51840/60000 (86%)]      Loss: 0.313375
    Train Epoch: 1 [52480/60000 (87%)]      Loss: 0.563074
    Train Epoch: 1 [53120/60000 (88%)]      Loss: 0.639663
    Train Epoch: 1 [53760/60000 (90%)]      Loss: 0.598785
    Train Epoch: 1 [54400/60000 (91%)]      Loss: 0.604863
    Train Epoch: 1 [55040/60000 (92%)]      Loss: 0.433123
    Train Epoch: 1 [55680/60000 (93%)]      Loss: 0.479933
    Train Epoch: 1 [56320/60000 (94%)]      Loss: 0.384417
    Train Epoch: 1 [56960/60000 (95%)]      Loss: 0.572573
    Train Epoch: 1 [57600/60000 (96%)]      Loss: 0.474651
    Train Epoch: 1 [58240/60000 (97%)]      Loss: 0.420517
    Train Epoch: 1 [58880/60000 (98%)]      Loss: 0.468604
    Train Epoch: 1 [59520/60000 (99%)]      Loss: 0.532335
    
    Test set: Avg. loss: 0.2090, Accuracy: 9399/10000 (94%)
    
    Train Epoch: 2 [0/60000 (0%)]   Loss: 0.366734
    Train Epoch: 2 [640/60000 (1%)] Loss: 0.722454
    Train Epoch: 2 [1280/60000 (2%)]        Loss: 0.336176
    Train Epoch: 2 [1920/60000 (3%)]        Loss: 0.748185
    Train Epoch: 2 [2560/60000 (4%)]        Loss: 0.519431
    Train Epoch: 2 [3200/60000 (5%)]        Loss: 0.469481
    Train Epoch: 2 [3840/60000 (6%)]        Loss: 0.484743
    Train Epoch: 2 [4480/60000 (7%)]        Loss: 0.242931
    Train Epoch: 2 [5120/60000 (9%)]        Loss: 0.713406
    Train Epoch: 2 [5760/60000 (10%)]       Loss: 0.832393
    Train Epoch: 2 [6400/60000 (11%)]       Loss: 0.454071
    Train Epoch: 2 [7040/60000 (12%)]       Loss: 0.420207
    Train Epoch: 2 [7680/60000 (13%)]       Loss: 0.510488
    Train Epoch: 2 [8320/60000 (14%)]       Loss: 0.472568
    Train Epoch: 2 [8960/60000 (15%)]       Loss: 0.380424
    Train Epoch: 2 [9600/60000 (16%)]       Loss: 0.344593
    Train Epoch: 2 [10240/60000 (17%)]      Loss: 0.400106
    Train Epoch: 2 [10880/60000 (18%)]      Loss: 0.357579
    Train Epoch: 2 [11520/60000 (19%)]      Loss: 0.244068
    Train Epoch: 2 [12160/60000 (20%)]      Loss: 0.329613
    Train Epoch: 2 [12800/60000 (21%)]      Loss: 0.395200
    Train Epoch: 2 [13440/60000 (22%)]      Loss: 0.680711
    Train Epoch: 2 [14080/60000 (23%)]      Loss: 0.523370
    Train Epoch: 2 [14720/60000 (25%)]      Loss: 0.399706
    Train Epoch: 2 [15360/60000 (26%)]      Loss: 0.374576
    Train Epoch: 2 [16000/60000 (27%)]      Loss: 0.401027
    Train Epoch: 2 [16640/60000 (28%)]      Loss: 0.667782
    Train Epoch: 2 [17280/60000 (29%)]      Loss: 0.325726
    Train Epoch: 2 [17920/60000 (30%)]      Loss: 0.424247
    Train Epoch: 2 [18560/60000 (31%)]      Loss: 0.499896
    Train Epoch: 2 [19200/60000 (32%)]      Loss: 0.451446
    Train Epoch: 2 [19840/60000 (33%)]      Loss: 0.439135
    Train Epoch: 2 [20480/60000 (34%)]      Loss: 0.487537
    Train Epoch: 2 [21120/60000 (35%)]      Loss: 0.441139
    Train Epoch: 2 [21760/60000 (36%)]      Loss: 0.402886
    Train Epoch: 2 [22400/60000 (37%)]      Loss: 0.291915
    Train Epoch: 2 [23040/60000 (38%)]      Loss: 0.636113
    Train Epoch: 2 [23680/60000 (39%)]      Loss: 0.269374
    Train Epoch: 2 [24320/60000 (41%)]      Loss: 0.392922
    Train Epoch: 2 [24960/60000 (42%)]      Loss: 0.266148
    Train Epoch: 2 [25600/60000 (43%)]      Loss: 0.380597
    Train Epoch: 2 [26240/60000 (44%)]      Loss: 0.344070
    Train Epoch: 2 [26880/60000 (45%)]      Loss: 0.477857
    Train Epoch: 2 [27520/60000 (46%)]      Loss: 0.606165
    Train Epoch: 2 [28160/60000 (47%)]      Loss: 0.551639
    Train Epoch: 2 [28800/60000 (48%)]      Loss: 0.558916
    Train Epoch: 2 [29440/60000 (49%)]      Loss: 0.749660
    Train Epoch: 2 [30080/60000 (50%)]      Loss: 0.332171
    Train Epoch: 2 [30720/60000 (51%)]      Loss: 0.375500
    Train Epoch: 2 [31360/60000 (52%)]      Loss: 0.362925
    Train Epoch: 2 [32000/60000 (53%)]      Loss: 0.306489
    Train Epoch: 2 [32640/60000 (54%)]      Loss: 0.299393
    Train Epoch: 2 [33280/60000 (55%)]      Loss: 0.208521
    Train Epoch: 2 [33920/60000 (57%)]      Loss: 0.348526
    Train Epoch: 2 [34560/60000 (58%)]      Loss: 0.508665
    Train Epoch: 2 [35200/60000 (59%)]      Loss: 0.190389
    Train Epoch: 2 [35840/60000 (60%)]      Loss: 0.291090
    Train Epoch: 2 [36480/60000 (61%)]      Loss: 0.357614
    Train Epoch: 2 [37120/60000 (62%)]      Loss: 0.316668
    Train Epoch: 2 [37760/60000 (63%)]      Loss: 0.311942
    Train Epoch: 2 [38400/60000 (64%)]      Loss: 0.323030
    Train Epoch: 2 [39040/60000 (65%)]      Loss: 0.327689
    Train Epoch: 2 [39680/60000 (66%)]      Loss: 0.471894
    Train Epoch: 2 [40320/60000 (67%)]      Loss: 0.242763
    Train Epoch: 2 [40960/60000 (68%)]      Loss: 0.430212
    Train Epoch: 2 [41600/60000 (69%)]      Loss: 0.180888
    Train Epoch: 2 [42240/60000 (70%)]      Loss: 0.276546
    Train Epoch: 2 [42880/60000 (71%)]      Loss: 0.266703
    Train Epoch: 2 [43520/60000 (72%)]      Loss: 0.253300
    Train Epoch: 2 [44160/60000 (74%)]      Loss: 0.492046
    Train Epoch: 2 [44800/60000 (75%)]      Loss: 0.615155
    Train Epoch: 2 [45440/60000 (76%)]      Loss: 0.344643
    Train Epoch: 2 [46080/60000 (77%)]      Loss: 0.308926
    Train Epoch: 2 [46720/60000 (78%)]      Loss: 0.338160
    Train Epoch: 2 [47360/60000 (79%)]      Loss: 0.440104
    Train Epoch: 2 [48000/60000 (80%)]      Loss: 0.481084
    Train Epoch: 2 [48640/60000 (81%)]      Loss: 0.326619
    Train Epoch: 2 [49280/60000 (82%)]      Loss: 0.298519
    Train Epoch: 2 [49920/60000 (83%)]      Loss: 0.357107
    Train Epoch: 2 [50560/60000 (84%)]      Loss: 0.274719
    Train Epoch: 2 [51200/60000 (85%)]      Loss: 0.345895
    Train Epoch: 2 [51840/60000 (86%)]      Loss: 0.241214
    Train Epoch: 2 [52480/60000 (87%)]      Loss: 0.243374
    Train Epoch: 2 [53120/60000 (88%)]      Loss: 0.191087
    Train Epoch: 2 [53760/60000 (90%)]      Loss: 0.287821
    Train Epoch: 2 [54400/60000 (91%)]      Loss: 0.237997
    Train Epoch: 2 [55040/60000 (92%)]      Loss: 0.274627
    Train Epoch: 2 [55680/60000 (93%)]      Loss: 0.253250
    Train Epoch: 2 [56320/60000 (94%)]      Loss: 0.309018
    Train Epoch: 2 [56960/60000 (95%)]      Loss: 0.277265
    Train Epoch: 2 [57600/60000 (96%)]      Loss: 0.276998
    Train Epoch: 2 [58240/60000 (97%)]      Loss: 0.459611
    Train Epoch: 2 [58880/60000 (98%)]      Loss: 0.516984
    Train Epoch: 2 [59520/60000 (99%)]      Loss: 0.311498
    
    Test set: Avg. loss: 0.1252, Accuracy: 9623/10000 (96%)
    
    Train Epoch: 3 [0/60000 (0%)]   Loss: 0.454000
    Train Epoch: 3 [640/60000 (1%)] Loss: 0.274891
    Train Epoch: 3 [1280/60000 (2%)]        Loss: 0.390851
    Train Epoch: 3 [1920/60000 (3%)]        Loss: 0.156189
    Train Epoch: 3 [2560/60000 (4%)]        Loss: 0.419210
    Train Epoch: 3 [3200/60000 (5%)]        Loss: 0.300367
    Train Epoch: 3 [3840/60000 (6%)]        Loss: 0.256845
    Train Epoch: 3 [4480/60000 (7%)]        Loss: 0.418629
    Train Epoch: 3 [5120/60000 (9%)]        Loss: 0.171152
    Train Epoch: 3 [5760/60000 (10%)]       Loss: 0.321221
    Train Epoch: 3 [6400/60000 (11%)]       Loss: 0.424148
    Train Epoch: 3 [7040/60000 (12%)]       Loss: 0.143154
    Train Epoch: 3 [7680/60000 (13%)]       Loss: 0.488087
    Train Epoch: 3 [8320/60000 (14%)]       Loss: 0.402844
    Train Epoch: 3 [8960/60000 (15%)]       Loss: 0.417593
    Train Epoch: 3 [9600/60000 (16%)]       Loss: 0.276311
    Train Epoch: 3 [10240/60000 (17%)]      Loss: 0.503877
    Train Epoch: 3 [10880/60000 (18%)]      Loss: 0.387446
    Train Epoch: 3 [11520/60000 (19%)]      Loss: 0.612089
    Train Epoch: 3 [12160/60000 (20%)]      Loss: 0.430010
    Train Epoch: 3 [12800/60000 (21%)]      Loss: 0.300433
    Train Epoch: 3 [13440/60000 (22%)]      Loss: 0.346322
    Train Epoch: 3 [14080/60000 (23%)]      Loss: 0.187743
    Train Epoch: 3 [14720/60000 (25%)]      Loss: 0.330318
    Train Epoch: 3 [15360/60000 (26%)]      Loss: 0.201379
    Train Epoch: 3 [16000/60000 (27%)]      Loss: 0.264247
    Train Epoch: 3 [16640/60000 (28%)]      Loss: 0.305077
    Train Epoch: 3 [17280/60000 (29%)]      Loss: 0.386015
    Train Epoch: 3 [17920/60000 (30%)]      Loss: 0.267599
    Train Epoch: 3 [18560/60000 (31%)]      Loss: 0.245534
    Train Epoch: 3 [19200/60000 (32%)]      Loss: 0.407366
    Train Epoch: 3 [19840/60000 (33%)]      Loss: 0.231558
    Train Epoch: 3 [20480/60000 (34%)]      Loss: 0.134505
    Train Epoch: 3 [21120/60000 (35%)]      Loss: 0.332575
    Train Epoch: 3 [21760/60000 (36%)]      Loss: 0.298820
    Train Epoch: 3 [22400/60000 (37%)]      Loss: 0.283264
    Train Epoch: 3 [23040/60000 (38%)]      Loss: 0.267990
    Train Epoch: 3 [23680/60000 (39%)]      Loss: 0.274185
    Train Epoch: 3 [24320/60000 (41%)]      Loss: 0.258080
    Train Epoch: 3 [24960/60000 (42%)]      Loss: 0.235735
    Train Epoch: 3 [25600/60000 (43%)]      Loss: 0.313570
    Train Epoch: 3 [26240/60000 (44%)]      Loss: 0.248482
    Train Epoch: 3 [26880/60000 (45%)]      Loss: 0.292335
    Train Epoch: 3 [27520/60000 (46%)]      Loss: 0.327348
    Train Epoch: 3 [28160/60000 (47%)]      Loss: 0.283135
    Train Epoch: 3 [28800/60000 (48%)]      Loss: 0.441134
    Train Epoch: 3 [29440/60000 (49%)]      Loss: 0.311160
    Train Epoch: 3 [30080/60000 (50%)]      Loss: 0.424866
    Train Epoch: 3 [30720/60000 (51%)]      Loss: 0.355469
    Train Epoch: 3 [31360/60000 (52%)]      Loss: 0.434249
    Train Epoch: 3 [32000/60000 (53%)]      Loss: 0.205223
    Train Epoch: 3 [32640/60000 (54%)]      Loss: 0.270712
    Train Epoch: 3 [33280/60000 (55%)]      Loss: 0.189404
    Train Epoch: 3 [33920/60000 (57%)]      Loss: 0.182121
    Train Epoch: 3 [34560/60000 (58%)]      Loss: 0.214576
    Train Epoch: 3 [35200/60000 (59%)]      Loss: 0.315236
    Train Epoch: 3 [35840/60000 (60%)]      Loss: 0.367852
    Train Epoch: 3 [36480/60000 (61%)]      Loss: 0.402626
    Train Epoch: 3 [37120/60000 (62%)]      Loss: 0.494649
    Train Epoch: 3 [37760/60000 (63%)]      Loss: 0.473171
    Train Epoch: 3 [38400/60000 (64%)]      Loss: 0.465688
    Train Epoch: 3 [39040/60000 (65%)]      Loss: 0.401424
    Train Epoch: 3 [39680/60000 (66%)]      Loss: 0.252164
    Train Epoch: 3 [40320/60000 (67%)]      Loss: 0.283355
    Train Epoch: 3 [40960/60000 (68%)]      Loss: 0.214266
    Train Epoch: 3 [41600/60000 (69%)]      Loss: 0.197134
    Train Epoch: 3 [42240/60000 (70%)]      Loss: 0.411207
    Train Epoch: 3 [42880/60000 (71%)]      Loss: 0.330771
    Train Epoch: 3 [43520/60000 (72%)]      Loss: 0.207687
    Train Epoch: 3 [44160/60000 (74%)]      Loss: 0.296396
    Train Epoch: 3 [44800/60000 (75%)]      Loss: 0.447102
    Train Epoch: 3 [45440/60000 (76%)]      Loss: 0.151577
    Train Epoch: 3 [46080/60000 (77%)]      Loss: 0.134489
    Train Epoch: 3 [46720/60000 (78%)]      Loss: 0.292283
    Train Epoch: 3 [47360/60000 (79%)]      Loss: 0.256751
    Train Epoch: 3 [48000/60000 (80%)]      Loss: 0.130260
    Train Epoch: 3 [48640/60000 (81%)]      Loss: 0.217153
    Train Epoch: 3 [49280/60000 (82%)]      Loss: 0.496362
    Train Epoch: 3 [49920/60000 (83%)]      Loss: 0.567312
    Train Epoch: 3 [50560/60000 (84%)]      Loss: 0.246096
    Train Epoch: 3 [51200/60000 (85%)]      Loss: 0.368138
    Train Epoch: 3 [51840/60000 (86%)]      Loss: 0.300753
    Train Epoch: 3 [52480/60000 (87%)]      Loss: 0.341426
    Train Epoch: 3 [53120/60000 (88%)]      Loss: 0.091202
    Train Epoch: 3 [53760/60000 (90%)]      Loss: 0.310372
    Train Epoch: 3 [54400/60000 (91%)]      Loss: 0.188257
    Train Epoch: 3 [55040/60000 (92%)]      Loss: 0.238355
    Train Epoch: 3 [55680/60000 (93%)]      Loss: 0.366855
    Train Epoch: 3 [56320/60000 (94%)]      Loss: 0.340705
    Train Epoch: 3 [56960/60000 (95%)]      Loss: 0.322747
    Train Epoch: 3 [57600/60000 (96%)]      Loss: 0.388191
    Train Epoch: 3 [58240/60000 (97%)]      Loss: 0.154319
    Train Epoch: 3 [58880/60000 (98%)]      Loss: 0.207342
    Train Epoch: 3 [59520/60000 (99%)]      Loss: 0.458232
    
    Test set: Avg. loss: 0.1009, Accuracy: 9685/10000 (97%)

## 7.手写图片验证 ##

### 1.进行多次训练 ###

注释掉训练日志打印，基于已有参数训练，多调用几次脚本
    训练次数已经没什么效果了

![在这里插入图片描述][a5828922dd1042e688f93b680259382f.png]

### 2.测试 ###

通过上面查看数据方法，去测试集截一张图用来验证  
![在这里插入图片描述][7bf6363831b6489e84aa7bc09479ba23.png]  
第一个 8 输出了 5  
![在这里插入图片描述][3224a4e418d64383b2e83b044ce18986.png]

第三行 第三列的 8 输出了 8

![在这里插入图片描述][559267acdc9e496493a3630300bb9930.png]

自己手写数字，截图 8 进行识别

![在这里插入图片描述][66d4c64d355145d0b2c2ac0aeedbf0e6.png]

自己手写数字，截图 2 进行识别

![在这里插入图片描述][c6447033e3de4dcdb320b985baf7e1ad.png]

自己手写数字，截图 5 进行识别

![在这里插入图片描述][85485453b66742218d3880fd18f65241.png]

数字 8 的效果不太理想，感觉可能和图片尺寸有关，有时间可以在细究一下

[MINST]: http://yann.lecun.com/exdb/mnist/
[Pytorch]: https://pytorch.org/docs/stable/torch.html
[1a369ace77db48319ddb0e2db075e8e0.png]: https://img-blog.csdnimg.cn/1a369ace77db48319ddb0e2db075e8e0.png
[f98b75b2a7ec47e0b142dc2d6cc1cd39.png]: https://img-blog.csdnimg.cn/f98b75b2a7ec47e0b142dc2d6cc1cd39.png
[2612dc3043d44021861d799009972451.png]: https://img-blog.csdnimg.cn/2612dc3043d44021861d799009972451.png
[5bad35c60aba4cbbba6d62097478a759.png]: https://img-blog.csdnimg.cn/5bad35c60aba4cbbba6d62097478a759.png
[d67747dcced64159835c7d6a734cad7d.png]: https://img-blog.csdnimg.cn/d67747dcced64159835c7d6a734cad7d.png
[a5828922dd1042e688f93b680259382f.png]: https://img-blog.csdnimg.cn/a5828922dd1042e688f93b680259382f.png
[7bf6363831b6489e84aa7bc09479ba23.png]: https://img-blog.csdnimg.cn/7bf6363831b6489e84aa7bc09479ba23.png
[3224a4e418d64383b2e83b044ce18986.png]: https://img-blog.csdnimg.cn/3224a4e418d64383b2e83b044ce18986.png
[559267acdc9e496493a3630300bb9930.png]: https://img-blog.csdnimg.cn/559267acdc9e496493a3630300bb9930.png
[66d4c64d355145d0b2c2ac0aeedbf0e6.png]: https://img-blog.csdnimg.cn/66d4c64d355145d0b2c2ac0aeedbf0e6.png
[c6447033e3de4dcdb320b985baf7e1ad.png]: https://img-blog.csdnimg.cn/c6447033e3de4dcdb320b985baf7e1ad.png
[85485453b66742218d3880fd18f65241.png]: https://img-blog.csdnimg.cn/85485453b66742218d3880fd18f65241.png