多层神经网络识别猫图片

爱被打了一巴掌 2023-06-01 06:20 15阅读 0赞

例子是之前博客"通过简单神经网络识别猫图片"的例子，现在改为多层神经网络结构

之前博客地址：[https://blog.csdn.net/qq\_33873431/article/details/101672423][https_blog.csdn.net_qq_33873431_article_details_101672423]

代码位置：[https://github.com/lilihongjava/deep\_learning/tree/master/%E5%A4%9A%E5%B1%82%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E8%AF%86%E5%88%AB%E7%8C%AB][https_github.com_lilihongjava_deep_learning_tree_master_E5_A4_9A_E5_B1_82_E7_A5_9E_E7_BB_8F_E7_BD_91_E7_BB_9C_E8_AF_86_E5_88_AB_E7_8C_AB]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzODczNDMx_size_16_color_FFFFFF_t_70][]

### 参数维度 ###

W - 权重矩阵，维度为(n\[L\]，n\[L−1\])  
b  - 偏向量，维度为(n\[L\]，1)  
A  - 激活值的维度为(n\[L\]，m)

### 前向传播函数 ###

一共L层，带有RELU的linear\_activation\_forward）L-1次，然后用一个层为SIGMOID做二分类

公式为：

![Z^\{\[l\]\} = W^\{\[l\]\}A^\{\[l-1\]\} +b^\{\[l\]\}\\tag\{3\}][Z_l_ _ W_l_A_l-1_ _b_l_tag_3]

激活函数公式：

**Sigmoid**：![\\sigma(Z) = \\sigma(W A + b) = \\frac\{1\}\{ 1 + e^\{-(W A + b)\}\}][sigma_Z_ _ _sigma_W A _ b_ _ _frac_1_ 1 _ e_-_W A _ b]

**ReLU：![A = RELU(Z) = max(0, Z)][A _ RELU_Z_ _ max_0_ Z]**

### 计算成本函数 ###

![\-\\frac\{1\}\{m\} \\sum\\limits\_\{i = 1\}^\{m\} (y^\{(i)\}\\log\\left(a^\{\[L\] (i)\}\\right) + (1-y^\{(i)\})\\log\\left(1- a^\{\[L\](i)\}\\right))][-_frac_1_m_ _sum_limits_i _ 1_m_ _y_i_log_left_a_L_ _i_right_ _ _1-y_i_log_left_1- a_L_i_right]

### 反向传播 ###

使用![dZ^\{\[l\]\}][dZ_l]来计算三个输出![(dW^\{\[l\]\}, db^\{\[l\]\}, dA^\{\[l\]\})][dW_l_ db_l_ dA_l]，公式：

![dW^\{\[l\]\} = \\frac\{\\partial \\mathcal\{L\} \}\{\\partial W^\{\[l\]\}\} = \\frac\{1\}\{m\} dZ^\{\[l\]\} A^\{\[l-1\] T\}][dW_l_ _ _frac_partial _mathcal_L_ _partial W_l_ _ _frac_1_m_ dZ_l_ A_l-1_ T]

![db^\{\[l\]\} = \\frac\{\\partial \\mathcal\{L\} \}\{\\partial b^\{\[l\]\}\} = \\frac\{1\}\{m\} \\sum\_\{i = 1\}^\{m\} dZ^\{\[l\](i)\}][db_l_ _ _frac_partial _mathcal_L_ _partial b_l_ _ _frac_1_m_ _sum_i _ 1_m_ dZ_l_i]

![dA^\{\[l-1\]\} = \\frac\{\\partial \\mathcal\{L\} \}\{\\partial A^\{\[l-1\]\}\} = W^\{\[l\] T\} dZ^\{\[l\]\}][dA_l-1_ _ _frac_partial _mathcal_L_ _partial A_l-1_ _ W_l_ T_ dZ_l]

对于激活函数，求导公式：

![dZ^\{\[l\]\} = dA^\{\[l\]\} \* g'(Z^\{\[l\]\}) \\tag\{8\}][dZ_l_ _ dA_l_ _ g_Z_l_ _tag_8]

最后一层为Sigmoid，成本函数L先对a求导：

**![da=\\frac\{\\partial L\}\{\\partial a\}=-\\frac ya+\\frac\{1-y\}\{1-a\}][da_frac_partial L_partial a_-_frac ya_frac_1-y_1-a]**

# python代码
    dAL = - (np.divide(Y, AL) - np.divide(1 - Y, 1 - AL))

[https_blog.csdn.net_qq_33873431_article_details_101672423]: https://blog.csdn.net/qq_33873431/article/details/101672423
[https_github.com_lilihongjava_deep_learning_tree_master_E5_A4_9A_E5_B1_82_E7_A5_9E_E7_BB_8F_E7_BD_91_E7_BB_9C_E8_AF_86_E5_88_AB_E7_8C_AB]: https://github.com/lilihongjava/deep_learning/tree/master/%E5%A4%9A%E5%B1%82%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E8%AF%86%E5%88%AB%E7%8C%AB
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzODczNDMx_size_16_color_FFFFFF_t_70]: /images/20230601/018fce9befd840a8afdda29d0adc9848.png
[Z_l_ _ W_l_A_l-1_ _b_l_tag_3]: https://private.codecogs.com/gif.latex?Z%5E%7B%5Bl%5D%7D%20%3D%20W%5E%7B%5Bl%5D%7DA%5E%7B%5Bl-1%5D%7D%20&plus;b%5E%7B%5Bl%5D%7D%5Ctag%7B3%7D
[sigma_Z_ _ _sigma_W A _ b_ _ _frac_1_ 1 _ e_-_W A _ b]: https://private.codecogs.com/gif.latex?%5Csigma%28Z%29%20%3D%20%5Csigma%28W%20A%20&plus;%20b%29%20%3D%20%5Cfrac%7B1%7D%7B%201%20&plus;%20e%5E%7B-%28W%20A%20&plus;%20b%29%7D%7D
[A _ RELU_Z_ _ max_0_ Z]: https://private.codecogs.com/gif.latex?A%20%3D%20RELU%28Z%29%20%3D%20max%280%2C%20Z%29
[-_frac_1_m_ _sum_limits_i _ 1_m_ _y_i_log_left_a_L_ _i_right_ _ _1-y_i_log_left_1- a_L_i_right]: https://private.codecogs.com/gif.latex?-%5Cfrac%7B1%7D%7Bm%7D%20%5Csum%5Climits_%7Bi%20%3D%201%7D%5E%7Bm%7D%20%28y%5E%7B%28i%29%7D%5Clog%5Cleft%28a%5E%7B%5BL%5D%20%28i%29%7D%5Cright%29%20&plus;%20%281-y%5E%7B%28i%29%7D%29%5Clog%5Cleft%281-%20a%5E%7B%5BL%5D%28i%29%7D%5Cright%29%29
[dZ_l]: https://private.codecogs.com/gif.latex?dZ%5E%7B%5Bl%5D%7D
[dW_l_ db_l_ dA_l]: https://private.codecogs.com/gif.latex?%28dW%5E%7B%5Bl%5D%7D%2C%20db%5E%7B%5Bl%5D%7D%2C%20dA%5E%7B%5Bl%5D%7D%29
[dW_l_ _ _frac_partial _mathcal_L_ _partial W_l_ _ _frac_1_m_ dZ_l_ A_l-1_ T]: https://private.codecogs.com/gif.latex?dW%5E%7B%5Bl%5D%7D%20%3D%20%5Cfrac%7B%5Cpartial%20%5Cmathcal%7BL%7D%20%7D%7B%5Cpartial%20W%5E%7B%5Bl%5D%7D%7D%20%3D%20%5Cfrac%7B1%7D%7Bm%7D%20dZ%5E%7B%5Bl%5D%7D%20A%5E%7B%5Bl-1%5D%20T%7D
[db_l_ _ _frac_partial _mathcal_L_ _partial b_l_ _ _frac_1_m_ _sum_i _ 1_m_ dZ_l_i]: https://private.codecogs.com/gif.latex?db%5E%7B%5Bl%5D%7D%20%3D%20%5Cfrac%7B%5Cpartial%20%5Cmathcal%7BL%7D%20%7D%7B%5Cpartial%20b%5E%7B%5Bl%5D%7D%7D%20%3D%20%5Cfrac%7B1%7D%7Bm%7D%20%5Csum_%7Bi%20%3D%201%7D%5E%7Bm%7D%20dZ%5E%7B%5Bl%5D%28i%29%7D
[dA_l-1_ _ _frac_partial _mathcal_L_ _partial A_l-1_ _ W_l_ T_ dZ_l]: https://private.codecogs.com/gif.latex?dA%5E%7B%5Bl-1%5D%7D%20%3D%20%5Cfrac%7B%5Cpartial%20%5Cmathcal%7BL%7D%20%7D%7B%5Cpartial%20A%5E%7B%5Bl-1%5D%7D%7D%20%3D%20W%5E%7B%5Bl%5D%20T%7D%20dZ%5E%7B%5Bl%5D%7D
[dZ_l_ _ dA_l_ _ g_Z_l_ _tag_8]: https://private.codecogs.com/gif.latex?dZ%5E%7B%5Bl%5D%7D%20%3D%20dA%5E%7B%5Bl%5D%7D%20*%20g%27%28Z%5E%7B%5Bl%5D%7D%29%20%5Ctag%7B8%7D
[da_frac_partial L_partial a_-_frac ya_frac_1-y_1-a]: https://private.codecogs.com/gif.latex?da%3D%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20a%7D%3D-%5Cfrac%20ya&plus;%5Cfrac%7B1-y%7D%7B1-a%7D