矩阵求导的相关知识

小咪咪 2022-02-02 03:59 732阅读 0赞

## 一、矩阵求导 ##

一般来讲，我们约定  x = ( x 1 , x 2 , . . . x n ) T x = (x\_\{1\},x\_\{2\},...x\_\{n\})^T x=(x1,x2,...xn)T，这是分母布局。常见的矩阵求导方式有：向量对向量求导，标量对向量求导，向量对标量求导。  
**六种矩阵最整齐的求导结果：**  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70]  
**1.向量对向量求导**  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 1]  
**2.标量对向量求导**  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 2]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 3]  
**3、向量对标量求导**  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 4]

## 二、几种重要的矩阵 ##

**1、梯度（Gradient）**  
![在这里插入图片描述][20190510160729683.png]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 5]  
**梯度的转置是一个行向量：**  
![在这里插入图片描述][20190510160612327.png]  
**2、雅克比矩阵（Jacobian matrix）【雅可比矩阵本质上是一阶梯度，向量对向量微分】**  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 6]  
**3、海森矩阵（Hessian matrix）【海森矩阵是二阶梯度】**  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 7]

## 三、常用的矩阵求导公式 ##

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 8]  
参考：  
[https://blog.csdn.net/daaikuaichuan/article/details/80620518][https_blog.csdn.net_daaikuaichuan_article_details_80620518]  
[http://blog.sina.com.cn/s/blog\_8eac0b290101fsqb.html][http_blog.sina.com.cn_s_blog_8eac0b290101fsqb.html]  
[https://blog.csdn.net/yc461515457/article/details/49682473][https_blog.csdn.net_yc461515457_article_details_49682473]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70]: /images/20220202/a49e840c3f1d4a44ade876eef3aed06c.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 1]: /images/20220202/85694a6ab0ad492d8c2fe39a813a2d8f.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 2]: /images/20220202/26fe5224c874412c873de646aa15810b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 3]: /images/20220202/68246b49968f45949c152bac09cab655.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 4]: /images/20220202/ad506de3ad8e43818e9bcbb29549dec1.png
[20190510160729683.png]: /images/20220202/b7b3a380ea1e46f2b6cfaabe2e255eee.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 5]: /images/20220202/1a79965f12ad4834bb9b5d89578e4ffb.png
[20190510160612327.png]: /images/20220202/54d27b77edd64ea3ac30e81586927f2e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 6]: /images/20220202/7164d2a5af7c40d1813c036007d7b539.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 7]: /images/20220202/3c80920f8de64250a18810a4a33a77c4.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FzMTc4MDkyNTk3MTU_size_16_color_FFFFFF_t_70 8]: /images/20220202/9f9c6ec0f2624b15ab93f00d08ddb9c5.png
[https_blog.csdn.net_daaikuaichuan_article_details_80620518]: https://blog.csdn.net/daaikuaichuan/article/details/80620518
[http_blog.sina.com.cn_s_blog_8eac0b290101fsqb.html]: http://blog.sina.com.cn/s/blog_8eac0b290101fsqb.html
[https_blog.csdn.net_yc461515457_article_details_49682473]: https://blog.csdn.net/yc461515457/article/details/49682473