机器学习入门-第五天

布满荆棘的人生 2023-03-12 10:47 147阅读 0赞

# Logistic Regression逻辑回归 #

## 1,伯努利分布及交叉熵 ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70][]

**交叉熵参考:** 一文搞懂交叉熵在机器学习中的使用，透彻理解交叉熵背后的直觉[https://blog.csdn.net/tsyccnh/article/details/79163834][https_blog.csdn.net_tsyccnh_article_details_79163834]

## 2,逻辑回归的model ##

![format_png][]

## 3,逻辑回归的Loss Function ##

![format_png 1][]

**我们令![20200512150253695.png][]可以将对数方程改写成含y head的方程:**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 1][]

之所以能将对数方程改写成含y^的,是因为例如第3个式子,y^=1,而1-y^=0,则![20200512152957205.png][]就是等于![20200512153025501.png][]

![format_png 2][]

为什么要将对数方程改写成含y^的,这是为了让它出现两个符合伯努利的分布的及其交叉熵:

![format_png 3][]

![20200512154215972.png][]

![format_png 4][]

现在我们要做的就是找到使loss function即交叉熵之和最小的那组参数w,b就行了，这里用gradient descent的方法进行运算就ok

![20200512172111767.png][]

![format_png 5][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 2][]

**为什么不是用平方差作为LF(损失函数)而用交叉熵?**

![format_png 6][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 3][]

![20200512173927333.png][]

![format_png 7][]

**参考:**

[https://sakura-gh.github.io/ML-notes/ML-notes-html/7\_Logistic-Regression.html][https_sakura-gh.github.io_ML-notes_ML-notes-html_7_Logistic-Regression.html]

[https://www.cnblogs.com/xxlad/p/11221997.html][https_www.cnblogs.com_xxlad_p_11221997.html]

[https://datawhalechina.github.io/leeml-notes/\#/chapter11/chapter11][https_datawhalechina.github.io_leeml-notes_chapter11_chapter11]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70]: /images/20230312/7f95a7a338be4766a83cd3bea2853b24.png
[https_blog.csdn.net_tsyccnh_article_details_79163834]: https://blog.csdn.net/tsyccnh/article/details/79163834
[format_png]: /images/20230312/0fd19ed4487f496db8dc1fafdc1bc6d2.png
[format_png 1]: /images/20230312/530cd0ca9bc74c30a1b27bbe734bd383.png
[20200512150253695.png]: /images/20230312/624620db275c40a5a89c886b831ce513.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 1]: /images/20230312/ea5ba522c7df4fe7a7399ccda4a1da8a.png
[20200512152957205.png]: https://img-blog.csdnimg.cn/20200512152957205.png
[20200512153025501.png]: /images/20230312/1e76ed4ab54249c09e111c8ba465184e.png
[format_png 2]: /images/20230312/0f9deb3273e54e639c1f8f7f7a944a1b.png
[format_png 3]: /images/20230312/669cb15f93494272ba26a5372f4f7ef3.png
[20200512154215972.png]: /images/20230312/562206ad83f1453dbbdb42eb68db78f7.png
[format_png 4]: /images/20230312/395d4071f559406aa14044ae772eacf4.png
[20200512172111767.png]: /images/20230312/b2db6e351a8c4d7d80efc924ddbbee54.png
[format_png 5]: /images/20230312/03393cc6ad42420692c17049861b9315.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 2]: /images/20230312/377658bd242f414d96e6953babd5712c.png
[format_png 6]: /images/20230312/6900862bcfef4c9db126f21e6c87b1b0.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2tpbmc1MjExMzE0MTMxNA_size_16_color_FFFFFF_t_70 3]: /images/20230312/8f9e19f136324e9f90c51fd5498f01a6.png
[20200512173927333.png]: /images/20230312/cdcefd339cca4f9d80af93c90c056fbf.png
[format_png 7]: /images/20230312/cd349bea79704b848b061db08e73a182.png
[https_sakura-gh.github.io_ML-notes_ML-notes-html_7_Logistic-Regression.html]: https://sakura-gh.github.io/ML-notes/ML-notes-html/7_Logistic-Regression.html
[https_www.cnblogs.com_xxlad_p_11221997.html]: https://www.cnblogs.com/xxlad/p/11221997.html
[https_datawhalechina.github.io_leeml-notes_chapter11_chapter11]: https://datawhalechina.github.io/leeml-notes/#/chapter11/chapter11