L1和L2正则化

蔚落 2022-03-02 08:42 290阅读 0赞

**什么是正则化？**  
正则化就是在损失函数后加上一个正则化项（惩罚项），其实就是常说的结构风险最小化策略，即经验风险（损失函数）加上正则化。一般模型越复杂，正则化值越大。  
**常见的正则化有L1正则化和L2正则化：**

*  L1正则化（L1范数）指的是权重参数W的各项元素绝对值之和，即![This is the rendered form of the equation. You can not edit this directly. Right click will give you the option to save the image, and in most browsers you can drag the image onto your desktop or another program.][This is the rendered form of the equation. You can not edit this directly. Right click will give you the option to save the image_ and in most browsers you can drag the image onto your desktop or another program.],记作![This is the rendered form of the equation. You can not edit this directly. Right click will give you the option to save the image, and in most browsers you can drag the image onto your desktop or another program.][This is the rendered form of the equation. You can not edit this directly. Right click will give you the option to save the image_ and in most browsers you can drag the image onto your desktop or another program. 1]
 *  L2（L2范数）权重参数W的各项元素的平方和的开方，

**正则化有什么作用：**

*  L1正则化可以产生稀疏权重矩阵，即大部分w为0，只有少数w非0，可以用于特征选择
 *  L2正则化可以防止模型过拟合

**自己的理解：**  
L1正则化：加上正则惩罚项之后为了使得损失函数最小，所以在求绝对值之和的时候就会有很多0，这样就会产生L!正则化的作用，产生了稀疏权重矩阵，所以可用于特征选择。  
L2正则化：加上正则惩罚项之后同样为了使得损失函数最小，也就是求权重的平方和最小，就会导致权重都变小，权重的变小就会导致拟合曲线更平滑，继而就会防止过拟合。

[This is the rendered form of the equation. You can not edit this directly. Right click will give you the option to save the image_ and in most browsers you can drag the image onto your desktop or another program.]: http://latex.codecogs.com/gif.latex?%5Csum%20%5Cleft%20%7C%20w_%7Bi%7D%20%5Cright%20%7C
[This is the rendered form of the equation. You can not edit this directly. Right click will give you the option to save the image_ and in most browsers you can drag the image onto your desktop or another program. 1]: /images/20220302/b52d9c41b8d449fa9e5795d9d69293f3.png