机器学习之线性模型一

雨点打透心脏的1/2处 2022-06-05 07:48 264阅读 0赞

机器学习之线性模型(Generalized Linear Models)一

本篇主要针对于Ordinary Least Squares，周志华的机器学习中关于线性模型章节3.2首先提到了对数据数字化的过程，原文是：对离散属性，若属性值间存在“序”关系，可以通过连续化将其转化为连续值，例如值属性“身高”的取值“高”“矮”可以转化为\{1.0,0.0\}，三值属性“高度”的取值“高”“中”“低”可以转化为\{1.0,0.5,0.0\}；若属性间不存在序关系，假定有k个属性值，则通常转化为k维向量，列如属性“瓜类”的取值“西瓜”“南瓜”“黄瓜”可转化为(0,0,1)，(0,1,0)，(1,0,0)。值得注意的点是留白处提到的问题：若将无序属性连续化，则会不恰当地引入序关系，对后续处理如距离计算等造成误导。

给定由d个属性描述的示例 ![Center][]列向量，其中xi是x在第i个属性上的取值，线性模型 ![Center 1][]试图学得通过属性的线性组合来进行预测的函数。向量形式 ![Center 2][]，其中 ![Center 3][]列向量。采用如下公式度量真实值和预测值之间的误差：

![Center 4][]

求解w和b即是使 ![Center 5][]最小化的过程。旁注中指的凸函数和国内关于凸函数的定义是相反的。对w和b求导，得：

![Center 6][]令两式为零可得到w和b最优解的闭式解： ![Center 7][] ![Center 8][]其中 ![Center 9][]

scikit-learn库中的Ordinary Least Squares

线性回归试图学的： ![Center 10][]其中 ![Center 11][]为系数（coef\_），W0为截距（intercept\_）。如何确定w,关键在于衡量真实值f(x)和估计值y之间的差别。若以均方差作为性能度量函数，则问题被转化为让均方误差最小化，即 ![Center 12][]。

`LinearRegression`  will take in its  `fit`  method arrays X, y and will store the coefficients   of the linear model in its  `coef_`  member

LinearRegression类的成员函数 fit 以数组X和y为输入，并将线性模型的系数  存储在其成员变量coef\_ 中

>>> from sklearn import linear_model
    >>> reg = linear_model.LinearRegression()
    >>> reg.fit ([[0, 0], [1, 1], [2, 2]], [0, 1, 2])
    LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)
    >>> reg.coef_
    array([ 0.5,  0.5])

函数原型：class `sklearn.linear_model.``LinearRegression`(fit\_intercept=True, normalize=False, copy\_X=True, n\_jobs=1)

参考地址：http://scikit-learn.org/stable/modules/generated/sklearn.linear\_model.LinearRegression.html\#sklearn.linear\_model.LinearRegression

主要关注两个参数fit\_intercept和normalize。fit\_intercept用来确定是否计算模型的截距，normalize在不计算模型截距情况下被忽略，如果该参数为True,回归因子X在回归前将被归一化（通过减去均值除去L2范数）。返回值coef\_和intercept\_。intercept\_是线性模型中的独立项，coef\_（线性回归问题估计的参数）可选类型有array, shape (n\_features, ) 或 (n\_targets, n\_features)。

这边不是很清楚，占坑学习However, coefficient estimates for Ordinary Least Squares rely on the independence of the model terms. When terms are correlated and the columns of the design matrix   have an approximate linear dependence, the design matrix becomes close to singular and as a result, the least-squares estimate becomes highly sensitive to random errors in the observed response, producing a large variance. This situation of  multicollinearity  can arise, for example, when data are collected without an experimental design.

然而，最小二乘的系数估计依赖于模型特征项的独立性。当特征项相关并且设计矩阵 的列近似的线性相关时，设计矩阵便接近于一个奇异矩阵，因此最小二乘估计对观测点中的随机误差变得高度敏感，产生大的方差。例如，当没有试验设计的收集数据时，可能会出现这种多重共线性(multicollinearity )的情况。

import pandas   
import matplotlib.pyplot as plt  
  
\#read data from csv  
pga = pandas.read\_csv("pga.csv")  
  
\# Normalize the data  
pga.distance = (pga.distance - pga.distance.mean())/pga.distance.std()  
pga.accuracy = (pga.accuracy - pga.accuracy.mean())/pga.accuracy.std()  
  
\# show relationship between distance and accuracy  
plt.scatter(pga.distance, pga.accuracy)                   \#散点图  
plt.xlabel('normalized distance')  
plt.ylabel('normalized accuracy')  
plt.show()  
  
\# accuracyi=θ1distancei+θ0+ϵ  
\# scikit-learn 0.18.1  
from sklearn.linear\_model import LinearRegression  
import numpy as np  
  
\# We can add a dimension to an array by using np.newaxis  \#使用numpy.newaxis给数组增加一维  
print("Shape of the series:", pga.distance.shape)         \#矩阵形状  
print("Shape with new axis",pga.distance\[:,np.newaxis\].shape)  
  
\# The X variable in LinearRegression.fit() must have 2 dimensions  
lm = LinearRegression()  
lm.fit(pga.distance\[:, np.newaxis\], pga.accuracy)  
theta1 = lm.coef\_\[0\]

print(theta1)

![Center 13][]

\# The cost function of a single variable linear model  
def cost(theta0, theta1, x, y):  
    \# Initialize cost   
    J = 0  
    \# The number of observation  
    m = len(x)  
    \# Loop through each observation  
    for i in range(m):  
        \# Compute the hypothesis  
        h = theta1 \* x\[i\] +theta0  
        \# Add to cost  
        J += (h - y\[i\])\*\*2  
    \# Average and normalize cost  
    J /= (2\*m)  
    return J  
  
  
\# The cost for theta0 = 0 and theta1 = 1  
\#print(cost(0, 1, pga.distance, pga.accuracy))  
  
  
theta0 = 100  
thetals = np.linspace(-3,2,100)  
costs = \[\]  
for theta1 in thetals:  
    costs.append(cost(theta0, theta1, pga.distance, pga.accuracy))  
  
  
plt.plot(thetals, costs)  
plt.show()

![Center 14][]

import numpy as np  
from mpl\_toolkits.mplot3d import Axes3D

theta0s = np.linspace(-2,2,100)  
theta1s = np.linspace(-2,2,100)  
COST = np.empty(shape=(100,100))  
\# Meshgrid for paramaters  
T0S, T1S = np.meshgrid(theta0s, theta1s)  
\#for each parameter combination compute the cost  
for i in range(100):  
    for j in range(100):  
        COST\[i,j\] = cost(T0S\[0,i\], T1S\[j,0\], pga.distance, pga.accuracy)  
  
  
\# make 3d plot  
fig2 = plt.figure()  
ax = fig2.gca(projection='3d')  
ax.plot\_surface(X=T0S,Y=T1S,Z=COST)  
plt.show()

![Center 15][]

参考书籍：周志华 机器学习

参考文档：http://scikit-learn.org/stable/modules/linear\_model.html针对scikit-learn v0.19.1的User guide

中文 http://cwiki.apachecn.org/pages/viewpage.action?pageId=10814293\#GeneralizedLinearModels(广义线性模型)-OrdinaryLeastSquares(普通最小二乘法)

[Center]: https://img-blog.csdn.net/20171122095016853?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvYXNtYXJ0a2lsbGVy/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center
[Center 1]: https://img-blog.csdn.net/20171122095206739?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvYXNtYXJ0a2lsbGVy/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center
[Center 2]: https://img-blog.csdn.net/20171122095447009?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvYXNtYXJ0a2lsbGVy/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center
[Center 3]: /images/20220605/a5d33e0d2f8544b9a4b1a2210076aec8.png
[Center 4]: /images/20220605/06b6840d712d4ef4959e1e353ad09bfb.png
[Center 5]: /images/20220605/f2d033d4339e461f8152663bdac3f350.png
[Center 6]: https://img-blog.csdn.net/20171122160216645?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvYXNtYXJ0a2lsbGVy/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center
[Center 7]: https://img-blog.csdn.net/20171122160414730?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvYXNtYXJ0a2lsbGVy/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center
[Center 8]: https://img-blog.csdn.net/20171122160431372?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvYXNtYXJ0a2lsbGVy/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center
[Center 9]: /images/20220605/77b579430d2b421f8d0a62852b8e2673.png
[Center 10]: https://img-blog.csdn.net/20171121201443571?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvYXNtYXJ0a2lsbGVy/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center
[Center 11]: https://img-blog.csdn.net/20171121201703076?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvYXNtYXJ0a2lsbGVy/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center
[Center 12]: /images/20220605/488890274aed4caaa3f679827e13e527.png
[Center 13]: /images/20220605/ff29a508697249b2987ee8ebf9bf94b6.png
[Center 14]: /images/20220605/3f2d893c77b84a52a8d90c8422ab9f39.png
[Center 15]: /images/20220605/9ff0a410f198499586b7d4d5ff8d8ecd.png