神经网络通俗讲，麻瓜变大神(二)-解释非线性

爱被打了一巴掌 2022-05-27 12:29 169阅读 0赞

系列目录:  
[神经网络通俗讲，麻瓜变大神(一)-初识神经网络][-]  
[神经网络通俗讲，麻瓜变大神(二)-解释非线性][- 1]  
[神经网络通俗讲，麻瓜变大神(三)-卷积神经网络][- 2]

下面是很常见的神经网络代码，3层神经网络，输入层(300,2),隐藏层(2,3),激活函数ReLu,输出层(3,3)，epoch=5000:

import numpy as np
    import matplotlib.pyplot as plt
    
    np.random.seed(0)
    N = 100 # number of points per class
    D = 2 # dimensionality
    K = 3 # number of classes
    X = np.zeros((N*K,D))
    y = np.zeros(N*K, dtype='uint8')
    for j in xrange(K):
      ix = range(N*j,N*(j+1))
      r = np.linspace(0.0,1,N) # radius
      t = np.linspace(j*4,(j+1)*4,N) + np.random.randn(N)*0.2 # theta
      X[ix] = np.c_[r*np.sin(t), r*np.cos(t)]
      y[ix] = j
    
    h = 3 # size of hidden layer
    W1 = 0.01 * np.random.randn(D,h)
    b1 = np.zeros((1,h))
    W2 = 0.01 * np.random.randn(h,K)
    b2 = np.zeros((1,K))
    
    # some hyperparameters
    step_size = 1e-0
    reg = 1e-3 # regularization strength
    
    # gradient descent loop
    num_examples = X.shape[0]
    #开始训练
    for i in xrange(5000):
    
      # evaluate class scores, [N x K]
      score1 = np.maximum(0, np.dot(X, W1) + b1) # note, ReLU activation hidden_layer
      #print hidden_layer.shape
      score2 = np.dot(score1, W2) + b2  #scores
      #print scores.shape
      # compute the class probabilities
      exp_scores = np.exp(score2)
      probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True) # [N x K]
      #print probs.shape
    
      # compute the loss: average cross-entropy loss and regularization
      corect_logprobs = -np.log(probs[range(num_examples),y])
      data_loss = np.sum(corect_logprobs)/num_examples
      reg_loss = 0.5*reg*np.sum(W*W) + 0.5*reg*np.sum(W2*W2)
      loss = data_loss + reg_loss
      if i % 100 == 0:
        print "iteration %d: loss %f" % (i, loss)
    
      # compute the gradient on scores，开始反向传播
      dscore2 = probs
      dscore2[range(num_examples),y] -= 1
      dscore2 /= num_examples
    
      # backpropate the gradient to the parameters
      # first backprop into parameters W2 and b2
      dW2 = np.dot(score1.T, dscore2)
      db2 = np.sum(dscore2, axis=0, keepdims=True)
      # next backprop into hidden layer
      dscore1 = np.dot(dscore2, W2.T)
      # backprop the ReLU non-linearity
      dscore1[score1 <= 0] = 0
      # finally into W,b
      dW = np.dot(X.T, dscore1)
      db = np.sum(dscore1, axis=0, keepdims=True)
    
      # add regularization gradient contribution
      dW2 += reg * W2
      dW1 += reg * W1
    
      # perform a parameter update
      W1 += -step_size * dW1
      b1 += -step_size * db1
      W2 += -step_size * dW2
      b2 += -step_size * db2
    #后面这里只是用于画图了
    hidden_layer = np.maximum(0, np.dot(X, W) + b)
    scores = np.dot(hidden_layer, W2) + b2
    predicted_class = np.argmax(scores, axis=1)
    print 'training accuracy: %.2f' % (np.mean(predicted_class == y))
    
    
    h = 0.02
    x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
    y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
                         np.arange(y_min, y_max, h))
    Z = np.dot(np.maximum(0, np.dot(np.c_[xx.ravel(), yy.ravel()], W) + b), W2) + b2
    Z = np.argmax(Z, axis=1)
    Z = Z.reshape(xx.shape)
    fig = plt.figure()
    plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral, alpha=0.8)
    plt.scatter(X[:, 0], X[:, 1], c=y, s=40, cmap=plt.cm.Spectral)
    plt.xlim(xx.min(), xx.max())
    plt.ylim(yy.min(), yy.max())
    plt.show()

最后输出各层权重和分类结果图:

![这里写图片描述][70]  
![这里写图片描述][70 1]  
经过四舍五入，得到各个参数矩阵:

w1=\[−1.951.632.741.262.49−1.26\]  w 1 = \[ − 1.95 2.74 2.49 1.63 1.26 − 1.26 \]  b1=\[−1.361.100.92\]  b 1 = \[ − 1.36 1.10 0.92 \]

w2=⎡⎣⎢−1.54−0.23−0.180.471.58−1.221.07−1.351.4⎤⎦⎥  w 2 = \[ − 1.54 0.47 1.07 − 0.23 1.58 − 1.35 − 0.18 − 1.22 1.4 \]  b2=\[2.59−1.98−0.61\]  b 2 = \[ 2.59 − 1.98 − 0.61 \]

根据矩阵得到3个方程:  
(1)z1 = -1.54\*max(0,-1.95x+1.63y+1.36)-0.23\*max(0,2.74x+1.26y+1.10)-0.18\*max(0,2.49x-1.26y+0.92)+2.59  
(2)z2 = 0.47\*max(0,-1.95x+1.63y+1.36)+1.58\*max(0,2.74x+1.26y+1.10)-1.22\*max(0,2.49x-1.26y+0.92)-1.98  
(3)z3 = 1.07\*max(0,-1.95x+1.63y+1.36)-1.35\*max(0,2.74x+1.26y+1.10)+1.4\*max(0,2.49x-1.26y+0.92)-0.61

再看这3个方程图像,画图工具为ipad app,Pocket CAS:  
![这里写图片描述][70 2]  
![这里写图片描述][70 3]

[-]: https://blog.csdn.net/u011311291/article/details/79841748
[- 1]: https://blog.csdn.net/u011311291/article/details/79996630
[- 2]: https://blog.csdn.net/u011311291/article/details/80273394
[70]: /images/20220527/c479fa86749e4a7f8a5e247c76121f0f.png
[70 1]: /images/20220527/14a29a29ea864078acc1f29b081f9b5f.png
[70 2]: /images/20220527/1135d83259cb44d4afd2164de0303837.png
[70 3]: /images/20220527/8e08008a707843a2b5f8ac085d283dd7.png