【MOCC：人工智能实践：Tensorflow学习笔记】3.2、3.3 Tensflow框架-前向、后向传播

傷城~ 2022-03-24 18:04 219阅读 0赞

**目录**

参数

神经网络的实现过程

前向传播

后向传播

综合示例

搭建神经网络的八股

--------------------

# 参数 #

参数就神经元上的权重W，用变量表示，随机给初始值。

随机生成一些变量的方法：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BsU29uZ19DU0RO_size_16_color_FFFFFF_t_70][]

![20190122160159848.png][]

# 神经网络的实现过程 #

1.  准备数据集。提取特征。作为输入喂给神经网络（Neural Network，NN）
2.  搭建NN结构，从输入到输出（先搭建计算图，再用会话执行）（NN前向传播算法→计算输出）
3.  大量特征数据喂给NN，迭代优化NN参数（NN反向传播算法→优化参数训练模型）
4.  使用训练好的模型进行预测和分类

前三步：训练，第四步：测试

# 前向传播 #

搭建一个模型，实现推理（以全连接网络为例）

例：生产一批零件，将体积x1和重量x2为特征，输入到NN，通过NN后，得到输出：

先看下面的结构

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BsU29uZ19DU0RO_size_16_color_FFFFFF_t_70 1][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BsU29uZ19DU0RO_size_16_color_FFFFFF_t_70 2][]

神经网络的层数从第一个计算层开始，比如，输入不是计算出，所以a是第一层。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BsU29uZ19DU0RO_size_16_color_FFFFFF_t_70 3][]

看一下代码实现

#coding:utf-8
    #代码中如果有中文，用utf-8编码
    #两层简单神经网络
    import tensorflow as tf
    
    #定义输入和参数
    x = tf.constant([[0.7,0.5]])#一行两列
    w1 = tf.Variable(tf.random_normal([2,3],stddev=1,seed=1))
    w2 = tf.Variable(tf.random_normal([3,1],stddev=1,seed=1))
    
    #定义前向传播过程
    a = tf.matmul(x,w1)#矩阵乘法
    y = tf.matmul(a,w2)
    
    #用会话进行计算
    with tf.Session() as sess:
        init_op = tf.global_variables_initializer()
        sess.run(init_op)
        print("y in tf3_3.py is:",sess.run(y))

执行结果

![20190122162953509.png][]

下面是使用placeholder，一次性喂入多个数据。

#coding:utf-8
    #代码中如果有中文，用utf-8编码
    #两层简单神经网络
    import tensorflow as tf
    
    #定义输入和参数
    #使用placeholder实现输入定义
    x = tf.placeholder(tf.float32,shape=(None,2))
    w1 = tf.Variable(tf.random_normal([2,3],stddev=1,seed=1))
    w2 = tf.Variable(tf.random_normal([3,1],stddev=1,seed=1))
    
    #定义前向传播过程
    a = tf.matmul(x,w1)
    y = tf.matmul(a,w2)
    
    #用会话进行计算
    with tf.Session() as sess:
        init_op = tf.global_variables_initializer()
        sess.run(init_op)
        print("y in tf3_3.py is:\n",sess.run(y,feed_dict={x:[[0.7,0.5],[0.2,0.4],[4.0,2.0]]}))

# 后向传播 #

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BsU29uZ19DU0RO_size_16_color_FFFFFF_t_70 4][]

# 综合示例 #

建立一个完整的神经网络，输入体积和重量，当“体积+重量>1 合格，否则不合格”

#coding:utf-8
    #导入模块
    import numpy as np
    import tensorflow as tf
    batch_size = 8
    seed = 23455
    
    #基于seed产生随机数
    rng = np.random.RandomState(seed)
    
    #随机数返回32行2列的矩阵，表示32组 体积和重量数据，作为输入数据集
    X = rng.rand(32,2)
    
    #取出一行，判断如果小于1，y = 1，否则，y=0， 作为数据的标签（正确答案）
    Y = [[int((x0+x1)<1)] for (x0,x1) in X]
    
    #定义神经网络的输入，参数和输出，定义前向传播的过程
    x = tf.placeholder(tf.float32,shape=(None,2))
    y_ = tf.placeholder(tf.float32,shape=(None,1))
    
    #初始化参数（权重）
    w1 = tf.Variable(tf.random_normal([2,3],stddev=1, seed=1))
    w2 = tf.Variable(tf.random_normal([3,1],stddev=1,seed=1))
    
    a = tf.matmul(x,w1)
    y = tf.matmul(a,w2)
    
    #定义损失函数及反向传播算法
    loss = tf.reduce_mean(tf.square(y-y_)) #均方误差
    train_step = tf.train.GradientDescentOptimizer(0.001).minimize(loss) # 梯度下降法，0.001是学习率
    #train_step = tf.train.MomentumOptimizer(0.001,0.9).minimize(loss)
    #train_step = tf.train.AdamOptimizer(0.001,0.9).minimize(loss)
    
    #生成会话，训练steps轮
    with tf.Session() as sess:
        init_op = tf.global_variables_initializer()
        sess.run(init_op)
        print("w1:\n",sess.run(w1))
        print("w2:\n",sess.run(w2))
    
        #开始训练
        steps = 10000
        for i in range(steps):
            start = (i*batch_size) % 32
            end = start + batch_size
            sess.run(train_step,feed_dict={x:X[start:end],y_:Y[start:end]})
            if i % 500 == 0:
                total_loss = sess.run(loss,feed_dict={x:X,y_:Y})
                print("After %d training steps,loss on all data is %g"%(i,total_loss))
            
        #输出训练好的参数值
        print("\n")
        print("w1:\n",sess.run(w1))
        print("w2:\n",sess.run(w2))

执行结果：

> w1:  
>  \[\[-0.8113182   1.4845988   0.06532937\]  
>  \[-2.4427042   0.0992484   0.5912243 \]\]  
> w2:  
>  \[\[-0.8113182 \]  
>  \[ 1.4845988 \]  
>  \[ 0.06532937\]\]  
> After 0 training steps,loss on all data is 5.13118  
> After 500 training steps,loss on all data is 0.429111  
> After 1000 training steps,loss on all data is 0.409789  
> After 1500 training steps,loss on all data is 0.399923  
> After 2000 training steps,loss on all data is 0.394146  
> After 2500 training steps,loss on all data is 0.390597  
> After 3000 training steps,loss on all data is 0.388336  
> After 3500 training steps,loss on all data is 0.386855  
> After 4000 training steps,loss on all data is 0.385863  
> After 4500 training steps,loss on all data is 0.385186  
> After 5000 training steps,loss on all data is 0.384719  
> After 5500 training steps,loss on all data is 0.384391  
> After 6000 training steps,loss on all data is 0.38416  
> After 6500 training steps,loss on all data is 0.383995  
> After 7000 training steps,loss on all data is 0.383877  
> After 7500 training steps,loss on all data is 0.383791  
> After 8000 training steps,loss on all data is 0.383729  
> After 8500 training steps,loss on all data is 0.383684  
> After 9000 training steps,loss on all data is 0.383652  
> After 9500 training steps,loss on all data is 0.383628
> 
>   
> w1:  
>  \[\[-0.6916735   0.81592685  0.0962934 \]  
>  \[-2.3433847  -0.1074271   0.5854707 \]\]  
> w2:  
>  \[\[-0.08710451\]  
>  \[ 0.81644475\]  
>  \[-0.05229824\]\]

# 搭建神经网络的八股 #

准备、前传、反传、迭代

1.  准备： import模块、常量定义、生成数据集
2.  前向传播：定义输入、参数（权重）、输出， x= ，y\_= ,w1= ,w2 =  ,a=  ,y =
3.  反向传播:定义损失函数、反向传播算法， loss = ，   train\_step = 
4.  生成会话，进行迭代steps轮。

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BsU29uZ19DU0RO_size_16_color_FFFFFF_t_70]: /images/20220323/6e2cda5fcd26456bbf7aaba176a97b6a.png
[20190122160159848.png]: /images/20220323/455ba6d2e77b43b89041334a2ff9d14e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BsU29uZ19DU0RO_size_16_color_FFFFFF_t_70 1]: /images/20220323/828c900dcc0a4f4ebbab6a7e37c778a6.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BsU29uZ19DU0RO_size_16_color_FFFFFF_t_70 2]: /images/20220323/b1ec380e428d4a0e9409f3cb20dbafef.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BsU29uZ19DU0RO_size_16_color_FFFFFF_t_70 3]: /images/20220323/0d7b398ef6404d3b99aac1af7683e507.png
[20190122162953509.png]: /images/20220323/c76ee9c7018b4918bd6fcaea55906dae.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BsU29uZ19DU0RO_size_16_color_FFFFFF_t_70 4]: /images/20220323/7fefb6a856ae4bb19be4356bc3538ac9.png