Tensorflow快速入门——知识点总结

骑猪看日落 2023-06-27 05:30 4阅读 0赞

# Tensorflow快速入门 #

## 知识点总结 ##

*  梯度下降方法总结
 *  模型保存和恢复

前两篇博文我们实现了线性回归与手写数字识别  
[Tensorflow快速入门——线性回归][Tensorflow]  
[Tensorflw快速入门——运用逻辑回归算法手写数字识别][Tensorflw]  
本文具体数据集与源代码可从我的GitHub地址获取  
[https://github.com/liuzuoping/Deep\_Learning\_note][https_github.com_liuzuoping_Deep_Learning_note]

### 梯度下降方法总结 ###

*  Tensorflow直接计算，满足公式：  
     θ = θ − η ▽ θ M S E ( θ ) \\theta = \\theta- \\eta \\bigtriangledown\_\{\\theta\} MSE(\\theta) θ=θ−η▽θMSE(θ)
 *  tf.random\_uniform()：在图中创建一个包含随机值的节点，类似于NumPy中的random()函数
 *  tf.assign()：创建一个将新值赋给变量的一个节点，为variable更新值  
    更加具体的可以回顾上篇博文  
    [梯度下降法详解][Link 1]

#--coding--:utf-8
    import numpy as np
    import tensorflow as tf
    from sklearn.datasets import fetch_california_housing
    from sklearn.preprocessing import StandardScaler
    scaler = StandardScaler()
    
    # 获取数据
    housing = fetch_california_housing()
    m,n = housing.data.shape
    print (housing.keys())
    print (housing.feature_names)
    print (housing.target)
    print (housing.DESCR)

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdXp1b3Bpbmc_size_16_color_FFFFFF_t_70]

housing_data_plus_bias = np.c_[np.ones((m,1)), housing.data]
    scaled_data = scaler.fit_transform(housing.data)
    data = np.c_[np.ones((m,1)),scaled_data]
    
    # 设置参数
    n_epoch = 1000
    learning_rate = 0.01
    
    # 设置placeholder
    X = tf.constant(data,dtype = tf.float32,name = "X")
    y = tf.constant(housing.target.reshape(-1,1),dtype=tf.float32,name='y')
    
    theta = tf.Variable(tf.random_uniform([n+1, 1], -1, 1),name='theta')
    y_pred = tf.matmul(X,theta,name='prediction')
    error = y_pred - y
    mse = tf.reduce_mean(tf.square(error),name='mse') #Mean Squared Error 
    
    # 计算梯度公式
    gradient = 2/m * tf.matmul(tf.transpose(X),error)
    training_op = tf.assign(theta,theta - learning_rate * gradient)
    
    init = tf.global_variables_initializer()
    with tf.Session() as sess:
        sess.run(init)
        for epoch in range(n_epoch):
            if epoch % 100 == 0:
                print ("Epoch",epoch, "MSE =", mse.eval())
    
        print ('best theta:',theta.eval())

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdXp1b3Bpbmc_size_16_color_FFFFFF_t_70 1]

### 自动求导autodiff ###

手动求导方法有两个缺陷：

*  深度神经网络中公式较长
 *  计算效率较低  
    可以采取以下命令进行自动求导：

gradients = tf.gradients(mse,[theta])[0]

gradient有两个参数：

*  op，损失函数，这里是mse
 *  variable lists，变量列表，这里是theta值

### 使用Optimizer ###

可以使用各种优化器,如梯度下降优化器

optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)
    training_op = optimizer.minimize(mse)

还有很多其他优化器，例如收敛更快的MomentumOptimizer优化器等

### 梯度下降中传输数据的方式 ###

*  mini-batchbatch
 *  方法：使用占位符placeholder

# 传统方式
    A = tf.placeholder(tf.float32,shape=(None,3))
    B = A + 5
    with tf.Session() as sess:
        test_b_1 = B.eval(feed_dict={ A:[[1,2,3]]})
        test_b_2 = B.eval(feed_dict={ A:[[4,5,6],[7,8,9]]})
    print(test_b_1)  
    print(test_b_2)

> \[\[6. 7. 8.\]\]  
> \[\[ 9. 10. 11.\]  
> \[12. 13. 14.\]\]

# 使用mini-batch方式
    X = tf.placeholder(tf.float32, shape=(None, n + 1), name="X")
    y = tf.placeholder(tf.float32, shape=(None, 1), name="y")
    
    # 与上文例子相同，这里采用的是使用optimizer计算梯度
    n_epochs = 100
    display_step = 10
    learning_rate = 0.01
    batch_size = 100
    n_batches = int(np.ceil(m / batch_size))
    theta = tf.Variable(tf.random_uniform([n + 1, 1], -1.0, 1.0, seed=42), name="theta") #X0
    y_pred = tf.matmul(X, theta, name="predictions")
    error = y_pred - y
    mse = tf.reduce_mean(tf.square(error), name="mse")
    optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)
    training_op = optimizer.minimize(mse)
    
    init = tf.global_variables_initializer()
    
    
    # 定义mini-batch取数据方式
    def fetch_batch(epoch, batch_index, batch_size):
        np.random.seed(epoch * n_batches + batch_index) 
        indices = np.random.randint(m, size=batch_size)  
        X_batch = data[indices] 
        y_batch = housing.target.reshape(-1, 1)[indices] 
        return X_batch, y_batch
    
    # mini-batch计算过程
    with tf.Session() as sess:
        sess.run(init)
        for epoch in range(n_epochs):#迭代的次数
            avg_cost = 0.
            for batch_index in range(n_batches):
                X_batch, y_batch = fetch_batch(epoch, batch_index, batch_size)
                sess.run(training_op, feed_dict={ X: X_batch, y: y_batch})
    
                if epoch % 10 == 0:
                    total_loss = 0
                    acc_train = mse.eval(feed_dict={ X: X_batch, y: y_batch})
                    total_loss += acc_train 
                    #print acc_train,total_loss
                    print ( "epoch:",epoch, "batch_id:",batch_index, "Batch loss:", acc_train)

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdXp1b3Bpbmc_size_16_color_FFFFFF_t_70 2]

## 模型的保存和恢复 ##

*  在创建图阶段创建一个Saver的节点，在执行阶段需要保存模型的地方调用Save()函数即可

# 模型的保存
    init = tf.global_variables_initializer()
    saver = tf.train.Saver()
    with tf.Session() as sess:
        sess.run(init)
        for epoch in range(n_epochs):
            if epoch % 100 == 0:
                print("Epoch", epoch, "MSE =", mse.eval())  # 保存运行过程
                save_path = saver.save(sess, "/my_model.ckpt")
            sess.run(training_op)
        
        best_theta = theta.eval()
        save_path = saver.save(sess, "/my_model_final.ckpt")#保存最后的结果
        
    # 模型的加载
    with tf.Session() as sess:
        saver.restore(sess, "/tmp/my_model_final.ckpt")
        best_theta_restored = theta.eval() 
        
    saver = tf.train.Saver({ "weights":theta})
    
    
    reset_graph()
    saver = tf.train.import_meta_graph("/tmp/my_model_final.ckpt.meta")
    theta = tf.get_default_graph().get_tensor_by_name("theta:0")

[Tensorflow]: https://blog.csdn.net/liuzuoping/article/details/103819763
[Tensorflw]: https://blog.csdn.net/liuzuoping/article/details/103820293
[https_github.com_liuzuoping_Deep_Learning_note]: https://github.com/liuzuoping/Deep_Learning_note
[Link 1]: https://blog.csdn.net/liuzuoping/article/details/103818991
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdXp1b3Bpbmc_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200103150943886.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdXp1b3Bpbmc=,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdXp1b3Bpbmc_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20200103151658552.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdXp1b3Bpbmc=,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdXp1b3Bpbmc_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/2020010315232596.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdXp1b3Bpbmc=,size_16,color_FFFFFF,t_70