tf.nn.dynamic_rnn的输出outputs和state含义

古城微笑少年丶 2022-02-21 08:24 247阅读 0赞

## 转载自[https://blog.csdn.net/u010960155/article/details/81707498][https_blog.csdn.net_u010960155_article_details_81707498] ##

## **一、 tf.nn.dynamic\_rnn的输出** ##

tf.nn.dynamic\_rnn的输入参数如下：

tf.nn.dynamic\_rnn(  
cell,  
inputs,  
sequence\_length=None,  
initial\_state=None,  
dtype=None,  
parallel\_iterations=None,  
swap\_memory=False,  
time\_major=False,  
scope=None  
)

tf.nn.dynamic\_rnn的返回值有两个：outputs和state

为了描述输出的形状，先介绍几个变量，batch\_size是输入的这批数据的数量，max\_time就是这批数据中序列的最长长度，如果输入的三个句子，那max\_time对应的就是最长句子的单词数量，cell.output\_size其实就是rnn cell中神经元的个数。

outputs. outputs是一个tensor  
如果time\_major==True，outputs形状为 \[max\_time, batch\_size, cell.output\_size \]（要求rnn输入与rnn输出形状保持一致）  
如果time\_major==False（默认），outputs形状为 \[ batch\_size, max\_time, cell.output\_size \]  
state. state是一个tensor。state是最终的状态，也就是序列中最后一个cell输出的状态。一般情况下state的形状为 \[batch\_size, cell.output\_size \]，但当输入的cell为BasicLSTMCell时，state的形状为\[2，batch\_size, cell.output\_size \]，其中2也对应着LSTM中的cell state和hidden state  
那为什么state输出形状会有变化呢？state和output又有什么关系呢？

二、state含义  
对于第一问题“state”形状为什么会发生变化呢？  
![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2R1bGluZ3Rpbmd6aQ_size_16_color_FFFFFF_t_70][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2R1bGluZ3Rpbmd6aQ_size_16_color_FFFFFF_t_70 1][]

## 三、实验 ##

我们写点代码来具体感觉下outputs和state是什么，代码如下

import tensorflow as tf  
import numpy as np  
  
def dynamic\_rnn(rnn\_type='lstm'):  
\# 创建输入数据,3代表batch size,6代表输入序列的最大步长(max time),8代表每个序列的维度  
X = np.random.randn(3, 6, 4)  
  
\# 第二个输入的实际长度为4  
X\[1, 4:\] = 0  
  
\#记录三个输入的实际步长  
X\_lengths = \[6, 4, 6\]  
  
rnn\_hidden\_size = 5  
if rnn\_type == 'lstm':  
cell = tf.contrib.rnn.BasicLSTMCell(num\_units=rnn\_hidden\_size, state\_is\_tuple=True)  
else:  
cell = tf.contrib.rnn.GRUCell(num\_units=rnn\_hidden\_size)  
  
outputs, last\_states = tf.nn.dynamic\_rnn(  
cell=cell,  
dtype=tf.float64,  
sequence\_length=X\_lengths,  
inputs=X)  
  
with tf.Session() as session:  
session.run(tf.global\_variables\_initializer())  
o1, s1 = session.run(\[outputs, last\_states\])  
print(np.shape(o1))  
print(o1)  
print(np.shape(s1))  
print(s1)  
  
  
if \_\_name\_\_ == '\_\_main\_\_':  
dynamic\_rnn(rnn\_type='lstm')  
实验一：cell类型为LSTM，我们看看输出是什么样子，如下图所示，输入的形状为 \[ 3, 6, 4 \]，经过tf.nn.dynamic\_rnn后outputs的形状为 \[ 3, 6, 5 \]，state形状为 \[ 2, 3, 5 \]，其中state第一部分为c，代表cell state;第二部分为h，代表hidden state。可以看到hidden state 与 对应的outputs的最后一行是相等的。另外需要注意的是输入一共有三个序列，但第二个序列的长度只有4，可以看到outputs中对应的两行值都为0，所以hidden state对应的是最后一个不为0的部分。tf.nn.dynamic\_rnn通过设置sequence\_length来实现这一逻辑。  
(3, 6, 5)  
\[\[\[ 0.0146346 -0.04717453 -0.06930042 -0.06065602 0.02456717\]  
\[-0.05580321 0.08770171 -0.04574306 -0.01652854 -0.04319528\]  
\[ 0.09087799 0.03535907 -0.06974291 -0.03757408 -0.15553619\]  
\[ 0.10003044 0.10654698 0.21004055 0.13792148 -0.05587583\]  
\[ 0.13547596 -0.014292 -0.0211154 -0.10857875 0.04461256\]  
\[ 0.00417564 -0.01985144 0.00050634 -0.13238986 0.14323784\]\]  
  
\[\[ 0.04893576 0.14289175 0.17957205 0.09093887 -0.0507192 \]  
\[ 0.17696126 0.09929577 0.21185635 0.20386451 0.11664373\]  
\[ 0.15658667 0.03952745 -0.03425637 0.00773833 -0.03546742\]  
\[-0.14002582 -0.18578786 -0.08373584 -0.25964601 0.04090167\]  
\[ 0. 0. 0. 0. 0. \]  
\[ 0. 0. 0. 0. 0. \]\]  
  
\[\[ 0.18564152 0.01531695 0.13752453 0.17188506 0.19555427\]  
\[ 0.13703949 0.14272294 0.21313036 0.07417354 0.0477547 \]  
\[ 0.23021792 0.04455495 0.10204565 0.17159792 0.34148467\]  
\[ 0.0386402 0.0387848 0.02134559 0.00110381 0.08414687\]  
\[ 0.01386241 -0.02629686 -0.0733538 -0.03194245 0.13606553\]  
\[ 0.01859433 -0.00585316 -0.04007138 0.03811594 0.21708331\]\]\]  
(2, 3, 5)  
LSTMStateTuple(c=array(\[\[ 0.00909146, -0.03747076, 0.0008946 , -0.23459786, 0.29565899\],  
\[-0.18409266, -0.30463044, -0.28033809, -0.49032542, 0.12597639\],  
\[ 0.04494702, -0.01359631, -0.06706629, 0.06766361, 0.40794032\]\]), h=array(\[\[ 0.00417564, -0.01985144, 0.00050634, -0.13238986, 0.14323784\],  
\[-0.14002582, -0.18578786, -0.08373584, -0.25964601, 0.04090167\],  
\[ 0.01859433, -0.00585316, -0.04007138, 0.03811594, 0.21708331\]\]))  
实验二：cell类型为GRU，我们看看输出是什么样子，如下图所示，输入的形状为 \[ 3, 6, 4 \]，经过tf.nn.dynamic\_rnn后outputs的形状为 \[ 3, 6, 5 \]，state形状为 \[ 3, 5 \]。可以看到 state 与 对应的outputs的最后一行是相等的。  
(3, 6, 5)  
\[\[\[-0.05190962 -0.13519617 0.02045928 -0.0821183 0.28337528\]  
\[ 0.0201574 0.03779418 -0.05092804 0.02958051 0.12232347\]  
\[ 0.14884441 -0.26075898 0.1821795 -0.03454954 0.18424161\]  
\[-0.13854156 -0.26565378 0.09567164 -0.03960079 0.14000589\]  
\[-0.2605973 -0.39901657 0.12495693 -0.19295695 0.52423598\]  
\[-0.21596414 -0.63051687 0.20837501 -0.31775378 0.77519457\]\]  
  
\[\[-0.1979659 -0.30253523 0.0248779 -0.17981144 0.41815343\]  
\[ 0.34481129 -0.05256187 0.1643036 0.00739746 0.27384158\]  
\[ 0.49703664 0.22241165 0.27344766 0.00093435 0.09854949\]  
\[ 0.23312444 0.156997 0.25482553 0.0138156 -0.02302272\]  
\[ 0. 0. 0. 0. 0. \]  
\[ 0. 0. 0. 0. 0. \]\]  
  
\[\[-0.06401732 0.08605342 -0.03936866 -0.02287695 0.16947652\]  
\[-0.1775206 -0.2801672 -0.0387468 -0.20264583 0.58125297\]  
\[ 0.39408762 -0.44066425 0.25826641 -0.18851604 0.36172166\]  
\[ 0.0536013 -0.29902928 0.08891931 -0.03930039 0.0743423 \]  
\[ 0.02304702 -0.0612499 0.09113458 -0.05169013 0.29876455\]  
\[-0.06711324 0.014125 -0.05856332 -0.05632359 -0.00390189\]\]\]  
(3, 5)  
\[\[-0.21596414 -0.63051687 0.20837501 -0.31775378 0.77519457\]  
\[ 0.23312444 0.156997 0.25482553 0.0138156 -0.02302272\]  
\[-0.06711324 0.014125 -0.05856332 -0.05632359 -0.00390189\]\]

所以一个样本对应一个state，而一个时刻有一个output

Reference

[http://colah.github.io/posts/2015-08-Understanding-LSTMs/][http_colah.github.io_posts_2015-08-Understanding-LSTMs]

[https://www.tensorflow.org/api\_docs/python/tf/nn/dynamic\_rnn][https_www.tensorflow.org_api_docs_python_tf_nn_dynamic_rnn]

[https_blog.csdn.net_u010960155_article_details_81707498]: https://blog.csdn.net/u010960155/article/details/81707498
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2R1bGluZ3Rpbmd6aQ_size_16_color_FFFFFF_t_70]: /images/20220221/14135fd6ca644b5e97aefc983f926cce.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2R1bGluZ3Rpbmd6aQ_size_16_color_FFFFFF_t_70 1]: /images/20220221/1e93772961de48cbb9e25d452678bf40.png
[http_colah.github.io_posts_2015-08-Understanding-LSTMs]: http://colah.github.io/posts/2015-08-Understanding-LSTMs/
[https_www.tensorflow.org_api_docs_python_tf_nn_dynamic_rnn]: https://www.tensorflow.org/api_docs/python/tf/nn/dynamic_rnn