ValueError: Variable net_-1/basic_lstm_cell/weights does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?

[Jiankai-Sun]

Environment:
Python 3.5.2
TensorFlow 1.2.1

Traceback (most recent call last):
  File "a3c.py", line 70, in <module>
    device = device)
  File "/home/jack/Applications/async_deep_reinforce/a3c_training_thread.py", line 51, in __init__
    self.gradients )
  File "/home/jack/Applications/async_deep_reinforce/rmsprop_applier.py", line 103, in apply_gradients
    clipped_accum_grad = tf.clip_by_norm(accum_grad, self._clip_norm)
  File "/home/jack/python3/venv/lib/python3.5/site-packages/tensorflow/python/ops/clip_ops.py", line 107, in clip_by_norm
    t = ops.convert_to_tensor(t, name="t")
  File "/home/jack/python3/venv/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 676, in convert_to_tensor
    as_ref=False)
  File "/home/jack/python3/venv/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 741, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/home/jack/python3/venv/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 113, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/home/jack/python3/venv/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 102, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/home/jack/python3/venv/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py", line 364, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.
(venv) jack@jack:~/Applications/async_deep_reinforce$ python a3c.py 
Traceback (most recent call last):
  File "a3c.py", line 50, in <module>
    global_network = GameACLSTMNetwork(ACTION_SIZE, -1, device)
  File "/home/jack/Applications/async_deep_reinforce/game_ac_network.py", line 218, in __init__
    self.W_lstm = tf.get_variable("basic_lstm_cell/weights")
  File "/home/jack/python3/venv/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 1065, in get_variable
    use_resource=use_resource, custom_getter=custom_getter)
  File "/home/jack/python3/venv/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 962, in get_variable
    use_resource=use_resource, custom_getter=custom_getter)
  File "/home/jack/python3/venv/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 367, in get_variable
    validate_shape=validate_shape, use_resource=use_resource)
  File "/home/jack/python3/venv/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 352, in _true_getter
    use_resource=use_resource)
  File "/home/jack/python3/venv/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 682, in _get_single_variable
    "VarScope?" % name)
ValueError: Variable net_-1/basic_lstm_cell/weights does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?

[MogicianWu]

I had the same issue when running a3c.py with LSTM = True.

[MogicianWu]

You need to change these two lines in game_ac_network.py:

self.W_lstm = tf.get_variable("basic_lstm_cell/kernel")
self.b_lstm = tf.get_variable("basic_lstm_cell/bias")

This is mentioned in issue #33 .