Function and usage of TF. Control ﹣ dependencies()



In some machine learning programs, we want to specify the dependency of certain operation execution. At this time, we can use TF. Control ﹣ dependencies().  
Control dependencies (control inputs) returns a context manager that controls dependencies. Using the with keyword, all operations in this context can be performed in control inputs.

with g.control_dependencies([a, b, c]):
  # `d` and `e` will only run after `a`, `b`, and `c` have executed.
  d = ...
  e = ...
  • 1
  • 2
  • 3
  • 4

You can nest control dependencies to use

with g.control_dependencies([a, b]):
  # Ops constructed here run after `a` and `b`.
  with g.control_dependencies([c, d]):
    # Ops constructed here run after `a`, `b`, `c`, and `d`.
  • 1
  • 2
  • 3
  • 4

You can pass in None to eliminate the dependency:

with g.control_dependencies([a, b]):
  # Ops constructed here run after `a` and `b`.
  with g.control_dependencies(None):
    # Ops constructed here run normally, not waiting for either `a` or `b`.
    with g.control_dependencies([c, d]):
      # Ops constructed here run after `c` and `d`, also not waiting
      # for either `a` or `b`.
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7

Note:
Control dependency is only valid for those operations established in context. It is useless to use only one operation or tensor in context

# WRONG
def my_func(pred, tensor):
  t = tf.matmul(tensor, tensor)
  with tf.control_dependencies([pred]):
    # The matmul op is created outside the context, so no control
    # dependency will be added.
    return t

# RIGHT
def my_func(pred, tensor):
  with tf.control_dependencies([pred]):
    # The matmul op is created in the context, so a control dependency
    # will be added.
    return tf.matmul(tensor, tensor)
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14

Example:
When training the model, we may perform two operations for each step of training, op a, b. then we can use the following code:

with tf.control_dependencies([a, b]):
    c= tf.no_op(name='train')#Tf.no-op; do nothing
sess.run(c)
  • 1
  • 2
  • 3

Under such simple requirements, you can replace the above code with:

c= tf.group([a, b])
sess.run(c)

Posted on Sun, 03 May 2020 01:39:37 -0700 by metuin