How to get the global_step when restoring checkpoints in Tensorflow?

In TensorFlow, global_step is a crucial variable used to track the number of iterations during training. Retrieving this variable is often useful when restoring model checkpoints to resume training from where it was previously stopped.

Assume you have already trained a model and saved checkpoints. To restore checkpoints and retrieve global_step in TensorFlow, follow these steps:

1. Import necessary libraries: First, ensure TensorFlow is imported along with any other required libraries.

   python
   import tensorflow as tf

2. Create or build the model: Construct or rebuild your model architecture based on your requirements. This step is necessary because a model architecture is required to load checkpoint data.

3. Create or obtain the Saver object: The Saver object is used to load model weights. Ensure the model is defined before creating the Saver object.

   python
   saver = tf.train.Saver()

4. Create a session (Session): All operations in TensorFlow must be performed within a session.

   python
   with tf.Session() as sess:

5. Restore checkpoints: Within the session, use the saver.restore() method to load model weights. Provide the session object and the path to the checkpoint file.

   python
   ckpt_path = 'path/to/your/checkpoint'
   saver.restore(sess, ckpt_path)

6. Retrieve global_step: global_step is typically obtained or created using tf.train.get_or_create_global_step() during initialization. Once the model is restored, evaluate this variable to obtain the current step count.

   python
   global_step = tf.train.get_or_create_global_step()
   current_step = sess.run(global_step)
   print("Current global step is: {}".format(current_step))

By following these steps, you not only restore the model weights but also successfully retrieve the current global_step, enabling you to resume training from where it was previously stopped or perform other operations.

A concrete example might involve training a deep learning model for image classification, where you save models at each epoch and resume training from the last saved epoch when needed. Using global_step helps track the number of completed epochs.