How to predict from a SavedModel with TensorFlow？

When using TensorFlow to perform predictions from a SavedModel, the process can be broken down into several steps. I will now provide a detailed explanation of each step, along with a simple example to illustrate the process.

Step 1: Load SavedModel

First, use TensorFlow's tf.saved_model.load function to load the saved model. This step reads the model's architecture and trained parameters.

python
import tensorflow as tf

# Assume the model is saved at saved_model_path
saved_model_path = '/path/to/saved_model'
loaded_model = tf.saved_model.load(saved_model_path)

Step 2: Access the Model's Prediction Function

After loading the model, access its provided functions via the signatures attribute. Typically, the function used for prediction is 'serving_default', which is the default signature set during model export.

python
infer = loaded_model.signatures['serving_default']

Step 3: Prepare Input Data

Before prediction, prepare the input data. Ensure its format and data type match those used during model training. Assuming our model expects a tensor of floating-point numbers.

python
import numpy as np

# Create an example input data
input_data = np.array([[1.0, 2.0, 3.0]], dtype=np.float32)

Step 4: Execute Prediction

Now, use the loaded prediction function infer to perform prediction. Input data must be provided as TensorFlow tensors to this function.

python
# Convert the numpy array to a tensor using TensorFlow
input_tensor = tf.convert_to_tensor(input_data)

# Perform prediction
output = infer(input_tensor)

Step 5: Process Output Results

The prediction function returns a dictionary containing the output results. Extract the required prediction results from this dictionary.

python
predicted_result = output['output_0'].numpy()
print("Predicted result:", predicted_result)

Example Summary

This example demonstrates how to load a model from a SavedModel, prepare input data, execute prediction, and process output results. This process applies to various TensorFlow models as long as you know the input and output formats.