Introduction to the

TensorFlow models are available in a variety of formats and can be used for different scenarios. Any model that meets the specification can be easily deployed to online services or mobile devices. Here are a few examples.

  • Checkpoint: saves the weight of the model, which is mainly used for backup of parameters during model training and hot start of model training.
  • GraphDef: The Graph that holds the model, excluding the weight of the model, and checkpoint to get all the information about the model going online.
  • ExportModel: A model file exported using the Exportor interface, containing the model Graph and weights, can be used directly for online, but has been officially marked deprecated and is recommended to use SavedModel.
  • SavedModel: The model file exported using the saved_model interface, containing the model Graph and permissions, can be used directly for on-line. This model format is recommended for TensorFlow and Keras models.
  • FrozenGraph: Checkpoint and GraphDef are integrated and optimized using freeze_graph.py, and can be deployed directly on Android, iOS and other mobile devices.
  • TFLite: Optimized model based on Flatbuf, can be directly deployed to Android, iOS and other mobile devices, using interface and FrozenGraph slightly different.

Model format

It is recommended that both the TensorFlow and Keras models be exported to SavedModel format so that the general TensorFlow Serving service can be used directly and the model export is on-line without any code changes. The key of the string can be named arbitrarily and will only be used when the client requests it, as shown in the following code example.

Note that models currently using tf.py_func() cannot be exported directly and all structure recommendations for models are implemented using OP.

TensorFlow model export

import os
import tensorflow as tf
from tensorflow.python.saved_model import builder as saved_model_builder
from tensorflow.python.saved_model import (
    signature_constants, signature_def_utils, tag_constants, utils)
from tensorflow.python.util import compat

model_path = "model"
model_version = 1
model_signature = signature_def_utils.build_signature_def(
    inputs={
        "keys": utils.build_tensor_info(keys_placeholder),
        "features": utils.build_tensor_info(inference_features)
    },
    outputs={
        "keys": utils.build_tensor_info(keys_identity),
        "prediction": utils.build_tensor_info(inference_op),
        "softmax": utils.build_tensor_info(inference_softmax),
    },
    method_name=signature_constants.PREDICT_METHOD_NAME)
export_path = os.path.join(compat.as_bytes(model_path), compat.as_bytes(str(model_version)))
legacy_init_op = tf.group(tf.tables_initializer(), name='legacy_init_op')
 
builder = saved_model_builder.SavedModelBuilder(export_path)
builder.add_meta_graph_and_variables(
    sess, [tag_constants.SERVING],
    clear_devices=True,
    signature_def_map={
        signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
        model_signature,
    },
    legacy_init_op=legacy_init_op)
 
builder.save() 
Copy the code

Keras model export

import os
import tensorflow as tf
from tensorflow.python.util import compat
 
def export_savedmodel(model):
  model_path = "model"
  model_version = 1
  model_signature = tf.saved_model.signature_def_utils.predict_signature_def(
      inputs={'input': model.input}, outputs={'output': model.output})
  export_path = os.path.join(compat.as_bytes(model_path), compat.as_bytes(str(model_version)))
 
  builder = tf.saved_model.builder.SavedModelBuilder(export_path)
  builder.add_meta_graph_and_variables(
      sess=K.get_session(),
      tags=[tf.saved_model.tag_constants.SERVING],
      clear_devices=True,
      signature_def_map={
          tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
          model_signature
      })
  builder.save()
Copy the code

SavedModel Model structure

After exporting the SavedModel model using the TensorFlow API, you can examine the directory structure of the model as follows, and then load the service directly using the open source tool.


Model online

Deploying online Services

Using the HTTP interface see tobegit3hub/ simple_tensorFLOW_SERVING.

Using the gRPC interface refer to tensorflow/ Serving.

Deploying offline Devices

For deployment to Android, see medium.com/@tobe_ml/al… .

Deployed to the iOS zhuanlan.zhihu.com/p/33715219 for reference.