The Wide & Deep model is the model mentioned by Google in a paper published in 2016. In the paper, Google combined LR model with deep neural network as a recommendation for Google Play and achieved certain results. After this paper, Youtube, Meituan and other companies also made corresponding attempts and made their work public (see related links at the bottom of this article).

The official Wide & Deep model (WD model for short) tutorials are feature engineering using TensorFlow (TF) built-in functions, and the model is also packaged. In this article, WE will show you how to build a clean and customizable WD model using TF. For example, we will use sklearn, NUMpy, and PANDAS to do the feature engineering. After the WD model is well trained, we also need to quickly see the effect of model prediction, so in this paper, we use Docker to quickly deploy a TensorFlow model available for service, which can also provide service API.

Therefore, the content of this paper is as follows:

  • Use TF to build WD network structure
  • Use Docker to quickly deploy the model

The corresponding code address is github.com/edvardHua/A… Welcome to star

The structure of WD model implemented in this paper is shown in the figure below:





The network constructed in this paper

It is not difficult to see that the side of Wide model is actually an LR model, while the part of Deep model on the right is a neural network with three hidden layers. The number of neurons in these three hidden layers is 256-12-64 respectively. Finally, the results of Wide model and Deep model are added together and the prediction results are output through ReLu activation function. OK, take a look at the code for the Wide model section

Def wide_model(input_data): """ "LR :param input_data: :return: """ input_len = int(input_data.shape[1]) with tf.name_scope("wide"): # modify the way to initialize weights, The output layer has only one weights = tf.variable (tf.truncated_normal([input_len, 1], stddev=1.0 / math.sqrt(float(input_len))), Name ="weights") output = tf.matmul(input_data, weights) # Name ="reduce_sum") 0 output = tf. 0 (Output, [-1, 1]) return outputCopy the code

Now look at the code for the Deep model.

Def deep_model(input_data, hidden1_units, hidden2_units, hidden3_units): "" 2-D tensor :param hidden1_units: int :param hidden2_units: int :param hidden3_units: int :return: Shape = int(input_data.shape[1]) with tf.name_scope("hidden1"): Weights = tf.Variable(tf.truncated_normal([input_len, hidden1_units], Stddev =1.0 / math.sqrt(float(input_len))), name="weights1") biases = tf.variable (tf.zeros([hidden1_units]), Name ='biases1') hidden1 = tf.nn.relu(tf.matmul(input_data, weights)) + biases ·· With tf.name_scope("output"): Weights = tf.variable (tf.truncated_normal([hidden3_units, 1], stddev=1.0 / math.sqrt(float(input_len))), name="weights4" ) biases = tf.Variable(tf.zeros([1]), name='biases4') output = tf.nn.relu(tf.matmul(hidden3, weights) + biases) return tf.nn.relu(output)Copy the code

Although there is some redundancy in the Deep model code, it makes it easy to modify and adjust the structure of the network.

Finally, the results of Wide model and Deep model are added together and the predicted results are output by ReLu activation function.

Def build_wdl(deep_input, wide_input, y): """ def build_wdl(deep_input, wide_input, y): """ " """ central_bias = tf.Variable([np.random.randn()], name="central_bias") dmodel = deep_model(deep_input, 256, 128, Dmodel_weight = tf.variable (tf.truncated_normal([1, 1]), name="dmodel_weight") wmodel_weight = tf.Variable(tf.truncated_normal([1, 1]), name="wmodel_weight") network = tf.add( tf.matmul(dmodel, dmodel_weight), tf.matmul(wmodel, wmodel_weight) ) prediction = tf.nn.sigmoid(tf.add(network, central_bias), name="prediction") loss = tf.reduce_mean( tf.nn.sigmoid_cross_entropy_with_logits(labels=y, Logits =prediction) train_step = tf.train.AdamOptimizer(0.001). Minimize (loss) return train_step, loss, predictionCopy the code

After setting up the structure, we can generate some random data to test the Wide & Deep model. Here we randomly generate 1000 samples, each with a dimension of 10, as training samples. For the sake of simplicity, no verification samples are created. The training only iterates once, that is, the training sample is traversed only once. Here, the label value of each sample is 0 or 1, so the objective function is cross entropy, and the code is as follows:

Def build_and_saved_wdl(): """ Train and save the model :return: 00 x_deep_data = np.random. Rand (10000) X_deep_data = x_deep_data. 00 10) x_wide_data = np.random.rand(10000) x_wide_data = x_wide_data.reshape(-1, 10) x_deep = tf.placeholder(tf.float32, [None, 10]) x_wide = tf.placeholder(tf.float32, [None, 10]) y = tf.placeholder(tf.float32, [None, 0]) y_data = Np.array ([random.randint(0, 1) for I in range(1000)]) y_data = y_data. 0 Loss train_step, Loss, prediction = build_wDL (x_deep, x_wide, y) sess = tf.Session() sess.run(tf.global_variables_initializer()) sess.run(train_step, feed_dict={x_deep: x_deep_data, x_wide: x_wide_data, y: y_data})Copy the code

After the training, you need to save the model. To be used in TensorFlow Serving, you need to save the model using SavedModelBuilder as follows:

def build_and_saved_wdl(): ... # to save the trained model in the current folder builder. = tf saved_model. Builder. SavedModelBuilder (join (". / model_name ", MODEL_VERSION)) inputs = { "x_wide": tf.saved_model.utils.build_tensor_info(x_wide), "x_deep": tf.saved_model.utils.build_tensor_info(x_deep) } output = {"output": tf.saved_model.utils.build_tensor_info(prediction)} prediction_signature = tf.saved_model.signature_def_utils.build_signature_def( inputs=inputs, outputs=output, method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME ) builder.add_meta_graph_and_variables( sess, [tf.saved_model.tag_constants.SERVING], {tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: prediction_signature} ) builder.save()Copy the code

Note that MODEL_VERSION must be a number (representing the version of the model). TF Serving only loads the model with the largest number by default. For example, after executing the code here, the folder where model_name is saved is as follows:

12. λ root [tf_demo/servering/model_name] → tree. ├── ├─ variables ├─ data └─ variables. Index 2 directories, 3 filesCopy the code

After saving the model, here we use the container to deploy the model, of course you can also choose to configure the relevant environment on the machine, we use the image provided by Bitnami (Dockerhub address please click here), when you need to deploy the model, Simply map the path of the model to the /bitnami/model-data path in the container by typing the following command

λ edvard [tf_demo/servering/model_name] → docker run -it -p 55:9000 --volume /root/tf_demo/servering/model_name:/bitnami/model-data bitnami/tensorflow-serving Welcome to the Bitnami tensorflow-serving container ... 2017-11-01 03:43:55.983106: I tensorflow_serving/core/ loader_harps.cc :86] Successfully loaded servable version {name: Inception Version: 1} 2017-11-01 03:43:55.986130: I tensorFLOW_serving/model_Servers /main.cc:288] Running ModelServer at 0.0.0.0:9000...Copy the code

Here you may need some docker-related knowledge, I provide a very good introduction to Gitbook in Resources, you can take a look.

After mapping the services in the container to port 5000 on the host, let’s test the API interface. The code is as follows:

Def test_servable_api(): """ 00 x_deep_data = Np.random.rand (100). 0 (-1, 10) X_WIde_data = NP.random.rand (100). 0 = 10) channel implementations. Insecure_channel (127.0.0.1, Int (5000)) stub = prediction_service_pb2.beta_create_PredictionService_stub(channel) # predict_pb2.PredictRequest() request.model_spec.name = 'inception' request.model_spec.signature_name = tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY request.inputs[INPUT_WIDE_KEY].CopyFrom( tf.contrib.util.make_tensor_proto(x_wide_data, shape=[10, WIDE_DIM], dtype=tf.float32)) request.inputs[INPUT_DEEP_KEY].CopyFrom( tf.contrib.util.make_tensor_proto(x_deep_data, shape=[10, Outputs [OUTPUT_KEY] : outputs[OUTPUT_KEY] : outputs[OUTPUT_KEY] : outputs[OUTPUT_KEY]Copy the code

The structure of the output prediction results is as follows:

Dtype: DT_FLOAT tensor_shape {dim {size: 10} DIM {size: 1}} float_VAL: 0.355874538422 float_val: Float_val: 0.3225004673 float_val: 0.32104665041 Float_val: 0.233089879155 Float_val: 0.376621931791 float_val: 0.3225004673 float_val: 0.32104665041 float_val: 0.233089879155 0.144557282329 Float_VAL: 0.34686845541 Float_VAL: 0.304817527533 Float_VAL: 0.367866277695 Float_VAL: 0.393035560846Copy the code

The resources

Docker from The Beginning to the Practice (Gitbook)