[모델 배포하기(2/2)] TF-Serving 예제

출처 : http://solarisailab.com/archives/2703

36. 텐서플로우 서빙(TensorFlow Serving)을 이용한 딥러닝(Deep Learning) 모델 추론을 위한 REST API 서버 구

docker pull tensorflow/serving

docker run -t --rm -p 8501:8501 \ -v "/home/solaris/Desktop/tf_serving/saved_model:/models/fashion_model" \ -e MODEL_NAME=fashion_model \ tensorflow/serving &

-p : 서버에서 데이터를 주고받는데 사용할 포트(Port) 번호를 지정한다. (위 예시의 경우, 8501 포트)
-v :  불러올 모델이 SavedModel 포맷으로 저장된 전체 경로(full path)를 의미한다. (위 예시의 경우, home/solaris/Desktop/tf_serving/saved_model 경로에서 모델 파일을 불러온다.), : 뒤에는 모델을 실행할 REST API URL을 의미한다. (위 예시의 경우, models/fashion_model)

POST Request를 통한 이미지 데이터 전송 및 예측 결과 시각화

지정한 URL(models/fashion_model)로 인풋 데이터를 전송한 후, 해당 인풋 데이터에 대한 예측 결과값을 반환받을 수 있다.

http://localhost:8501/v1/models/fashion_model:predict

만약 버전에 대한 정보까지 지정해서 요청하고자 할 경우, 요청 URL은 아래와 같다.

http://localhost:8501/v1/models/fashion_model/versions/1:predict

model.py

# TensorFlow Serving Example (Simple REST API using Fashion MNIST)
# Reference : https://www.tensorflow.org/tfx/tutorials/serving/rest_simple

import tensorflow as tf
from tensorflow import keras
import os

def make_directory(target_path):
  if not os.path.exists(target_path):
    os.mkdir(target_path)
    print('Directory ', target_path, ' Created ')
  else:
    print('Directory ', target_path, ' already exists')

print('TensorFlow version: {}'.format(tf.__version__))

fashion_mnist = keras.datasets.fashion_mnist
(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()

# scale the values to 0.0 to 1.0
train_images = train_images / 255.0
test_images = test_images / 255.0

# reshape for feeding into the model
train_images = train_images.reshape(train_images.shape[0], 28, 28, 1)
test_images = test_images.reshape(test_images.shape[0], 28, 28, 1)

class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

print('\ntrain_images.shape: {}, of {}'.format(train_images.shape, train_images.dtype))
print('test_images.shape: {}, of {}'.format(test_images.shape, test_images.dtype))

model = keras.Sequential([
  keras.layers.Conv2D(input_shape=(28,28,1), filters=8, kernel_size=3, 
                      strides=2, activation='relu', name='Conv1'),
  keras.layers.Flatten(),
  keras.layers.Dense(10, activation=tf.nn.softmax, name='Softmax')
])
model.summary()

epochs = 5

model.compile(optimizer='adam', 
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])
model.fit(train_images, train_labels, epochs=epochs)

test_loss, test_acc = model.evaluate(test_images, test_labels)
print('\nTest accuracy: {}'.format(test_acc))

# Fetch the Keras session and save the model
# The signature definition is defined by the input and output tensors,
# and stored with the default serving key
SAVED_MODEL_PATH = './saved_model'
make_directory(SAVED_MODEL_PATH)
MODEL_DIR = SAVED_MODEL_PATH

version = 1
export_path = os.path.join(MODEL_DIR, str(version))
print('export_path = {}\n'.format(export_path))

tf.keras.models.save_model(
  model,
  export_path,
  overwrite=True,
  include_optimizer=True,
  save_format=None,
  signatures=None,
  options=None
)
print('\nSaved model:')

Client.py 

from tensorflow import keras
import matplotlib.pyplot as plt
import numpy as np
import random
import json
import requests

def show(idx, title):
  plt.figure(figsize=(12, 3))
  plt.imshow(test_images[idx].reshape(28,28))
  plt.axis('off')
  plt.title('\n\n{}'.format(title), fontdict={'size': 16})
  plt.show()

fashion_mnist = keras.datasets.fashion_mnist
(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()

# scale the values to 0.0 to 1.0
test_images = test_images / 255.0

# reshape for feeding into the model
test_images = test_images.reshape(test_images.shape[0], 28, 28, 1)

class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

print('test_images.shape: {}, of {}'.format(test_images.shape, test_images.dtype))

rando = random.randint(0,len(test_images)-1)
show(rando, 'An Example Image: {}'.format(class_names[test_labels[rando]]))

data = json.dumps({"signature_name": "serving_default", "instances": test_images[0:3].tolist()})
print('Data: {} ... {}'.format(data[:50], data[len(data)-52:]))


# send data using POST request and receive prediction result
headers = {"content-type": "application/json"}
json_response = requests.post('http://localhost:8501/v1/models/fashion_model:predict', data=data, headers=headers)
predictions = json.loads(json_response.text)['predictions']
# show first prediction result
show(0, 'The model thought this was a {} (class {}), and it was actually a {} (class {})'.format(
  class_names[np.argmax(predictions[0])], np.argmax(predictions[0]), class_names[test_labels[0]], test_labels[0]))

# set model version and send data using POST request and receive prediction result
json_response = requests.post('http://localhost:8501/v1/models/fashion_model/versions/1:predict', data=data, headers=headers)
predictions = json.loads(json_response.text)['predictions']
# show all prediction result
for i in range(0,3):
  show(i, 'The model thought this was a {} (class {}), and it was actually a {} (class {})'.format(
    class_names[np.argmax(predictions[i])], np.argmax(predictions[i]), class_names[test_labels[i]], test_labels[i]))