R Serving with FastAPI

This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.

Dockerfile

• The dockerfile defines the environment in which our server will be executed.
• Below, you can see that the entrypoint for our container will be deploy.R

Code: deploy.R

deploy.R handles the following steps

• Loads the R libraries used by the server.
• Loads a pretrained xgboost model that has been trained on the classical Iris dataset.
  • Dua, D. and Graff, C. (2019). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.
• Defines an inference function that takes a matrix of iris features and returns predictions for those iris examples.
• Wraps the inference function to make it thread-safe for passing to python through reticulate.
• Finally, it generates the endpoints.py from python and launches the FastAPI server app using those endpoint definitions.

Code: endpoints.py

endpoints.py defines two routes:

• /ping returns a status of 'Alive' to indicate that the application is healthy
• /invocations applies the previously defined inference function to the input features from the request body

Note, that FastAPI is typed. The Example class define the type of the input that we expect to receive from the request.

For more information about the requirements for building your own inference container, see: Use Your Own Inference Code with Hosting Services

Build the Serving Image

Launch the Serving Container

Define Simple Python Client

Define Example Inputs

We define example inputs from the Iris dataset.

Plumber

Stop All Serving Containers

Finally, we will shut down the serving container we launched for the test.

Notebook CI Test Results

This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.