Vaclav Kosar's face photo
Vaclav Kosar
Software And Machine Learning Blog

BentoML vs Cortex - ML Serving Showdown

To find the best model serving tool, compare open-source MLOps platforms BentoML and Cortex.

BentoML and Cortex logo

Updated 2021-02-21. Note, that I have mainly experience with Cortex and BentoML view is based on cursory overview of their documentation.

Common

  • both deploy and serve models via API
  • both support major ML frameworks (TensorFlow, Pytorch)
  • both have good documentation

Comparison

Feature BentoML.org Cortex.dev
Language fully Python - easier to modify? Go & Python wrapper (Updated)
Deployment Delegated to other tools: Docker, Kubernetes, AWS Lambda, SageMaker, GCP, Azule ML and more. Works currently only with local Docker and AWS EKS, GCP (Updated)
Service configuration from Python from Python (Updated)
Service packaging and distribution Can be packaged, saved via Python command to management repository with a web dashboard or PyPI Packaging only via Docker images without explicit support
Horizontal scaling Configured separately in other clustering tools. Working on an opinionated Kubernetes deployment. Configurable in Cortex. May be less flexible (private cloud deploy may require custom scripts)
User interface CLI, Web UI CLI
Metrics Prometheus metrics Prometheus metrics (Updated)
API Auto-Docs Swagger/OpenAPI N/A
User support Responsive unpaid Slack Channel, but Slack is not the best tool for support Very responsive Gitter and now Slack
Suggest anything else?

My Experience with Cortex

Here is a blog post on Cortex use at GLAMI. It is a bit outdated take as Cortex now has its own wrapper. Consider using this Cortex client for Python, which is a Python wrapper around Cortex CLI that we use at GLAMI for MLOps. It has couple of extra features, that keep us using it for now.

Discussions

11 May 2020