All Machine Learning (ML) models being developed go through several steps in the ML lifecycle. Such steps include data versioning, pre-processing, validation, model training, analysis and model development. Often, these steps are repetitive. Therefore, it becomes critical for data scientists and ML engineers to save time by automating such repetitive tasks. As a result, MLOps platforms have emerged to take charge of this cause. Today, companies and individuals like you have several MLOps platforms to choose from, raising the need to find the best option.

While several considerations go into making a choice, we have decided to help by comparing these MLOps platforms comprehensively.

Kubeflow and SageMaker have emerged as the two most popular end-to-end MLOps platforms. Kubeflow is the first entrant on the open-source side, and SageMaker has a robust ecosystem through AWS. In this article, we will compare the differences and similarities between these two platforms.

Components of Kubeflow

Kubeflow is a free and open-source ML platform that allows you to use ML pipelines to orchestrate complicated workflows running on Kubernetes. This solution was based on Google's method of deploying TensorFlow models, that is, TensorFlow Extended. The logical components that makeup Kubeflow include the following:

• Kubeflow Pipelines: Empower you to build and deploy portable, scalable machine learning workflows based on Docker containers. It consists of a user interface to manage jobs, an engine to schedule multi-step ML workflows, an SDK to define and manipulate pipelines, and notebooks to interact with the system via SDK.
• KFServing: Enables serverless inferencing on Kubernetes. It also provides performant and high abstraction interfaces for ML frameworks like PyTorch, TensorFlow, scikit-learn, and XGBoost.
• Multi-tenancy: Simplifies user operations to allow users to view and edit only the Kubeflow components and model artifacts in their configuration. Key concepts under this Kubeflow's multi-user isolation include authentication, authorization, administrator, user and profile.
• Training Operators: Enables you to train ML models through operators. For instance, it provides Tensorflow training (TFJob) that runs TensorFlow model training on Kubernetes, PyTorchJob for Pytorch model training, etc.
• Notebooks: Kubeflow deployment provides services for managing and spawning Jupyter notebooks. Each Kubeflow deployment can include multiple notebook servers and each notebook server can include multiple notebooks.

Components of SageMaker

The built-in SageMaker Studio is a core part of the SageMaker experience.

SageMaker is a fully-managed service that provides data scientists and machine learning engineers with the ability and resources to seamlessly prepare, build, train, and deploy ML models. Amazon SageMaker components can be described under four major categories as follows:

• Collect and Prepare: Under this category, SageMaker Data Wrangler helps you to connect to data sources, prepare data and create model features. Amazon SageMaker Clarify allows you to improve model quality via bias detection during data preparation and after training. SageMaker Ground Truth console helps you to develop accurate training datasets for ML and use built-in data labeling workflows to label your data quickly. With SageMaker Feature Store, you can securely store, discover, and share ML serving features in real-time or batches. Finally, Amazon SageMaker Processing allows you to connect to the existing storage, run your job, and save the output to the persistent storage.
• Build: Here, SageMaker Studio Notebooks, which are one-click Jupyter notebooks, enable you to spin up or down any available resources. Amazon SageMaker JumpStart empowers you to get started with ML using pre-built solutions that can be easily deployed. Amazon SageMaker Autopilot automatically builds, trains, and tunes machine learning models for you.
• Train and Tune: Use Amazon SageMaker Experiments to track any iteration made to machine learning models by capturing the input parameters, configurations and results, and storing them as 'experiments'. Amazon SageMaker Debugger allows you to capture metrics and profiles training jobs in real-time. With SageMaker, you can automatically tune your model by altering several combinations of parameters to get the most accurate predictions.
• Deploy: SageMaker Pipelines make CI/CD easy as you can build fully automated workflows for your ML lifecycle. SageMaker Model Monitor automatically detects any concept drift in your deployed models and gives alerts to identify the problems as well as improve model quality. Amazon Augmented AI offers built-in human review workflows for the most common ML use cases.

Similarities between Kubeflow and SageMaker

The first similarity between Kubeflow and SageMaker is that they can be used to automate and manage ML workflows, and they both have tools for model exploration. However, there is more; this section will explore other major similarities between the two platforms.

1. Both platforms are fully-fledged MLOps platforms built for managing the entire machine learning lifecycle. They both have robust feature sets that include pipeline orchestration, storing metadata and model deployment.
2. Just like Kubeflow's modules and add-ons, SageMaker equally has different tools that vary in maturity levels. As a result, they cover a lot of use cases but may not always be presented in a cohesive user experience due to the modular nature.
3. Both platforms have support for the most common Python-based machine learning frameworks.
4. Both platforms are prevalent and come with plenty of integrations and guides.

Differences between Kubeflow and SageMaker

Kubeflow is the ML toolkit for Kubernetes, while SageMaker is a fully-managed service that offers an IDE for ML model deployment and workflow management. Based on this foundational difference, the core differences between Kubeflow and SageMaker include the following:

1. Kubeflow is free and open-source, and thus it may be more accessible and adaptable to fit your needs. On the other hand, SageMaker is not free. Although you can access some components like the SageMaker Studio for free, you will need to pay for the AWS services that you use within the studio.
2. Unlike Kubeflow, SageMaker is mostly built around its own IDE, which provides the tools you need. Therefore, if you have existing tools that you are familiar with, you may have to drop them while adopting SageMaker to get the entire user experience.
3. Amazon has been building plenty of new modules for SageMaker. And today, it covers more of the data engineering side than Kubeflow with the Feature Store and Data Wrangler parts.

Summary

Amazon SageMaker and Kubeflow allow data scientists and developers to prepare, build, train, and deploy quality ML models. While Amazon SageMaker offers you a fully-managed service, including a studio, to automate ML workflows, Kubeflow offers a complete toolkit to manage workflows and deploy ML models on Kubernetes.

If you are familiar with AWS and you don't mind paying some charges to automate your workflows, you can choose SageMaker. However, if you are highly comfortable with Kubernetes, choose Kubeflow; an additional benefit you get with Kubeflow is that it is free.

Valohai as an Alternative for Kubeflow and SageMaker

[CAUTION: Opinions ahead] Kubeflow and SageMaker are currently the most popular MLOps platforms in the market. However, both have distinct weaknesses that we think Valohai doesn't. Whether these weaknesses matter to you is, of course, dependent on your use case.

	Valohai	Kubeflow	SageMaker
Type	Managed	Open-source	Managed
Languages and frameworks	Any	Python and popular frameworks (Any with customization)	Python and popular frameworks
Infrastructure	AWS GCP Azure OpenStack On-premise	Any (with Kubernetes)	AWS
Features	Experiment tracking Model management Machine orchestration Pipeline automation	Experiment tracking Model management Machine orchestration Pipeline automation	Experiment tracking Model management Machine orchestration Pipeline automation Feature store
Setup time	Hours	Days to weeks	Hours
Maintenance	None	Required	None
Cost	Based on # of users	Based on time spent	Based on usage

Business model

Kubeflow is an open-source platform that is one of its strengths and one of its core weaknesses. Due to the complexity of the platform, it can be a pain to set up and manage without a dedicated team. Unfortunately, the free price tag comes at a cost.

SageMaker, on the other hand, is a managed platform, and if you are already in the AWS ecosystem, utilizing it is simple and free for the first two months. The SageMaker price tag increases with the services you use. Enias Cailliau from radix.ai points to a 40% cost increase compared to running purely on EC2 instances. This can make costs rise rather rapidly once you use SageMaker for production pipelines.

Valohai is also a managed platform with a 2-week free trial period. We'll set up the platform on your infrastructure during the trial and go through an onboarding session to get you started. Our pricing model is based on the amount users, so the cost doesn't scale with your utilization of the platform, saving you plenty in the long run. In addition, the computation you use will be billed by whatever cloud vendor you use and we never try to get a cut from that.

Technology agnosticism

Kubeflow and SageMaker both have some limitations regarding what you can run and where you can run them. For Kubeflow, the whole platform is built on top of Kubernetes which can be a pain for teams that aren't using it already and have no other need for it. Additionally, Kubeflow is quite limited to Python and ML libraries built on it. You can, of course, supplement this by writing your own training operators.

SageMaker shares this limitation with Kubeflow that the focus is on the popular Python libraries but with SageMaker, you can't supplement these capabilities. However, the most glaring weakness is that SageMaker is only available for AWS, which means you'll have no choice but to use their compute resources. For some organizations, this is simply a dealbreaker. For others, it can be just an expensive inconvenience as you cannot optimize for cost among the clouds and on-prem hardware.

Valohai, on the other hand, is built to be as technology agnostic as possible. So anything you can run in a Docker container on Linux, you can run on Valohai. On top of that, Valohai can orchestrate most of the popular clouds out there and most on-premise setups.

So if you're looking for an MLOps platform that is technology agnostic and has a transparent business model, Valohai should be on your list.

• Valohai Free Trial
• Valohai Product
• Valohai and Kubeflow Comparison
• MLOps Platform: Build vs Buy
• MLOps Platforms Compared

More Kubeflow comparisons

This article continues our series on common tools teams are comparing for various machine learning tasks. You can check out some of our previous Kubeflow comparison articles:

Other Kubeflow comparison articles
Kubeflow and MLflow
Kubeflow and Airflow
Kubeflow and Metaflow
Kubeflow and Databricks
Kubeflow and SageMaker
Kubeflow and Argo