What is the AIIA blueprint and how does Valohai fit into it?
by Henrik Skogström | on August 24, 2021The MLOps field is still an emerging one. While a few companies have a long history of machine learning in internal and production use cases; Most are playing catch-up with best practices and tooling. This hasn't gone unnoticed, and many vendors and open-source initiatives have emerged to fulfill these needs.
The tooling space has evolved rather quickly and organically, with most authors drawing from their own experiences. Valohai has a similar story as one of our founders was building machine learning infrastructure at a company pioneering gesture recognition way before the term MLOps was coined. While it's an exciting time in the space, the lack of unified terminology and vision makes it rather challenging to navigate as a user.
For this reason, AIIA has pushed to create a blueprint that would establish terminology and structure to the tooling space. The initiative draws inspiration from other popularized tech stacks, such as the LAMP stack.
What is the AI Infrastructure Alliance?
The rebel alliance for AI interoperability, as VentureBeat put it, the AI Infrastructure Alliance is an organization driving to build the canonical stack of machine learning. Its members include AI infrastructure vendors, solutions integrators, and other partners.
What are the AIIA blueprints and why are they important?
The main goal of the AIIA blueprint is to create a shared vocabulary among the multitude of vendors and users to drive interoperability. Ideally, it'd be easy for anyone developing machine learning models to pick and choose a stack of tools that suit their specific needs.
As we've seen, the requirements for different types of machine learning applications can vary -- a lot. For example, structured and unstructured data have different data backends, real-time and edge inference have different deployment solutions, and so forth.
Another critical aspect of AIIA and the blueprints is to spark collaboration between the solution vendors (like we've done with Superb AI) and drive towards open platforms that can be integrated.
How does Valohai fit into the AIIA blueprint?
The AIIA blueprint consists of two parts, the Stack diagram and the Workflow diagram.
The Stack diagram focuses on feature sets, and it highlights Valohai's positioning well. The Valohai platform covers the data science areas, while data engineering is handled primarily outside of the platform. Our principle has been to remain unopinionated towards how the data is handled. We want to support integrations with any established data platforms (and support both structured and unstructured data).
The Workflow diagram shows the complete workflow between raw data and a deployed model. Similar to the Stack diagram, it highlights our focus on the data science pipeline. This is not to say that you couldn't run data transformation or even ingestion in Valohai (because you can run almost anything on Valohai), but those aspects require more custom code on the users' part.
Conclusion
The AIIA blueprint is an excellent starting resource for teams looking to implement their stack for machine learning development. Whether you want to mix and match the best-in-breed solutions or cover as much ground as possible with a single platform, mapping these capabilities is immensely helpful.
Check out the AIIA blueprints on GitHub: github.com/ai-infrastructure-alliance/blueprints/ Check out the AI Infrastructure Alliance: ai-infrastructure.org