Product

MLOps has become increasingly complicated in recent years, but in reality, ML teams have most of the infrastructure they need. Aqueduct helps untangle the knot by integrating with and empowering your cloud.

Vikram Sreekanti

March 2, 2023

Building the Aqueduct March Madness Workflow

While Airflow is the default orchestrator for machine learning today, it has some key shortcomings: it doesn't care about data, it's not built for fast iteration, and it exacerbates infrastructure complexity. In this post, we discuss how Aqueduct's Airflow integration enhances how Airflow works for machine learning.

March 24, 2023

Predicting March Madness with Aqueduct

It's March, and that means it's the NCAA tournament! See how you can build a workflow to predict the NCAA tournament with Aqueduct.

March 17, 2023

Understanding the MLOps Knot: Metadata Drift

MLOps has become increasingly complex as machine learning has matured. One of the key reasons is ML teams have to build, extend, and maintain far too much infrastructure. We dive into how and why this happens.

Vikram Sreekanti

February 23, 2023

Understanding the MLOps Knot: Infrastructure Proliferation

Vikram Sreekanti

February 16, 2023

Visualizing ML pipelines with React and elk

This tutorial walks you through how Aqueduct uses React Flow and elk to build interactive visualizations of machine learning pipelines.

Andre Giron

February 10, 2023

Unlock machine learning in your cloud: Aqueduct v0.2

Aqueduct v0.2 unlocks machine learning in your cloud. This release takes big steps forward on running ML workloads on existing cloud infrastructure and on providing visibility into what's running.

Vikram Sreekanti

February 1, 2023

The MLOps Knot

Machine learning has seen a rapid proliferation of powerful infrastructure. However, the adoption of all these tools has led to a tangled mess of infrastructure leaving most teams without a way to track the code, data, and metadata for their machine learning workloads.

Vikram Sreekanti

January 26, 2023

Managing Python Environments in Aqueduct

Managing Python environments is a critical task for workflow orchestration. When done naively, it can lead to incorrect execution or extremely high performance overheads. In this post, we describe how Aqueduct optimizes workflow execution within intelligently managed Conda environments.

Wei Chen

January 19, 2023

Enhancing Airflow for machine learning

Saurav Chhatrapati

January 12, 2023

Stop using Airflow for data science

Airflow has become the default data engineering orchestrator in recent years, and data science teams have inherited it. Unfortunately, Airflow wasn't designed for machine learning workflows — using Airflow for ML results in significant overhead and tech debt.

Vikram Sreekanti

January 5, 2023

Predictions are just data

In conversations with over 200 data teams, we've found that the "typical" solution to ML model deployments is wrong. Most people don't need REST endpoints — they just need predictions as data.

Vikram Sreekanti

October 27, 2022

Aqueduct v0.1: A simpler way to run data science workflows

We're excited to share Aqueduct v0.1. Aqueduct allows you to easily construct robust data & ML pipelines that work with your cloud infrastructure.

Vikram Sreekanti

October 18, 2022

Aqueduct: Taking Data Science to Production

As ML has become widely adopted, the next critical challenge for data teams is in generating value from data science & machine learning. Production data science infrastructure is the missing link that will enable data science and machine learning to succeed, by abstracting away low-level cloud infrastructure. Aqueduct is the world's first production data science platform; it enables data scientists to run models anywhere, publish predictions everywhere, and ensure prediction quality.

Vikram Sreekanti

August 17, 2022

MLOps: Right Problem, Wrong Solution

The fundamental problem with MLOps is that it mixes together tools for two very different concerns — (1) ensuring high-quality predictions and (2) deploying & managing cloud infrastructure. As a consequence, this requires data teams to have expertise in both data science and also in low-level cloud infrastructure.

Vikram Sreekanti

August 2, 2022

The Real Challenge in (Useful) Machine Learning isn’t Learning

This post discusses research from the UC Berkeley RISE Lab around building scalable prediction infrastructure, and why that wasn't the problem the world needed solved.