Prefect with Jeremiah Lowin

00:00:00
/
00:48:59

October 21st, 2020

48 mins 59 secs

Your Host

About this Episode

Eric Anderson (@ericmander) and Jeremiah Lowin (@jlowin) discuss Prefect, a workflow management system and data orchestration tool under development as an open-source project. Jeremiah initially created Prefect to solve a technical challenge specific to his own work, but soon realized that it was appealing to a very wide range of different clients. Listen to today’s episode to learn why Jeremiah believes most attempts to build a unified framework for solving data orchestration fail.

In this episode we discuss:

  • Solving the “negative engineering problem”
  • Learning from the complaints of data engineers at Apache Airflow
  • The difficulty of having a product that serves two masters
  • How COVID changed the direction of Prefect

Links:

People mentioned: