Introduction
In a world where complexity is often seen as a necessary evil, simplifying can be revolutionary. Durable workflows are crucial for the reliability of programs, but the question arises: why burden ourselves with external orchestrators when Postgres can handle it efficiently? In this article, we'll explore how Postgres can be used to manage durable workflows without the intervention of an external orchestrator.
Understanding Durable Workflows
Durable workflows allow for the state of a program to be saved at regular intervals in a database. This way, if a crash occurs, the program can be restored from its last checkpoint. It works like a save system in video games: frequent saves allow you to resume where you left off in case of a problem.
Traditionally, these workflows are managed by external orchestration systems like Temporal, Airflow, or AWS Step Functions. These systems create a record for each workflow in a data store and coordinate their execution via workers.
Why Postgres?
Postgres is a powerful relational database, known for its scalability and reliability. With its rich ecosystem of plugins and extensions, it stands as an ideal solution to orchestrate durable workflows. By using Postgres, the need for a separate orchestrator is eliminated, simplifying the architecture and reducing potential points of failure.
The Example of DBOS Transact
DBOS Transact is a concrete example of using Postgres for durable workflows. This open-source project uses Postgres to store the state of workflows, eliminating the need for an external orchestrator. According to DBOS data, this solution can handle up to 4 billion workflows per day, demonstrating the efficiency and scalability of this approach.
Advantages of Using Postgres
- Simplicity: Removing the orchestrator reduces system complexity.
- Cost: Fewer components mean lower infrastructure costs.
- Reliability: Postgres is a proven solution with robust backup and recovery mechanisms.
- Rich Ecosystem: With its numerous extensions, Postgres can be tailored to various specific needs.
Implementing Durable Workflows with Postgres
To implement this approach, one simply leverages Postgres' transaction and logging capabilities. Each step of a workflow is treated as a transaction, and the state is regularly updated in the database. In case of a problem, the most recent state can be retrieved thanks to transaction logging.
Conclusion
Using Postgres for durable workflows is not just about simplification. It is an approach that aligns with the KISS (Keep It Simple, Stupid) philosophy while offering scalability and reliability. If you're looking to streamline your processes while reducing costs, Postgres might just be the ideal solution.
Let's discuss your project in 15 minutes.