What is a pipeline?

Here you will understand what a pipeline is and how it works.

Pipelines

Pipelines are the representation of a workflow composed of different nodes that help us define the logic or flow through which our process will pass. Each node represents a specific action, condition, or operation, allowing you to build complex workflows in an intuitive and modular fashion.

Typically, pipelines consist of a start, intermediate steps, and an end. However, it's important to note that this isn't strictly linear; a pipeline can have multiple triggers to initiate tasks and multiple possible endings based on the conditions and logic you define.

Pipeline example

An illustration of a pipeline with multiple triggers and endpoints.

Key Characteristics of Pipelines

  • Multiple Triggers: Pipelines can begin through various events, providing flexibility in how processes are initiated. Examples include:

    • File Creation Trigger: The pipeline starts when a new file is created in a specific location.
    • Webhook Call Trigger: Initiated by an external webhook call, allowing integration with other services and applications.
  • Intermediate Steps: These are the actions, conditions, automations, and integrations that occur between the start and end of the pipeline. They shape the workflow and determine the path based on specific criteria.

  • Multiple Endings: Pipelines can conclude in different ways depending on the workflow's outcome. Possible endings include:

    • Quality Approved End: The process ends with a task being approved after meeting certain quality standards.
    • Quality Rejected End: The task doesn't meet the required standards and is rejected, triggering alternative actions.
    • File Storage End: Concludes by storing a file in a designated repository or database.

Examples of What Can Be Achieved with Pipelines

  • Automated Approval Processes: Start the pipeline when a user submits a form. The data goes through validation steps, and if approved, it ends by sending a confirmation email and updating the database.

  • Content Management Workflows: Begin with the creation of a draft article. The pipeline moves the article through editing and review stages, ending with publication on the website or rejection with feedback.

  • Data Processing Tasks: Initiate when new data is received via an API webhook. Process the data through transformation steps and conclude by storing the processed data or triggering alerts if anomalies are detected.


By leveraging pipelines, you can design and implement complex workflows that mirror real-world business processes. The ability to have multiple triggers and endpoints allows for greater flexibility and efficiency, ensuring that your workflows are both robust and adaptable to various scenarios.