Skip to main content

​
What is a Project

A project in Bringits Stream UI is a container for organizing and managing your data extraction workflows. Projects serve as the top-level organizational unit that groups together related steps, commands, and configurations. Projects enable you to:
  • Organize workflows by purpose or domain
  • Manage project-level settings and variables
  • Control execution and scheduling at a project level
  • Organize content using folders and tags
Projects are the foundation of your data extraction workflows. Each project can contain multiple steps, and each step contains a sequence of commands called a Blueprint.

​
Key Concepts

Understanding these core concepts will help you work effectively with Bringits Stream projects:

​
Projects

Projects are containers that hold your complete data extraction workflow. They include:
  • Project-level configuration (name, description, browser type, miss tolerance)
  • Project-level variables accessible across all steps
  • Organization through folders and tags
  • Execution control and scheduling

​
Steps

A step is a self-contained unit within a project that executes a sequence of commands (called a Blueprint). Each step:
  • Runs synchronously - commands execute one after another
  • Passes output from one command to the next
  • Can be configured with its own interval and variables
  • Represents a specific task in your data extraction workflow

​
Commands

Commands are the building blocks of a Blueprint. Each command:
  • Performs a single specific operation
  • Executes synchronously in sequence
  • Chains output to the next command
  • Supports various operations (HTTP requests, browser automation, DOM manipulation, data parsing)

​
Blueprints

A Blueprint is the sequence of commands within a step. Commands in a Blueprint execute sequentially, with each command’s output available to subsequent commands.

​
Sinks

Sinks are output destinations where your extracted data is published. Bringits Stream supports various sink types including:
  • Message queues (RabbitMQ)
  • Databases
  • APIs

​
Variable Types

Bringits Stream supports several types of variables that serve different purposes in your workflows:

​
Project-Level Variables

Project-level variables are global variables accessible across all steps in a project. They use the @{VARIABLE_NAME} syntax:
VariableDescription
@{PROJECT_ID}Unique identifier for the project
@{RUN_ID}Unique identifier for the current execution run
@{START_TIMESTAMP}Timestamp when the execution started

​
Secrets

Secrets are encrypted storage for sensitive information like API keys and credentials. They use the %{SECRET_NAME} syntax:
Secret ExampleDescription
%{SINK_DB_PASSWORD}Encrypted database password
%{PROXY_PASSWORD}Encrypted proxy authentication password
Secrets are stored securely and never exposed in logs or output. Use them for any sensitive credentials.

​
Session Tracking

Session variables maintain state across multi-step flows. They use the #{UUID} syntax:
VariableDescription
#{UUID}Session identifier that persists across commands and steps

​
Event-Specific Variables

Event-specific variables are scoped to individual events or executions. They use the ${variable_name} syntax:
VariableDescription
${event_id}Identifier for the current event being processed

​
Dynamic Parameters

Dynamic parameters generate values at runtime. They use the #{FUNCTION(PARAMS)} syntax:
ParameterDescription
#{DATE(TIMESTAMP)}Generates formatted date/time values based on timestamp
Variable interpolation occurs before command execution. Ensure variables are properly defined before use.

​
Variable Scope

Understanding variable scope helps you use variables effectively:
  • Project-level variables (@{...}) - Available to all steps in the project
  • Step-specific variables - Local variables that override project-level variables within a step
  • Session variables (#{...}) - Maintain state across multi-step flows
  • Event variables (${...}) - Scoped to individual event processing
  • Secrets (%{...}) - Encrypted credentials accessible project-wide

​
Project Organization

Projects can be organized using folders and tags for easier management:

​
Folders

Folders help you group related projects together. The UI displays projects grouped by folder structure, making it easier to navigate large numbers of projects.

​
Tags

Tags provide additional categorization and filtering options. You can assign multiple tags to a project to organize them by:
  • Purpose or use case
  • Team or department
  • Status or priority
  • Technology stack

​
Ownership

Projects display ownership and organizational structure, allowing team members to access relevant projects based on permissions. This helps maintain proper access control and project organization.

​
How Components Work Together

Understanding the relationship between projects, steps, commands, and sinks: 
Project
  ├── Step 1 (with interval)
  │   ├── Command 1 (HTTP Fetch)
  │   ├── Command 2 (Parse)
  │   └── Command 3 (Transform)
  ├── Step 2 (with interval)
  │   ├── Command 1 (Browser Open)
  │   ├── Command 2 (Click)
  │   └── Command 3 (GetText)
  └── Sink (RabbitMQ)
      └── Publishes final output
Each step runs independently based on its configured interval. Steps can pass data to subsequent steps or directly to sinks.

​
Next Steps

Now that you understand the key concepts, you’re ready to create your first project.