Creating Your First Project

Prerequisites:

Access to Bringits Stream UI
Organization dashboard access
Understanding of key concepts (projects, steps, commands, variables)

This guide walks you through creating your first project in Bringits Stream UI, from initial setup to running your first data extraction workflow.

Project Naming & Settings

Accessing Bringits Stream UI

Open Bringits Stream UI

Access the Bringits Stream UI from your organization’s dashboard or navigate directly using the provided Bringits Stream URL.

Navigate to Projects

From the main sidebar menu, select “Projects” to display existing projects and project groups.

Create New Project

Click “Create Project” located in the upper-right corner of the Projects page.

Project Configuration Fields

When creating a project, you’ll configure the following fields:

Field	Description	Required
Project Name	Choose a descriptive, unique name for your project	Yes
Description	Document the project’s purpose and use case	No
Tags/Grouping	Organize projects using optional tags	No
Browser Type	Select from available browsers for your workflow	Yes
Miss Tolerance	Set an integer value for error handling at the project level	Yes

Use descriptive project names that clearly indicate the project’s purpose. This makes it easier to find and manage projects later.

Variable Management

Bringits Stream supports predefined variables at the project level. For detailed information about variable types, see the Introduction to Projects. Project-level variables (@{PROJECT_ID}, @{RUN_ID}, @{START_TIMESTAMP}) are global variables accessible across all steps. Secrets (%{SINK_DB_PASSWORD}, %{PROXY_PASSWORD}) provide encrypted storage for API keys and credentials. Session tracking (#{UUID}) maintains state across multi-step flows.

Folder and Ownership Management

Projects can be organized using folders and tags for easier management. The UI displays projects grouped by ownership and organizational structure, allowing team members to access relevant projects based on permissions.

Folder organization helps manage large numbers of projects. Consider creating folders by team, use case, or project type.

Creating Your First Step

Step Configurations

After creating your project, you’ll add steps to define your workflow:

Add a Step

Within your project, click to add a new step. Each step represents a specific task in your data extraction workflow.

Configure Step Name

Provide a descriptive identifier for the step that clearly indicates its purpose in the workflow.

Set Cycle Configuration

Each cycle has a name for tracking. This helps you identify and monitor different execution cycles.

Configure Step-Specific Variables

Define local variables that override project-level variables if needed. These variables are scoped to this specific step.

Define Output Handling

Specify how results pass to subsequent steps or sinks.

Step Intervals Explained

Step interval configuration makes each step a repeatable job:

Intervals are per step - Each step runs independently on its configured schedule
After interpolation - The system processes variables before executing the step
Repeatable jobs - Steps automatically re-execute based on the interval
Last update tracking - Each item job tracks its execution time separately from the parent

When changing or deleting a project, all repeatable jobs for that project’s steps are stopped. Child steps that are removed have their repeatable jobs automatically stopped.

The interval ensures continuous data collection without manual intervention.

Variable Scope in Steps

Steps support various variable scopes:

${event_id} - Event-specific variables
#{UUID} - Session identifiers for maintaining state
#{DATE(TIMESTAMP)} - Dynamic date/time parameters
@{PROJECT_ID}, @{RUN_ID} - Runtime parameters inherited from project level

Adding Commands to Your Step

Command Configurations

Each command includes standard parameters:

Parameter	Description	Example
Command Type	Specifies the operation (e.g., Click, GetText, Parse)	`"command": "GetTextContents"`
Timeout	Maximum wait time in milliseconds	`"timeout": 10000` (10 seconds)
Element Locators	CSS selectors to target specific elements	`"selector": ".article-title"`

The UI provides info icons for each command that explain the command purpose, available parameters, usage examples, and validation requirements.

Advanced Command Features

Parameter Interpolation - Use variables like @{PROJECT_ID} in command configurations:

{
  "command": "http-fetch",
  "params": {
    "url": "https://api.example.com/data?project=@{PROJECT_ID}",
    "timeout": 10000
  }
}

Session Management - Use #{UUID} for persistent sessions across commands:

{
  "command": "browser-open",
  "params": {
    "sessionId": "#{UUID}",
    "url": "https://example.com"
  }
}

Secrets - Reference encrypted credentials with %{SECRET_NAME}:

{
  "command": "http-fetch",
  "params": {
    "url": "https://api.example.com/data",
    "headers": {
      "Authorization": "Bearer %{API_KEY}"
    }
  }
}

Example: HTTP Fetch Command

HTTP Fetch commands retrieve data from web APIs:

{
  "command": "http-fetch",
  "params": {
    "url": "https://example.com/api/data",
    "method": "GET",
    "timeout": 10000
  }
}

Example: Parse Commands

Bringits Stream includes several Parse commands for data extraction:

JSONPathCommand

Extracts data from JSON responses using JSONPath expressions:

{
  "command": "JSONPathCommand",
  "params": {
    "jsonPath": "$.data.items[*].title",
    "input": "@{PREVIOUS_OUTPUT}"
  }
}

GetTextContentsCommand

Retrieves text content from DOM elements:

{
  "command": "GetTextContents",
  "params": {
    "elementsLocator": {
      "selector": ".article-title",
      "timeout": 10000
    }
  }
}

Features:

CSS Selectors - Target elements using standard selectors
Filtering - Filter by innerText, hasChildren, or index
Chained Locators - Use continueWith for nested searches
Returns Array - Text content from all matching elements

GetAttributesCommand

Extracts attributes from DOM elements for detailed scraping operations:

{
  "command": "GetAttributes",
  "params": {
    "elementsLocator": {
      "selector": "a.article-link",
      "timeout": 10000
    },
    "attributes": ["href", "title"]
  }
}

Connecting a Sink

Supported Sink Types

The SINK command allows publishing messages directly from any step. Bringits Stream supports various output destinations: RabbitMQ (Message Queue):

Publish messages directly to a RabbitMQ exchange
No Redis storage required
Data sent as part of the message itself
Primary method for robot communication

Configuring a Sink

Add SINK Command

Place the SINK command where you want to publish a message. Typically, this is the final command in the last step of your workflow.

Configure Exchange and Routing Key

Set the exchange name and routing key for message routing:

{
  "command": "SINK",
  "params": {
    "exchange": "your-exchange-name",
    "routingKey": "your-routing-key"
  }
}

Verify Consumption

Run the robot and confirm messages are being consumed successfully from the message queue.

Key Differences

No Orchestrator Flow - Direct publishing bypasses the bringits-stream-orchestrator
No Redis Storage - All data included in the message
Immediate Publishing - Messages sent as soon as the step completes

Test your sink configuration with a simple message before running the full workflow to ensure proper routing and consumption.

Running Your Project

Turning On the Project

Save Configuration

Ensure all steps and commands are properly configured and saved.

Enable Project

Toggle the project status to “Active” or “On” to begin execution.

Monitor Execution

The system begins executing steps based on configured intervals. Monitor the execution logs to verify proper operation.

Execution Behavior:

Each step runs as a repeatable job based on its interval
Variable interpolation occurs before each execution
Parent-child relationships - Child steps inherit context from parent executions
Automatic scheduling - No manual triggering required once active

Testing and Debugging Your Project

For comprehensive testing and debugging guidance, see the Debugging Your Project guide.

The Bringits Stream ecosystem includes debugging tools and sandbox environments for testing projects before production deployment.

Next Steps

Now that you’ve created your first project, explore:

Debugging Your Project - Learn testing and troubleshooting techniques
Monitor Your Stream - Track project performance
Advanced command configurations
Multi-step workflows
Error handling and retry logic
Performance optimization

Quick Start

Commands Glossary

Creating Your First Project

Project Naming & Settings

Accessing Bringits Stream UI

Project Configuration Fields

Variable Management

Folder and Ownership Management

Creating Your First Step

Step Configurations

Step Intervals Explained

Variable Scope in Steps

Adding Commands to Your Step

Command Configurations

Advanced Command Features

Example: HTTP Fetch Command

Example: Parse Commands

JSONPathCommand

GetTextContentsCommand

GetAttributesCommand

Connecting a Sink

Supported Sink Types

Configuring a Sink

Key Differences

Running Your Project

Turning On the Project

Testing and Debugging Your Project

Next Steps

Quick Start

Commands Glossary

​Project Naming & Settings

​Accessing Bringits Stream UI

​Project Configuration Fields

​Variable Management

​Folder and Ownership Management

​Creating Your First Step

​Step Configurations

​Step Intervals Explained

​Variable Scope in Steps

​Adding Commands to Your Step

​Command Configurations

​Advanced Command Features

​Example: HTTP Fetch Command

​Example: Parse Commands

​JSONPathCommand

​GetTextContentsCommand

​GetAttributesCommand

​Connecting a Sink

​Supported Sink Types

​Configuring a Sink

​Key Differences

​Running Your Project

​Turning On the Project

​Testing and Debugging Your Project

​Next Steps

Project Naming & Settings

Accessing Bringits Stream UI

Project Configuration Fields

Variable Management

Folder and Ownership Management

Creating Your First Step

Step Configurations

Step Intervals Explained

Variable Scope in Steps

Adding Commands to Your Step

Command Configurations

Advanced Command Features

Example: HTTP Fetch Command

Example: Parse Commands

JSONPathCommand

GetTextContentsCommand

GetAttributesCommand

Connecting a Sink

Supported Sink Types

Configuring a Sink

Key Differences

Running Your Project

Turning On the Project

Testing and Debugging Your Project

Next Steps