Skip to main content
Sink commands enable you to send processed data to various output destinations including message queues, databases, and APIs. These commands are typically used at the end of your workflow to publish extracted data.

Overview

Sink commands provide functionality for:
  • Publishing messages to message queues (RabbitMQ)
  • Sending data to databases
  • Outputting data to APIs and webhooks
  • Configuring exchange and routing keys
  • Managing data output formats

Available Commands

Common Use Cases

  • Message Queue Publishing: Publish messages directly to RabbitMQ exchanges
  • Data Pipeline Integration: Send data to downstream processing systems
  • Robot Communication: Primary method for robot communication
  • Event Streaming: Stream events to message brokers
  • Data Archival: Send processed data to storage systems

Key Features

Direct Publishing

Sink commands enable direct publishing without requiring orchestrator flow:
  • No Orchestrator Flow: Direct publishing bypasses the bringits-stream-orchestrator
  • No Redis Storage: All data included in the message itself
  • Immediate Publishing: Messages sent as soon as the step completes

Message Configuration

Configure message routing and delivery:
ParameterTypeDescriptionRequired
exchangestringRabbitMQ exchange nameYes
routingKeystringRouting key for message routingYes
dataobjectData payload to sendYes

Command Parameters

Sink commands typically support the following parameters:
ParameterTypeDescriptionRequired
exchangestringExchange name for message routingYes
routingKeystringRouting key for message routingYes
dataobjectData payload to publishYes

Variable Support

Sink commands support variable interpolation in exchange names, routing keys, and data payloads:
{
  "command": "sink",
  "params": {
    "exchange": "your-exchange-name",
    "routingKey": "your-routing-key",
    "data": {
      "projectId": "@{PROJECT_ID}",
      "runId": "@{RUN_ID}",
      "timestamp": "@{START_TIMESTAMP}"
    }
  }
}

Best Practices

  • Place at End of Workflow: Typically use sink commands as the final command in your last step
  • Test Configuration: Test sink configuration with simple messages before full workflow
  • Use Descriptive Routing Keys: Use clear routing keys for message organization
  • Include Metadata: Include project and run identifiers in message payloads
  • Verify Consumption: Confirm messages are being consumed successfully

Configuration Example

{
  "command": "SINK",
  "params": {
    "exchange": "your-exchange-name",
    "routingKey": "your-routing-key"
  }
}

Troubleshooting

Messages Not Being Consumed

If messages aren’t being consumed:
  1. Verify Exchange Exists: Ensure the exchange is configured in RabbitMQ
  2. Check Routing Key: Verify routing key matches consumer configuration
  3. Test Connection: Ensure RabbitMQ connection is working
  4. Review Message Format: Verify message format matches consumer expectations

Common Issues

  • Exchange Not Found: Verify exchange name is correct and exists
  • Routing Key Mismatch: Ensure routing key matches consumer bindings
  • Connection Issues: Check RabbitMQ connection and credentials
  • Message Format Errors: Verify data payload format is correct