Skip to main content
Validation commands check that the data flowing through your scraper pipeline meets expected conditions. When a validation fails, the command throws an error and the rest of the step’s Blueprint does not run. The step run then counts as invalid in the step’s statistics, and no results from that run reach child steps or sinks.
A rising invalid count on a step after you add a validation usually means the guard is catching bad responses. Check the step statistics and the test panel output to see which value failed.

Overview

Validation commands provide functionality for:
  • Checking that required strings are present in the data
  • Checking that forbidden strings are absent from the data
  • Combining multiple checks with OR or AND logic
  • Validating the payload against a named tenant JSON Schema

Available Commands

Command Validation

Validate whether specific strings are present in or absent from data

Schema Validation

Validate the previous command payload against a named JSON Schema

Common Use Cases

  • Response Guard: Confirm a success token exists before parsing an HTTP response
  • Error Detection: Stop the step early, and mark the run invalid, if an error string is detected
  • Completeness Check: Ensure all required field names are present in the raw payload
  • Contract Guard: Ensure the transformed payload matches an agreed JSON Schema before a Sink