Skip to main content
Version: 2.0.0 (Latest)

Advanced

Management → Advanced is the component runtime lifecycle surface for Tasks, Sources, and Sinks on the currently selected Core. Operators start and stop workload execution, inspect grid state, and drive execution control for deployed components—not registry authoring or pipeline-wide graphs.

Architecture context

LayerRole
UISends runtime lifecycle commands and renders operational state returned from the API.
CoreExecutes connector and task workloads after activation; reports runtime health and errors.

Runtime scope

  • One Core at a time — tabs and row actions apply to the selected Core instance in the UI header.
  • Execution state is Core-specific — the same component name can show different runtime operations results on another engine; switch Core to compare.
  • Grids reuse familiar Configuration list patterns (search, filters, import/export where offered), but the contract here is runtime management, not defining new objects from scratch.

Operational responsibility

SurfaceResponsibility
Tasks, Sources, Sinks, PipelinesDefinitions — what exists in the registry and how pipelines are wired.
Management → Cores, Management → PipelinesDeployment coordination — assignment, Deploy All, deployed topology, and component deploy parity.
This page (Advanced)Execution stateworkload activation (start / stop) for deployed tasks and connectors on the bound Core.
Control TowerLive pipeline graph, per-stage runtime health, and graph-level controls for attached pipelines.

Working with Tasks

  • Tasks run PDL over stream data and publish results to configured outputs.
  • A task row assumes the definition is already deployed to the Core; start / stop only change runtime execution.
  • If a task stays inactive or errors, triage upstream streams, Pipelines deployment, and Cores placement—not only this grid.
  • Execution control for the row applies only to the selected Core.

Working with Sources

  • Sources drive ingest runtime: connectors read external systems and publish events into streams.
  • Workload activation assumes deployment succeeded and stream bindings exist; starting a source does not fix missing topology.
  • Runtime health often depends on upstream reachability (network, credentials, endpoint rate limits)—use logs and metrics when the row shows running but volume is wrong.
  • Actions are scoped to the selected Core like other tabs on this page.

Working with Sinks

  • Sinks control outbound delivery: connectors consume from streams and write to destinations.
  • Deployment state and execution state are independent — a sink can be deployed but stopped, or running yet failing delivery.
  • Delivery failures (backpressure, auth, destination outages) may surface as errors or lag even when deployment is healthy; correlate with streams, tasks, and external systems.

Runtime lifecycle

  • Deploying a component or pipeline does not automatically start execution.
  • Start and stop can be applied independently of fresh registry edits until you redeploy—expect stale behavior if definitions drift.
  • Deployed components may remain inactive indefinitely (stopped) while still present on the Core.
  • All runtime lifecycle outcomes here are evaluated per Core; there is no cross-Core bulk execution control on this screen.

Deployment prerequisites

Before start succeeds in practice, typically:

  • The component is assigned and deployed via Cores (and related flows).
  • Pipeline relationships and stream wiring exist—see Pipelines and Streams.
  • Advanced does not create deployment topology or registry rows; it only issues runtime commands against what is already on the engine.

Cross-page workflow

Placement and needs deploy start on Cores; pipeline synchronization uses Pipelines; graph-level runtime work continues on Control Tower.

  • Tasks — task definitions and modes (processing vs detection)
  • Sources — source connector definitions and parameters
  • Sinks — sink connector definitions and parameters
  • Pipelines — registry pipeline topology before assign/deploy
  • Streamsstream durability and buffering for ingest/egress
  • Management → Cores — component assignment and deploy to engines
  • Management → PipelinesDeploy All and deployed topology across Cores
  • Control Towerlive execution graph and pipeline-level runtime operations