Scanner Log Format Migration Guide for Real-Time Log Streaming

Note: Contact your Customer Success Manager or Mend Support to enable Log Streaming.

Overview

Enabling the Mend SCA Orchestrator (MEND_SCA_ORCHESTRATOR_ENABLED=true) and turning on log streaming changes both the structure and content of your logs.

This page explains:

What new log messages and states you’ll see
How the log line format and metadata change
How Unified Agent (UA) output behaves when streaming is enabled
Concrete before/after examples you can compare with your own logs

The goal is to help you quickly recognize Orchestrator-driven behavior, debug issues faster, and integrate these logs into your existing observability stack.

Note: Sending logs to the console on self-hosted repository integrations can be activated using EXTERNAL_LOG_IN_CONSOLE=true

1. New Orchestrator Log Messages and States

With the Mend SCA Orchestrator enabled, your logs no longer show only raw engine execution. They also reflect the Orchestrator’s lifecycle and decisions.

1.1 High-Level Orchestrator Messages

When the Orchestrator starts managing an SCA scan, you’ll see a clear, high-level message such as:

Starting SCA scan...

This makes it easier to identify the start of an SCA run in CI/CD logs or shared logging systems.

1.2 Orchestrator “state” Indicators

Standard scanner logs typically expose low-level engine commands without orchestration context. With the Orchestrator active, log lines include state metadata that indicate which phase the Orchestrator is currently in.

Examples of states:

state=READING_ENVIRONMENT_CONFIG
state=ROUTER_CONFIG
state=ROUTER_DECISION_MAKING
state=ENGINE_RUN

These states help you understand:

Where you are in the orchestration flow
Whether issues relate to configuration, routing, or engine execution
How long each phase takes (when combined with timestamps from your logging system)

2. Log Line Structure and Metadata Changes

When the Orchestrator is enabled, individual log lines become more structured and machine-friendly.

2.1 Standard Logs (Without Orchestrator)

In a non-orchestrated setup, log lines usually follow a simple, free-form pattern, such as:

TEXT

[Date/Time] [Log Level] [Package] - [Context Tags] Message

This is human-readable, but harder to parse reliably across tools.

2.2 Orchestrator Logs (With Orchestrator Enabled)

With the Orchestrator, log entries are enriched with key-value style metadata embedded in the message body. This makes it easier to:

Filter and query logs in Kibana, Datadog, Splunk, etc.
Correlate events across scans, agents, and engines
Build dashboards or alerts around specific states or components

Examples of new metadata keys:

agent=sca
agent-version
state
command
engine

In addition, context tags that previously appeared in a less structured way—such as CTX, SCAN_CTX, and SCAN_ID—are often consolidated at the beginning of these metadata blocks. This helps you:

Quickly isolate a specific scan (SCAN_ID)
Group related log lines by context (CTX, SCAN_CTX)
Use structured filters rather than free-text search

3. Log Streaming: Line-by-Line Unified Agent (UA) Output

The most noticeable operational change for users is how Unified Agent output behaves once log streaming is enabled.

3.1 Behavior Without Streaming

When log streaming is disabled:

The Orchestrator captures UA output during the run.
The captured output is printed only at the end of the scan.
In your logs, this typically appears as a single, large block of text (a “continuous sentence”) under one timestamp.

Implications:

You don’t see real-time progress from the UA.
Errors or warnings appear only after the entire scan completes.
Parsing line-by-line events is harder because they’re grouped into a single chunk of output.

3.2 Behavior with Streaming Enabled

When log streaming is enabled, the Orchestrator streams all engine printouts line by line in real-time.

Key changes:

Per-Line Timestamping

Each line of UA output is emitted as a separate log entry with its own scanner timestamp. This allows you to:

Monitor scan progress in real time
Correlate specific UA events with other system events
Identify delays or bottlenecks at a finer granularity

Engine Line Tagging

To distinguish engine output from Orchestrator messages, streamed lines from the Unified Agent are explicitly tagged, for example:

[Unified-Agent] – regular UA output
[Unified-Agent:err] – error output from the UA

This makes it easy to:

Filter engine logs vs. Orchestrator logs in your logging platform
Build alerts specifically on UA errors ([Unified-Agent:err])
Separate high-level orchestration flow from low-level engine details

Real-Time Error Visibility

With streaming enabled, errors produced by the engine are visible in real time, instead of only at the end of the run. This is especially useful in:

CI/CD pipelines, where you want to fail fast and see failures immediately
Long-running scans, where early warnings or configuration issues should be surfaced as soon as they appear
Interactive troubleshooting, where you tail logs while debugging environment or configuration problems

Examples: Before and After

The following examples illustrate how log output changes when you enable the Mend SCA Orchestrator and log streaming.

Standard Log vs. Orchestrator Log

Without Orchestrator:

TEXT

2025-01-10 10:15:03 INFO  com.mend.sca.ScanRunner - CTX=default SCAN_ID=12345 Starting scan

With Orchestrator enabled:

TEXT

2025-01-10 10:15:03 INFO  com.mend.orchestrator.Main -
  CTX=default SCAN_ID=12345
  agent=sca agent-version=2.3.0
  state=READING_ENVIRONMENT_CONFIG command=start-scan engine=unified-agent
  message="Starting SCA scan..."

Key differences:

Additional Orchestrator metadata (agent, state, command, engine)
Consolidated context tags (CTX, SCAN_ID) at the front of the metadata block
Clear orchestration phase (state=READING_ENVIRONMENT_CONFIG)

Unified Agent Output: Non-Streaming vs. Streaming

UA output without streaming (printed once at the end):

TEXT

2025-01-10 10:15:45 INFO  com.mend.orchestrator.EngineRunner -
  [Unified-Agent] Initializing...
  [Unified-Agent] Analyzing dependencies...
  [Unified-Agent] Generating report...
  [Unified-Agent] Scan completed successfully.

All UA output is grouped under a single timestamp, making it difficult to see real-time progress.

UA output with streaming enabled:

TEXT

2025-01-10 10:15:20 INFO  com.mend.orchestrator.EngineRunner - [Unified-Agent] Initializing...
2025-01-10 10:15:25 INFO  com.mend.orchestrator.EngineRunner - [Unified-Agent] Analyzing dependencies...
2025-01-10 10:15:40 INFO  com.mend.orchestrator.EngineRunner - [Unified-Agent] Generating report...
2025-01-10 10:15:45 INFO  com.mend.orchestrator.EngineRunner - [Unified-Agent] Scan completed successfully.

Error example with streaming:

TEXT

2025-01-10 10:15:22 ERROR com.mend.orchestrator.EngineRunner - [Unified-Agent:err] Failed to read configuration file: config.yaml

Benefits:

Each UA line has its own timestamp.
Error lines are clearly tagged ([Unified-Agent:err]) and can be filtered or alerted on.
You can follow the scan lifecycle in real time in your CI logs or logging platform.