コンテンツにスキップ

Migration from Elastic

This guide helps Elastic Stack (ELK / Elastic Security) users move to CSIRT-Pro. It maps KQL and Elasticsearch concepts to their PRQL equivalents and lays out a practical migration workflow.

Concept Mapping

Most Elastic concepts map cleanly onto CSIRT-Pro. Queries move from KQL or Lucene to PRQL, and detection logic moves into Playbooks.

Elastic Concept CSIRT-Pro Equivalent Notes
Index / Data Stream Pipeline Each Pipeline maps to its own log store
Index Pattern Pipeline name Queries target the pipeline name directly
KQL / Lucene PRQL Pipelined syntax, strongly typed
Kibana Dashboard Dashboard 12-column grid, PDF export, JSON import/export
Detection Rule Playbook (schedule trigger) More flexible: full Python/JS logic
Alert Case (opened by Playbook/UEBA) Threaded investigation workflow
Saved Search Saved Visualization Query and chart config are persisted
Index Lifecycle Policy (ILM) Pipeline TTL Simple day-based retention
Ingest Pipeline Pipeline parse settings Regex or JSON extraction at query time
Kibana Lens / Visualize Search visualization Multiple chart types, right-click to switch
Elastic Agent / Beats API client / log shipper Any HTTP-capable agent works
Spaces Organizations Full multi-tenant isolation
Cross-Cluster Search SharedAccess Fine-grained cross-org data sharing
Elastic SIEM Built-in (Search + Cases + UEBA) Unified in one product
Elastic SOAR Playbook Visual editor with Python/JS nodes

KQL to PRQL Mapping

action: "DENY"

from `firewall_logs`
filter action == "DENY"

Multiple Conditions (AND)

action: "DENY" and src_ip: "192.168.1.100"

from `firewall_logs`
filter action == "DENY"
filter src_ip == "192.168.1.100"

OR Conditions

action: "DENY" or action: "DROP"

from `firewall_logs`
filter (action == "DENY" || action == "DROP")

Wildcard / Partial Match

message: *failed*

from `auth_logs`
filter message ~= "failed"

Numeric Range

status >= 400 and status < 500

from `web_access`
filter status >= 400
filter status < 500

Negation

NOT action: "ALLOW"

from `firewall_logs`
filter action != "ALLOW"

Aggregation: Count by Field

{
  "size": 0,
  "aggs": {
    "by_src_ip": {
      "terms": { "field": "src_ip", "size": 10 }
    }
  }
}

from `firewall_logs`
group {src_ip} (
  aggregate {count = count this}
)
sort {-count}
take 10

Date Histogram

{
  "size": 0,
  "aggs": {
    "over_time": {
      "date_histogram": {
        "field": "@timestamp",
        "calendar_interval": "1h"
      }
    }
  }
}

from `firewall_logs`

Automatic Bucketing

The CSIRT-Pro Timeline visualization applies date-histogram bucketing automatically. Select the Timeline chart type after running your query.

Operator Reference

KQL / Elastic PRQL Equivalent Notes
field: "value" filter field == "value" Exact match
field: *pattern* filter field ~= "pattern" Regex partial match
field >= 100 filter field >= 100 Numeric comparison
NOT field: "value" filter field != "value" Negation
AND Multiple filter lines Sequential filters are AND-ed
OR \|\| inside a filter filter (a == 1 \|\| b == 2)
_source: [fields] select {fields} Field projection
sort: [{"@timestamp": "desc"}] sort {-__time} Descending sort
size: N take N Limit results
terms aggregation group {field} (aggregate {...}) Group and aggregate
sum aggregation aggregate {total = sum field} Sum aggregation
avg aggregation aggregate {avg = average field} Average aggregation
min / max aggregation aggregate {min field} / aggregate {max field} Min/Max

Ingest Pipeline Mapping

In Elastic, parsing happens at ingest time through ingest-pipeline processors. In CSIRT-Pro, you define the parse rule on the pipeline, and extraction runs at query time (schema-on-read). A Grok processor maps to a regex parse rule with the matching field names.

Elastic Ingest Pipeline (Grok Processor)

{
  "processors": [
    {
      "grok": {
        "field": "message",
        "patterns": ["%{IP:src_ip} %{WORD:action} %{NUMBER:bytes}"]
      }
    }
  ]
}

CSIRT-Pro Pipeline Parse Settings

druid_parse: 

(\S+) (\w+) (\d+)

druid_parse_field: 

src_ip, action, bytes(Int64)

A Grok numeric field maps to a typed parse field with name(Type) (e.g. bytes(Int64)), so numeric comparison and aggregation work without conversion functions. Untyped names default to String. Typing works in both regex and JSON mode.

Converting Grok patterns

You do not have to translate Grok patterns by hand. Paste a sample log line into AI Parse Generation: it accepts Grok-style input and produces the regex and field names, which you review and apply.

Data Migration Workflow

Phase 1: Set Up Pipelines

  1. Map each Elasticsearch index or data stream to a CSIRT-Pro Pipeline.
  2. Configure parse settings to match your existing ingest-pipeline processors.
  3. Use AI Parse Generation to draft rules quickly from sample logs.
  4. Set TTL values to match your existing ILM hot/warm/delete phases.

Phase 2: Parallel Ingestion

  1. Configure your log shippers to send data to both Elastic and CSIRT-Pro at the same time.
  2. Confirm the data in the CSIRT-Pro Search screen.
  3. Compare record counts and field-extraction accuracy.

Phase 3: Historical Data Export

Export data from Elasticsearch and import it into CSIRT-Pro.

Using the Scroll API

# Export from Elasticsearch
curl -X POST "http://elasticsearch:9200/firewall-logs/_search?scroll=5m" \
  -H "Content-Type: application/json" \
  -d '{"size": 1000, "query": {"match_all": {}}}' \
  > export.json

# Convert to NDJSON and import to CSIRT-Pro
curl -X POST "https://<domain>/api/v2/ingest/batch" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: <api_key>" \
  -d '{
    "pipeline_id": "<pipeline_id>",
    "records": [ ... ]
  }'

Using Elasticdump (for large datasets)

# Export
elasticdump --input=http://elasticsearch:9200/my-index --output=data.json --type=data

# Transform and import to CSIRT-Pro
# (use a script to convert to the CSIRT-Pro batch format)

Phase 4: Migrate Dashboards and Detection Rules

  1. Recreate Kibana dashboards by converting KQL queries to PRQL.
  2. Convert Elastic Detection Rules to CSIRT-Pro Playbooks with schedule triggers and PRQL-based detection logic.
  3. Map Elastic alerting actions to Playbook notification nodes (Slack, email, webhook).

Phase 5: Cutover

  1. Redirect all log shippers to CSIRT-Pro only.
  2. Scale down or decommission the Elasticsearch nodes.
  3. Keep Elasticsearch in read-only mode until CSIRT-Pro retention covers your compliance window.

Common Questions

How does CSIRT-Pro handle schemaless data?

JSON logs are ingested without a schema definition. When the parse rule is empty, the pipeline treats messages as JSON and extracts fields at query time, and a value that does not match its declared type becomes NULL. Field names can carry type annotations (name(Type)) in both JSON mode and regex mode; untyped fields default to String. For structured text logs, configure a regex parse rule per pipeline, optionally typing fields the same way.

Can I still use Sigma rules?

Yes. CSIRT-Pro includes a Sigma rule conversion endpoint (POST /api/sigma) that translates Sigma YAML into PRQL queries.

What about ECS (Elastic Common Schema)?

CSIRT-Pro does not require a fixed schema. You can keep using ECS field names in your logs, and they are preserved as written.

How does pricing compare?

CSIRT-Pro uses predictable pricing rather than node-based or resource-based licensing. Capacity planning is straightforward, and there is no cluster sizing to tune.