Migrate from Snowplow

Snowplow produces validated, enriched events in a warehouse or lake. Its canonical event model is broader than GraphJSON's event body, and self-describing events and entities can create many warehouse columns.

Select the product and operational facts needed in GraphJSON. Keep the Snowplow warehouse or another customer-controlled archive when you need a complete replayable history.

Start from the atomic event data#

Snowplow documents one immutable row per event in the atomic.events table for most warehouse destinations. Redshift stores self-describing event and entity data in related tables.

Inventory:

• warehouse or lake destination
• data model and loader version
• atomic event date range
• self-describing event schemas
• entity schemas
• custom enrichments
• failed-event storage
• derived dbt or SQL models
• consent and deletion workflows

Do not export from an aggregate dashboard when atomic event data is available.

Select business events#

Snowplow can contain page views, page pings, media events, ecommerce events, structured events, and custom self-describing events.

Create a mapping:

Do not import every enrichment and context because it exists.

Preserve canonical identity#

Snowplow's canonical event-field reference defines the standard fields and self-describing event identifiers.

Do not merge domain_userid, network user identity, application user_id, and session identity into one field.

Choose event time explicitly#

Snowplow records several times for collection, device occurrence, enrichment, and processing. Use the same occurrence-time policy as the governed models you are replacing.

Migration worksheet:

primary occurrence field:
fallback field:
clock correction policy:
processing-delay field:
time zone:
known bad interval:

Compare the chosen field with known user actions and existing reports. Convert to whole Unix seconds only after the policy is fixed.

Do not substitute warehouse load time for event time.

Preserve schema information#

For self-describing events, keep:

{
  "source_event_vendor": "com.example",
  "source_event_name": "report_exported",
  "source_event_version": "1-0-2"
}

Map known schema fields into a stable GraphJSON body:

{
  "event": "report_exported",
  "event_id": "snowplow:550e8400-e29b-41d4-a716-446655440000",
  "user_id": "usr_91",
  "account_id": "acct_42",
  "report_type": "retention",
  "format": "csv",
  "source_event_vendor": "com.example",
  "source_event_name": "report_exported",
  "source_event_version": "1-0-2",
  "source": "snowplow",
  "schema_version": 1
}

Do not copy complete self-describing event or entity JSON into one GraphJSON event. Map approved fields and keep raw history in the source warehouse.

Extract through a staging model#

Create one staging query per event family:

SELECT
  event_id,
  COALESCE(derived_tstamp, collector_tstamp) AS occurred_at,
  user_id,
  domain_userid,
  domain_sessionid,
  app_id,
  platform,
  event_vendor,
  event_name,
  event_version
FROM atomic.events
WHERE
  event_vendor = 'com.example'
  AND event_name = 'report_exported'
  AND collector_tstamp >= TIMESTAMP '2026-01-01 00:00:00'
  AND collector_tstamp < TIMESTAMP '2026-07-27 00:00:00'

The example assumes derived_tstamp matches the chosen policy and uses generic SQL notation. Adapt quoting, self-describing columns, and timestamp syntax to the actual warehouse.

Materialize the normalized result before delivery. Record source query, warehouse snapshot, schema versions, and row controls.

Handle entities and contexts#

Snowplow entities can describe web pages, users, products, consent, or other reusable context. Decide for each entity:

• copy selected event-time fields
• map to a separate GraphJSON event collection
• join in the warehouse and publish a compact result
• exclude from GraphJSON

Avoid one-to-many joins that duplicate the root event. If an event has several product entities, either model items deliberately or publish a normalized order-level event plus controlled item events.

Keep failed events separate#

Snowplow may store failed events outside the validated atomic table. Do not import them as successful product facts.

Classify:

• repairable schema or transformation issue
• permanently invalid payload
• duplicate
• privacy violation
• unsupported event

Repair into a new validated staging dataset with lineage to the failure record. Quarantine anything that still violates the target contract.

Import safely#

Transform staging rows into:

{"timestamp":1785110400,"body":{"event":"report_exported","event_id":"snowplow:...","schema_version":1}}

Validate:

• whole Unix-second timestamp
• stable event ID
• event name allowlist
• JSON size
• required user or account grain
• field types
• sensitive-data rules

Send up to 50 events per GraphJSON bulk request. Keep live delivery separate from the backfill queue.

Design the live path#

Options:

• emit explicit GraphJSON events from the application
• consume the same durable stream that feeds Snowplow
• use a Snowplow event forwarder or stream consumer through an adapter

Snowplow documents real-time event forwarding for selected events. Use an adapter to create the GraphJSON request, protect the workspace key, batch, retry, and quarantine.

Do not make a browser tracker call GraphJSON directly with the workspace key.

Reconcile#

For each event family:

• atomic rows selected
• unique Snowplow event IDs
• normalized rows
• accepted and quarantined rows
• counts by event schema and day
• min and max chosen event time
• distinct users, sessions, and accounts
• required-property completeness
• key governed metrics

Existing Snowplow dbt models may apply sessionization, bot filters, attribution, and late-event handling not present in raw rows. Document which logic is reproduced and which definition intentionally changes.

Cut over#

1. deploy the live GraphJSON path
2. run shadow delivery
3. reconcile source and target over settled windows
4. stop the historical extraction at an exact boundary
5. complete the backfill
6. switch dashboards and consumers deliberately
7. retain the warehouse/archive for required history
8. retire Snowplow components only after other consumers are inventoried

Migration checklist#

• Atomic event data is the historical source.
• Business events are selected instead of copying every Snowplow signal.
• Event, user, browser, session, application, and account IDs remain distinct.
• Occurrence-time and fallback policy are explicit.
• Source event vendor, name, and version are preserved.
• Self-describing data is mapped through approved fields.
• One-to-many entities do not duplicate root events accidentally.
• Failed events are repaired or quarantined, not imported as successes.
• Backfill batches and live delivery remain operationally separate.
• Atomic, normalized, accepted, and quarantined counts reconcile.
• Existing modeled definitions are compared explicitly.
• Cutover preserves the warehouse or archive where required.