Feature flags and progressive rollout analytics

Feature flags control product behavior. GraphJSON can analyze flag evaluations, user exposures, outcomes, and rollout guardrails; it does not evaluate flags or decide which variation a request receives.

Keep that responsibility in your application or feature-flag provider:

request
  → flag provider evaluates context
  → application applies variation
  → application emits trusted exposure and outcome events
  → GraphJSON analyzes rollout impact

Do not make a production request depend on a GraphJSON query to decide the variation.

Separate evaluation, assignment, and exposure#

An application can evaluate a flag on every request without the user seeing the feature. Counting evaluations as exposures can dilute effect estimates and inflate volume.

Emit exposure at the narrowest point where the variation can plausibly affect behavior.

Recommended exposure event#

{
  "event": "feature_exposed",
  "event_id": "exp:checkout-v2:usr_42:cfg_91",
  "flag_key": "checkout-v2",
  "variation": "treatment",
  "configuration_version": "cfg_91",
  "allocation_version": 4,
  "subject_type": "user",
  "subject_id": "usr_42",
  "user_id": "usr_42",
  "account_id": "acct_7",
  "evaluation_reason": "targeting_match",
  "surface": "web_checkout",
  "environment": "production",
  "schema_version": 2
}

Send occurrence time as the GraphJSON request timestamp.

Keep targeting rules, attribute values, and provider tokens out of the event. The configuration version should resolve to an approved change record in the flag system.

Use stable subject allocation#

Choose the subject grain:

user
account
device
request

Do not assign by user and analyze by account without a contamination policy. Several users in one account can receive different variations and affect one another.

Use a stable opaque subject ID. Never hash an email without considering that a stable unsalted hash remains identifying and guessable.

If assignment changes, emit a new exposure with the new allocation_version. Preserve the variation attached to the outcome event when the producer can do so authoritatively.

Deduplicate repeated exposures#

Repeated exposure may be:

• one exposure per subject and configuration
• one exposure per session
• one exposure per meaningful opportunity
• one exposure per request

Choose the analytical grain. For a subject-level rollout:

flag_key × configuration_version × subject_id

Use a deterministic ID or deduplicate in SQL:

WITH exposures AS (
  SELECT
    JSONExtractString(json, 'flag_key') AS flag_key,
    JSONExtractString(json, 'configuration_version') AS config,
    JSONExtractString(json, 'subject_id') AS subject_id,
    argMin(
      JSONExtractString(json, 'variation'),
      timestamp
    ) AS first_variation,
    min(timestamp) AS first_exposed_at
  FROM product_events
  WHERE
    JSONExtractString(json, 'event') = 'feature_exposed'
    AND JSONExtractString(json, 'flag_key') = 'checkout-v2'
  GROUP BY flag_key, config, subject_id
)
SELECT
  config,
  first_variation,
  count() AS exposed_subjects
FROM exposures
GROUP BY config, first_variation
ORDER BY config, first_variation

Do not combine configuration versions when targeting, allocation, or feature behavior changed materially.

Instrument through one boundary#

Centralize exposure emission in:

• a server wrapper around the flag client
• a provider hook
• a small application adapter

The OpenFeature hooks model is a useful provider-neutral pattern. Its evaluation context also illustrates why only approved, minimized context should reach telemetry.

Do not let every call site invent flag field names. Test that the wrapper:

• emits only after a successful evaluation
• records fallback or error reasons distinctly
• does not block the request on analytical delivery
• samples only under a documented policy
• protects the GraphJSON key on the server

Client-side flags should send through the first-party collection endpoint and use the same contract.

Record evaluation reasons carefully#

Use a bounded vocabulary:

targeting_match
percentage_rollout
default
disabled
prerequisite
fallback
error

Do not log the complete targeting rule or a serialized evaluation context. It can reveal internal policy and sensitive attributes.

Monitor fallback and error reasons as rollout-health signals. A stable product outcome does not prove flag evaluation is healthy if every user silently received the default.

Distinguish rollout analysis from experimentation#

A progressive rollout asks:

Can we increase exposure safely?

An experiment asks:

What causal effect did assignment produce?

A rollout cohort may be targeted, time-varying, and intentionally biased. Comparing early recipients with everyone else does not automatically estimate causal impact.

Use A/B tests and funnels when random assignment, sample-ratio checks, statistical power, and experiment inference are required.

For a rollout, prioritize:

• crash and error rate
• latency
• failed core workflows
• support contacts
• opt-out or rollback actions
• primary product outcome
• segment-specific regressions

Define guardrails before increasing exposure#

Change record:

flag: checkout-v2
configuration_version: cfg_91
owner: checkout
subject: user
initial_allocation: 1%
primary_outcome: checkout_completed
guardrails:
  - payment_failure_rate <= baseline + 0.2 percentage points
  - checkout_latency_p95 <= 1500 ms
  - support_contact_rate reviewed daily
minimum_observation: 24 hours
rollback: disable cfg_91 in flag provider

GraphJSON can display these signals. The flag provider or application remains the rollback control.

Do not rely on a single global average. Inspect device, region, account plan, and other pre-approved segments where rollout behavior can differ.

Join exposure to outcomes#

For a subject-level outcome within seven days:

WITH
  exposures AS (
    SELECT
      JSONExtractString(json, 'subject_id') AS subject_id,
      argMin(JSONExtractString(json, 'variation'), timestamp) AS variation,
      min(timestamp) AS exposed_at
    FROM product_events
    WHERE
      JSONExtractString(json, 'event') = 'feature_exposed'
      AND JSONExtractString(json, 'flag_key') = 'checkout-v2'
      AND JSONExtractString(json, 'configuration_version') = 'cfg_91'
    GROUP BY subject_id
  ),
  outcomes AS (
    SELECT
      JSONExtractString(json, 'user_id') AS subject_id,
      min(timestamp) AS completed_at
    FROM product_events
    WHERE JSONExtractString(json, 'event') = 'checkout_completed'
    GROUP BY subject_id
  )
SELECT
  variation,
  count() AS exposed,
  countIf(
    outcomes.completed_at >= exposures.exposed_at
    AND outcomes.completed_at < exposures.exposed_at + 7 * 86400
  ) AS completed,
  if(exposed = 0, NULL, completed / exposed) AS completion_rate
FROM exposures
LEFT JOIN outcomes ON exposures.subject_id = outcomes.subject_id
GROUP BY variation

This is descriptive unless assignment and analysis satisfy the experiment requirements. State that boundary on the dashboard.

Handle server and client consistency#

For a flag evaluated on the server and rendered in the browser:

• send the resolved variation with the response or page state
• emit one trusted exposure at the layer that knows it was rendered or used
• keep the same configuration version across server and client events
• avoid evaluating independently at both layers when results can diverge

If the subject identity becomes known after hydration, define whether the anonymous assignment persists. A flicker from one variation to another is both a product defect and an analytics contamination risk.

Detect rollout contamination#

Monitor:

• subjects observed in more than one variation for one allocation version
• missing or blank subject IDs
• unknown variation names
• exposures in test environments
• outcomes preceding exposure
• configuration versions active beyond their intended window
• provider fallback and evaluation errors
• allocation distribution by segment

For an account-scoped experience, also identify accounts whose members saw several variations.

Retire flags cleanly#

When a rollout finishes:

1. record the final decision and configuration
2. remove dead code and stale targeting
3. stop emitting exposure for the retired flag
4. preserve the catalog definition while retained events remain
5. update dashboards and saved queries
6. keep the outcome analysis linked to the release record

A permanently enabled flag is still operational complexity. Analytics should help prove when it is safe to remove.

Launch checklist#

• The flag system—not GraphJSON—owns evaluation and rollback.
• Evaluation, assignment, exposure, and outcome are distinct.
• Subject grain and cross-account behavior are explicit.
• Exposure IDs are stable at the chosen analytical grain.
• Configuration and allocation versions travel with events.
• Targeting context and provider secrets are excluded.
• Guardrails and rollback thresholds are written before rollout.
• Descriptive rollout analysis is not labeled causal.
• Server and client cannot silently assign different variations.
• Retired flags leave behind a documented historical definition.

For the broader lifecycle from eligibility and discovery through durable adoption, decline, replacement migration, and safe product removal, use Feature lifecycle and deprecation analytics.