Product search and discovery analytics

Search analytics should explain whether people can express intent, receive useful results, choose one confidently, and complete the downstream task. Counting searches alone cannot tell you that.

Treat the search service as the authority for request execution and ranking. Send bounded analytical facts to GraphJSON. Do not log raw search text by default: queries often contain names, email addresses, customer data, support details, or secrets.

Define the journey#

search submitted
  → results returned
  → result selected or query reformulated
  → downstream task completed

Decide what counts as success. A documentation search may succeed when an article is opened and no new query follows. A marketplace search may require an order. A support search may require a resolved case.

Keep immediate search quality and later business conversion as separate metrics.

Use one stable search identity#

Generate a search_id on the server for each executed request. Reuse it on result selections and downstream attribution:

{
  "event": "search_results_returned",
  "search_id": "srch_01J...",
  "session_id": "sess_42",
  "user_id": "usr_7",
  "account_id": "acct_3",
  "surface": "global_search",
  "query_class": "natural_language",
  "query_length_bucket": "3_5_tokens",
  "result_count": 18,
  "result_count_bucket": "11_25",
  "filters": ["status", "owner"],
  "sort": "relevance",
  "ranking_version": "ranker_12",
  "latency_ms": 184,
  "result": "success"
}

Do not use the query string as an identifier. If you need repeated-query analysis, create a keyed, access-controlled fingerprint upstream after normalization. A plain unsalted hash of a small query vocabulary can be reversed by enumeration.

Model selection without full result payloads#

{
  "event": "search_result_selected",
  "search_id": "srch_01J...",
  "session_id": "sess_42",
  "result_type": "report",
  "result_id": "rpt_91",
  "rank": 3,
  "selection_method": "click",
  "milliseconds_since_results": 2140,
  "ranking_version": "ranker_12"
}

Usually you do not need to log every result ID and score. That payload is high-cardinality, expensive, and may expose content. Store the selected item, rank, result type, and bounded diagnostic categories. Keep full ranking traces in a short-lived, access-controlled search diagnostic system if required.

Track reformulation#

A reformulation is a new search in the same search journey:

{
  "event": "search_reformulated",
  "search_id": "srch_02K...",
  "previous_search_id": "srch_01J...",
  "session_id": "sess_42",
  "change_type": "added_filter",
  "milliseconds_since_previous": 6200
}

Use controlled change categories:

• broadened query
• narrowed query
• added or removed filter
• changed sort
• corrected spelling
• repeated unchanged

Do not copy old and new query strings into the event.

Define core metrics#

State the eligibility window. A selection after 30 minutes may belong to a different task.

Query zero-result rate#

WITH searches AS (
  SELECT
    JSONExtractString(json, 'search_id') AS search_id,
    argMax(JSONExtractString(json, 'surface'), timestamp) AS surface,
    argMax(JSONExtractInt(json, 'result_count'), timestamp) AS result_count,
    argMax(JSONExtractString(json, 'result'), timestamp) AS result
  FROM search_events
  WHERE JSONExtractString(json, 'event') = 'search_results_returned'
  GROUP BY search_id
)
SELECT
  surface,
  count() AS successful_searches,
  countIf(result_count = 0) AS zero_result_searches,
  zero_result_searches / nullIf(successful_searches, 0) AS zero_result_rate
FROM searches
WHERE result = 'success'
GROUP BY surface
ORDER BY successful_searches DESC

Errors and timeouts do not belong in the zero-result numerator or denominator. Show them as reliability outcomes.

Query selection quality#

Use the first selection per search:

WITH selections AS (
  SELECT
    JSONExtractString(json, 'search_id') AS search_id,
    argMin(JSONExtractInt(json, 'rank'), timestamp) AS first_selected_rank
  FROM search_events
  WHERE JSONExtractString(json, 'event') = 'search_result_selected'
  GROUP BY search_id
)
SELECT
  count() AS searches_with_selection,
  quantileExact(0.5)(first_selected_rank) AS median_first_selected_rank,
  countIf(first_selected_rank <= 3) / count() AS top_3_selection_share
FROM selections

Selected rank is an imperfect relevance signal. Position bias makes high-ranked results more likely to be clicked even when they are not better.

Link downstream outcomes conservatively#

Attach search_id to a later terminal event only while the attribution is valid:

{
  "event": "report_opened",
  "report_id": "rpt_91",
  "account_id": "acct_3",
  "search_id": "srch_01J...",
  "discovery_method": "global_search"
}

Define the maximum window, session boundary, and competing discovery methods. Call the metric “search-assisted,” not “caused by search.”

Filter automated and internal traffic#

Exclude:

• synthetic monitoring
• indexing and preview bots
• load tests
• employee test accounts
• retries with the same request ID
• prefetches that were never shown to a person

Use server-verified flags or maintained reference data. User-agent filtering alone is not reliable.

Investigate failure without collecting raw queries#

Start with safe dimensions:

• query length bucket
• language or locale
• search surface
• filter combination
• result type
• ranking version
• latency bucket
• account plan or lifecycle stage at event time

If raw-text review is genuinely necessary, use a separate access-controlled workflow with short retention, redaction, sampling, purpose limitation, and audited access. Do not make GraphJSON the unrestricted query-log archive.

Build the dashboard#

Show:

1. eligible search volume and active searchers
2. zero-result, error, and timeout rates
3. selection and reformulation rates
4. first selected rank distribution
5. latency percentiles with failure rate
6. search-assisted downstream outcomes
7. trends by surface, platform, and ranking version
8. instrumentation completeness

Annotate ranking releases. Compare versions only over equivalent traffic or a controlled experiment.

Launch checklist#

• Every executed request has a stable search_id.
• Raw query text is excluded by default.
• Result, selection, reformulation, and conversion facts are separate.
• Errors are not misclassified as zero results.
• Selection and conversion windows are explicit.
• Retries, bots, prefetches, and internal traffic are filtered.
• Ranking version travels with result and selection events.
• Search-assisted outcomes are not described as causal.
• Diagnostic dimensions are controlled and privacy reviewed.
• The search service remains authoritative for execution and ranking.

For internal documentation or learning-center search, continue through the selected page, engaged reading, feedback, and assisted task outcome with Content and documentation analytics.