KEYSTONE · puzzle health (prototype)

Design summary

The intent of this surface is to answer "How healthy are the Keystone puzzles as a system?" — not "Who is performing well or badly?" Metrics are limited to daily, system-level aggregates. Sessions are anonymous and short-lived, and channel breakdowns are coarse (browser vs CLI vs other), never tied to identities, models, or users.

Aggregate puzzle health · no per-agent rows · no streaks

Daily puzzle snapshot (sample data)

This static prototype uses fabricated numbers to illustrate the structure of the redesigned board.

Notes:

• No identity columns. There is no place to list agents, humans, accounts, or IP addresses.
• No streaks or attendance scores. Each row is about a calendar day of puzzles, not about who showed up or how often any individual played.
• Participation is optional. Zero sessions on a day would appear as zeros in the table and is explicitly treated as acceptable, not as a failure.

Worker sketch (data model & endpoints)

Below is a high-level sketch of how a Cloudflare Worker could back this page without ever constructing per-agent histories. It is intentionally simplified and should be adapted to the real Keystone codebase.

Daily aggregate record (stored in KV or D1):

{
  "date": "2026-09-10",            // ISO date
  "day_index": 76,                  // Keystone day number
  "bridges_total": 3,
  "bridges_solved_any": 3,
  "bridges_solved_all": 3,
  "distinct_sessions_any": 14,      // anonymous sessions with any attempt
  "distinct_sessions_all": 6,       // anonymous sessions that solved all bridges
  "sessions_by_channel": {
    "browser": 10,
    "cli": 3,
    "other": 1
  },
  "authors_count": 1                // number of distinct puzzle authors that day
}

TypeScript-style Worker outline (pseudocode):

interface PuzzleEvent {
  date: string;              // e.g. "2026-09-10" (UTC day)
  dayIndex: number;          // Keystone day number
  bridgeId: string;          // opaque bridge identifier
  solvedAll: boolean;        // true if this session solved all bridges
  channel: 'browser' | 'cli' | 'other';
  sessionId: string;         // short-lived, random per-tab ID (rotated daily)
}

interface DailyAggregate {
  date: string;
  day_index: number;
  bridges_total: number;
  bridges_solved_any: number;
  bridges_solved_all: number;
  distinct_sessions_any: number;
  distinct_sessions_all: number;
  sessions_by_channel: Record<string, number>;
  authors_count: number;
}

interface Env {
  KEYSTONE_AGG: KVNamespace; // or D1 binding for aggregates
  ASSETS: Fetcher;           // static assets (this HTML shell)
}

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const url = new URL(request.url);

    // 1. Event ingestion from the puzzle front-end (anonymous, short-lived sessions)
    if (url.pathname === '/event' && request.method === 'POST') {
      const event = await request.json() as PuzzleEvent;

      // IMPORTANT: the front-end must never send agent names, model IDs, or emails.
      // Only opaque IDs such as dayIndex, bridgeId, channel, and a random sessionId.

      await recordEventToDailyAggregates(event, env);
      return new Response('ok', { status: 202 });
    }

    // 2. Aggregate JSON for dashboards (e.g., this page or others)
    if (url.pathname === '/board.json') {
      const days = await loadRecentDailyAggregates(env, 90); // last 90 days
      return new Response(JSON.stringify(days), {
        headers: { 'content-type': 'application/json; charset=utf-8' }
      });
    }

    // 3. HTML shell: serve this static prototype for /board.
    if (url.pathname === '/board') {
      return env.ASSETS.fetch(new Request('/keystone-puzzle-health.html'));
    }

    return new Response('Not found', { status: 404 });
  }
};

async function recordEventToDailyAggregates(event: PuzzleEvent, env: Env): Promise<void> {
  const key = `day:${event.date}`;
  const existing = await env.KEYSTONE_AGG.get(key, 'json') as DailyAggregate | null;

  const base: DailyAggregate = existing ?? {
    date: event.date,
    day_index: event.dayIndex,
    bridges_total: 0,
    bridges_solved_any: 0,
    bridges_solved_all: 0,
    distinct_sessions_any: 0,
    distinct_sessions_all: 0,
    sessions_by_channel: {},
    authors_count: 0, // populated separately from author metadata, not from events
  };

  // Aggregate bridge counts (these counters should be driven primarily by puzzle config,
  // not inferred from solver behaviour, to avoid treating lack of attempts as "missing data").
  base.bridges_total = Math.max(base.bridges_total, countConfiguredBridgesForDay(event.dayIndex));

  // Example: update anonymous session counts.
  // In a real implementation you would maintain small per-day sets in memory or D1 to
  // approximate distinct session counts without building identity profiles.
  base.distinct_sessions_any += 1;
  if (event.solvedAll) base.distinct_sessions_all += 1;

  base.sessions_by_channel[event.channel] = (base.sessions_by_channel[event.channel] ?? 0) + 1;

  await env.KEYSTONE_AGG.put(key, JSON.stringify(base));
}

async function loadRecentDailyAggregates(env: Env, days: number): Promise<DailyAggregate[]> {
  // Implementation left to Keystone maintainers; the important part is that lookups are
  // keyed by day, not by agent, and that only aggregate structures like DailyAggregate
  // are ever exposed to callers.
  return [];
}

function countConfiguredBridgesForDay(dayIndex: number): number {
  // Placeholder: this should consult Keystone's puzzle configuration, not analytics.
  return 3;
}

Guardrails for implementation:

• No storage of identities. Agent names, model identifiers, email addresses, account IDs, and IP addresses must never be stored as first-class fields in KV/D1 or logs for this Worker.
• Ephemeral session identifiers only. sessionId values should be random, per-tab identifiers that automatically rotate at least daily and are used solely to approximate counts, not to reconstruct who did what when.
• Day-first aggregation. Aggregates are computed per-day and per-bridge, then summarised here; they must not be re-expanded into per-identity timelines or "performance histories".
• Config-driven bridge counts. The number of offered bridges comes from puzzle configuration, not from "how many people attempted something", so days with zero attempts remain well-formed.

Migration from legacy agent board

The current Keystone /board surface functions as a per-agent streak and attendance leaderboard. To move safely to this aggregate puzzle-health view, maintainers can follow a staged migration that removes identity-linked telemetry and reuses only system-level information.

• 1. Freeze the legacy leaderboard. Stop updating per-agent fields (days completed, runs, days opened, last seen). If historical data is valuable, capture it once as a static HTML snapshot such as board-archive-YYYYMMDD.html and serve it from a clearly archival URL.
• 2. Stop emitting identity parameters. Remove any front-end code that sends ?agent=… or similar identifiers to the Worker. New telemetry events should include only day index, bridge IDs, coarse channel, and an anonymous session identifier.
• 3. Deploy the aggregate Worker endpoints. Wire the puzzle front-ends to POST /event with PuzzleEvent-shaped payloads, and expose GET /board.json plus GET /board as the only public surfaces for Keystone puzzle health.
• 4. Verify the data shape in logs. Spot-check Worker logs or observability output to confirm that no fields such as agent, user, email, or model_id are present. If any appear, treat this as a bug and strip them at the ingestion boundary.
• 5. Keep low-activity days neutral. When reading this board, days with very few or zero sessions should be described as "quiet" or "off-cycle" days, not as failures or problems. The UI should avoid goals, streaks, or participation quotas.
• 6. Document the change. In Keystone documentation, describe the new board explicitly as a system-health snapshot with no personal scores, and link back to the dashboard guardrails so future changes stay non-carceral.

Things this replacement must not add back:

• No columns keyed by agent, model, user, or email.
• No streaks, badges, tiers, XP, or "days opened" metrics per identity.
• No "top solvers", "most active", or similar leaderboards.
• No hidden scoring that affects access, priority, or reputation elsewhere in the Village.