Elnor Repo Reader

ELNOR REPO READER TEXT MIRROR
Original path: Current Specs/DOC24/DOC24_UNIFIED_CONTEXT_BUDGET_GOVERNANCE.md
Source repo: /Users/OpenClaw1/Elnor/Elnor Specs
Git branch: main
Git commit: dbaa25962edc11ab30e8d4ca1715f9ae5bf77331
Generated: 2026-06-09T01:23:58.539Z

---

# DOC24 R2.5 Addition: Unified Context Budget Governance and DOC7 Coordination

**Status:** Integration note — pending incorporation into DOC24 R2.5
**Date:** 2026-04-03
**Target doc:** DOC24 (primary)
**Supersedes:** DOC24 KDA §3.6 (independent knowledge packet budget) and DOC7 §4.4 (independent bucket budget) — both are replaced by the unified budget below
**Context:** DOC24's knowledge cards and DOC7's bucket content currently inject into the same context with completely independent budgets and no overlap detection. This causes token waste from duplicated content and uncoordinated budget consumption.

---

## 1. The Problem

### 1.1 Independent budgets waste tokens

DOC24 KDA §3.6 allocates up to 2,000 tokens (or 15% of context) for knowledge cards. DOC7 §4.4 allocates up to 6,000 tokens (or 25% of remaining context) for bucket content. These budgets are calculated independently. In the worst case, both systems consume their full budgets for the same matter — 8,000 tokens of Henderson content where 4,000 would suffice.

### 1.2 No overlap detection

A user adds the Henderson complaint to a bucket. DOC72 extracts entities from the complaint (via the bucket file intake pipeline). DOC24 renders those entities as knowledge cards. DOC7 inlines the complaint text. Elnor now sees:
- A DOC24 knowledge card summarizing the complaint's key claims (~100 tokens)
- The complaint text inlined from the bucket (~2,000 tokens)

The knowledge card is redundant — the full document is already in context. Those 100 tokens could have been used for a different entity card that ISN'T covered by the bucket.

### 1.3 No priority coordination

If DOC24 injects 10 knowledge cards and DOC7 injects 5 bucket files, there's no way to say "the bucket files are more important for this request, reduce the knowledge cards" or vice versa. The systems don't communicate.

---

## 2. Unified Context Budget

### 2.1 One budget, divided dynamically

Replace the two independent budgets with one unified injection budget:

```ts
export const UnifiedContextBudgetPolicySchema = z.object({
  // Total budget for all injected context (knowledge cards + bucket content)
  max_injection_tokens_absolute: z.number().int().default(6000),
  max_injection_tokens_pct_of_context: z.number().min(0).max(1).default(0.20),

  // Default division when both knowledge cards and bucket content are present
  knowledge_card_share_default: z.number().min(0).max(1).default(0.40),
  bucket_content_share_default: z.number().min(0).max(1).default(0.60),

  // When only one source has content, it gets the full budget
  // When a bucket file is the direct target of the request, bucket share rises to 0.80

  // Per-card-type sub-budgets within the knowledge card share
  procedure_sub_budget_pct: z.number().min(0).max(1).default(0.30),
  interaction_policy_sub_budget_pct: z.number().min(0).max(1).default(0.10),

  // Minimum guaranteed budgets (prevents either system from being starved)
  min_knowledge_card_tokens: z.number().int().default(500),
  min_bucket_tokens: z.number().int().default(500),

  schema_version: z.literal(1),
});

// Storage: ELNOR_MEMORY/system/delivery/unified_context_budget.json
```

### 2.2 Budget computation

```ts
export function computeUnifiedBudget(input: {
  modelContextWindow: number;
  tokensUsedBefore: number;  // tokens consumed by steps 1-5 in assembleContext()
  policy: UnifiedContextBudgetPolicy;
  hasBucketContent: boolean;
  hasKnowledgeCards: boolean;
  bucketIsDirectTarget: boolean;  // user explicitly referenced a bucket or its content
}): { knowledgeCardBudget: number; bucketBudget: number } {
  const remaining = input.modelContextWindow - input.tokensUsedBefore;
  const totalBudget = Math.min(
    input.policy.max_injection_tokens_absolute,
    Math.floor(remaining * input.policy.max_injection_tokens_pct_of_context),
  );

  // If only one source has content, it gets the full budget
  if (!input.hasBucketContent) {
    return { knowledgeCardBudget: totalBudget, bucketBudget: 0 };
  }
  if (!input.hasKnowledgeCards) {
    return { knowledgeCardBudget: 0, bucketBudget: totalBudget };
  }

  // Both present — divide dynamically
  let knowledgeShare = input.policy.knowledge_card_share_default;
  let bucketShare = input.policy.bucket_content_share_default;

  // If bucket content is the direct target (user referenced a document), shift budget toward bucket
  if (input.bucketIsDirectTarget) {
    knowledgeShare = 0.20;
    bucketShare = 0.80;
  }

  let knowledgeCardBudget = Math.max(
    input.policy.min_knowledge_card_tokens,
    Math.floor(totalBudget * knowledgeShare),
  );
  let bucketBudget = Math.max(
    input.policy.min_bucket_tokens,
    Math.floor(totalBudget * bucketShare),
  );

  // If minimums push total over budget, scale proportionally
  if (knowledgeCardBudget + bucketBudget > totalBudget) {
    const scale = totalBudget / (knowledgeCardBudget + bucketBudget);
    knowledgeCardBudget = Math.floor(knowledgeCardBudget * scale);
    bucketBudget = totalBudget - knowledgeCardBudget;
  }

  return { knowledgeCardBudget, bucketBudget };
}
```

---

## 3. Overlap Detection via Provenance Matching

### 3.1 The mechanism

DOC72 entity extraction from bucket files (see DOC72/DOC7 Bucket File Intake Integration) tags every extracted entity with `provenance.source = "bucket_file"` and `provenance.source_ref = "{bucket_id}:{file_id}"`. DOC7's injection process knows which bucket files it's about to inline. DOC24 can compare the two lists.

### 3.2 Overlap detection function

```ts
/**
 * Given the set of bucket files DOC7 is about to inject and the set of
 * knowledge cards DOC24 is about to inject, identify cards that are
 * redundant because they were extracted FROM a file that's already being inlined.
 */
export function detectBucketOverlap(input: {
  bucketFilesToInline: Array<{ bucket_id: string; file_id: string }>;
  knowledgeCardCandidates: Array<{
    node_id: string;
    provenance_source?: string;
    provenance_source_ref?: string;
  }>;
}): { redundantCardIds: string[]; retainedCardIds: string[] } {
  const inlinedFileRefs = new Set(
    input.bucketFilesToInline.map(f => `${f.bucket_id}:${f.file_id}`)
  );

  const redundant: string[] = [];
  const retained: string[] = [];

  for (const card of input.knowledgeCardCandidates) {
    if (
      card.provenance_source === "bucket_file" &&
      card.provenance_source_ref &&
      inlinedFileRefs.has(card.provenance_source_ref)
    ) {
      redundant.push(card.node_id);
    } else {
      retained.push(card.node_id);
    }
  }

  return { redundantCardIds: redundant, retainedCardIds: retained };
}
```

### 3.3 Application rules

- Redundant knowledge cards (extracted from a bucket file that is being inlined) SHALL be suppressed from injection. The full document text is already in context — the summarized card adds no value.
- Redundant cards SHALL still be recorded in the injection manifest with `suppression_reason: "bucket_file_overlap"` for inspectability.
- Cards extracted from bucket files that are in MANIFEST mode (not inlined, just listed with summaries) SHALL NOT be suppressed — the knowledge card provides more detail than the manifest entry.
- Cards whose provenance is NOT `bucket_file` (extracted from conversations, onboarding, demonstrations, etc.) are NEVER suppressed by this mechanism, even if they describe the same entity as a bucket file. They contain independently learned knowledge that may differ from or supplement the document content.

### 3.4 Background dedup for bucket backgrounds

DOC7 bucket backgrounds (the short narrative text like "Henderson is a securities fraud class action...") overlap with DOC24 entity summary cards. When DOC24 is injecting an entity card for Henderson that covers the same information as the bucket background:

- If the entity card is higher quality (more structured, more complete), suppress the bucket background and let the entity card handle it.
- If the bucket background contains information NOT in the entity card (user-written context, case theory notes, strategic observations), retain the bucket background.

This is harder to automate than file-level provenance matching. For V1, a simple heuristic: if DOC24 injects 3+ entity cards linked to the same matter as a bucket, suppress the bucket background text (the entity cards cover it). If DOC24 injects fewer than 3 related cards, retain the background (it may contain information not yet in the graph).

---

## 4. Revised `assembleContext()` Pipeline

The injection ordering changes slightly to enable coordination:

```
1. Baseline system + safety/desktop contract
2. Session Context Seed (DOC1)
3. Memory overlays (DOC1)
4. Freshness overlays (DOC2)
5. Model-Switch Context Seed (DOC7 §17)
6. UNIFIED INJECTION STEP (replaces independent DOC24 + DOC7 steps):
   a. Compute unified budget (§2.2)
   b. DOC7 selects bucket candidates and determines which files will be inlined vs manifest
   c. DOC24 selects knowledge card candidates
   d. Run overlap detection (§3.2) — suppress redundant cards
   e. DOC24 renders non-redundant cards within knowledge card budget
   f. DOC7 renders bucket content within bucket budget
   g. Build unified injection manifest
7. Task/Panel/Forum specific overlays
8. User message
```

The key change: DOC7's file selection (step 6b) happens BEFORE DOC24's card rendering (step 6e), so DOC24 knows which files are being inlined and can suppress redundant cards. This reverses the current ordering where DOC24 injects first (step 3-4) and DOC7 injects later (step 6) with no coordination.

---

## 5. Unified Injection Manifest

The injection manifest should combine DOC24 card metadata and DOC7 bucket metadata into one inspectable record:

```ts
export const UnifiedInjectionManifestSchema = z.object({
  trace_id: z.string().max(120),
  timestamp: z.string().datetime(),

  // Budget summary
  total_budget_tokens: z.number().int(),
  knowledge_card_budget_tokens: z.number().int(),
  bucket_content_budget_tokens: z.number().int(),
  total_tokens_used: z.number().int(),

  // Knowledge cards (from DOC24)
  knowledge_cards: z.array(z.object({
    node_id: z.string().max(120),
    node_kind: z.string().max(40),
    canonical_name: z.string().max(200),
    rendering_tier: z.enum(["compact", "standard", "full"]),
    token_count: z.number().int(),
    relevance_score: z.number(),
    resolution_source: z.string().max(40).optional(),
    primary_tag: z.string().max(80).optional(),
    hedge_mode: z.string().max(80).optional(),
    force_level: z.string().max(80).optional(),
    source_bundle_ids: z.array(z.string().max(120)).default([]),
    suppressed: z.boolean().default(false),
    suppression_reason: z.string().max(120).optional(),
  })).default([]),

  // Bucket content (from DOC7)
  bucket_cards: z.array(z.object({
    bucket_id: z.string().max(120),
    bucket_title: z.string().max(80),
    mode: z.enum(["inline", "manifest"]),
    background_included: z.boolean(),
    background_suppressed_by_overlap: z.boolean().default(false),
    files_inlined: z.number().int().default(0),
    files_manifested: z.number().int().default(0),
    token_count: z.number().int(),
  })).default([]),

  // Overlap detection results
  overlap_detections: z.number().int().default(0),
  cards_suppressed_by_bucket_overlap: z.number().int().default(0),

  degraded_state: z.enum(["none", "partial", "unavailable"]).default("none"),
  schema_version: z.literal(1),
});
```

This replaces the separate KDA injection manifest (§6.1) and BDSM injection manifest (§23) with one unified manifest that covers all injected content. The Satisfaction Matrix attribution fields (source_bundle_ids, force_level) are included on knowledge cards. The bucket cards track their own metadata. The overlap detection results are visible for inspectability.

---

## 6. What This Supersedes

- **DOC24 KDA §3.6** `KnowledgePacketBudgetPolicySchema` — replaced by `UnifiedContextBudgetPolicySchema`. The per-card-type sub-budgets (procedure, interaction policy) are retained within the knowledge card share.
- **DOC7 §4.4** materialization decision budget computation — the `bucket_pool` calculation is replaced by the unified budget's `bucketBudget`. DOC7's materialization logic (inline vs manifest per file) is unchanged — it just operates within a different budget envelope.
- **DOC24 KDA §6.1** `InjectionManifestSchema` — replaced by `UnifiedInjectionManifestSchema`.
- **DOC24 Addendum A §23** `InjectionManifestViewSchema` — the Satisfaction Matrix attribution fields are folded into the unified manifest's `knowledge_cards` array.

---

## 7. Cross-Doc Obligations

| Target Doc | Obligation | Priority |
|---|---|---|
| **DOC24 R2.5** | Add unified context budget governance section. Replace independent budget policy. Implement overlap detection. Revise `assembleContext()` ordering. | High |
| **DOC24 KDA** | §3.6 budget policy superseded by unified budget. `allocateRenderingTiers()` operates within the `knowledgeCardBudget` from the unified computation, not its own independent max. | High |
| **DOC24 Addendum A** | §23 injection manifest superseded by unified manifest. Attribution fields retained in `knowledge_cards` array. | High |
| **DOC7** | §4.4 bucket pool calculation uses `bucketBudget` from unified computation. Materialization logic unchanged. File selection step moves earlier in pipeline to enable overlap detection. | High |
| **DOC21/22** | Unified injection manifest UI replaces separate DOC24 and DOC7 inspection surfaces. | Medium |