Skip to content

Post Upload Ingest Memory Contract

Plan Metadata

  • Plan type: sub-plan
  • Parent plan: memory-upload-flow.md
  • Depends on: memory-orm.md
  • Status: documentation

System Intent

  • What is being built: A minimal ingest contract that receives a terminally-complete memory reference (all upload items terminal, at least one completed) and writes an embedding_vector onto that memory record.
  • Primary consumer(s): The upload completion flow that invokes ingest only after terminal-state accounting is complete, plus downstream memory retrieval/search paths that require embedding_vector to exist.
  • Boundary (black-box scope only): Validate ingest payload ownership and required fields, generate a vector, persist embedding_vector, and set memory_upload_status=processed. Out of scope: derived-memory creation, summary generation, additional enrichment pipelines, and event fan-out, and all-failed upload recovery behavior.

Stage Gate Tracker

  • [x] Stage 1 Mermaid approved
  • [x] Stage 2 I/O contracts approved
  • [x] Stage 3 pseudocode/technical details approved or skipped

1. Mermaid Diagram

Reference: .agent/skills/create-mermaid-diagram/SKILL.md

flowchart TD
  A[Upload Complete Contract Boundary - docs/plans/memory-upload-flow.md]
  B[Ingest Lambda Handler - main/server/memories/pipeline/ingest/app.py]
  C[memoryorm.updateMemoryEmbedding - docs/plans/memory-orm.md]

  A -->|ingest payload memory_id user_id key bucket when all uploads are terminal and at least one upload completed| B
  B --> C
  B -->|ingest result memory_id and memory_upload_status processed| A

  classDef unchanged fill:#d3d3d3,stroke:#666,stroke-width:1px;
  classDef updated fill:#ffe58a,stroke:#666,stroke-width:1px;
  classDef deleted fill:#f4a6a6,stroke:#666,stroke-width:1px;
  classDef created fill:#a8e6a3,stroke:#666,stroke-width:1px;

  class A,B,C unchanged;

2. Black-Box Inputs and Outputs

Current helper behavior note: this lambda uses invoke_lambda with require_auth_context=false; non-AppError exceptions (including ValueError) are returned as HTTP 500 with message-only error payload. Identity source note: ingest receives user_id from the complete-upload trigger payload (constructed from authenticated upload completion context), and enforces memory ownership before writing embeddings.

Global Types

Type source: reuse shared ORM contract types from docs/plans/memory-orm.md.

MemoryStatusValue {
  value: "started" | "in_progress" | "uploaded" | "failed" | "aborted" | "processed"
}

MemoryEmbeddingVector {
  value: number[384] (vector length must equal 384)
}

StandardError {
  status: number (HTTP or equivalent status)
  message: string (human-readable summary)
}

Flow: ingestMemory

  • Test files: main/server/tests/unit/test_memories_pipeline_ingest_app.py
  • Core files: main/server/memories/pipeline/ingest/app.py

Type Definitions

IngestMemoryPayload {
  memory_id: string (required)
  user_id: string (required)
  key: string (required; must start with "<user_id>/")
  bucket: string (required)
}

IngestMemoryResponse {
  memory_id: string
  memory_upload_status: MemoryStatusValue (for this flow, resolves to "processed")
}

Paths

path-name input output/expected state change path-type notes updated
ingestMemory.success IngestMemoryPayload IngestMemoryResponse; memoryorm.updateMemoryEmbedding persists embedding_vector[384] and memory_upload_status=processed according to docs/plans/memory-orm.md happy path embedding source is current deterministic placeholder vector mode; this is intentionally temporary until provider integration
ingestMemory.invalid-input IngestMemoryPayload StandardError error missing required ingest payload fields currently raise ValueError and are serialized by helper as HTTP 500 message-only
ingestMemory.ownership-error IngestMemoryPayload StandardError error key must be user-scoped to the supplied user_id; current helper behavior serializes ValueError as HTTP 500 message-only
ingestMemory.memory-not-found IngestMemoryPayload StandardError error memoryorm.updateMemoryEmbedding fails when memory_id is absent; current helper behavior returns HTTP 500
ingestMemory.placeholder-vector-invalid-dimension IngestMemoryPayload StandardError error current placeholder vector must still satisfy MemoryEmbeddingVector 384-dimension contract
ingestMemory.unexpected-runtime-error IngestMemoryPayload StandardError error generic lambda exception behavior from invoke_lambda currently returns HTTP 500 message-only envelope

3. Pseudocode / Technical Details for Critical Flows (Optional)

Skipped by request: Stage 2 contract details are sufficient for this sub-plan revision.

After all stages are approved, apply .agent/skills/reconcile-plans/SKILL.md to propagate contract updates across linked plans.