Skip to content

Audio Metadata Endpoint Missing

Failure Signature

Error message: "Could not load audio metadata"

Symptom: Tapping an audio memory in the memories feed shows a black screen with an error message. The AudioPlayer component fails to initialize because the backend API endpoint /memories/audio-metadata (and /memories/audio) does not exist, returning HTTP 404.

Reproduction: 1. Open the app with feature/audio-tile-player branch 2. Navigate to memories feed with audio-only memories 3. Tap on an audio tile to open the player 4. Player shows error: "Could not load audio metadata"

Root Cause Analysis

The audio-tile-player feature branch added the frontend AudioPlayer component and TypeScript API client functions (getMemoryAudio, getMemoryAudioMetadata) that call two backend endpoints:

  1. POST /memories/audio-metadata — fetch duration and processing status
  2. POST /memories/audio — fetch presigned S3 URL for audio file

However, the feature branch did not implement these backend endpoints. The Lambda functions and API Gateway routes were missing entirely, causing 404 errors when the frontend tried to fetch audio data.

This is a classic case of incomplete feature implementation where the frontend contract was created but the backend provider was omitted.

Error Messages & Logs

Frontend Error (AudioPlayer.tsx:200)

setError("Could not load audio metadata");
console.error("Failed to fetch audio metadata", err);

Caught exception from getMemoryAudioMetadata() axioscall: HTTP 404 (endpoint not found)

Backend Evidence

Template.yaml search: grep -i "MemoriesAudio" template.yaml — no matches before fix - MemoriesFeedFunction ✓ (exists) - MemoriesVideoFunction ✓ (exists) - MemoriesAudioFunction ✗ (missing) - MemoriesAudioMetadataFunction ✗ (missing)

Filesystem search: ls /main/server/api/memories/ — no audio/ directory before fix

System Context

  • Component: AudioPlayer.tsx (frontend React Native component)
  • API Layer: main/app/lib/api/memory/audio.ts (axios POST wrapper with 120s timeout)
  • Backend Layer: main/server/api/memories/ (SAM Lambda handler + API Gateway routes)
  • Database: WorldMMSegment table (segments have duration_seconds and processing_status)
  • Storage: S3 bucket encache-raw-memory (audio.wav files at sessions/{sessionId}/window_{windowIndex:03d}/audio.wav)

Fix Applied

Added missing backend endpoints by:

  1. Created /main/server/api/memories/audio/app.py — unified handler supporting both endpoints:
  2. Looks up segment by memory_id (tries source_session_id first for grouped feeds, falls back to segment id)
  3. Returns metadata (duration_seconds, processing_status) without requiring S3 file existence
  4. Generates presigned URL only if processing_status == "complete"

  5. Handles all processing states gracefully (pending, complete, failed)

  6. Updated template.yaml to register two Lambda functions:

  7. MemoriesAudioFunction — routes /memories/audio POST requests
  8. MemoriesAudioMetadataFunction — routes /memories/audio-metadata POST requests

  9. Both share same handler, timeout 120s, database + S3 access policies

  10. Response shapes:

  11. Metadata endpoint returns: { duration_seconds, processing_status, presigned_url? }
  12. Audio endpoint returns: { presigned_url, duration_seconds, processing_status }
  13. Frontend uses only relevant fields; gracefully ignores extras

Testing Strategy

Test file: main/server/tests/unit/test_memories_audio_api.py (to be added in follow-up)

Test coverage needed:

  1. Happy path: Memory with complete audio returns presigned_url + metadata
  2. Pending state: Memory with pending status returns metadata without presigned_url
  3. Failed state: Memory with failed processing returns metadata with failed status
  4. Missing segment: Non-existent memory_id returns 404 NotFoundError
  5. Grouped feed: memory_id=source_session_id finds first matching segment
  6. Individual segment: memory_id=segment_id finds by primary key

Deployment Notes

  • No database migrations required (WorldMMSegment already has duration_seconds, processing_status)
  • Timeout: 120s (matches Lambda Timeout in template, accounts for cold-start delays during audio processing)
  • VPC config required (database access needed to query WorldMMSegment)
  • S3 access policy needed (head_object + generate_presigned_url)
  • Video tile player works correctly (MemoriesVideoFunction exists and is properly configured)
  • Pattern used here matches existing video endpoint implementation
  • Presigned URL TTL matches other endpoints: 3600 seconds

Commit

Branch: feature/audio-tile-player Commit: e5bfab06b905d37690a144c40f3cedd14c30a306 Message: "fix(audio): implement missing /memories/audio and /memories/audio-metadata endpoints"

Status

✅ RESOLVED — Backend endpoints implemented and committed. Audio metadata now loads successfully and AudioPlayer UI displays audio with playback controls or appropriate processing status overlays.