Skip to content

API Gateway Audio and Video Endpoints Authorization

Metadata

  • System type: infrastructure

System Intent

  • What this is: Authorization configuration pattern for API Gateway endpoints (/memories/audio, /memories/audio-metadata, /memories/video) in the SAM template. Documents why the audio endpoints return 403 Forbidden when first deployed, the root cause (missing explicit Event-level Auth: config), how the fix resolves it, and how this compares to the working /memories/video endpoint.

Mermaid Diagram

flowchart TD
  Client["Mobile client\nAuthorization: Bearer <token>"] -->|POST /memories/audio-metadata| APIGW["API Gateway\n(REST API)"]
  APIGW -->|Checks route authorizer| AuthCheck{Route has\nCognitoAuth\nauthorizer?}
  AuthCheck -->|Yes| Cognito["Cognito User Pool\n/encache/auth/user_pool_id"]
  Cognito -->|Validates token,\ninjects claims| Lambda["MemoriesAudioMetadataFunction\napi/memories/audio/app.py"]
  Lambda -->|DB query by user_id| DB["WorldMMSegment\n(PostgreSQL)"]
  Lambda -->|generate_presigned_url| S3["encache-raw-memory\nsessions/.../audio.wav"]
  Lambda -->|200 + presigned_url| Client
  AuthCheck -->|No authorizer attached| Forbidden["403 Forbidden\n{message: Forbidden}\nLambda never invoked"]

Flows

Flow: getAudioMetadata

  • Core files: main/server/api/memories/audio/app.py, main/server/template.yaml, main/server/layers/shared/python/shared/lambda_helpers.py
  • Frontend: main/app/lib/api/memory/audio.ts

Types

AudioMetadataRequest {
  memory_id: string (required) — source_session_id or segment id
}

AudioMetadataResponse {
  presigned_url: string | null — null when processing is not yet complete
  duration_seconds: number
  processing_status: "pending" | "complete" | "failed"
}

StandardError {
  status: number (HTTP status)
  code: string (stable machine-readable code)
  message: string (human-readable summary)
}

Paths

path input output path-type notes
getAudioMetadata.success AudioMetadataRequest AudioMetadataResponse happy path CognitoAuth authorizer attached to route
getAudioMetadata.forbidden any {"message": "Forbidden"} status=403 error API Gateway returns 403 before Lambda invokes; authorizer not attached to route
getAudioMetadata.not-found AudioMetadataRequest StandardError status=404 code=MEMORY_NOT_FOUND error No segment in DB for user_id + memory_id
getAudioMetadata.audio-not-found AudioMetadataRequest StandardError status=404 code=AUDIO_NOT_FOUND error source_session_id or source_window_index missing on segment
getAudioMetadata.pending AudioMetadataRequest AudioMetadataResponse presigned_url=null happy path processing_status != "complete"; presigned URL skipped

Pseudocode

# API Gateway route for POST /memories/audio-metadata
# MUST have explicit Auth: Authorizer: CognitoAuth in template.yaml Event properties
# Without it, SAM may create the route without an authorizer → 403 from API Gateway

getAuthContext(event):
  # API Gateway injects validated claims into requestContext.authorizer.claims
  # This avoids a JWKS fetch inside the VPC (no NAT gateway)
  agw_claims = event.requestContext.authorizer.claims
  if agw_claims.sub:
    return { user_id: agw_claims.sub, ... }
  # Fallback: verify JWT directly (local testing only)

implementation(payload, auth):
  memory_id = payload.memory_id
  segment = query WorldMMSegment WHERE user_id=auth.user_id AND source_session_id=memory_id
  if not segment: raise NotFoundError
  if not segment.source_session_id or segment.source_window_index is None:
    raise NotFoundError("AUDIO_NOT_FOUND")
  audio_key = "sessions/{source_session_id}/window_{source_window_index:03d}/audio.wav"
  if segment.processing_status == "complete":
    head_object(audio_key)  # raises 404 if missing
    presigned_url = generate_presigned_url(audio_key, ExpiresIn=3600)
  return { presigned_url, duration_seconds, processing_status }

Flow: getAudioPresignedUrl

  • Core files: main/server/api/memories/audio/app.py, main/server/template.yaml
  • Same handler (api/memories/audio/app.py) serves both /memories/audio and /memories/audio-metadata

Types

Same as getAudioMetadata — both endpoints share the same Lambda handler.

Paths

path input output path-type notes
getAudioPresignedUrl.success AudioMetadataRequest AudioMetadataResponse happy path Same as getAudioMetadata
getAudioPresignedUrl.forbidden any {"message": "Forbidden"} status=403 error Same root cause as getAudioMetadata

Flow: getVideo

  • Core files: main/server/api/memories/video/app.py, main/server/template.yaml
  • Frontend: main/app/lib/api/memory/video.ts

Types

VideoRequest {
  memory_id: string (required)
}

VideoResponse {
  presigned_url: string  — empty string ("") when processing_status != "complete"
  duration_seconds: number (30)
  frame_count?: number   — count of frame_*.jpg objects in S3; returned for both complete and non-complete responses when the segment has an s3_frames_key (always true in practice, since segments without one are filtered out before this response is built)
  processing_status: "complete" | "pending" | "failed"
}

StandardError {
  status: number (HTTP status)
  code: string (stable machine-readable code)
  message: string (human-readable summary)
}

Paths

path input output path-type notes
getVideo.success VideoRequest, segment.processing_status = "complete" VideoResponse presigned_url=<url>, processing_status="complete" happy path FFmpeg encodes frames + audio, uploads MP4, returns presigned URL
getVideo.pending VideoRequest, segment.processing_status != "complete" VideoResponse presigned_url="", processing_status=<status> happy path skips encoding; counts frames via S3 list if s3_frames_key set; returns immediately
getVideo.not-found VideoRequest StandardError status=404 code=MEMORY_NOT_FOUND error no segment with s3_frames_key for user
getVideo.frames-not-found VideoRequest StandardError status=404 code=FRAMES_NOT_FOUND error segment complete but S3 has no frame objects
getVideo.audio-not-found VideoRequest StandardError status=404 code=AUDIO_NOT_FOUND error audio.wav missing from S3
getVideo.encoding-failed VideoRequest StandardError status=500 code=ENCODING_FAILED error FFmpeg fails or times out at 90 s

Pseudocode

implementation(payload, auth):
  segment = _find_visual_segment(payload.memory_id, auth.user_id)
  if not segment: raise NotFoundError("MEMORY_NOT_FOUND")

  processing_status = segment.processing_status or "pending"
  if processing_status != "complete":
    # Count frames without downloading (lets frontend show metadata)
    frame_count = count s3 objects at frames_dir/frame_* if s3_frames_key set
    return {
      presigned_url: "",
      duration_seconds: 30.0,
      processing_status: processing_status,
      frame_count: frame_count,  # omitted if None
    }

  # processing_status == "complete": download frames + audio, encode, upload, sign
  frames_dir = segment.s3_frames_key.rsplit("/", 1)[0]
  download all frame_*.jpg from S3 to tmpdir
  download audio.wav from S3 to tmpdir
  run FFmpeg: frames + audio → output.mp4 (libx264, aac, -shortest, timeout=90s)
  upload output.mp4 to temp/videos/<uuid>.mp4
  return {
    presigned_url: generate_presigned_url(temp_key, ExpiresIn=3600),
    duration_seconds: 30.0,
    frame_count: len(frame_keys),
    processing_status: "complete",
  }

Authorization: Root Cause of 403 on Audio Endpoints

Why 403 Occurs

API Gateway checks whether each route has an authorizer attached. When a route is deployed without an authorizer, API Gateway returns 403 Forbidden immediately — before the Lambda is ever invoked. The client receives {"message": "Forbidden"} and CloudWatch shows no Lambda invocation.

Root Cause: Missing Event-Level Auth: in template.yaml

The SAM Globals define a default authorizer:

# main/server/template.yaml, lines 64-69
Globals:
  Api:
    Auth:
      DefaultAuthorizer: CognitoAuth
      Authorizers:
        CognitoAuth:
          UserPoolArn: !Sub "arn:aws:cognito-idp:${AWS::Region}:${AWS::AccountId}:userpool/{{resolve:ssm:/encache/auth/user_pool_id}}"
      AddDefaultAuthorizerToCorsPreflight: false

The audio endpoint Events on the feature/fix-audio-403 branch are missing an explicit Auth: block:

# CURRENT (missing Auth — may not get CognitoAuth attached on first deploy)
Events:
  MemoriesAudio:
    Type: Api
    Properties:
      Path: /memories/audio
      Method: post
      # No Auth: block

  MemoriesAudioMetadata:
    Type: Api
    Properties:
      Path: /memories/audio-metadata
      Method: post
      # No Auth: block

SAM inherits DefaultAuthorizer from Globals in most cases, but this inheritance can fail on new routes being added to an existing API Gateway stage for the first time. The result: the route is created without the Cognito authorizer attached, and API Gateway returns 403 on every request.

Fix: Add Explicit Event-Level Auth

# FIXED
Events:
  MemoriesAudio:
    Type: Api
    Properties:
      Path: /memories/audio
      Method: post
      Auth:
        Authorizer: CognitoAuth    # explicit — no ambiguity on first deploy

  MemoriesAudioMetadata:
    Type: Api
    Properties:
      Path: /memories/audio-metadata
      Method: post
      Auth:
        Authorizer: CognitoAuth    # explicit — no ambiguity on first deploy

How the Fix Resolves It

With Auth: Authorizer: CognitoAuth explicit at the Event level, SAM generates a CloudFormation resource that directly attaches the named authorizer to the route — regardless of whether it is a first deploy or an update. API Gateway validates the token before invoking Lambda, injects requestContext.authorizer.claims, and the Lambda's getAuthContext() reads claims["sub"] as the user_id.

Comparison: Audio vs Video

Aspect /memories/video (working) /memories/audio, /memories/audio-metadata (403 risk)
Lambda handler api/memories/video/app.py api/memories/audio/app.py
require_auth_context True True
IAM Policies DatabaseSsmPolicy, S3AccessPolicy DatabaseSsmPolicy, S3AccessPolicy
VPC Config Yes Yes
Lambda Timeout 120s 120s
Lambda Memory 1024 MB 512 MB
Event-level Auth: Not set (relies on Global default) Not set (relies on Global default)
Deployment status On master — deployed; authorizer already attached On feature branch — new routes; authorizer attachment depends on SAM inheritance
403 risk Low (authorizer already attached from prior deploys) High (new routes may not inherit authorizer on first deploy)

The video endpoint works despite also lacking explicit Event-level Auth: because its route was created in API Gateway during earlier deploys when the Global default applied correctly. Newly created routes for audio endpoints on a fresh branch deploy do not have this history.

Auth Flow After Fix

Client → POST /memories/audio-metadata (Authorization: Bearer <cognito-id-token>)
  → API Gateway: validates token against Cognito User Pool
  → API Gateway: injects requestContext.authorizer.claims = { sub, token_use, ... }
  → Lambda invoked with enriched event
  → getAuthContext(): reads agw_claims.sub as user_id (no JWKS fetch needed inside VPC)
  → implementation(): queries WorldMMSegment WHERE user_id=sub AND source_session_id=memory_id
  → returns { presigned_url, duration_seconds, processing_status }

To Make an Endpoint Public (No Auth)

Use Authorizer: NONE explicitly:

Auth:
  Authorizer: NONE

This pattern is used by /auth/start, /auth/verify, /auth/handoff, and OPTIONS preflight routes.

Logs

Source Location
Audio Lambda CloudWatch: /aws/lambda/MemoriesAudioFunction
Audio Metadata Lambda CloudWatch: /aws/lambda/MemoriesAudioMetadataFunction
Video Lambda CloudWatch: /aws/lambda/MemoriesVideoFunction
API Gateway execution CloudWatch: /aws/apigateway/<api-id>/<stage>

To distinguish a 403 from API Gateway vs a 403 from Lambda: if the Lambda log group shows no invocation, the 403 came from API Gateway (authorizer not attached). If the Lambda log shows an invocation that returned 403, the authorizer was attached but the token was invalid or missing.

Deployment

  • Mechanism: SAM
  • Deploy command:
    cd main/server
    sam build
    sam deploy
    
  • Notes:
  • Always add explicit Auth: Authorizer: CognitoAuth to new Event definitions before deploying. Do not rely on Global inheritance for new routes.
  • After deployment, verify in the API Gateway console that the route shows "CognitoAuth" in the Authorization column.
  • Endpoints requiring no auth must use Auth: Authorizer: NONE explicitly — omitting Auth on a public endpoint also risks inconsistent behavior.