API Gateway Audio and Video Endpoints Authorization
Metadata
- System type:
infrastructure
System Intent
- What this is: Authorization configuration pattern for API Gateway endpoints (
/memories/audio,/memories/audio-metadata,/memories/video) in the SAM template. Documents why the audio endpoints return 403 Forbidden when first deployed, the root cause (missing explicit Event-levelAuth:config), how the fix resolves it, and how this compares to the working/memories/videoendpoint.
Mermaid Diagram
flowchart TD
Client["Mobile client\nAuthorization: Bearer <token>"] -->|POST /memories/audio-metadata| APIGW["API Gateway\n(REST API)"]
APIGW -->|Checks route authorizer| AuthCheck{Route has\nCognitoAuth\nauthorizer?}
AuthCheck -->|Yes| Cognito["Cognito User Pool\n/encache/auth/user_pool_id"]
Cognito -->|Validates token,\ninjects claims| Lambda["MemoriesAudioMetadataFunction\napi/memories/audio/app.py"]
Lambda -->|DB query by user_id| DB["WorldMMSegment\n(PostgreSQL)"]
Lambda -->|generate_presigned_url| S3["encache-raw-memory\nsessions/.../audio.wav"]
Lambda -->|200 + presigned_url| Client
AuthCheck -->|No authorizer attached| Forbidden["403 Forbidden\n{message: Forbidden}\nLambda never invoked"] Flows
Flow: getAudioMetadata
- Core files:
main/server/api/memories/audio/app.py,main/server/template.yaml,main/server/layers/shared/python/shared/lambda_helpers.py - Frontend:
main/app/lib/api/memory/audio.ts
Types
AudioMetadataRequest {
memory_id: string (required) — source_session_id or segment id
}
AudioMetadataResponse {
presigned_url: string | null — null when processing is not yet complete
duration_seconds: number
processing_status: "pending" | "complete" | "failed"
}
StandardError {
status: number (HTTP status)
code: string (stable machine-readable code)
message: string (human-readable summary)
}
Paths
| path | input | output | path-type | notes |
|---|---|---|---|---|
getAudioMetadata.success | AudioMetadataRequest | AudioMetadataResponse | happy path | CognitoAuth authorizer attached to route |
getAudioMetadata.forbidden | any | {"message": "Forbidden"} status=403 | error | API Gateway returns 403 before Lambda invokes; authorizer not attached to route |
getAudioMetadata.not-found | AudioMetadataRequest | StandardError status=404 code=MEMORY_NOT_FOUND | error | No segment in DB for user_id + memory_id |
getAudioMetadata.audio-not-found | AudioMetadataRequest | StandardError status=404 code=AUDIO_NOT_FOUND | error | source_session_id or source_window_index missing on segment |
getAudioMetadata.pending | AudioMetadataRequest | AudioMetadataResponse presigned_url=null | happy path | processing_status != "complete"; presigned URL skipped |
Pseudocode
# API Gateway route for POST /memories/audio-metadata
# MUST have explicit Auth: Authorizer: CognitoAuth in template.yaml Event properties
# Without it, SAM may create the route without an authorizer → 403 from API Gateway
getAuthContext(event):
# API Gateway injects validated claims into requestContext.authorizer.claims
# This avoids a JWKS fetch inside the VPC (no NAT gateway)
agw_claims = event.requestContext.authorizer.claims
if agw_claims.sub:
return { user_id: agw_claims.sub, ... }
# Fallback: verify JWT directly (local testing only)
implementation(payload, auth):
memory_id = payload.memory_id
segment = query WorldMMSegment WHERE user_id=auth.user_id AND source_session_id=memory_id
if not segment: raise NotFoundError
if not segment.source_session_id or segment.source_window_index is None:
raise NotFoundError("AUDIO_NOT_FOUND")
audio_key = "sessions/{source_session_id}/window_{source_window_index:03d}/audio.wav"
if segment.processing_status == "complete":
head_object(audio_key) # raises 404 if missing
presigned_url = generate_presigned_url(audio_key, ExpiresIn=3600)
return { presigned_url, duration_seconds, processing_status }
Flow: getAudioPresignedUrl
- Core files:
main/server/api/memories/audio/app.py,main/server/template.yaml - Same handler (
api/memories/audio/app.py) serves both/memories/audioand/memories/audio-metadata
Types
Same as getAudioMetadata — both endpoints share the same Lambda handler.
Paths
| path | input | output | path-type | notes |
|---|---|---|---|---|
getAudioPresignedUrl.success | AudioMetadataRequest | AudioMetadataResponse | happy path | Same as getAudioMetadata |
getAudioPresignedUrl.forbidden | any | {"message": "Forbidden"} status=403 | error | Same root cause as getAudioMetadata |
Flow: getVideo
- Core files:
main/server/api/memories/video/app.py,main/server/template.yaml - Frontend:
main/app/lib/api/memory/video.ts
Types
VideoRequest {
memory_id: string (required)
}
VideoResponse {
presigned_url: string — empty string ("") when processing_status != "complete"
duration_seconds: number (30)
frame_count?: number — count of frame_*.jpg objects in S3; returned for both complete and non-complete responses when the segment has an s3_frames_key (always true in practice, since segments without one are filtered out before this response is built)
processing_status: "complete" | "pending" | "failed"
}
StandardError {
status: number (HTTP status)
code: string (stable machine-readable code)
message: string (human-readable summary)
}
Paths
| path | input | output | path-type | notes |
|---|---|---|---|---|
getVideo.success | VideoRequest, segment.processing_status = "complete" | VideoResponse presigned_url=<url>, processing_status="complete" | happy path | FFmpeg encodes frames + audio, uploads MP4, returns presigned URL |
getVideo.pending | VideoRequest, segment.processing_status != "complete" | VideoResponse presigned_url="", processing_status=<status> | happy path | skips encoding; counts frames via S3 list if s3_frames_key set; returns immediately |
getVideo.not-found | VideoRequest | StandardError status=404 code=MEMORY_NOT_FOUND | error | no segment with s3_frames_key for user |
getVideo.frames-not-found | VideoRequest | StandardError status=404 code=FRAMES_NOT_FOUND | error | segment complete but S3 has no frame objects |
getVideo.audio-not-found | VideoRequest | StandardError status=404 code=AUDIO_NOT_FOUND | error | audio.wav missing from S3 |
getVideo.encoding-failed | VideoRequest | StandardError status=500 code=ENCODING_FAILED | error | FFmpeg fails or times out at 90 s |
Pseudocode
implementation(payload, auth):
segment = _find_visual_segment(payload.memory_id, auth.user_id)
if not segment: raise NotFoundError("MEMORY_NOT_FOUND")
processing_status = segment.processing_status or "pending"
if processing_status != "complete":
# Count frames without downloading (lets frontend show metadata)
frame_count = count s3 objects at frames_dir/frame_* if s3_frames_key set
return {
presigned_url: "",
duration_seconds: 30.0,
processing_status: processing_status,
frame_count: frame_count, # omitted if None
}
# processing_status == "complete": download frames + audio, encode, upload, sign
frames_dir = segment.s3_frames_key.rsplit("/", 1)[0]
download all frame_*.jpg from S3 to tmpdir
download audio.wav from S3 to tmpdir
run FFmpeg: frames + audio → output.mp4 (libx264, aac, -shortest, timeout=90s)
upload output.mp4 to temp/videos/<uuid>.mp4
return {
presigned_url: generate_presigned_url(temp_key, ExpiresIn=3600),
duration_seconds: 30.0,
frame_count: len(frame_keys),
processing_status: "complete",
}
Authorization: Root Cause of 403 on Audio Endpoints
Why 403 Occurs
API Gateway checks whether each route has an authorizer attached. When a route is deployed without an authorizer, API Gateway returns 403 Forbidden immediately — before the Lambda is ever invoked. The client receives {"message": "Forbidden"} and CloudWatch shows no Lambda invocation.
Root Cause: Missing Event-Level Auth: in template.yaml
The SAM Globals define a default authorizer:
# main/server/template.yaml, lines 64-69
Globals:
Api:
Auth:
DefaultAuthorizer: CognitoAuth
Authorizers:
CognitoAuth:
UserPoolArn: !Sub "arn:aws:cognito-idp:${AWS::Region}:${AWS::AccountId}:userpool/{{resolve:ssm:/encache/auth/user_pool_id}}"
AddDefaultAuthorizerToCorsPreflight: false
The audio endpoint Events on the feature/fix-audio-403 branch are missing an explicit Auth: block:
# CURRENT (missing Auth — may not get CognitoAuth attached on first deploy)
Events:
MemoriesAudio:
Type: Api
Properties:
Path: /memories/audio
Method: post
# No Auth: block
MemoriesAudioMetadata:
Type: Api
Properties:
Path: /memories/audio-metadata
Method: post
# No Auth: block
SAM inherits DefaultAuthorizer from Globals in most cases, but this inheritance can fail on new routes being added to an existing API Gateway stage for the first time. The result: the route is created without the Cognito authorizer attached, and API Gateway returns 403 on every request.
Fix: Add Explicit Event-Level Auth
# FIXED
Events:
MemoriesAudio:
Type: Api
Properties:
Path: /memories/audio
Method: post
Auth:
Authorizer: CognitoAuth # explicit — no ambiguity on first deploy
MemoriesAudioMetadata:
Type: Api
Properties:
Path: /memories/audio-metadata
Method: post
Auth:
Authorizer: CognitoAuth # explicit — no ambiguity on first deploy
How the Fix Resolves It
With Auth: Authorizer: CognitoAuth explicit at the Event level, SAM generates a CloudFormation resource that directly attaches the named authorizer to the route — regardless of whether it is a first deploy or an update. API Gateway validates the token before invoking Lambda, injects requestContext.authorizer.claims, and the Lambda's getAuthContext() reads claims["sub"] as the user_id.
Comparison: Audio vs Video
| Aspect | /memories/video (working) | /memories/audio, /memories/audio-metadata (403 risk) |
|---|---|---|
| Lambda handler | api/memories/video/app.py | api/memories/audio/app.py |
require_auth_context | True | True |
| IAM Policies | DatabaseSsmPolicy, S3AccessPolicy | DatabaseSsmPolicy, S3AccessPolicy |
| VPC Config | Yes | Yes |
| Lambda Timeout | 120s | 120s |
| Lambda Memory | 1024 MB | 512 MB |
Event-level Auth: | Not set (relies on Global default) | Not set (relies on Global default) |
| Deployment status | On master — deployed; authorizer already attached | On feature branch — new routes; authorizer attachment depends on SAM inheritance |
| 403 risk | Low (authorizer already attached from prior deploys) | High (new routes may not inherit authorizer on first deploy) |
The video endpoint works despite also lacking explicit Event-level Auth: because its route was created in API Gateway during earlier deploys when the Global default applied correctly. Newly created routes for audio endpoints on a fresh branch deploy do not have this history.
Auth Flow After Fix
Client → POST /memories/audio-metadata (Authorization: Bearer <cognito-id-token>)
→ API Gateway: validates token against Cognito User Pool
→ API Gateway: injects requestContext.authorizer.claims = { sub, token_use, ... }
→ Lambda invoked with enriched event
→ getAuthContext(): reads agw_claims.sub as user_id (no JWKS fetch needed inside VPC)
→ implementation(): queries WorldMMSegment WHERE user_id=sub AND source_session_id=memory_id
→ returns { presigned_url, duration_seconds, processing_status }
To Make an Endpoint Public (No Auth)
Use Authorizer: NONE explicitly:
This pattern is used by /auth/start, /auth/verify, /auth/handoff, and OPTIONS preflight routes.
Logs
| Source | Location |
|---|---|
| Audio Lambda | CloudWatch: /aws/lambda/MemoriesAudioFunction |
| Audio Metadata Lambda | CloudWatch: /aws/lambda/MemoriesAudioMetadataFunction |
| Video Lambda | CloudWatch: /aws/lambda/MemoriesVideoFunction |
| API Gateway execution | CloudWatch: /aws/apigateway/<api-id>/<stage> |
To distinguish a 403 from API Gateway vs a 403 from Lambda: if the Lambda log group shows no invocation, the 403 came from API Gateway (authorizer not attached). If the Lambda log shows an invocation that returned 403, the authorizer was attached but the token was invalid or missing.
Deployment
- Mechanism:
SAM - Deploy command:
- Notes:
- Always add explicit
Auth: Authorizer: CognitoAuthto new Event definitions before deploying. Do not rely on Global inheritance for new routes. - After deployment, verify in the API Gateway console that the route shows "CognitoAuth" in the Authorization column.
- Endpoints requiring no auth must use
Auth: Authorizer: NONEexplicitly — omitting Auth on a public endpoint also risks inconsistent behavior.