iotlcm/worker-svr/internal/fullsync/contract/README.md

Full-sync shared contracts

This package establishes the boundary for later queue accounting, archive, replay, and HTTP work. It does not change running service behavior. Production code in this package does not import the queue package; compatibility tests compare its models and keys with the existing queue implementation.

Compatibility and counters

• RunMeta retains all existing JSON fields and adds optional fields. Existing zset.RunMeta, zset.Task, and zset.FailureRecord remain unchanged; adapters are a later integration step. The new RunStatus is a named string type.
• Validate accepts missing extension fields on old records. ValidateAccounting is opt-in for runs with known extended counters and enforces enqueuedCount = successCount + failedCount + activeCount + nextRoundCount. inflightCount is a subset of activeCount.
• Source cleanup changes pendingFailureCount, never historical failedCount.
• Round statistics have no task/card/channel dimensions. ErrorCounts normalizes both JSON input and output to eight canonical categories, merging unknown names into other and rejecting negative/overflowing totals.
• Existing failure payloads retain their free-form classification and message. The HTTP Failure projection normalizes categories and bounds messages to 2,048 UTF-8 bytes. Historical blocked tasks may have round zero.

Status domains

Domain	Wire values
Run	producing, running, completed, completed_with_failures
Sync type	normal, replay
Archive	disabled, pending, processing, succeeded, failed
Source replay disposition	none, pending, processing, completed, needs_attention
Replay action	accepted, processing, cleanup_pending, receipt_pending, completed, needs_attention
Error category	timeout, rate_limited, authentication, validation, upstream, transport, internal, other

The source field is replayDisposition, distinct from the replay action's status. Lease tokens, record versions, action revision, idempotency key, and request fingerprint are internal and excluded from HTTP JSON.

Interfaces and persistence obligations

HTTP consumers depend on RunReader, FailurePager, ReplayRequester, and ActionReader. Separate live/archive readers support fallback only after a live ErrNotFound; storage failures must remain visible. An archived summary cannot provide per-card details: use ErrFailureDetailsUnavailable when live details are gone. Repositories and HTTP both apply the default page limit 100 and maximum 500. Cursors remain opaque and are bound to the original query. An empty scan page with a next cursor does not signal completion.

Archive discovery, scheduling, snapshot loading, and result registration use separate narrow interfaces. Registration is conditional on an opaque work version. Archives contain run and round aggregates only. Archive receipts use a schema version, storage-owned path, lowercase SHA-256 checksum of stored bytes, and publication time. Archive schema version starts at 1.

Replay acceptance atomically reserves (sourceRunId, idempotencyKey) with a SHA-256 fingerprint of the source and exact requestedBy body value. Repeating the request returns its original action/target; changing the request returns ErrConflict. The action fixes its target ID before target creation/enqueue.

Replay work discovery, lease management, progress updates, source pagination, target creation/enqueue, and source cleanup are separate interfaces. Progress updates are fenced and use a revision check. Target enqueue distinguishes durably present duplicates from rejected writes; a zero insert count by itself does not authorize source cleanup. Cleanup checks exact source record versions and target acceptance evidence. Implementations must persist enough evidence to recover partial batches; a scan cursor by itself is insufficient. Source pagination must remain correct while already-enqueued entries are deleted.

ErrNotFound, ErrConflict, ErrInvalidArgument, ErrFailureDetailsUnavailable, ErrLeaseLost, and ErrBusy support errors.Is through wrapping. HTTP error envelopes and status mappings remain the HTTP implementation's responsibility.

Redis naming

All names are prefixed with the configured namespace (flow: by default; sim-status: for status sync). Existing run/round/task-state/failure names are preserved. New keys include:

Suffix after namespace	Scope
{runId}:round:N:stats	Run slot
{runId}:archive	Run slot
runs:all	Global index
archive:pending	Global index
{replay}:action:actionId:meta	Replay slot
{replay}:action:actionId:lease	Replay slot
{replay}:idempotency:SHA256(sourceRunId,key)	Replay slot; tuple encoded as a JSON array before hashing
{replay}:pending	Replay slot

Replay action creation, idempotency, and discovery can be atomic within their shared slot. Source/target runs and global indexes can occupy different slots; their coordination requires recoverable registrations/receipts rather than a single cross-slot Lua transaction. New identifier validation rejects hash-tag or path delimiters without rewriting legacy JSON on decode.

HTTP surface

Method	Route	Response
GET	/internal/v1/full-sync-runs?syncKind=&limit=	RunList merged from active Redis Runs and run-<runId>.json archives
GET	/internal/v1/full-sync-runs/{runId}	RunView with source, run, optional rounds
GET	/internal/v1/full-sync-runs/{runId}/failures?cursor=&limit=	FailurePage with items, nextCursor
POST	/internal/v1/full-sync-runs/{runId}/replay	HTTP 202 ReplayAccepted
GET	/internal/v1/full-sync-actions/{actionId}	ReplayAction

POST requires Idempotency-Key and JSON {"requestedBy":"..."}. Authentication, HTTP handlers, storage, replay execution, archive files, and service wiring are intentionally left to subsequent implementations.

Verification

gofmt -l internal/fullsync/contract
go test ./internal/fullsync/contract
go test ./internal/queue/zset ./internal/consumer/zset ./internal/producer/simstatus ./internal/producer/monthlyusage ./internal/service/simstatus ./internal/service/monthlyusage

Tests cover legacy schema and JSON compatibility, existing key compatibility, cluster key boundaries, bounded categories/messages, overflow, run accounting, pagination limits, status domains, replay identity, internal field exclusion, and receipt/progress validation. Regression tests use the repository's isolated Redis helper and require redis-server or REDIS_SERVER_BINARY.