chore: 并入 iotlcm-worker-svr 历史到 worker-svr/(原提交 96f0f4e)

This commit is contained in:
MiFaZhan
2026-11-02 13:28:03 +08:00
422 changed files with 80993 additions and 0 deletions
@@ -0,0 +1,124 @@
# Full-sync shared contracts
This package establishes the boundary for later queue accounting, archive,
replay, and HTTP work. It does not change running service behavior. Production
code in this package does not import the queue package; compatibility tests
compare its models and keys with the existing queue implementation.
## Compatibility and counters
- `RunMeta` retains all existing JSON fields and adds optional fields. Existing
`zset.RunMeta`, `zset.Task`, and `zset.FailureRecord` remain unchanged; adapters
are a later integration step. The new `RunStatus` is a named string type.
- `Validate` accepts missing extension fields on old records.
`ValidateAccounting` is opt-in for runs with known extended counters and
enforces `enqueuedCount = successCount + failedCount + activeCount + nextRoundCount`.
`inflightCount` is a subset of `activeCount`.
- Source cleanup changes `pendingFailureCount`, never historical `failedCount`.
- Round statistics have no task/card/channel dimensions. `ErrorCounts`
normalizes both JSON input and output to eight canonical categories, merging
unknown names into `other` and rejecting negative/overflowing totals.
- Existing failure payloads retain their free-form classification and message.
The HTTP `Failure` projection normalizes categories and bounds messages to
2,048 UTF-8 bytes. Historical blocked tasks may have round zero.
## Status domains
| Domain | Wire values |
| --- | --- |
| Run | `producing`, `running`, `completed`, `completed_with_failures` |
| Sync type | `normal`, `replay` |
| Archive | `disabled`, `pending`, `processing`, `succeeded`, `failed` |
| Source replay disposition | `none`, `pending`, `processing`, `completed`, `needs_attention` |
| Replay action | `accepted`, `processing`, `cleanup_pending`, `receipt_pending`, `completed`, `needs_attention` |
| Error category | `timeout`, `rate_limited`, `authentication`, `validation`, `upstream`, `transport`, `internal`, `other` |
The source field is `replayDisposition`, distinct from the replay action's
`status`. Lease tokens, record versions, action revision, idempotency key, and
request fingerprint are internal and excluded from HTTP JSON.
## Interfaces and persistence obligations
HTTP consumers depend on `RunReader`, `FailurePager`, `ReplayRequester`, and
`ActionReader`. Separate live/archive readers support fallback only after a live
`ErrNotFound`; storage failures must remain visible. An archived summary cannot
provide per-card details: use `ErrFailureDetailsUnavailable` when live details
are gone. Repositories and HTTP both apply the default page limit 100 and
maximum 500. Cursors remain opaque and are bound to the original query. An empty
scan page with a next cursor does not signal completion.
Archive discovery, scheduling, snapshot loading, and result registration use
separate narrow interfaces. Registration is conditional on an opaque work
version. Archives contain run and round aggregates only. Archive receipts use
a schema version, storage-owned path, lowercase SHA-256 checksum of stored
bytes, and publication time. Archive schema version starts at 1.
Replay acceptance atomically reserves `(sourceRunId, idempotencyKey)` with a
SHA-256 fingerprint of the source and exact requestedBy body value. Repeating
the request returns its original action/target; changing the request returns
`ErrConflict`. The action fixes its target ID before target creation/enqueue.
Replay work discovery, lease management, progress updates, source pagination,
target creation/enqueue, and source cleanup are separate interfaces. Progress
updates are fenced and use a revision check. Target enqueue distinguishes
durably present duplicates from rejected writes; a zero insert count by itself
does not authorize source cleanup. Cleanup checks exact source record versions
and target acceptance evidence. Implementations must persist enough evidence
to recover partial batches; a scan cursor by itself is insufficient. Source
pagination must remain correct while already-enqueued entries are deleted.
`ErrNotFound`, `ErrConflict`, `ErrInvalidArgument`,
`ErrFailureDetailsUnavailable`, `ErrLeaseLost`, and `ErrBusy` support
`errors.Is` through wrapping. HTTP error envelopes and status mappings remain
the HTTP implementation's responsibility.
## Redis naming
All names are prefixed with the configured namespace (`flow:` by default;
`sim-status:` for status sync). Existing run/round/task-state/failure names are
preserved. New keys include:
| Suffix after namespace | Scope |
| --- | --- |
| `{runId}:round:N:stats` | Run slot |
| `{runId}:archive` | Run slot |
| `runs:all` | Global index |
| `archive:pending` | Global index |
| `{replay}:action:actionId:meta` | Replay slot |
| `{replay}:action:actionId:lease` | Replay slot |
| `{replay}:idempotency:SHA256(sourceRunId,key)` | Replay slot; tuple encoded as a JSON array before hashing |
| `{replay}:pending` | Replay slot |
Replay action creation, idempotency, and discovery can be atomic within their
shared slot. Source/target runs and global indexes can occupy different slots;
their coordination requires recoverable registrations/receipts rather than a
single cross-slot Lua transaction. New identifier validation rejects hash-tag
or path delimiters without rewriting legacy JSON on decode.
## HTTP surface
| Method | Route | Response |
| --- | --- | --- |
| GET | `/internal/v1/full-sync-runs?syncKind=&limit=` | `RunList` merged from active Redis Runs and `run-<runId>.json` archives |
| GET | `/internal/v1/full-sync-runs/{runId}` | `RunView` with `source`, `run`, optional `rounds` |
| GET | `/internal/v1/full-sync-runs/{runId}/failures?cursor=&limit=` | `FailurePage` with `items`, `nextCursor` |
| POST | `/internal/v1/full-sync-runs/{runId}/replay` | HTTP 202 `ReplayAccepted` |
| GET | `/internal/v1/full-sync-actions/{actionId}` | `ReplayAction` |
POST requires `Idempotency-Key` and JSON `{"requestedBy":"..."}`. Authentication,
HTTP handlers, storage, replay execution, archive files, and service wiring are
intentionally left to subsequent implementations.
## Verification
```text
gofmt -l internal/fullsync/contract
go test ./internal/fullsync/contract
go test ./internal/queue/zset ./internal/consumer/zset ./internal/producer/simstatus ./internal/producer/monthlyusage ./internal/service/simstatus ./internal/service/monthlyusage
```
Tests cover legacy schema and JSON compatibility, existing key compatibility,
cluster key boundaries, bounded categories/messages, overflow, run accounting,
pagination limits, status domains, replay identity, internal field exclusion,
and receipt/progress validation. Regression tests use the repository's isolated
Redis helper and require `redis-server` or `REDIS_SERVER_BINARY`.
@@ -0,0 +1,46 @@
package contract
import "time"
const ArchiveSchemaVersion = 1
// ArchiveReceipt identifies a durably published archive object. Path is a
// storage-owned reference, never a client-supplied destination. Checksum is
// lowercase hex SHA-256 of the final stored bytes; version is the schema.
type ArchiveReceipt struct {
Version int `json:"version"`
Path string `json:"path"`
Checksum string `json:"checksum"`
ArchivedAt time.Time `json:"archivedAt"`
}
func (r ArchiveReceipt) Validate() error {
if r.Version < 1 || r.Path == "" || r.ArchivedAt.IsZero() || len(r.Checksum) != 64 {
return invalid("incomplete archive receipt")
}
for _, c := range r.Checksum {
if !((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f')) {
return invalid("archive checksum must be lowercase SHA-256 hex")
}
}
return nil
}
// RunArchive is summary-only: never embed per-card task/failure records.
type RunArchive struct {
Version int `json:"version"`
Run RunMeta `json:"run"`
Rounds []RoundStats `json:"rounds"`
}
// ArchiveWork.Version is an opaque concurrency token for registration; it is
// distinct from the archive schema version. Work discovery must survive restarts.
type ArchiveWork struct {
RunID string `json:"runId"`
Version string `json:"-"`
}
type ArchiveWorkPage struct {
Items []ArchiveWork `json:"items"`
NextCursor Cursor `json:"nextCursor"`
}
@@ -0,0 +1,107 @@
package contract_test
import (
"encoding/json"
"reflect"
"testing"
"time"
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract"
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset"
)
// Check every old field, including zero/omitempty behavior, so future adapters
// can adopt the contract without silently changing the persisted wire format.
func TestLegacyFieldSchemas(t *testing.T) {
for _, pair := range []struct{ old, next any }{
{zset.RunMeta{}, contract.RunMeta{}},
{zset.FailureRecord{}, contract.FailureRecord{}},
{zset.Task{}, contract.Task{}},
} {
old, next := reflect.TypeOf(pair.old), reflect.TypeOf(pair.next)
for i := 0; i < old.NumField(); i++ {
field := old.Field(i)
replacement, ok := next.FieldByName(field.Name)
if !ok || replacement.Tag.Get("json") != field.Tag.Get("json") || replacement.Type.Kind() != field.Type.Kind() {
t.Errorf("%s.%s lost its legacy schema: old=%+v new=%+v", old.Name(), field.Name, field, replacement)
}
}
}
}
func TestLegacyRunJSONRoundTrip(t *testing.T) {
now := time.Date(2026, 9, 20, 1, 2, 3, 456, time.UTC)
legacy := zset.RunMeta{
RunID: "legacy", Operation: "current_month_usage_full_sync", QueryMonth: "202609",
Status: "completed_with_failures", ProducerDone: true, CurrentRound: 5,
SuccessCount: 9, FailedCount: 1, CreatedAt: now, ProducerCompletedAt: now, CompletedAt: now,
}
data := mustJSON(t, legacy)
var decoded contract.RunMeta
if err := json.Unmarshal(data, &decoded); err != nil {
t.Fatal(err)
}
if err := decoded.Validate(); err != nil {
t.Fatalf("legacy run rejected: %v", err)
}
if got := string(mustJSON(t, decoded)); got != string(data) {
t.Fatalf("legacy wire format changed:\n%s\n%s", data, got)
}
// New fields do not prevent old readers from consuming the same metadata.
decoded.SyncType = contract.SyncReplay
decoded.SourceRunID = "source"
decoded.ReplayCycle = 1
decoded.EnqueuedCount = 10
decoded.PendingFailureCount = 1
var oldReader zset.RunMeta
if err := json.Unmarshal(mustJSON(t, decoded), &oldReader); err != nil {
t.Fatal(err)
}
oldReader.SyncType = ""
oldReader.SourceRunID = ""
oldReader.ReplayCycle = 0
oldReader.EnqueuedCount = 0
oldReader.PendingFailureCount = 0
if !reflect.DeepEqual(oldReader, legacy) {
t.Fatalf("old reader lost fields: %+v", oldReader)
}
}
func TestLegacyFailureAndTaskJSON(t *testing.T) {
legacy := zset.FailureRecord{
Version: 1, TaskID: "source:op:42", OriginalRunID: "source", LatestRunID: "latest",
QueryMonth: "202609", Operation: "op", CardID: 42, ICCID: "iccid", MSISDN: "msisdn", IMSI: "imsi",
ChanKey: "chan", TempKey: "template", Carrier: "mobile", FailureType: "mobile_busy_exhausted",
Round: 5, Attempts: 7, Code: 13000, ErrCode: "BUSY", Message: "try again",
ReviewStatus: "pending", ReviewRemark: "remark", ReviewedBy: "operator", ReviewedAt: "2026-09-20",
RetryCycle: 2, ReplayRunID: "replayed", FirstFailedAt: time.Unix(1, 0).UTC(), LastFailedAt: time.Unix(2, 0).UTC(),
}
var record contract.FailureRecord
if err := json.Unmarshal(mustJSON(t, legacy), &record); err != nil {
t.Fatal(err)
}
if string(mustJSON(t, record)) != string(mustJSON(t, legacy)) {
t.Fatal("failure record wire format changed")
}
if summary := record.Summary(); summary.ErrorCategory != contract.ErrorRateLimited || summary.TaskID != legacy.TaskID || summary.FailedAt != legacy.LastFailedAt {
t.Fatalf("incorrect failure projection: %+v", summary)
}
task := zset.Task{TaskID: "r:op:42", RunID: "r", Operation: "op", QueryMonth: "202609", ChanKey: "c", TempKey: "t", Operator: 2, CardID: 42, ICCID: "i", MSISDN: "m", IMSI: "s", NextExecuteAt: 1234}
var newTask contract.Task
if err := json.Unmarshal(mustJSON(t, task), &newTask); err != nil {
t.Fatal(err)
}
newTask.NextExecuteAt = 9999
if string(mustJSON(t, newTask)) != string(mustJSON(t, task)) {
t.Fatal("task wire format or scheduling exclusion changed")
}
}
func mustJSON(t *testing.T, value any) []byte {
t.Helper()
data, err := json.Marshal(value)
if err != nil {
t.Fatal(err)
}
return data
}
@@ -0,0 +1,14 @@
// Package contract defines the shared full-sync domain and service boundaries.
// It contains no Redis, filesystem, replay execution, or HTTP handler code.
//
// Legacy queue models remain in internal/queue/zset until adapters adopt this
// package. Their existing JSON fields are retained here; new run fields are
// optional so historical records still decode. Missing extended counters mean
// unknown, not a reconstructed total. FailedCount is the historical terminal
// count for a run; cleaning source failures changes PendingFailureCount only.
//
// Implementations must preserve these boundaries: run-local Redis keys share
// one hash tag, global indexes do not; archive summaries never contain card
// details; cursors are opaque and scoped to the original query; replay writes
// are idempotent and fenced by a lease before source cleanup is allowed.
package contract
@@ -0,0 +1,112 @@
package contract
import (
"encoding/json"
"math"
"strings"
"unicode/utf8"
)
type ErrorCategory string
const (
ErrorTimeout ErrorCategory = "timeout"
ErrorRateLimited ErrorCategory = "rate_limited"
ErrorAuthentication ErrorCategory = "authentication"
ErrorValidation ErrorCategory = "validation"
ErrorUpstream ErrorCategory = "upstream"
ErrorTransport ErrorCategory = "transport"
ErrorInternal ErrorCategory = "internal"
ErrorOther ErrorCategory = "other"
MaxErrorCategories = 8
MaxErrorMessageBytes = 2048
)
// NormalizeErrorCategory maps the existing consumer classifications into the
// approved, bounded vocabulary. Never use an upstream message/code as a key.
func NormalizeErrorCategory(value string) ErrorCategory {
switch strings.ToLower(strings.TrimSpace(value)) {
case "timeout":
return ErrorTimeout
case "rate_limited", "mobile_busy", "mobile_busy_exhausted", "channel_limited", "channel_blocked":
return ErrorRateLimited
case "authentication":
return ErrorAuthentication
case "validation":
return ErrorValidation
case "upstream", "business", "carrier_retry":
return ErrorUpstream
case "transport", "network":
return ErrorTransport
case "internal", "canceled":
return ErrorInternal
default:
return ErrorOther
}
}
func (c ErrorCategory) MarshalJSON() ([]byte, error) {
return json.Marshal(string(NormalizeErrorCategory(string(c))))
}
func (c *ErrorCategory) UnmarshalJSON(data []byte) error {
var value string
if err := json.Unmarshal(data, &value); err != nil {
return invalid("error category must be a string: %v", err)
}
*c = NormalizeErrorCategory(value)
return nil
}
// ErrorCounts is normalized on both JSON input and output, including map keys
// (encoding/json does not invoke ErrorCategory.MarshalJSON for map keys).
// Counts are per category, not card IDs or individual error messages.
type ErrorCounts map[ErrorCategory]int64
func (counts ErrorCounts) Normalized() (ErrorCounts, error) {
normalized := make(ErrorCounts)
for category, count := range counts {
key := NormalizeErrorCategory(string(category))
if count < 0 || count > math.MaxInt64-normalized[key] {
return nil, invalid("error counts must be nonnegative and must not overflow")
}
normalized[key] += count
}
return normalized, nil
}
func (counts ErrorCounts) MarshalJSON() ([]byte, error) {
normalized, err := counts.Normalized()
if err != nil {
return nil, err
}
type wireCounts map[ErrorCategory]int64
return json.Marshal(wireCounts(normalized))
}
func (counts *ErrorCounts) UnmarshalJSON(data []byte) error {
var decoded map[ErrorCategory]int64
if err := json.Unmarshal(data, &decoded); err != nil {
return invalid("invalid error counts: %v", err)
}
normalized, err := ErrorCounts(decoded).Normalized()
if err != nil {
return err
}
*counts = normalized
return nil
}
// BoundedErrorMessage makes new summary/action error messages valid UTF-8 and
// caps their byte length. It does not modify legacy FailureRecord.Message.
func BoundedErrorMessage(message string) string {
message = strings.ToValidUTF8(message, "\uFFFD")
if len(message) <= MaxErrorMessageBytes {
return message
}
end := MaxErrorMessageBytes
for !utf8.RuneStart(message[end]) {
end--
}
return message[:end]
}
@@ -0,0 +1,91 @@
package contract_test
import (
"encoding/json"
"errors"
"fmt"
"math"
"strings"
"testing"
"unicode/utf8"
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract"
)
func TestBoundedErrorCategories(t *testing.T) {
for raw, want := range map[string]contract.ErrorCategory{
"timeout": contract.ErrorTimeout, " RATE_LIMITED ": contract.ErrorRateLimited,
"authentication": contract.ErrorAuthentication, "validation": contract.ErrorValidation,
"upstream": contract.ErrorUpstream, "transport": contract.ErrorTransport,
"internal": contract.ErrorInternal, "other": contract.ErrorOther,
"network": contract.ErrorTransport, "mobile_busy": contract.ErrorRateLimited,
"mobile_busy_exhausted": contract.ErrorRateLimited, "channel_blocked": contract.ErrorRateLimited,
"channel_limited": contract.ErrorRateLimited, "business": contract.ErrorUpstream,
"carrier_retry": contract.ErrorUpstream, "canceled": contract.ErrorInternal,
"": contract.ErrorOther, "card-898600-unique-error": contract.ErrorOther,
} {
if got := contract.NormalizeErrorCategory(raw); got != want {
t.Errorf("normalize %q = %q, want %q", raw, got, want)
}
var decoded contract.ErrorCategory
if err := json.Unmarshal(mustJSON(t, raw), &decoded); err != nil || decoded != want {
t.Errorf("decode %q = %q, err %v", raw, decoded, err)
}
if got := string(mustJSON(t, contract.ErrorCategory(raw))); got != string(mustJSON(t, string(want))) {
t.Errorf("encode %q = %s", raw, got)
}
}
}
func TestErrorCountsCannotGrowWithCardinality(t *testing.T) {
counts := contract.ErrorCounts{contract.ErrorTimeout: 2, contract.ErrorOther: 3, "network": 4}
for i := 0; i < 1000; i++ {
counts[contract.ErrorCategory(fmt.Sprintf("card-error-%d", i))] = 1
}
var decoded contract.ErrorCounts
if err := json.Unmarshal(mustJSON(t, counts), &decoded); err != nil {
t.Fatal(err)
}
if len(decoded) != 3 || len(decoded) > contract.MaxErrorCategories || decoded[contract.ErrorOther] != 1003 || decoded[contract.ErrorTransport] != 4 {
t.Fatalf("unbounded or lost counts: %+v", decoded)
}
if err := json.Unmarshal([]byte(`{"unknown":2,"different":3,"other":4,"network":5}`), &decoded); err != nil {
t.Fatal(err)
}
if len(decoded) != 2 || decoded[contract.ErrorOther] != 9 || decoded[contract.ErrorTransport] != 5 {
t.Fatalf("input normalization failed: %+v", decoded)
}
}
func TestErrorCountsRejectInvalidNumbers(t *testing.T) {
for _, raw := range []string{`{"timeout":-1}`, `{"one":9223372036854775807,"two":1}`, `{"other":1.5}`, `{"other":"1"}`} {
var counts contract.ErrorCounts
if err := json.Unmarshal([]byte(raw), &counts); !errors.Is(err, contract.ErrInvalidArgument) {
t.Errorf("invalid input %s returned %v", raw, err)
}
}
for _, counts := range []contract.ErrorCounts{{"timeout": -1}, {"one": math.MaxInt64, "two": 1}} {
if _, err := json.Marshal(counts); !errors.Is(err, contract.ErrInvalidArgument) {
t.Errorf("invalid output counts returned %v", err)
}
}
}
func TestBoundedFailureMessage(t *testing.T) {
for _, message := range []string{"short", strings.Repeat("x", 3000), strings.Repeat("卡", 1000), strings.Repeat("x", 2047) + "卡", "invalid\xfftext"} {
got := contract.BoundedErrorMessage(message)
if len(got) > contract.MaxErrorMessageBytes || !utf8.ValidString(got) {
t.Fatalf("invalid bounded output: bytes=%d", len(got))
}
if utf8.ValidString(message) && len(message) <= contract.MaxErrorMessageBytes && got != message {
t.Fatal("short message changed")
}
}
record := contract.FailureRecord{TaskID: "task", Message: strings.Repeat("卡", 1000)}
if err := record.Summary().Validate(); err != nil {
t.Fatal(err)
}
if len(record.Message) != 3000 {
t.Fatal("summary mutated legacy failure message")
}
}
@@ -0,0 +1,129 @@
package contract
import (
"time"
"unicode/utf8"
)
// Cursor is an opaque repository-issued continuation token. Clients must not
// parse, increment, or reuse it for another run/filter. Empty starts a scan;
// an empty nextCursor ends it. An empty page with a cursor is not end-of-scan.
// Repositories validate token contents and query binding with ErrInvalidArgument.
type Cursor string
const (
DefaultPageLimit = 100
MaxPageLimit = 500
)
type PageRequest struct {
Cursor Cursor `json:"cursor,omitempty"`
Limit int `json:"limit,omitempty"`
}
// Normalized must be called at both HTTP and repository boundaries. Limits
// outside the bounds are rejected, not silently truncated.
func (p PageRequest) Normalized() (PageRequest, error) {
if p.Limit == 0 {
p.Limit = DefaultPageLimit
}
if p.Limit < 1 || p.Limit > MaxPageLimit {
return PageRequest{}, invalid("limit must be between 1 and %d", MaxPageLimit)
}
return p, nil
}
// Failure is the bounded public projection, not the replay payload. Legacy
// free-form failureType/message fields remain available in FailureRecord.
type Failure struct {
TaskID string `json:"taskId"`
CardID int64 `json:"cardId"`
ErrorCategory ErrorCategory `json:"errorCategory"`
ErrorMessage string `json:"errorMessage"`
FailedAt time.Time `json:"failedAt"`
Round int `json:"round"`
}
func (f Failure) Validate() error {
if f.TaskID == "" || f.Round < 0 || len(f.ErrorMessage) > MaxErrorMessageBytes || !utf8.ValidString(f.ErrorMessage) {
return invalid("invalid failure summary")
}
return nil
}
type FailurePage struct {
Items []Failure `json:"items"`
NextCursor Cursor `json:"nextCursor"`
}
// FailureRecord preserves the existing persisted payload without changing
// review semantics or discarding fields needed to reconstruct replay tasks.
type FailureRecord struct {
Version int `json:"version"`
TaskID string `json:"taskId"`
OriginalRunID string `json:"originalRunId"`
LatestRunID string `json:"latestRunId"`
QueryMonth string `json:"queryMonth"`
Operation string `json:"operation"`
CardID int64 `json:"cardId"`
ICCID string `json:"iccid"`
MSISDN string `json:"msisdn"`
IMSI string `json:"imsi"`
ChanKey string `json:"chanKey"`
TempKey string `json:"tempKey"`
// PackageCode is additive; older failure records decode it as empty.
PackageCode string `json:"packageCode,omitempty"`
Carrier string `json:"carrier"`
FailureType string `json:"failureType"`
Round int `json:"round"`
Attempts int `json:"attempts"`
Code int `json:"code"`
ErrCode string `json:"errCode"`
Message string `json:"message"`
ReviewStatus string `json:"reviewStatus"`
ReviewRemark string `json:"reviewRemark"`
ReviewedBy string `json:"reviewedBy"`
ReviewedAt string `json:"reviewedAt"`
RetryCycle int `json:"retryCycle"`
ReplayRunID string `json:"replayRunId"`
FirstFailedAt time.Time `json:"firstFailedAt"`
LastFailedAt time.Time `json:"lastFailedAt"`
}
func (f FailureRecord) Summary() Failure {
return Failure{
TaskID: f.TaskID, CardID: f.CardID, ErrorCategory: NormalizeErrorCategory(f.FailureType),
ErrorMessage: BoundedErrorMessage(f.Message), FailedAt: f.LastFailedAt, Round: f.Round,
}
}
// FailureEntry carries an opaque version for conditional source cleanup. It
// must identify the precise record read, not just its reusable task ID.
type FailureEntry struct {
Record FailureRecord `json:"record"`
Version string `json:"-"`
}
type FailureRecordPage struct {
Items []FailureEntry `json:"items"`
NextCursor Cursor `json:"nextCursor"`
}
// Task keeps the existing queue member serialization. The scheduling score
// must never be part of the JSON member or producer retries lose idempotency.
type Task struct {
TaskID string `json:"taskId"`
RunID string `json:"runId"`
Operation string `json:"operation"`
QueryMonth string `json:"queryMonth"`
ChanKey string `json:"chanKey"`
TempKey string `json:"tempKey"`
Operator int `json:"operator,omitempty"`
CardID int64 `json:"cardId"`
ICCID string `json:"iccid"`
MSISDN string `json:"msisdn"`
IMSI string `json:"imsi"`
// PackageCode is additive; older members decode it as empty.
PackageCode string `json:"packageCode,omitempty"`
NextExecuteAt int64 `json:"-"`
}
@@ -0,0 +1,28 @@
package contract
import "net/url"
const (
ListRunsRoute = "/internal/v1/full-sync-runs"
GetRunRoute = "/internal/v1/full-sync-runs/{runId}"
GetFailuresRoute = "/internal/v1/full-sync-runs/{runId}/failures"
PostReplayRoute = "/internal/v1/full-sync-runs/{runId}/replay"
GetActionRoute = "/internal/v1/full-sync-actions/{actionId}"
IdempotencyHeader = "Idempotency-Key"
)
// ReplayAccepted is the 202 response from a successful ReplayRequester call.
// A repeated request returns the original action and its current status.
type ReplayAccepted struct {
ActionID string `json:"actionId"`
TargetRunID string `json:"targetRunId"`
Status ActionStatus `json:"status"`
StatusURL string `json:"statusUrl"`
}
func ActionStatusURL(actionID string) (string, error) {
if err := ValidateID(actionID); err != nil {
return "", err
}
return "/internal/v1/full-sync-actions/" + url.PathEscape(actionID), nil
}
@@ -0,0 +1,145 @@
package contract
import (
"context"
"time"
)
// RunReader is the HTTP lookup boundary. Fall back to archive only when the
// live reader returns ErrNotFound, not on transient live storage failures.
type RunReader interface {
GetRun(ctx context.Context, runID string) (RunView, error)
}
type RunListFilter struct {
SyncKind SyncKind
Limit int
}
type RunList struct {
Items []RunView `json:"items"`
Total int `json:"total"`
}
type RunLister interface {
ListRuns(ctx context.Context, filter RunListFilter) (RunList, error)
}
type LiveRunReader interface {
GetRunMeta(ctx context.Context, runID string) (RunMeta, error)
}
type ArchivedRunReader interface {
GetArchivedRun(ctx context.Context, runID string) (RunArchive, error)
}
type RoundStatsReader interface {
GetRoundStats(ctx context.Context, runID string) ([]RoundStats, error)
}
// FailurePager returns live details only. An archived summary without live
// details returns ErrFailureDetailsUnavailable, never a misleading empty page.
// Items must serialize as [] on empty pages; NextCursor alone signals EOF.
type FailurePager interface {
ListFailures(ctx context.Context, runID string, page PageRequest) (FailurePage, error)
}
type ReplayRequester interface {
RequestReplay(ctx context.Context, request ReplayRequest) (ReplayAccepted, error)
}
type ActionReader interface {
GetReplayAction(ctx context.Context, actionID string) (ReplayAction, error)
}
// ArchiveWorkDiscovery returns pending/retryable work from a durable index.
// Terminal run registration must not depend on an in-memory notification.
type ArchiveWorkDiscovery interface {
ListArchiveWork(ctx context.Context, page PageRequest) (ArchiveWorkPage, error)
}
// RegisterArchiveWork idempotently schedules a terminal run/version. Since the
// index can occupy another Redis slot, callers must reconcile registration
// after a crash between completing the run and updating the index.
type ArchiveWorkRegistrar interface {
RegisterArchiveWork(ctx context.Context, work ArchiveWork) error
}
type ArchiveWorkLoader interface {
LoadArchiveWork(ctx context.Context, work ArchiveWork) (RunArchive, error)
}
// ArchiveRegistrar conditionally records a durable publication against the
// work version. Repeating the same receipt is a no-op; a different version or
// conflicting receipt returns ErrConflict. Failure retains discoverable work.
type ArchiveRegistrar interface {
RegisterArchive(ctx context.Context, work ArchiveWork, receipt ArchiveReceipt) error
RegisterArchiveFailure(ctx context.Context, work ArchiveWork, message string) error
}
// ReplayActionCreator atomically reserves (sourceRunId, idempotencyKey) with
// its fingerprint and action. Same key/request returns the original action
// regardless of candidate IDs; different request returns ErrConflict. The
// candidate starts accepted and fixes the target ID before any target writes.
type ReplayActionCreator interface {
GetOrCreateReplayAction(ctx context.Context, request ReplayRequest, candidate ReplayAction) (ReplayActionCreation, error)
}
// Discovery includes accepted, processing, cleanup_pending and receipt_pending
// actions after crashes. Completed/needs_attention actions require no automatic
// work. Lease acquisition, not listing, chooses an exclusive owner.
type ReplayWorkDiscovery interface {
ListReplayWork(ctx context.Context, page PageRequest) (ReplayActionPage, error)
}
type ReplayLeaseStore interface {
AcquireReplayLease(ctx context.Context, actionID, owner string, ttl time.Duration) (ReplayLease, error)
RenewReplayLease(ctx context.Context, lease ReplayLease, ttl time.Duration) (ReplayLease, error)
ReleaseReplayLease(ctx context.Context, lease ReplayLease) error
}
// All updates are fenced and compare ExpectedRevision. Expired/replaced leases
// return ErrLeaseLost; revision conflicts return ErrConflict. Implementations
// reject progress regression and preserve immutable action identity.
type ReplayProgressStore interface {
UpdateReplayAction(ctx context.Context, update ReplayActionUpdate) (ReplayAction, error)
}
// The implementation must preserve scan continuity while source entries are
// cleaned. This can use a durable selection/snapshot; deleting records must not
// make a later cursor skip an unprocessed failure. Tokens are bound to actionID.
type ReplayFailurePager interface {
ListReplayFailures(ctx context.Context, actionID string, page PageRequest) (FailureRecordPage, error)
}
// TargetRunCreator is idempotent: retrying the same target preserves counters;
// incompatible source/operation/month/cycle returns ErrConflict. Replay targets
// are distinct runs and must not mutate the source's historical FailedCount.
type TargetRunCreator interface {
EnsureTargetRun(ctx context.Context, meta RunMeta) (RunMeta, error)
}
// TargetEnqueueResult distinguishes a durable duplicate from a rejected write.
// A zero NewlyEnqueued count alone is never sufficient evidence for cleanup.
// Every input TaskID must be durably present on nil error; otherwise return an
// error and let the caller safely retry the identical batch. Both counts are
// unique task counts, and their sum equals the number of distinct input IDs.
type TargetEnqueueResult struct {
NewlyEnqueued int64
AlreadyPresent int64
}
type TargetRunEnqueuer interface {
// Tasks must match the target run and channel and use stable logical IDs.
EnqueueTargetBatch(ctx context.Context, lease ReplayLease, runID, chanKey string, tasks []Task, nextExecuteAt time.Time) (TargetEnqueueResult, error)
MarkTargetProducerDone(ctx context.Context, lease ReplayLease, runID string) error
}
// Cleanup is idempotent and conditional on the lease, exact source versions,
// and durable evidence of target acceptance. It changes pending failures only,
// never historical failedCount. Conflicts must remain visible for attention.
// Source keys and action/target keys may occupy different Redis Cluster slots;
// implementations need recoverable receipts, not cross-slot Lua assumptions.
type ReplayFailureCleaner interface {
CleanupReplayFailures(ctx context.Context, cleanup ReplayCleanup) (ReplayCleanupResult, error)
}
@@ -0,0 +1,116 @@
package contract
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"strings"
)
const (
DefaultFlowPrefix = "flow:"
DefaultSimStatusPrefix = "sim-status:"
)
// Keys is a naming contract only. It does not create indexes or storage.
// Existing run names are byte-for-byte compatible with zset.Store. Global
// run/archive indexes are outside run hash slots and cannot join run-local Lua.
type Keys struct{ prefix string }
func NewKeys(prefix string) (Keys, error) {
if strings.TrimSpace(prefix) == "" {
prefix = DefaultFlowPrefix
}
if strings.ContainsAny(prefix, "{}") {
return Keys{}, invalid("prefix must not contain Redis hash tags")
}
return Keys{prefix: prefix}, nil
}
// The zero value uses the legacy flow prefix, just like NewKeys("").
func (k Keys) Prefix() string {
if k.prefix == "" {
return DefaultFlowPrefix
}
return k.prefix
}
func (k Keys) runKey(runID, suffix string) (string, error) {
if err := ValidateID(runID); err != nil {
return "", err
}
return k.Prefix() + "{" + runID + "}:" + suffix, nil
}
func (k Keys) RunMetaKey(runID string) (string, error) { return k.runKey(runID, "meta") }
func (k Keys) RunChannelsKey(runID string) (string, error) { return k.runKey(runID, "channels") }
func (k Keys) TaskStateKey(runID string) (string, error) { return k.runKey(runID, "task-state") }
func (k Keys) FailedKey(runID string) (string, error) { return k.runKey(runID, "failed") }
func (k Keys) ArchiveStateKey(runID string) (string, error) { return k.runKey(runID, "archive") }
func (k Keys) roundKey(runID string, round int, suffix string) (string, error) {
if round < 1 {
return "", invalid("round must be positive")
}
return k.runKey(runID, fmt.Sprintf("round:%d:%s", round, suffix))
}
func (k Keys) RoundStatsKey(runID string, round int) (string, error) {
return k.roundKey(runID, round, "stats")
}
func (k Keys) roundChannelKey(runID string, round int, kind, chanKey string) (string, error) {
if strings.TrimSpace(chanKey) == "" {
return "", invalid("chanKey is empty")
}
return k.roundKey(runID, round, kind+":"+chanKey)
}
func (k Keys) RoundActiveKey(runID string, round int, chanKey string) (string, error) {
return k.roundChannelKey(runID, round, "active", chanKey)
}
func (k Keys) RoundWaitingKey(runID string, round int, chanKey string) (string, error) {
return k.roundChannelKey(runID, round, "waiting", chanKey)
}
func (k Keys) RoundLeaseKey(runID string, round int, chanKey string) (string, error) {
return k.roundChannelKey(runID, round, "lease", chanKey)
}
func (k Keys) FailedRunsKey() string { return k.Prefix() + "failed:runs" }
func (k Keys) ActiveRunsKey() string { return k.Prefix() + "runs:active" }
func (k Keys) AllRunsKey() string { return k.Prefix() + "runs:all" }
func (k Keys) PendingArchivesKey() string { return k.Prefix() + "archive:pending" }
// Replay action state, leases, idempotency reservations and pending index share
// a dedicated slot, allowing atomic acceptance/discovery. Source/target run
// writes remain in their own slots and require a recoverable replay protocol.
func (k Keys) replayKey(actionID, suffix string) (string, error) {
if err := ValidateID(actionID); err != nil {
return "", err
}
return k.Prefix() + "{replay}:action:" + actionID + ":" + suffix, nil
}
func (k Keys) ReplayActionKey(actionID string) (string, error) { return k.replayKey(actionID, "meta") }
func (k Keys) ReplayLeaseKey(actionID string) (string, error) { return k.replayKey(actionID, "lease") }
func (k Keys) PendingReplayActionsKey() string { return k.Prefix() + "{replay}:pending" }
// Hash the complete tuple so neither delimiter collisions nor arbitrary
// Idempotency-Key characters can change the Redis key or hash slot.
func (k Keys) ReplayIdempotencyKey(sourceRunID, idempotencyKey string) (string, error) {
if err := ValidateID(sourceRunID); err != nil {
return "", err
}
if err := ValidateIdempotencyKey(idempotencyKey); err != nil {
return "", err
}
data, err := json.Marshal([2]string{sourceRunID, idempotencyKey})
if err != nil {
return "", invalid("cannot encode idempotency scope: %v", err)
}
digest := sha256.Sum256(data)
return k.Prefix() + "{replay}:idempotency:" + hex.EncodeToString(digest[:]), nil
}
@@ -0,0 +1,116 @@
package contract_test
import (
"context"
"errors"
"strings"
"testing"
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract"
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/queue/zset"
"github.com/gogf/gf/v2/container/gvar"
)
type unusedRedis struct{}
func (unusedRedis) Do(context.Context, string, ...any) (*gvar.Var, error) {
panic("naming contracts must not access Redis")
}
func (unusedRedis) Eval(context.Context, string, int64, []string, []any) (*gvar.Var, error) {
panic("naming contracts must not access Redis")
}
func TestKeysMatchExistingQueue(t *testing.T) {
for _, prefix := range []string{"", "flow:", "sim-status:", "custom-prefix"} {
keys, err := contract.NewKeys(prefix)
if err != nil {
t.Fatal(err)
}
store, err := zset.New(unusedRedis{}, prefix)
if err != nil {
t.Fatal(err)
}
for _, pair := range []struct {
old, next func(string) (string, error)
}{
{store.RunMetaKey, keys.RunMetaKey}, {store.RunChannelsKey, keys.RunChannelsKey},
{store.TaskStateKey, keys.TaskStateKey}, {store.FailedKey, keys.FailedKey},
} {
old, oldErr := pair.old("run:1")
next, nextErr := pair.next("run:1")
if oldErr != nil || nextErr != nil || old != next || !strings.Contains(next, "{run:1}") {
t.Errorf("run naming changed: old=%q new=%q errors=%v/%v", old, next, oldErr, nextErr)
}
}
for _, pair := range []struct {
old, next func(string, int, string) (string, error)
}{
{store.RoundActiveKey, keys.RoundActiveKey}, {store.RoundWaitingKey, keys.RoundWaitingKey},
{store.RoundLeaseKey, keys.RoundLeaseKey},
} {
old, oldErr := pair.old("run:1", 3, "channel")
next, nextErr := pair.next("run:1", 3, "channel")
if oldErr != nil || nextErr != nil || old != next {
t.Errorf("round naming changed: old=%q new=%q errors=%v/%v", old, next, oldErr, nextErr)
}
}
if keys.ActiveRunsKey() != store.ActiveRunsKey() || keys.FailedRunsKey() != store.FailedRunsKey() {
t.Fatal("existing global index names changed")
}
}
}
func TestNewKeysAndClusterBoundaries(t *testing.T) {
keys, _ := contract.NewKeys("flow:")
stats, _ := keys.RoundStatsKey("run", 2)
archive, _ := keys.ArchiveStateKey("run")
if stats != "flow:{run}:round:2:stats" || archive != "flow:{run}:archive" {
t.Fatalf("new run keys changed: %s, %s", stats, archive)
}
action, _ := keys.ReplayActionKey("a")
lease, _ := keys.ReplayLeaseKey("a")
idempotency, _ := keys.ReplayIdempotencyKey("source", "a{different-slot}:key")
for _, key := range []string{action, lease, idempotency, keys.PendingReplayActionsKey()} {
if !strings.HasPrefix(key, "flow:{replay}:") || strings.Count(key, "{") != 1 {
t.Fatalf("replay key escaped action slot: %s", key)
}
}
if action != "flow:{replay}:action:a:meta" || lease != "flow:{replay}:action:a:lease" {
t.Fatal("action naming changed")
}
for _, key := range []string{keys.ActiveRunsKey(), keys.FailedRunsKey(), keys.AllRunsKey(), keys.PendingArchivesKey()} {
if strings.ContainsAny(key, "{}") {
t.Fatalf("global index unexpectedly shares run slot: %s", key)
}
}
first, _ := keys.ReplayIdempotencyKey("a:b", "c")
second, _ := keys.ReplayIdempotencyKey("a", "b:c")
repeat, _ := keys.ReplayIdempotencyKey("a:b", "c")
otherSource, _ := keys.ReplayIdempotencyKey("other", "c")
if first == second || first == otherSource || first != repeat {
t.Fatal("idempotency scope is ambiguous or unstable")
}
}
func TestInvalidKeyParts(t *testing.T) {
keys := contract.Keys{}
for _, id := range []string{"", " ", "trim ", "{run}", "run}", "run/path", "run\\path", "run\n", "invalid\xff"} {
if _, err := keys.RunMetaKey(id); !errors.Is(err, contract.ErrInvalidArgument) {
t.Errorf("invalid ID %q returned %v", id, err)
}
}
if _, err := contract.NewKeys("prefix:{wrong}:"); !errors.Is(err, contract.ErrInvalidArgument) {
t.Fatalf("invalid prefix returned %v", err)
}
if _, err := keys.RoundStatsKey("run", 0); !errors.Is(err, contract.ErrInvalidArgument) {
t.Fatalf("invalid round returned %v", err)
}
if _, err := keys.RoundActiveKey("run", 1, " "); !errors.Is(err, contract.ErrInvalidArgument) {
t.Fatalf("invalid channel returned %v", err)
}
if key, err := keys.RunMetaKey("run"); err != nil || key != "flow:{run}:meta" {
t.Fatalf("zero-value keys differ from legacy default: %s %v", key, err)
}
}
@@ -0,0 +1,172 @@
package contract
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"strings"
"time"
"unicode"
"unicode/utf8"
)
const MaxIdempotencyKeyBytes = 256
// ReplayRequest combines the path, header and body. Only requestedBy is in the
// HTTP body. Idempotency keys are scoped to the source run; fingerprints cover
// the source and exact request body, not the key or a generated target run ID.
type ReplayRequest struct {
SourceRunID string `json:"-"`
IdempotencyKey string `json:"-"`
RequestedBy string `json:"requestedBy"`
}
func ValidateIdempotencyKey(key string) error {
if strings.TrimSpace(key) == "" || len(key) > MaxIdempotencyKeyBytes || !utf8.ValidString(key) || strings.ContainsFunc(key, unicode.IsControl) {
return invalid("invalid Idempotency-Key")
}
return nil
}
func (r ReplayRequest) Validate() error {
if err := ValidateID(r.SourceRunID); err != nil {
return err
}
if err := ValidateIdempotencyKey(r.IdempotencyKey); err != nil {
return err
}
if strings.TrimSpace(r.RequestedBy) == "" || !utf8.ValidString(r.RequestedBy) {
return invalid("requestedBy is required")
}
return nil
}
func (r ReplayRequest) Fingerprint() (string, error) {
if err := r.Validate(); err != nil {
return "", err
}
// JSON avoids ambiguous delimiter concatenation and fixes field ordering.
data, err := json.Marshal(struct {
SourceRunID string `json:"sourceRunId"`
RequestedBy string `json:"requestedBy"`
}{r.SourceRunID, r.RequestedBy})
if err != nil {
return "", invalid("cannot encode replay request: %v", err)
}
digest := sha256.Sum256(data)
return hex.EncodeToString(digest[:]), nil
}
// Counters describe distinct source failures, not scan visits or retry calls.
// ScanCursor alone is not an enqueue/cleanup checkpoint: repositories must
// retain sufficient durable per-batch evidence to resume partial writes.
type ReplayProgress struct {
ScanCursor Cursor `json:"scanCursor"`
DiscoveredCount int64 `json:"discoveredCount"`
EnqueuedCount int64 `json:"enqueuedCount"`
CleanedCount int64 `json:"cleanedCount"`
FailureCount int64 `json:"failureCount"`
}
func (p ReplayProgress) Validate() error {
if err := nonnegative(p.DiscoveredCount, p.EnqueuedCount, p.CleanedCount, p.FailureCount); err != nil {
return err
}
if p.EnqueuedCount > p.DiscoveredCount || p.CleanedCount > p.EnqueuedCount {
return invalid("replay progress must satisfy cleaned <= enqueued <= discovered")
}
return nil
}
type ReplayAction struct {
ActionID string `json:"actionId"`
SourceRunID string `json:"sourceRunId"`
TargetRunID string `json:"targetRunId"`
RequestedBy string `json:"requestedBy"`
IdempotencyKey string `json:"-"`
Fingerprint string `json:"-"`
Status ActionStatus `json:"status"`
ReplayCycle int `json:"replayCycle"`
Progress ReplayProgress `json:"progress"`
LeaseOwner string `json:"-"`
LeaseToken string `json:"-"`
LeaseUntil time.Time `json:"-"`
Revision uint64 `json:"-"`
Error string `json:"error,omitempty"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
CompletedAt *time.Time `json:"completedAt,omitempty"`
Receipt *ArchiveReceipt `json:"receipt,omitempty"`
}
func (a ReplayAction) Validate() error {
for _, id := range []string{a.ActionID, a.SourceRunID, a.TargetRunID} {
if err := ValidateID(id); err != nil {
return err
}
}
if a.SourceRunID == a.TargetRunID || a.ReplayCycle < 1 || !a.Status.Valid() || strings.TrimSpace(a.RequestedBy) == "" {
return invalid("invalid replay action identity or status")
}
if len(a.Error) > MaxErrorMessageBytes || !utf8.ValidString(a.Error) {
return invalid("action error exceeds message bounds")
}
if err := a.Progress.Validate(); err != nil {
return err
}
if a.Receipt != nil {
if err := a.Receipt.Validate(); err != nil {
return err
}
}
if a.Status == ActionCompleted && (a.Receipt == nil || a.CompletedAt == nil || a.CompletedAt.IsZero()) {
return invalid("completed action requires a durable receipt and completedAt")
}
return nil
}
type ReplayActionPage struct {
Items []ReplayAction `json:"items"`
NextCursor Cursor `json:"nextCursor"`
}
type ReplayActionCreation struct {
Action ReplayAction
Created bool
}
// Lease token fences a particular acquisition, even if the same owner name is
// reused. Repository methods check expiry against their authoritative clock.
type ReplayLease struct {
ActionID string
Owner string
Token string
Until time.Time
}
// ReplayActionUpdate is a compare-and-swap of mutable action state. Identity,
// provenance, request fingerprint, and target run must never be changed.
type ReplayActionUpdate struct {
Lease ReplayLease
ExpectedRevision uint64
Status ActionStatus
Progress ReplayProgress
Error string
Receipt *ArchiveReceipt
CompletedAt *time.Time
}
// Cleanup contains the exact source versions durably accepted by the target.
// The engine must not request cleanup until target enqueue is confirmed.
type ReplayCleanup struct {
Lease ReplayLease
SourceRunID string
TargetRunID string
Entries []FailureEntry
}
type ReplayCleanupResult struct {
CleanedCount int64
AlreadyCleanedCount int64
ConflictCount int64
}
@@ -0,0 +1,103 @@
package contract_test
import (
"encoding/json"
"errors"
"reflect"
"strings"
"testing"
"time"
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract"
)
func TestReplayRequestIdentity(t *testing.T) {
request := contract.ReplayRequest{SourceRunID: "source", IdempotencyKey: "key", RequestedBy: "operator"}
fingerprint, err := request.Fingerprint()
if err != nil || len(fingerprint) != 64 {
t.Fatalf("fingerprint=%q, err=%v", fingerprint, err)
}
for name, mutate := range map[string]func(*contract.ReplayRequest){
"same": func(*contract.ReplayRequest) {},
"key": func(r *contract.ReplayRequest) { r.IdempotencyKey = "another-key" },
"source": func(r *contract.ReplayRequest) { r.SourceRunID = "another-source" },
"body": func(r *contract.ReplayRequest) { r.RequestedBy = "another-operator" },
} {
copy := request
mutate(&copy)
got, err := copy.Fingerprint()
if err != nil || (got == fingerprint) != (name == "same" || name == "key") {
t.Errorf("%s fingerprint=%s err=%v", name, got, err)
}
}
if got := string(mustJSON(t, request)); got != `{"requestedBy":"operator"}` {
t.Fatalf("path/header leaked into body: %s", got)
}
for _, bad := range []contract.ReplayRequest{
{SourceRunID: "source", RequestedBy: "operator"},
{SourceRunID: "source", IdempotencyKey: "key", RequestedBy: " "},
{SourceRunID: "{source}", IdempotencyKey: "key", RequestedBy: "operator"},
{SourceRunID: "source", IdempotencyKey: "key\r\ninjected", RequestedBy: "operator"},
{SourceRunID: "source", IdempotencyKey: strings.Repeat("x", 257), RequestedBy: "operator"},
} {
if _, err := bad.Fingerprint(); !errors.Is(err, contract.ErrInvalidArgument) {
t.Errorf("bad request returned %v", err)
}
}
}
func TestActionProgressAndReceiptValidation(t *testing.T) {
for _, progress := range []contract.ReplayProgress{
{DiscoveredCount: -1}, {DiscoveredCount: 1, EnqueuedCount: 2},
{DiscoveredCount: 2, EnqueuedCount: 1, CleanedCount: 2}, {FailureCount: -1},
} {
if err := progress.Validate(); !errors.Is(err, contract.ErrInvalidArgument) {
t.Fatalf("bad progress %+v returned %v", progress, err)
}
}
now := time.Now().UTC()
action := contract.ReplayAction{ActionID: "action", SourceRunID: "source", TargetRunID: "target", RequestedBy: "operator", ReplayCycle: 1, Status: contract.ActionAccepted}
if err := action.Validate(); err != nil {
t.Fatal(err)
}
action.Status = contract.ActionCompleted
if err := action.Validate(); !errors.Is(err, contract.ErrInvalidArgument) {
t.Fatalf("completed action without receipt returned %v", err)
}
action.Receipt = &contract.ArchiveReceipt{Version: 1, Path: "receipt.json", Checksum: strings.Repeat("a", 64), ArchivedAt: now}
action.CompletedAt = &now
if err := action.Validate(); err != nil {
t.Fatal(err)
}
action.Receipt.Checksum = "not-a-checksum"
if err := action.Validate(); !errors.Is(err, contract.ErrInvalidArgument) {
t.Fatalf("bad receipt returned %v", err)
}
}
func TestActionWireFormatHidesInternalState(t *testing.T) {
action := contract.ReplayAction{
ActionID: "action", SourceRunID: "source", TargetRunID: "target", RequestedBy: "operator", ReplayCycle: 1,
Status: contract.ActionCleanupPending, IdempotencyKey: "secret", Fingerprint: "internal-fingerprint",
LeaseOwner: "worker", LeaseToken: "token", LeaseUntil: time.Now(), Revision: 9,
Progress: contract.ReplayProgress{ScanCursor: "opaque", DiscoveredCount: 10, EnqueuedCount: 8, CleanedCount: 7, FailureCount: 2},
}
data := mustJSON(t, action)
for _, hidden := range []string{"idempotency", "fingerprint", "lease", "revision", "secret", "token"} {
if strings.Contains(strings.ToLower(string(data)), hidden) {
t.Errorf("internal %s leaked: %s", hidden, data)
}
}
var decoded contract.ReplayAction
if err := json.Unmarshal(data, &decoded); err != nil || !reflect.DeepEqual(decoded.Progress, action.Progress) || decoded.Status != contract.ActionCleanupPending {
t.Fatalf("action roundtrip lost state: %+v, %v", decoded, err)
}
statusURL, err := contract.ActionStatusURL("action%1")
if err != nil || statusURL != "/internal/v1/full-sync-actions/action%251" {
t.Fatalf("status URL=%s, %v", statusURL, err)
}
accepted := contract.ReplayAccepted{ActionID: "action", TargetRunID: "target", Status: contract.ActionAccepted, StatusURL: "/internal/v1/full-sync-actions/action"}
if got := string(mustJSON(t, accepted)); got != `{"actionId":"action","targetRunId":"target","status":"accepted","statusUrl":"/internal/v1/full-sync-actions/action"}` {
t.Fatalf("202 contract changed: %s", got)
}
}
@@ -0,0 +1,146 @@
package contract
import (
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/supply"
"time"
)
// RunMeta retains every existing queue metadata field and JSON tag. The
// optional fields are additive; zero extension fields are legal on old runs.
type RunMeta struct {
SupplyObservation *supply.Snapshot `json:"supplyObservation,omitempty"`
RunID string `json:"runId"`
Operation string `json:"operation"`
QueryMonth string `json:"queryMonth"`
Status RunStatus `json:"status"`
ProducerDone bool `json:"producerDone"`
CurrentRound int `json:"currentRound"`
ActiveCount int64 `json:"activeCount"`
InflightCount int64 `json:"inflightCount"`
NextRoundCount int64 `json:"nextRoundCount"`
SuccessCount int64 `json:"successCount"`
FailedCount int64 `json:"failedCount"`
CreatedAt time.Time `json:"createdAt"`
ProducerCompletedAt time.Time `json:"producerCompletedAt"`
CompletedAt time.Time `json:"completedAt"`
SyncType SyncType `json:"syncType,omitempty"`
SourceRunID string `json:"sourceRunId,omitempty"`
ReplayCycle int `json:"replayCycle,omitempty"`
TriggerSource string `json:"triggerSource,omitempty"`
ScannedCount int64 `json:"scannedCount,omitempty"`
EnqueuedCount int64 `json:"enqueuedCount,omitempty"`
UpstreamAttemptCount int64 `json:"upstreamAttemptCount,omitempty"`
BatchCount int64 `json:"batchCount,omitempty"`
MaxRound int `json:"maxRound,omitempty"`
PendingFailureCount int64 `json:"pendingFailureCount,omitempty"`
StartedAt *time.Time `json:"startedAt,omitempty"`
ScanCompletedAt *time.Time `json:"scanCompletedAt,omitempty"`
ArchiveStatus ArchiveStatus `json:"archiveStatus,omitempty"`
ArchiveVersion int `json:"archiveVersion,omitempty"`
ArchivePath string `json:"archivePath,omitempty"`
ArchiveChecksum string `json:"archiveChecksum,omitempty"`
ArchivedAt *time.Time `json:"archivedAt,omitempty"`
ReplayDisposition ReplayDisposition `json:"replayDisposition,omitempty"`
// Scope fields mirror zset.RunMeta. ScopeChannels == 0 means a full sync.
ScopeChannels int `json:"scopeChannels,omitempty"`
ScopeDigest string `json:"scopeDigest,omitempty"`
ScopeChannelsSample string `json:"scopeChannelsSample,omitempty"`
RoundStats []RoundStats `json:"rounds,omitempty"`
}
func (m RunMeta) Validate() error {
if err := ValidateID(m.RunID); err != nil {
return err
}
if m.Status != "" && !m.Status.Valid() {
return invalid("unknown run status")
}
if m.SyncType != "" && !m.SyncType.Valid() {
return invalid("unknown sync type")
}
if m.ArchiveStatus != "" && !m.ArchiveStatus.Valid() {
return invalid("unknown archive status")
}
if m.ReplayDisposition != "" && !m.ReplayDisposition.Valid() {
return invalid("unknown replay disposition")
}
if m.SourceRunID != "" {
if err := ValidateID(m.SourceRunID); err != nil {
return err
}
if m.SourceRunID == m.RunID {
return invalid("a run cannot replay itself")
}
}
if m.SyncType == SyncReplay && (m.SourceRunID == "" || m.ReplayCycle < 1) {
return invalid("replay requires sourceRunId and a positive replayCycle")
}
if m.SyncType == SyncNormal && (m.SourceRunID != "" || m.ReplayCycle != 0) {
return invalid("normal runs cannot have replay provenance")
}
if m.MaxRound > 0 && m.CurrentRound > m.MaxRound {
return invalid("currentRound exceeds maxRound")
}
return nonnegative(int64(m.CurrentRound), int64(m.MaxRound), int64(m.ReplayCycle), int64(m.ArchiveVersion), m.ActiveCount, m.InflightCount, m.NextRoundCount, m.SuccessCount, m.FailedCount, m.ScannedCount, m.EnqueuedCount, m.UpstreamAttemptCount, m.BatchCount, m.PendingFailureCount, int64(m.ScopeChannels))
}
// ValidateAccounting is opt-in for runs whose extended counters are known.
// Do not apply it to legacy metadata with a missing enqueuedCount. Inflight is
// a subset of active and is never added to the completion equation.
func (m RunMeta) ValidateAccounting() error {
if err := m.Validate(); err != nil {
return err
}
remaining := m.EnqueuedCount
for _, count := range []int64{m.SuccessCount, m.FailedCount, m.ActiveCount, m.NextRoundCount} {
if count > remaining {
return invalid("enqueuedCount does not balance run outcomes")
}
remaining -= count
}
if remaining != 0 || m.InflightCount > m.ActiveCount || m.PendingFailureCount > m.FailedCount {
return invalid("inconsistent run counters")
}
if m.Status.Terminal() && (!m.ProducerDone || m.ActiveCount != 0 || m.NextRoundCount != 0) {
return invalid("terminal run still has producer or queue work")
}
return nil
}
// RoundStats contains only bounded aggregate fields. It must never grow with
// the number of cards, channels, task IDs, upstream codes, or error messages.
type RoundStats struct {
RunID string `json:"runId"`
Round int `json:"round"`
EnqueuedCount int64 `json:"enqueuedCount"`
ActiveCount int64 `json:"activeCount"`
NextRoundCount int64 `json:"nextRoundCount"`
SuccessCount int64 `json:"successCount"`
FailedCount int64 `json:"failedCount"`
UpstreamAttemptCount int64 `json:"upstreamAttemptCount"`
BatchCount int64 `json:"batchCount"`
StartedAt *time.Time `json:"startedAt,omitempty"`
CompletedAt *time.Time `json:"completedAt,omitempty"`
ErrorCounts ErrorCounts `json:"errorCounts"`
}
func (s RoundStats) Validate() error {
if err := ValidateID(s.RunID); err != nil {
return err
}
if s.Round < 1 {
return invalid("round must be positive")
}
if err := nonnegative(s.EnqueuedCount, s.ActiveCount, s.NextRoundCount, s.SuccessCount, s.FailedCount, s.UpstreamAttemptCount, s.BatchCount); err != nil {
return err
}
_, err := s.ErrorCounts.Normalized()
return err
}
type RunView struct {
Source RunSource `json:"source"`
Run RunMeta `json:"run"`
Rounds []RoundStats `json:"rounds,omitempty"`
}
@@ -0,0 +1,137 @@
package contract
type RunStatus string
const (
RunProducing RunStatus = "producing"
RunRunning RunStatus = "running"
RunCompleted RunStatus = "completed"
RunCompletedWithFailures RunStatus = "completed_with_failures"
)
func (s RunStatus) Valid() bool {
return s == RunProducing || s == RunRunning || s.Terminal()
}
func (s RunStatus) Terminal() bool { return s == RunCompleted || s == RunCompletedWithFailures }
type SyncType string
const (
SyncNormal SyncType = "normal"
SyncReplay SyncType = "replay"
)
func (s SyncType) Valid() bool { return s == SyncNormal || s == SyncReplay }
type SyncKind string
const (
SyncKindTrafficUsage SyncKind = "traffic_usage"
SyncKindSimStatus SyncKind = "sim_status"
SyncKindTrafficUsageDaily SyncKind = "traffic_usage_daily"
SyncKindTrafficUsageMonthly SyncKind = "traffic_usage_monthly"
SyncKindImei SyncKind = "imei"
)
func (s SyncKind) Valid() bool {
return s == SyncKindTrafficUsage || s == SyncKindSimStatus || s == SyncKindTrafficUsageDaily || s == SyncKindTrafficUsageMonthly || s == SyncKindImei
}
func SyncKindForOperation(operation string) SyncKind {
switch operation {
case "current_month_usage_full_sync":
return SyncKindTrafficUsage
case "sim_status_queue_full_sync":
return SyncKindSimStatus
case "previous_day_usage_full_sync":
return SyncKindTrafficUsageDaily
case "previous_month_usage_full_sync":
return SyncKindTrafficUsageMonthly
case "imei_full_sync":
return SyncKindImei
default:
return ""
}
}
type ArchiveStatus string
const (
ArchiveDisabled ArchiveStatus = "disabled"
ArchivePending ArchiveStatus = "pending"
ArchiveProcessing ArchiveStatus = "processing"
ArchiveSucceeded ArchiveStatus = "succeeded"
ArchiveFailed ArchiveStatus = "failed"
)
func (s ArchiveStatus) Valid() bool {
switch s {
case ArchiveDisabled, ArchivePending, ArchiveProcessing, ArchiveSucceeded, ArchiveFailed:
return true
}
return false
}
// ReplayDisposition describes a source run, not an individual replay action.
type ReplayDisposition string
const (
ReplayNone ReplayDisposition = "none"
ReplayPending ReplayDisposition = "pending"
ReplayProcessing ReplayDisposition = "processing"
ReplayCompleted ReplayDisposition = "completed"
ReplayNeedsAttention ReplayDisposition = "needs_attention"
)
func (s ReplayDisposition) Valid() bool {
switch s {
case ReplayNone, ReplayPending, ReplayProcessing, ReplayCompleted, ReplayNeedsAttention:
return true
}
return false
}
type ActionStatus string
const (
ActionAccepted ActionStatus = "accepted"
ActionProcessing ActionStatus = "processing"
ActionCleanupPending ActionStatus = "cleanup_pending"
ActionReceiptPending ActionStatus = "receipt_pending"
ActionCompleted ActionStatus = "completed"
ActionNeedsAttention ActionStatus = "needs_attention"
)
func (s ActionStatus) Valid() bool {
switch s {
case ActionAccepted, ActionProcessing, ActionCleanupPending, ActionReceiptPending, ActionCompleted, ActionNeedsAttention:
return true
}
return false
}
// Terminal means automatic work has stopped; needs_attention may be resumed
// only by an explicit recovery policy, never by assuming it succeeded.
func (s ActionStatus) Terminal() bool { return s == ActionCompleted || s == ActionNeedsAttention }
type RunSource string
const (
SourceLive RunSource = "live"
SourceArchive RunSource = "archive"
)
func (s RunSource) Valid() bool { return s == SourceLive || s == SourceArchive }
// These values are already persisted by the queue protocol.
const (
TaskQueued = "queued"
TaskClaimed = "claimed"
TaskWaitingNextRound = "waiting_next_round"
TaskSucceeded = "succeeded"
TaskCovered = "covered"
TaskFailedPendingReview = "failed_pending_review"
TaskChannelBlocked = "channel_blocked"
ReviewPending = "pending"
)
@@ -0,0 +1,57 @@
package contract_test
import (
"testing"
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract"
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/model"
)
// TestSyncKindForDailyUsageOperation guards the mapping that makes a daily run
// discoverable. Catalog.ListRuns drops any run whose operation maps to an empty
// kind, so losing this case would make daily runs invisible in the monitor
// without any error.
func TestSyncKindForDailyUsageOperation(t *testing.T) {
if got := contract.SyncKindForOperation(model.OperationPreviousDayUsageFullSync); got != contract.SyncKindTrafficUsageDaily {
t.Fatalf("SyncKindForOperation(%q) = %q, want %q",
model.OperationPreviousDayUsageFullSync, got, contract.SyncKindTrafficUsageDaily)
}
if !contract.SyncKindTrafficUsageDaily.Valid() {
t.Fatal("SyncKindTrafficUsageDaily must be a valid kind")
}
if contract.SyncKindForOperation("some_unknown_operation") != "" {
t.Fatal("an unknown operation must not be assigned a sync kind")
}
}
// TestSyncKindForMonthlyHistoryOperation guards the mapping that makes a monthly
// history run discoverable. Catalog.ListRuns drops any run whose operation maps
// to an empty kind, so losing this case would make monthly history runs
// invisible in the monitor without any error.
func TestSyncKindForMonthlyHistoryOperation(t *testing.T) {
if got := contract.SyncKindForOperation(model.OperationPreviousMonthUsageFullSync); got != contract.SyncKindTrafficUsageMonthly {
t.Fatalf("SyncKindForOperation(%q) = %q, want %q",
model.OperationPreviousMonthUsageFullSync, got, contract.SyncKindTrafficUsageMonthly)
}
}
// TestDailyAndMonthlyUsageHaveDistinctSyncKinds keeps the usage runs separately
// filterable. There are three distinct usage runs now: the MySQL current-month
// snapshot, the TDengine daily history, and the TDengine monthly history.
func TestDailyAndMonthlyUsageHaveDistinctSyncKinds(t *testing.T) {
monthly := contract.SyncKindForOperation("current_month_usage_full_sync")
daily := contract.SyncKindForOperation(model.OperationPreviousDayUsageFullSync)
monthlyHistory := contract.SyncKindForOperation(model.OperationPreviousMonthUsageFullSync)
if monthly == daily {
t.Fatalf("current-month snapshot and daily history share the sync kind %q", daily)
}
if monthly == monthlyHistory {
t.Fatalf("current-month snapshot and monthly history share the sync kind %q", monthlyHistory)
}
if daily == monthlyHistory {
t.Fatalf("daily and monthly history share the sync kind %q", daily)
}
if !monthlyHistory.Valid() {
t.Fatal("SyncKindTrafficUsageMonthly must be a valid kind")
}
}
@@ -0,0 +1,85 @@
package contract_test
import (
"testing"
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract"
)
// TestSyncKindForOperationCoversEveryChain pins the operation→kind mapping for
// all five full-sync chains. A missing case makes the run invisible in the
// admin monitor, because the catalog filters runs by SyncKind.
func TestSyncKindForOperationCoversEveryChain(t *testing.T) {
cases := []struct {
operation string
want contract.SyncKind
}{
{"current_month_usage_full_sync", contract.SyncKindTrafficUsage},
{"sim_status_queue_full_sync", contract.SyncKindSimStatus},
{"previous_day_usage_full_sync", contract.SyncKindTrafficUsageDaily},
{"previous_month_usage_full_sync", contract.SyncKindTrafficUsageMonthly},
{"imei_full_sync", contract.SyncKindImei},
}
for _, tc := range cases {
got := contract.SyncKindForOperation(tc.operation)
if got != tc.want {
t.Fatalf("SyncKindForOperation(%q) = %q, want %q", tc.operation, got, tc.want)
}
if !got.Valid() {
t.Fatalf("%q 应通过 Valid()", got)
}
}
}
// TestSyncKindForOperationRejectsUnknown 未知 operation 返回空 kind,
// 由调用方决定是跳过还是报错。
func TestSyncKindForOperationRejectsUnknown(t *testing.T) {
for _, operation := range []string{"", "unknown", "IMEI_FULL_SYNC"} {
got := contract.SyncKindForOperation(operation)
if got != "" {
t.Fatalf("SyncKindForOperation(%q) = %q, want empty", operation, got)
}
if got.Valid() {
t.Fatalf("空 kind 不应通过 Valid()")
}
}
}
// TestSyncKindValidRejectsUnknown 只有五个已注册的 kind 有效。
func TestSyncKindValidRejectsUnknown(t *testing.T) {
valid := []contract.SyncKind{
contract.SyncKindTrafficUsage,
contract.SyncKindSimStatus,
contract.SyncKindTrafficUsageDaily,
contract.SyncKindTrafficUsageMonthly,
contract.SyncKindImei,
}
for _, kind := range valid {
if !kind.Valid() {
t.Fatalf("%q 应有效", kind)
}
}
invalid := []contract.SyncKind{"", "imei_full_sync", "IMEI", "traffic", "sim-status"}
for _, kind := range invalid {
if kind.Valid() {
t.Fatalf("%q 不应有效", kind)
}
}
}
// TestSyncKindValuesAreStable 枚举字符串是持久化契约(写入 Run 元数据、
// 归档 JSON 与 admin 查询参数),不得随意改动。
func TestSyncKindValuesAreStable(t *testing.T) {
cases := map[contract.SyncKind]string{
contract.SyncKindTrafficUsage: "traffic_usage",
contract.SyncKindSimStatus: "sim_status",
contract.SyncKindTrafficUsageDaily: "traffic_usage_daily",
contract.SyncKindTrafficUsageMonthly: "traffic_usage_monthly",
contract.SyncKindImei: "imei",
}
for kind, want := range cases {
if string(kind) != want {
t.Fatalf("kind = %q, want %q(持久化契约不得改动)", kind, want)
}
}
}
@@ -0,0 +1,42 @@
package contract
import (
"strings"
"unicode"
"unicode/utf8"
"github.com/gogf/gf/v2/errors/gerror"
)
// Implementations wrap these errors so callers can use errors.Is without
// inspecting error text. HTTP adapters choose the response envelope themselves.
var (
ErrNotFound = gerror.New("full-sync resource not found")
ErrConflict = gerror.New("full-sync conflict")
ErrInvalidArgument = gerror.New("invalid full-sync argument")
ErrFailureDetailsUnavailable = gerror.New("live failure details unavailable")
ErrLeaseLost = gerror.New("full-sync lease lost")
ErrBusy = gerror.New("full-sync resource busy")
)
func invalid(format string, args ...any) error {
return gerror.Wrapf(ErrInvalidArgument, format, args...)
}
// ValidateID checks new resource identifiers used both as Redis hash tags and
// HTTP path segments. It does not rewrite historical identifiers during decode.
func ValidateID(id string) error {
if id == "" || strings.TrimSpace(id) != id || !utf8.ValidString(id) || strings.ContainsAny(id, "{}/\\") || strings.ContainsFunc(id, unicode.IsControl) {
return invalid("invalid resource identifier")
}
return nil
}
func nonnegative(values ...int64) error {
for _, value := range values {
if value < 0 {
return invalid("counts must be nonnegative")
}
}
return nil
}
@@ -0,0 +1,121 @@
package contract_test
import (
"errors"
"math"
"testing"
"git.5iot.com/iotlcm-public/iotlcm-worker-svr/internal/fullsync/contract"
)
func TestRunAccounting(t *testing.T) {
valid := contract.RunMeta{
RunID: "run", Status: contract.RunRunning, CurrentRound: 1, MaxRound: 5,
EnqueuedCount: 10, ActiveCount: 3, InflightCount: 2, NextRoundCount: 2,
SuccessCount: 4, FailedCount: 1, PendingFailureCount: 1,
}
if err := valid.ValidateAccounting(); err != nil {
t.Fatal(err)
}
for name, mutate := range map[string]func(*contract.RunMeta){
"unbalanced": func(m *contract.RunMeta) { m.EnqueuedCount++ },
"negative": func(m *contract.RunMeta) { m.BatchCount = -1 },
"overflow": func(m *contract.RunMeta) {
m.EnqueuedCount = math.MaxInt64
m.SuccessCount = math.MaxInt64
m.FailedCount = math.MaxInt64
},
"inflight double count": func(m *contract.RunMeta) { m.InflightCount = m.ActiveCount + 1 },
"pending exceeds history": func(m *contract.RunMeta) { m.PendingFailureCount = 2 },
"terminal with work": func(m *contract.RunMeta) { m.Status = contract.RunCompletedWithFailures; m.ProducerDone = true },
"unknown status": func(m *contract.RunMeta) { m.Status = "succeeded" },
"unknown sync type": func(m *contract.RunMeta) { m.SyncType = "manual" },
"action status in archive": func(m *contract.RunMeta) { m.ArchiveStatus = "completed" },
"action status in disposition": func(m *contract.RunMeta) { m.ReplayDisposition = "cleanup_pending" },
"round out of range": func(m *contract.RunMeta) { m.CurrentRound = 6 },
"replay without source": func(m *contract.RunMeta) { m.SyncType = contract.SyncReplay },
"self replay": func(m *contract.RunMeta) {
m.SyncType = contract.SyncReplay
m.SourceRunID = m.RunID
m.ReplayCycle = 1
},
"normal with provenance": func(m *contract.RunMeta) { m.SyncType = contract.SyncNormal; m.SourceRunID = "source" },
} {
t.Run(name, func(t *testing.T) {
meta := valid
mutate(&meta)
if err := meta.ValidateAccounting(); !errors.Is(err, contract.ErrInvalidArgument) {
t.Fatalf("invalid accounting returned %v", err)
}
})
}
// Cleaned source failures never subtract from the historical failed total.
completed := contract.RunMeta{RunID: "source", Status: contract.RunCompletedWithFailures, ProducerDone: true, EnqueuedCount: 5, SuccessCount: 3, FailedCount: 2, PendingFailureCount: 0}
if err := completed.ValidateAccounting(); err != nil {
t.Fatal(err)
}
completed.ProducerDone = false
if err := completed.ValidateAccounting(); err == nil {
t.Fatal("accepted terminal run with unfinished producer")
}
if err := (contract.RunMeta{RunID: "old", Status: contract.RunRunning, SuccessCount: 42}).Validate(); err != nil {
t.Fatalf("historical run without new counters rejected: %v", err)
}
}
func TestRoundStatsValidation(t *testing.T) {
valid := contract.RoundStats{RunID: "run", Round: 1, ErrorCounts: contract.ErrorCounts{"unrecognized": 1}}
if err := valid.Validate(); err != nil {
t.Fatal(err)
}
for _, stats := range []contract.RoundStats{
{RunID: "run", Round: 0}, {RunID: "run", Round: 1, BatchCount: -1},
{RunID: "run", Round: 1, ErrorCounts: contract.ErrorCounts{contract.ErrorOther: -1}},
} {
if err := stats.Validate(); !errors.Is(err, contract.ErrInvalidArgument) {
t.Fatalf("invalid round stats returned %v", err)
}
}
}
func TestPageLimitsAndOpaqueCursor(t *testing.T) {
for _, limit := range []int{0, 1, 100, 500} {
request, err := (contract.PageRequest{Limit: limit, Cursor: "opaque+/=:%?next"}).Normalized()
if err != nil || request.Cursor != "opaque+/=:%?next" || request.Limit < 1 || request.Limit > contract.MaxPageLimit {
t.Fatalf("unexpected page: %+v, %v", request, err)
}
if limit == 0 && request.Limit != 100 {
t.Fatal("default limit changed")
}
}
for _, limit := range []int{-1, 501, math.MaxInt} {
if _, err := (contract.PageRequest{Limit: limit}).Normalized(); !errors.Is(err, contract.ErrInvalidArgument) {
t.Fatalf("limit %d returned %v", limit, err)
}
}
page := contract.FailurePage{Items: []contract.Failure{}, NextCursor: "continue"}
if got := string(mustJSON(t, page)); got != `{"items":[],"nextCursor":"continue"}` {
t.Fatalf("empty scan page lost its continuation: %s", got)
}
}
func TestStatusVocabulary(t *testing.T) {
for _, status := range []contract.ActionStatus{"accepted", "processing", "cleanup_pending", "receipt_pending", "completed", "needs_attention"} {
if !status.Valid() || status.Terminal() != (status == "completed" || status == "needs_attention") {
t.Errorf("invalid action status %q", status)
}
}
for _, status := range []contract.ArchiveStatus{"disabled", "pending", "processing", "succeeded", "failed"} {
if !status.Valid() {
t.Errorf("invalid archive status %q", status)
}
}
for _, status := range []contract.ReplayDisposition{"none", "pending", "processing", "completed", "needs_attention"} {
if !status.Valid() {
t.Errorf("invalid disposition %q", status)
}
}
if contract.ActionStatus("succeeded").Valid() || contract.ArchiveStatus("completed").Valid() || contract.ReplayDisposition("receipt_pending").Valid() {
t.Fatal("status domains were conflated")
}
}